1.\" Copyright (c) 2010 The FreeBSD Foundation 2.\" Copyright (c) 2010-2012 Pawel Jakub Dawidek <pawel@dawidek.net> 3.\" All rights reserved. 4.\" 5.\" This documentation was written by Pawel Jakub Dawidek under sponsorship from 6.\" the FreeBSD Foundation. 7.\" 8.\" Redistribution and use in source and binary forms, with or without 9.\" modification, are permitted provided that the following conditions 10.\" are met: 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in the 15.\" documentation and/or other materials provided with the distribution. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND 18.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 20.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE 21.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 23.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 24.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 25.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 26.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 27.\" SUCH DAMAGE. 28.\" 29.\" $FreeBSD$ 30.\" 31.Dd January 25, 2012 32.Dt HAST.CONF 5 33.Os 34.Sh NAME 35.Nm hast.conf 36.Nd configuration file for the 37.Xr hastd 8 38daemon and the 39.Xr hastctl 8 40utility 41.Sh DESCRIPTION 42The 43.Nm 44file is used by both 45.Xr hastd 8 46daemon 47and 48.Xr hastctl 8 49control utility. 50Configuration file is designed in a way that exactly the same file can be 51(and should be) used on both HAST nodes. 52Every line starting with # is treated as comment and ignored. 53.Sh CONFIGURATION FILE SYNTAX 54General syntax of the 55.Nm 56file is following: 57.Bd -literal -offset indent 58# Global section 59control <addr> 60listen <addr> 61replication <mode> 62checksum <algorithm> 63compression <algorithm> 64timeout <seconds> 65exec <path> 66metaflush on | off 67pidfile <path> 68 69on <node> { 70 # Node section 71 control <addr> 72 listen <addr> 73 pidfile <path> 74} 75 76on <node> { 77 # Node section 78 control <addr> 79 listen <addr> 80 pidfile <path> 81} 82 83resource <name> { 84 # Resource section 85 replication <mode> 86 checksum <algorithm> 87 compression <algorithm> 88 name <name> 89 local <path> 90 timeout <seconds> 91 exec <path> 92 metaflush on | off 93 94 on <node> { 95 # Resource-node section 96 name <name> 97 # Required 98 local <path> 99 metaflush on | off 100 # Required 101 remote <addr> 102 source <addr> 103 } 104 on <node> { 105 # Resource-node section 106 name <name> 107 # Required 108 local <path> 109 metaflush on | off 110 # Required 111 remote <addr> 112 source <addr> 113 } 114} 115.Ed 116.Pp 117Most of the various available configuration parameters are optional. 118If parameter is not defined in the particular section, it will be 119inherited from the parent section. 120For example, if the 121.Ic listen 122parameter is not defined in the node section, it will be inherited from 123the global section. 124In case the global section does not define the 125.Ic listen 126parameter at all, the default value will be used. 127.Sh CONFIGURATION FILE DESCRIPTION 128The 129.Aq node 130argument can be replaced either by a full hostname as obtained by 131.Xr gethostname 3 , 132only first part of the hostname, by node's UUID as found in the 133.Va kern.hostuuid 134.Xr sysctl 8 135variable 136or by node's hostid as found in the 137.Va kern.hostid 138.Xr sysctl 8 139variable. 140.Pp 141The following statements are available: 142.Bl -tag -width ".Ic xxxx" 143.It Ic control Aq addr 144.Pp 145Address for communication with 146.Xr hastctl 8 . 147Each of the following examples defines the same control address: 148.Bd -literal -offset indent 149uds:///var/run/hastctl 150unix:///var/run/hastctl 151/var/run/hastctl 152.Ed 153.Pp 154The default value is 155.Pa uds:///var/run/hastctl . 156.It Ic pidfile Aq path 157.Pp 158File in which to store the process ID of the main 159.Xr hastd 8 160process. 161.Pp 162The default value is 163.Pa /var/run/hastd.pid . 164.It Ic listen Aq addr 165.Pp 166Address to listen on in form of: 167.Bd -literal -offset indent 168protocol://protocol-specific-address 169.Ed 170.Pp 171Each of the following examples defines the same listen address: 172.Bd -literal -offset indent 1730.0.0.0 1740.0.0.0:8457 175tcp://0.0.0.0 176tcp://0.0.0.0:8457 177tcp4://0.0.0.0 178tcp4://0.0.0.0:8457 179.Ed 180.Pp 181Multiple listen addresses can be specified. 182By default 183.Nm hastd 184listens on 185.Pa tcp4://0.0.0.0:8457 186and 187.Pa tcp6://[::]:8457 188if kernel supports IPv4 and IPv6 respectively. 189.It Ic replication Aq mode 190.Pp 191Replication mode should be one of the following: 192.Bl -tag -width ".Ic xxxx" 193.It Ic memsync 194.Pp 195Report the write operation as completed when local write completes and 196when the remote node acknowledges the data receipt, but before it 197actually stores the data. 198The data on remote node will be stored directly after sending 199acknowledgement. 200This mode is intended to reduce latency, but still provides a very good 201reliability. 202The only situation where some small amount of data could be lost is when 203the data is stored on primary node and sent to the secondary. 204Secondary node then acknowledges data receipt and primary reports 205success to an application. 206However, it may happen that the secondary goes down before the received 207data is really stored locally. 208Before secondary node returns, primary node dies entirely. 209When the secondary node comes back to life it becomes the new primary. 210Unfortunately some small amount of data which was confirmed to be stored 211to the application was lost. 212The risk of such a situation is very small. 213The 214.Ic memsync 215replication mode is the default. 216.It Ic fullsync 217.Pp 218Mark the write operation as completed when local as well as remote 219write completes. 220This is the safest and the slowest replication mode. 221.It Ic async 222.Pp 223The write operation is reported as complete right after the local write 224completes. 225This is the fastest and the most dangerous replication mode. 226This mode should be used when replicating to a distant node where 227latency is too high for other modes. 228.El 229.It Ic checksum Aq algorithm 230.Pp 231Checksum algorithm should be one of the following: 232.Bl -tag -width ".Ic sha256" 233.It Ic none 234No checksum will be calculated for the data being send over the network. 235This is the default setting. 236.It Ic crc32 237CRC32 checksum will be calculated. 238.It Ic sha256 239SHA256 checksum will be calculated. 240.El 241.It Ic compression Aq algorithm 242.Pp 243Compression algorithm should be one of the following: 244.Bl -tag -width ".Ic none" 245.It Ic none 246Data send over the network will not be compressed. 247.It Ic hole 248Only blocks that contain all zeros will be compressed. 249This is very useful for initial synchronization where potentially many blocks 250are still all zeros. 251There should be no measurable performance overhead when this algorithm is being 252used. 253This is the default setting. 254.It Ic lzf 255The LZF algorithm by Marc Alexander Lehmann will be used to compress the data 256send over the network. 257LZF is very fast, general purpose compression algorithm. 258.El 259.It Ic timeout Aq seconds 260.Pp 261Connection timeout in seconds. 262The default value is 263.Va 20 . 264.It Ic exec Aq path 265.Pp 266Execute the given program on various HAST events. 267Below is the list of currently implemented events and arguments the given 268program is executed with: 269.Bl -tag -width ".Ic xxxx" 270.It Ic "<path> role <resource> <oldrole> <newrole>" 271.Pp 272Executed on both primary and secondary nodes when resource role is changed. 273.It Ic "<path> connect <resource>" 274.Pp 275Executed on both primary and secondary nodes when connection for the given 276resource between the nodes is established. 277.It Ic "<path> disconnect <resource>" 278.Pp 279Executed on both primary and secondary nodes when connection for the given 280resource between the nodes is lost. 281.It Ic "<path> syncstart <resource>" 282.Pp 283Executed on primary node when synchronization process of secondary node is 284started. 285.It Ic "<path> syncdone <resource>" 286.Pp 287Executed on primary node when synchronization process of secondary node is 288completed successfully. 289.It Ic "<path> syncintr <resource>" 290.Pp 291Executed on primary node when synchronization process of secondary node is 292interrupted, most likely due to secondary node outage or connection failure 293between the nodes. 294.It Ic "<path> split-brain <resource>" 295.Pp 296Executed on both primary and secondary nodes when split-brain condition is 297detected. 298.El 299.Pp 300The 301.Aq path 302argument should contain full path to executable program. 303If the given program exits with code different than 304.Va 0 , 305.Nm hastd 306will log it as an error. 307.Pp 308The 309.Aq resource 310argument is resource name from the configuration file. 311.Pp 312The 313.Aq oldrole 314argument is previous resource role (before the change). 315It can be one of: 316.Ar init , 317.Ar secondary , 318.Ar primary . 319.Pp 320The 321.Aq newrole 322argument is current resource role (after the change). 323It can be one of: 324.Ar init , 325.Ar secondary , 326.Ar primary . 327.It Ic metaflush on | off 328.Pp 329When set to 330.Va on , 331flush write cache of the local provider after every metadata (activemap) update. 332Flushing write cache ensures that provider will not reorder writes and that 333metadata will be properly updated before real data is stored. 334If the local provider does not support flushing write cache (it returns 335.Er EOPNOTSUPP 336on the 337.Cm BIO_FLUSH 338request), 339.Nm hastd 340will disable 341.Ic metaflush 342automatically. 343The default value is 344.Va on . 345.It Ic name Aq name 346.Pp 347GEOM provider name that will appear as 348.Pa /dev/hast/<name> . 349If name is not defined, resource name will be used as provider name. 350.It Ic local Aq path 351.Pp 352Path to the local component which will be used as backend provider for 353the resource. 354This can be either GEOM provider or regular file. 355.It Ic remote Aq addr 356.Pp 357Address of the remote 358.Nm hastd 359daemon. 360Format is the same as for the 361.Ic listen 362statement. 363When operating as a primary node this address will be used to connect to 364the secondary node. 365When operating as a secondary node only connections from this address 366will be accepted. 367.Pp 368A special value of 369.Va none 370can be used when the remote address is not yet known (eg. the other node is not 371set up yet). 372.It Ic source Aq addr 373.Pp 374Local address to bind to before connecting to the remote 375.Nm hastd 376daemon. 377Format is the same as for the 378.Ic listen 379statement. 380.El 381.Sh FILES 382.Bl -tag -width ".Pa /var/run/hastctl" -compact 383.It Pa /etc/hast.conf 384The default 385.Xr hastctl 8 386and 387.Xr hastd 8 388configuration file. 389.It Pa /var/run/hastctl 390Control socket used by the 391.Xr hastctl 8 392control utility to communicate with the 393.Xr hastd 8 394daemon. 395.El 396.Sh EXAMPLES 397The example configuration file can look as follows: 398.Bd -literal -offset indent 399listen tcp://0.0.0.0 400 401on hasta { 402 listen tcp://2001:db8::1/64 403} 404on hastb { 405 listen tcp://2001:db8::2/64 406} 407 408resource shared { 409 local /dev/da0 410 411 on hasta { 412 remote tcp://10.0.0.2 413 } 414 on hastb { 415 remote tcp://10.0.0.1 416 } 417} 418resource tank { 419 on hasta { 420 local /dev/mirror/tanka 421 source tcp://10.0.0.1 422 remote tcp://10.0.0.2 423 } 424 on hastb { 425 local /dev/mirror/tankb 426 source tcp://10.0.0.2 427 remote tcp://10.0.0.1 428 } 429} 430.Ed 431.Sh SEE ALSO 432.Xr gethostname 3 , 433.Xr geom 4 , 434.Xr hastctl 8 , 435.Xr hastd 8 436.Sh AUTHORS 437The 438.Nm 439was written by 440.An Pawel Jakub Dawidek Aq Mt pjd@FreeBSD.org 441under sponsorship of the FreeBSD Foundation. 442