1.\" Copyright (c) 2010 The FreeBSD Foundation 2.\" All rights reserved. 3.\" 4.\" This software was developed by Pawel Jakub Dawidek under sponsorship from 5.\" the FreeBSD Foundation. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions and the following disclaimer. 12.\" 2. Redistributions in binary form must reproduce the above copyright 13.\" notice, this list of conditions and the following disclaimer in the 14.\" documentation and/or other materials provided with the distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND 17.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 18.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 19.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE 20.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 21.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 22.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 23.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 24.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 25.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 26.\" SUCH DAMAGE. 27.\" 28.\" $FreeBSD$ 29.\" 30.Dd December 21, 2019 31.Dt HASTD 8 32.Os 33.Sh NAME 34.Nm hastd 35.Nd "Highly Available Storage daemon" 36.Sh SYNOPSIS 37.Nm 38.Op Fl dFh 39.Op Fl c Ar config 40.Op Fl P Ar pidfile 41.Sh DESCRIPTION 42The 43.Nm 44daemon is responsible for managing highly available GEOM providers. 45.Pp 46.Nm 47allows the transparent storage of data on two physically separated machines 48connected over a TCP/IP network. 49Only one machine (cluster node) can actively use storage provided by 50.Nm . 51This machine is called primary. 52The 53.Nm 54daemon operates on block level, which makes it transparent to file 55systems and applications. 56.Pp 57There is one main 58.Nm 59daemon which starts new worker process as soon as a role for the given 60resource is changed to primary or as soon as a role for the given 61resource is changed to secondary and remote (primary) node will 62successfully connect to it. 63Every worker process gets a new process title (see 64.Xr setproctitle 3 ) , 65which describes its role and resource it controls. 66The exact format is: 67.Bd -literal -offset indent 68hastd: <resource name> (<role>) 69.Ed 70.Pp 71If (and only if) 72.Nm 73operates in primary role for the given resource, a corresponding 74.Pa /dev/hast/<name> 75disk-like device (GEOM provider) is created. 76File systems and applications can use this provider to send I/O 77requests to. 78Every write, delete and flush operation 79.Dv ( BIO_WRITE , BIO_DELETE , BIO_FLUSH ) 80is sent to the local component and replicated on the remote (secondary) node 81if it is available. 82Read operations 83.Dv ( BIO_READ ) 84are handled locally unless an I/O error occurs or the local version of the data 85is not up-to-date yet (synchronization is in progress). 86.Pp 87The 88.Nm 89daemon uses the GEOM Gate class to receive I/O requests from the 90in-kernel GEOM infrastructure. 91The 92.Nm geom_gate.ko 93module is loaded automatically if the kernel was not compiled with the 94following option: 95.Bd -ragged -offset indent 96.Cd "options GEOM_GATE" 97.Ed 98.Pp 99The connection between two 100.Nm 101daemons is always initiated from the one running as primary to the one 102running as secondary. 103When the primary 104.Nm 105is unable to connect or the connection fails, it will try to re-establish 106the connection every few seconds. 107Once the connection is established, the primary 108.Nm 109will synchronize every extent that was modified during connection outage 110to the secondary 111.Nm . 112.Pp 113It is possible that in the case of a connection outage between the nodes the 114.Nm 115primary role for the given resource will be configured on both nodes. 116This in turn leads to incompatible data modifications. 117Such a condition is called a split-brain and cannot be automatically 118resolved by the 119.Nm 120daemon as this will lead most likely to data corruption or loss of 121important changes. 122Even though it cannot be fixed by 123.Nm 124itself, it will be detected and a further connection between independently 125modified nodes will not be possible. 126Once this situation is manually resolved by an administrator, the resource 127on one of the nodes can be initialized (erasing local data), which makes 128a connection to the remote node possible again. 129Connection of the freshly initialized component will trigger full resource 130synchronization. 131.Pp 132A 133.Nm 134daemon never picks its role automatically. 135The role has to be configured with the 136.Xr hastctl 8 137control utility by additional software like 138.Nm ucarp 139or 140.Nm heartbeat 141that can reliably manage role separation and switch secondary node to 142primary role in case of the primary's failure. 143.Pp 144The 145.Nm 146daemon can be started with the following command line arguments: 147.Bl -tag -width ".Fl P Ar pidfile" 148.It Fl c Ar config 149Specify alternative location of the configuration file. 150The default location is 151.Pa /etc/hast.conf . 152.It Fl d 153Print or log debugging information. 154This option can be specified multiple times to raise the verbosity 155level. 156.It Fl F 157Start the 158.Nm 159daemon in the foreground. 160By default 161.Nm 162starts in the background. 163.It Fl h 164Print the 165.Nm 166usage message. 167.It Fl P Ar pidfile 168Specify alternative location of a file where main process PID will be 169stored. 170The default location is 171.Pa /var/run/hastd.pid . 172.El 173.Sh FILES 174.Bl -tag -width ".Pa /var/run/hastd.pid" -compact 175.It Pa /etc/hast.conf 176The configuration file for 177.Nm 178and 179.Xr hastctl 8 . 180.It Pa /var/run/hastctl 181Control socket used by the 182.Xr hastctl 8 183control utility to communicate with 184.Nm . 185.It Pa /var/run/hastd.pid 186The default location of the 187.Nm 188PID file. 189.El 190.Sh EXIT STATUS 191Exit status is 0 on success, or one of the values described in 192.Xr sysexits 3 193on failure. 194.Sh EXAMPLES 195Launch 196.Nm 197on both nodes. 198Set role for resource 199.Nm shared 200to primary on 201.Nm nodeA 202and to secondary on 203.Nm nodeB . 204Create file system on 205.Pa /dev/hast/shared 206provider and mount it. 207.Bd -literal -offset indent 208nodeB# hastd 209nodeB# hastctl role secondary shared 210 211nodeA# hastd 212nodeA# hastctl role primary shared 213nodeA# newfs -U /dev/hast/shared 214nodeA# mount -o noatime /dev/hast/shared /shared 215.Ed 216.Sh SEE ALSO 217.Xr sysexits 3 , 218.Xr geom 4 , 219.Xr hast.conf 5 , 220.Xr ggatec 8 , 221.Xr ggated 8 , 222.Xr ggatel 8 , 223.Xr hastctl 8 , 224.Xr mount 8 , 225.Xr newfs 8 , 226.Xr g_bio 9 227.Sh HISTORY 228The 229.Nm 230utility appeared in 231.Fx 8.1 . 232.Sh AUTHORS 233The 234.Nm 235was developed by 236.An Pawel Jakub Dawidek Aq Mt pjd@FreeBSD.org 237under sponsorship of the FreeBSD Foundation. 238