1.\" Copyright (c) 2010 The FreeBSD Foundation 2.\" All rights reserved. 3.\" 4.\" This software was developed by Pawel Jakub Dawidek under sponsorship from 5.\" the FreeBSD Foundation. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions and the following disclaimer. 12.\" 2. Redistributions in binary form must reproduce the above copyright 13.\" notice, this list of conditions and the following disclaimer in the 14.\" documentation and/or other materials provided with the distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND 17.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 18.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 19.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE 20.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 21.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 22.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 23.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 24.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 25.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 26.\" SUCH DAMAGE. 27.\" 28.\" $FreeBSD$ 29.\" 30.Dd February 1, 2010 31.Dt HASTD 8 32.Os 33.Sh NAME 34.Nm hastd 35.Nd "Highly Available Storage daemon" 36.Sh SYNOPSIS 37.Nm 38.Op Fl dFh 39.Op Fl c Ar config 40.Op Fl P Ar pidfile 41.Sh DESCRIPTION 42The 43.Nm 44daemon is responsible for managing highly available GEOM providers. 45.Pp 46.Nm 47allows to transparently store data on two physically separated machines 48connected over the TCP/IP network. 49Only one machine (cluster node) can actively use storage provided by 50.Nm . 51This machine is called primary. 52The 53.Nm 54daemon operates on block level, which makes it transparent for file 55systems and applications. 56.Pp 57There is one main 58.Nm 59daemon which starts new worker process as soon as a role for the given 60resource is changed to primary or as soon as a role for the given 61resource is changed to secondary and remote (primary) node will 62successfully connect to it. 63Every worker process gets a new process title (see 64.Xr setproctitle 3 ) , 65which describes its role and resource it controls. 66The exact format is: 67.Bd -literal -offset indent 68hastd: <resource name> (<role>) 69.Ed 70.Pp 71When (and only when) 72.Nm 73operates in primary role for the given resource, corresponding 74.Pa /dev/hast/<name> 75disk-like device (GEOM provider) is created. 76File systems and applications can use this provider to send I/O 77requests to. 78Every write, delete and flush operation 79.Dv ( BIO_WRITE , BIO_DELETE , BIO_FLUSH ) 80is send to local component and synchronously replicated 81to the remote (secondary) node if it is available. 82Read operations 83.Dv ( BIO_READ ) 84are handled locally unless I/O error occurs or local version of the data 85is not up-to-date yet (synchronization is in progress). 86.Pp 87The 88.Nm 89daemon uses the GEOM Gate class to receive I/O requests from the 90in-kernel GEOM infrastructure. 91The 92.Nm geom_gate.ko 93module is loaded automatically if the kernel was not compiled with the 94following option: 95.Bd -ragged -offset indent 96.Cd "options GEOM_GATE" 97.Ed 98.Pp 99The connection between two 100.Nm 101daemons is always initiated from the one running as primary to the one 102running as secondary. 103When primary 104.Nm 105is unable to connect or connection fails, it will try to re-establish 106connection every few seconds. 107Once connection is established, primary 108.Nm 109will synchronize every extent that was modified during connection outage 110to the secondary 111.Nm . 112.Pp 113It is possible that in case of connection outage between the nodes 114.Nm 115primary role for the given resource will be configured on both nodes. 116This in turn leads to incompatible data modifications. 117Such condition is called split-brain and cannot be automatically 118resolved by the 119.Nm 120daemon as this will lead most likely to data corruption or lost of 121important changes. 122Even though it cannot be fixed by 123.Nm 124itself, it will be detected and further connection between independently 125modified nodes will not be possible. 126Once this situation is manually resolved by an administrator, resource 127on one of the nodes can be initialized (erasing local data), which makes 128connection to the remote node possible again. 129Connection of freshly initialized component will trigger full resource 130synchronization. 131.Pp 132The 133.Nm 134daemon itself never picks his role up automatically. 135The role has to be configured with the 136.Xr hastctl 8 137control utility by additional software like 138.Nm ucarp 139or 140.Nm heartbeat 141that can reliably manage role separation and switch secondary node to 142primary role in case of original primary failure. 143.Pp 144The 145.Nm 146daemon can be started with the following command line arguments: 147.Bl -tag -width ".Fl P Ar pidfile" 148.It Fl c Ar config 149Specify alternative location of the configuration file. 150The default location is 151.Pa /etc/hast.conf . 152.It Fl d 153Print or log debugging information. 154This option can be specified multiple times to raise the verbosity 155level. 156.It Fl F 157Start the 158.Nm 159daemon in the foreground. 160By default 161.Nm 162starts in the background. 163.It Fl h 164Print the 165.Nm 166usage message. 167.It Fl P Ar pidfile 168Specify alternative location of a file where main process PID will be 169stored. 170The default location is 171.Pa /var/run/hastd.pid . 172.El 173.Sh FILES 174.Bl -tag -width ".Pa /var/run/hastctl" -compact 175.It Pa /etc/hast.conf 176The configuration file for 177.Nm 178and 179.Xr hastctl 8 . 180.It Pa /var/run/hastctl 181Control socket used by the 182.Xr hastctl 8 183control utility to communicate with 184.Nm . 185.It Pa /var/run/hastd.pid 186The default location of the 187.Nm 188PID file. 189.El 190.Sh EXIT STATUS 191Exit status is 0 on success, or one of the values described in 192.Xr sysexits 3 193on failure. 194.Sh EXAMPLES 195Launch 196.Nm 197on both nodes. 198Set role for resource 199.Nm shared 200to primary on 201.Nm nodeA 202and to secondary on 203.Nm nodeB . 204Create file system on 205.Pa /dev/hast/shared 206provider and mount it. 207.Bd -literal -offset indent 208nodeB# hastd 209nodeB# hastctl role secondary shared 210 211nodeA# hastd 212nodeA# hastctl role primary shared 213nodeA# newfs -U /dev/hast/shared 214nodeA# mount -o noatime /dev/hast/shared /shared 215.Ed 216.Sh SEE ALSO 217.Xr sysexits 3 , 218.Xr geom 4 , 219.Xr hast.conf 5 , 220.Xr ggatec 8 , 221.Xr ggated 8 , 222.Xr ggatel 8 , 223.Xr hastctl 8 , 224.Xr mount 8 , 225.Xr newfs 8 , 226.Xr g_bio 9 . 227.Sh AUTHORS 228The 229.Nm 230was developed by 231.An Pawel Jakub Dawidek Aq pjd@FreeBSD.org 232under sponsorship of the FreeBSD Foundation. 233