1.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 2.\" 3.\" Copyright (c) 2003, Ryan McBride. All rights reserved. 4.\" Copyright (c) 2011, Gleb Smirnoff <glebius@FreeBSD.org> 5.\" 6.\" Redistribution and use in source and binary forms, with or without 7.\" modification, are permitted provided that the following conditions 8.\" are met: 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in the 13.\" documentation and/or other materials provided with the distribution. 14.\" 15.\" THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND 16.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 17.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 18.\" ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE 19.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 21.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 22.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 25.\" SUCH DAMAGE. 26.\" 27.\" $FreeBSD$ 28.\" 29.Dd May 30, 2012 30.Dt CARP 4 31.Os 32.Sh NAME 33.Nm carp 34.Nd Common Address Redundancy Protocol 35.Sh SYNOPSIS 36.Cd "device carp" 37.Sh DESCRIPTION 38The CARP allows multiple hosts on the same local network to share a set of 39IPv4 and/or IPv6 addresses. 40Its primary purpose is to ensure that these 41addresses are always available. 42.Pp 43To use 44.Nm , 45the administrator needs to configure at a minimum a common virtual host ID 46(vhid), and attach at least one IP address to this vhid on each machine which 47is to take part in the virtual group. 48Additional parameters can also be set on a per-vhid basis: 49.Cm advbase 50and 51.Cm advskew , 52which are used to control how frequently the host sends advertisements when it 53is the master for a virtual host, and 54.Cm pass 55which is used to authenticate 56.Nm 57advertisements. 58The 59.Cm advbase 60parameter stands for 61.Dq "advertisement base" . 62It is measured in seconds and specifies the base of the advertisement interval. 63The 64.Cm advskew 65parameter stands for 66.Dq "advertisement skew" . 67It is measured in 1/256 of seconds. 68It is added to the base advertisement interval to make one host advertise 69a bit slower that the other does. 70Both 71.Cm advbase 72and 73.Cm advskew 74are put inside CARP advertisements. 75These values can be configured using 76.Xr ifconfig 8 , 77or through the 78.Dv SIOCSVH 79.Xr ioctl 2 . 80.Pp 81CARP virtual hosts can be configured on multicast-capable interfaces: Ethernet, 82layer 2 VLAN, FDDI and Token Ring. 83An arbitrary number of virtual host IDs can be configured on an interface. 84An arbitrary number of IPv4 or IPv6 addresses can be attached to a particular 85vhid. 86It is important that all hosts participating in a vhid have the same list 87of prefixes configured on the vhid, since all prefixes are included in the 88cryptographic checksum supplied in each advertisement. 89Multiple vhids running on one interface participate in master/backup 90elections independently. 91.Pp 92Additionally, there are a number of global parameters which can be set using 93.Xr sysctl 8 : 94.Bl -tag -width ".Va net.inet.carp.ifdown_demotion_factor" 95.It Va net.inet.carp.allow 96Accept incoming 97.Nm 98packets. 99Enabled by default. 100.It Va net.inet.carp.preempt 101Allow virtual hosts to preempt each other. 102When enabled, a vhid in a backup state would preempt a master that 103is announcing itself with a lower advskew. 104Disabled by default. 105.It Va net.inet.carp.log 106Determines what events relating to 107.Nm 108vhids are logged. 109A value of 0 disables any logging. 110A value of 1 enables logging state changes of 111.Nm 112vhids. 113Values above 1 enable logging of bad 114.Nm 115packets. 116The default value is 1. 117.It Va net.inet.carp.demotion 118This value shows current level of CARP demotion. 119The value is added to the actual advskew sent in announcements for 120all vhids. 121At normal system operation the demotion factor is zero. 122However, problematic conditions raise its level: when 123.Nm 124experiences problem with sending announcements, when an interface 125running a vhid goes down, or while the 126.Xr pfsync 4 127interface is not synchronized. 128The demotion value is writable, so that user may alter it 129depending on some external conditions, for example on the status of some 130daemon utility. 131However, altering the value should be performed with care, so as to 132not conflict with subsystems that adjust demotion factor 133automatically: 134.Nm 135and 136.Xr pfsync 4 . 137.It Va net.inet.carp.ifdown_demotion_factor 138This value is added to 139.Va net.inet.carp.demotion 140when an interface running a vhid goes down. 141The default value is 240 (the maximum advskew value). 142.It Va net.inet.carp.senderr_demotion_factor 143This value is added to 144.Va net.inet.carp.demotion 145when 146.Nm 147experiences errors sending its announcements. 148The default value is 240 (the maximum advskew value). 149.El 150.\".Sh ARP level load balancing 151.\"A 152.\".Nm 153.\"interface has limited abilities for load balancing incoming connections 154.\"between hosts in an Ethernet network. 155.\"For load-balancing operation, one needs several CARP interfaces that 156.\"are configured to the same IP address, but to a different vhids. 157.\"Once an ARP request is received, the CARP protocol will use a hashing 158.\"function against the source IP address in the ARP request to determine 159.\"which vhid the request will be assigned to. 160.\"If the corresponding CARP interface is the current 161.\"master interface, a reply will 162.\"be sent to the ARP request; 163.\"otherwise it will be ignored. 164.\"See the 165.\".Sx EXAMPLES 166.\"section for a practical example of load balancing. 167.\".Pp 168.\"The ARP load balancing implemented in 169.\".Nm 170.\"has some limitations. 171.\"First, ARP balancing only works on the local network segment. 172.\"It cannot balance traffic that crosses a router, because the 173.\"router itself will always be balanced to the same virtual host. 174.\"Second, ARP load balancing can lead to asymmetric routing 175.\"of incoming and outgoing traffic, and thus combining it with 176.\".Xr pfsync 4 177.\"is dangerous, because this creates a race condition between 178.\"balanced routers and a host they are serving. 179.\"Imagine an incoming packet creating state on the first router, being 180.\"forwarded to its destination, and the destination replying faster 181.\"than the state information is packed and synced with the second router. 182.\"If the reply would be load balanced to second router, it will be 183.\"dropped since the second router has not yet received information about 184.\"the connection state. 185.Sh STATE CHANGE NOTIFICATIONS 186Sometimes it is useful to get notified about 187.Nm 188status change events. 189This can be accomplished by using 190.Xr devd 8 191hooks. 192Master/slave events are signalled under system 193.Dv CARP . 194The subsystem specifies the vhid and name of the interface where 195the master/slave event occurred. 196The type of the message displays the new state of the vhid. 197Please see 198.Xr devd.conf 5 199and the 200.Sx EXAMPLES 201section for more information. 202.Sh EXAMPLES 203For firewalls and routers with multiple interfaces, it is desirable to 204failover all of the addresses running 205.Nm 206together, when one of the physical interfaces goes down. 207This is achieved by the use of the preempt option. 208Enable it on both hosts A and B: 209.Pp 210.Dl sysctl net.inet.carp.preempt=1 211.Pp 212Assume that host A is the preferred master and we are running the 213192.168.1.0/24 prefix on em0 and 192.168.2.0/24 on em1. 214This is the setup for host A: 215.Bd -literal -offset indent 216ifconfig em0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 217ifconfig em1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 218.Ed 219.Pp 220The setup for host B is identical, but it has a higher 221.Cm advskew : 222.Bd -literal -offset indent 223ifconfig em0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 224ifconfig em1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 225.Ed 226.Pp 227When one of the physical interfaces of host A fails, 228.Cm advskew 229is demoted to a configured value on all its 230.Nm 231vhids. 232Due to the preempt option, host B would start announcing itself, and thus 233preempt host A on both interfaces instead of just the failed one. 234.\".Pp 235.\"In order to set up an ARP balanced virtual host, it is necessary to configure 236.\"one virtual host for each physical host which would respond to ARP requests 237.\"and thus handle the traffic. 238.\"In the following example, two virtual hosts are configured on two hosts to 239.\"provide balancing and failover for the IP address 192.168.1.10. 240.\".Pp 241.\"First the 242.\".Nm 243.\"interfaces on host A are configured. 244.\"The 245.\".Cm advskew 246.\"of 100 on the second virtual host means that its advertisements will be sent 247.\"out slightly less frequently. 248.\".Bd -literal -offset indent 249.\"ifconfig carp0 create 250.\"ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 251.\"ifconfig carp1 create 252.\"ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 253.\".Ed 254.\".Pp 255.\"The configuration for host B is identical, except the 256.\".Cm advskew 257.\"is on virtual host 1 rather than virtual host 2. 258.\".Bd -literal -offset indent 259.\"ifconfig carp0 create 260.\"ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 261.\"ifconfig carp1 create 262.\"ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 263.\".Ed 264.\".Pp 265.\"Finally, the ARP balancing feature must be enabled on both hosts: 266.\".Pp 267.\".Dl sysctl net.inet.carp.arpbalance=1 268.\".Pp 269.\"When the hosts receive an ARP request for 192.168.1.10, the source IP address 270.\"of the request is used to compute which virtual host should answer the request. 271.\"The host which is master of the selected virtual host will reply to the 272.\"request, the other(s) will ignore it. 273.\".Pp 274.\"This way, locally connected systems will receive different ARP replies and 275.\"subsequent IP traffic will be balanced among the hosts. 276.\"If one of the hosts fails, the other will take over the virtual MAC address, 277.\"and begin answering ARP requests on its behalf. 278.Pp 279Processing of 280.Nm 281status change events can be set up by using the following devd.conf rule: 282.Bd -literal -offset indent 283notify 0 { 284 match "system" "CARP"; 285 match "subsystem" "[0-9]+@"; 286 match "type" "(MASTER|BACKUP)"; 287 action "/root/carpcontrol.sh $subsystem $type"; 288}; 289.Ed 290.Sh SEE ALSO 291.Xr inet 4 , 292.Xr pfsync 4 , 293.Xr rc.conf 5 , 294.Xr devd.conf 5 , 295.Xr ifconfig 8 , 296.Xr sysctl 8 297.Sh HISTORY 298The 299.Nm 300device first appeared in 301.Ox 3.5 . 302The 303.Nm 304device was imported into 305.Fx 5.4 . 306In 307.Fx 10.0 , 308.Nm 309was significantly rewritten, and is no longer a pseudo-interface. 310