1.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 2.\" 3.\" Copyright (c) 2003, Ryan McBride. All rights reserved. 4.\" Copyright (c) 2011, Gleb Smirnoff <glebius@FreeBSD.org> 5.\" 6.\" Redistribution and use in source and binary forms, with or without 7.\" modification, are permitted provided that the following conditions 8.\" are met: 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in the 13.\" documentation and/or other materials provided with the distribution. 14.\" 15.\" THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND 16.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 17.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 18.\" ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE 19.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 21.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 22.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 25.\" SUCH DAMAGE. 26.\" 27.\" $FreeBSD$ 28.\" 29.Dd December 20, 2011 30.Dt CARP 4 31.Os 32.Sh NAME 33.Nm carp 34.Nd Common Address Redundancy Protocol 35.Sh SYNOPSIS 36.Cd "device carp" 37.Sh DESCRIPTION 38The CARP allows multiple hosts on the same local network to share a set of 39IPv4 and/or IPv6 addresses. 40Its primary purpose is to ensure that these 41addresses are always available. 42.Pp 43To use 44.Nm , 45the administrator needs to configure at minimum a common virtual host ID 46(vhid) and attach at least one IP address to this vhid on each machine which 47is to take part in the virtual group. 48Additional parameters can also be set on a per-vhid basis: 49.Cm advbase 50and 51.Cm advskew , 52which are used to control how frequently the host sends advertisements when it 53is the master for a virtual host, and 54.Cm pass 55which is used to authenticate 56.Nm 57advertisements. 58The 59.Cm advbase 60parameter stands for 61.Dq "advertisement base" . 62It is measured in seconds and specifies the base of the advertisement interval. 63The 64.Cm advskew 65parameter stands for 66.Dq "advertisement skew" . 67It is measured in 1/256 of seconds. 68It is added to the base advertisement interval to make one host advertise 69a bit slower that the other does. 70Both 71.Cm advbase 72and 73.Cm advskew 74are put inside CARP advertisements. 75These configurations can be done using 76.Xr ifconfig 8 , 77or through the 78.Dv SIOCSVH 79.Xr ioctl 2 . 80.Pp 81CARP virtual hosts can be configured on multicast capable interfaces: Ethernet, 82layer 2 VLAN, FDDI and Token Ring. 83An arbitrary number of virtual host IDs can be configured on an interface. 84An arbitrary number of IPv4 or IPv6 addresses can be attached to a particular 85vhid. 86It is important that all hosts participating in a vhid have the same list 87of prefixes configured on the vhid, since all prefixes are included in the 88cryptographic checksum supplied in each advertisement. 89Multiple vhids running on one interface participate in master/backup 90elections independently. 91.Pp 92Additionally, there are a number of global parameters which can be set using 93.Xr sysctl 8 : 94.Bl -tag -width ".Va net.inet.carp.preempt" 95.It Va net.inet.carp.allow 96Accept incoming 97.Nm 98packets. 99Enabled by default. 100.It Va net.inet.carp.preempt 101Allow virtual hosts to preempt each other. 102It is also used to failover 103.Nm 104interfaces as a group. 105When the option is enabled and one of the 106.Nm 107enabled physical interfaces 108goes down, 109.Cm advskew 110is changed to 240 on all 111.Nm 112interfaces. 113See also the first example. 114Disabled by default. 115.It Va net.inet.carp.log 116Value of 0 disables any logging. 117Value of 1 enables logging state changes of 118.Nm 119interfaces. 120Values above 1 enable logging of bad 121.Nm 122packets. 123Default value is 1. 124.It Va net.inet.carp.demotion 125This value shows current level of CARP demotion. 126The value is added to the actual advskew sent in announcements for 127all vhids. 128At normal system operation the demotion factor is zero. 129However, problematic conditions raise its level: when 130.Nm 131experiences problem with sending announcements, when an interface 132running a vhid goes down, or while the 133.Xr pfsync 4 134interface is not synchronized. 135The demotion value is writable, so that user may alter it 136depending on some external conditions, for example on status of some 137daemon utility. 138However, altering the value should be performed with care, do 139not conflict with subsystems that adjust demotion factor 140automatically: 141.Nm 142and 143.Xr pfsync 4 . 144.It Va net.inet.carp.ifdown_demotion_factor 145Value added to 146.Va net.inet.carp.demotion 147when interface running a vhid goes down. 148Default value is 240 (maximum advskew value). 149.It Va net.inet.carp.senderr_demotion_factor 150Value added to 151.Va net.inet.carp.demotion 152when 153.Nm 154experiences errors sending its announcements. 155Default value is 240 (maximum advskew value). 156.El 157.\".Sh ARP level load balancing 158.\"The 159.\".Nm 160.\"has limited abilities for load balancing the incoming connections 161.\"between hosts in Ethernet network. 162.\"For load balancing operation, one needs several CARP interfaces that 163.\"are configured to the same IP address, but to a different vhids. 164.\"Once an ARP request is received, the CARP protocol will use a hashing 165.\"function against the source IP address in the ARP request to determine 166.\"which vhid should this request belong to. 167.\"If the corresponding CARP interface is in master state, the ARP request 168.\"will be replied, otherwise it will be ignored. 169.\"See the 170.\".Sx EXAMPLES 171.\"section for a practical example of load balancing. 172.\".Pp 173.\"The ARP load balancing has some limitations. 174.\"First, ARP balancing only works on the local network segment. 175.\"It cannot balance traffic that crosses a router, because the 176.\"router itself will always be balanced to the same virtual host. 177.\"Second, ARP load balancing can lead to asymmetric routing 178.\"of incoming and outgoing traffic, and thus combining it with 179.\".Xr pfsync 4 180.\"is dangerous, because this creates a race condition between 181.\"balanced routers and a host they are serving. 182.\"Imagine an incoming packet creating state on the first router, being 183.\"forwarded to its destination, and destination replying faster 184.\"than the state information is packed and synced with the second router. 185.\"If the reply would be load balanced to second router, it will be 186.\"dropped due to no state. 187.Sh STATE CHANGE NOTIFICATIONS 188Sometimes it is useful to get notified about 189.Nm 190status change events. 191This can be accomplished by using 192.Xr devd 8 193hooks. 194Master/slave events are signalled under system 195.Dv CARP . 196Subsystem specifies vhid and name of interface, where event occured. 197Type of the message displays new state of vhid. 198Please see 199.Xr devd.conf 5 200and 201.Sx EXAMPLES 202section for more information. 203.Sh EXAMPLES 204For firewalls and routers with multiple interfaces, it is desirable to 205failover all of the 206.Nm 207interfaces together, when one of the physical interfaces goes down. 208This is achieved by the preempt option. 209Enable it on both host A and B: 210.Pp 211.Dl sysctl net.inet.carp.preempt=1 212.Pp 213Assume that host A is the preferred master and we are running the 214192.168.1.0/24 prefix on em0 and 192.168.2.0/24 on em1. 215This is the setup for host A: 216.Bd -literal -offset indent 217ifconfig em0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 218ifconfig em1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 219.Ed 220.Pp 221The setup for host B is identical, but it has a higher 222.Cm advskew : 223.Bd -literal -offset indent 224ifconfig em0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 225ifconfig em1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 226.Ed 227.Pp 228Because of the preempt option, when one of the physical interfaces of 229host A fails, 230.Cm advskew 231is adjusted to 240 on all its 232.Nm 233interfaces. 234This will cause host B to preempt on both interfaces instead of 235just the failed one. 236.\".Pp 237.\"In order to set up an ARP balanced virtual host, it is necessary to configure 238.\"one virtual host for each physical host which would respond to ARP requests 239.\"and thus handle the traffic. 240.\"In the following example, two virtual hosts are configured on two hosts to 241.\"provide balancing and failover for the IP address 192.168.1.10. 242.\".Pp 243.\"First the 244.\".Nm 245.\"interfaces on host A are configured. 246.\"The 247.\".Cm advskew 248.\"of 100 on the second virtual host means that its advertisements will be sent 249.\"out slightly less frequently. 250.\".Bd -literal -offset indent 251.\"ifconfig carp0 create 252.\"ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 253.\"ifconfig carp1 create 254.\"ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 255.\".Ed 256.\".Pp 257.\"The configuration for host B is identical, except the 258.\".Cm advskew 259.\"is on virtual host 1 rather than virtual host 2. 260.\".Bd -literal -offset indent 261.\"ifconfig carp0 create 262.\"ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 263.\"ifconfig carp1 create 264.\"ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 265.\".Ed 266.\".Pp 267.\"Finally, the ARP balancing feature must be enabled on both hosts: 268.\".Pp 269.\".Dl sysctl net.inet.carp.arpbalance=1 270.\".Pp 271.\"When the hosts receive an ARP request for 192.168.1.10, the source IP address 272.\"of the request is used to compute which virtual host should answer the request. 273.\"The host which is master of the selected virtual host will reply to the 274.\"request, the other(s) will ignore it. 275.\".Pp 276.\"This way, locally connected systems will receive different ARP replies and 277.\"subsequent IP traffic will be balanced among the hosts. 278.\"If one of the hosts fails, the other will take over the virtual MAC address, 279.\"and begin answering ARP requests on its behalf. 280.Pp 281Processing of 282.Nm 283status change events can be set up by using the following devd.conf rule: 284.Bd -literal -offset indent 285notify 0 { 286 match "system" "CARP"; 287 match "subsystem" "[0-9]+@"; 288 match "type" "(MASTER|BACKUP)"; 289 action "/root/carpcontrol.sh $subsystem $type"; 290}; 291.Ed 292.Sh SEE ALSO 293.Xr inet 4 , 294.Xr pfsync 4 , 295.Xr rc.conf 5 , 296.Xr devd.conf 5 , 297.Xr ifconfig 8 , 298.Xr sysctl 8 299.Sh HISTORY 300The 301.Nm 302device first appeared in 303.Ox 3.5 . 304The 305.Nm 306device was imported into 307.Fx 5.4 . 308In 309.Fx 10 310the 311.Nm 312was significantly rewritten, and is no longer a pseudo-interface. 313