1.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 2.\" 3.\" Copyright (c) 2003, Ryan McBride. All rights reserved. 4.\" Copyright (c) 2011, Gleb Smirnoff <glebius@FreeBSD.org> 5.\" 6.\" Redistribution and use in source and binary forms, with or without 7.\" modification, are permitted provided that the following conditions 8.\" are met: 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in the 13.\" documentation and/or other materials provided with the distribution. 14.\" 15.\" THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND 16.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 17.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 18.\" ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE 19.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 21.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 22.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 25.\" SUCH DAMAGE. 26.\" 27.\" $FreeBSD$ 28.\" 29.Dd April 22, 2012 30.Dt CARP 4 31.Os 32.Sh NAME 33.Nm carp 34.Nd Common Address Redundancy Protocol 35.Sh SYNOPSIS 36.Cd "device carp" 37.Sh DESCRIPTION 38The CARP allows multiple hosts on the same local network to share a set of 39IPv4 and/or IPv6 addresses. 40Its primary purpose is to ensure that these 41addresses are always available. 42.Pp 43To use 44.Nm , 45the administrator needs to configure at minimum a common virtual host ID 46(vhid) and attach at least one IP address to this vhid on each machine which 47is to take part in the virtual group. 48Additional parameters can also be set on a per-vhid basis: 49.Cm advbase 50and 51.Cm advskew , 52which are used to control how frequently the host sends advertisements when it 53is the master for a virtual host, and 54.Cm pass 55which is used to authenticate 56.Nm 57advertisements. 58The 59.Cm advbase 60parameter stands for 61.Dq "advertisement base" . 62It is measured in seconds and specifies the base of the advertisement interval. 63The 64.Cm advskew 65parameter stands for 66.Dq "advertisement skew" . 67It is measured in 1/256 of seconds. 68It is added to the base advertisement interval to make one host advertise 69a bit slower that the other does. 70Both 71.Cm advbase 72and 73.Cm advskew 74are put inside CARP advertisements. 75These configurations can be done using 76.Xr ifconfig 8 , 77or through the 78.Dv SIOCSVH 79.Xr ioctl 2 . 80.Pp 81CARP virtual hosts can be configured on multicast capable interfaces: Ethernet, 82layer 2 VLAN, FDDI and Token Ring. 83An arbitrary number of virtual host IDs can be configured on an interface. 84An arbitrary number of IPv4 or IPv6 addresses can be attached to a particular 85vhid. 86It is important that all hosts participating in a vhid have the same list 87of prefixes configured on the vhid, since all prefixes are included in the 88cryptographic checksum supplied in each advertisement. 89Multiple vhids running on one interface participate in master/backup 90elections independently. 91.Pp 92Additionally, there are a number of global parameters which can be set using 93.Xr sysctl 8 : 94.Bl -tag -width ".Va net.inet.carp.preempt" 95.It Va net.inet.carp.allow 96Accept incoming 97.Nm 98packets. 99Enabled by default. 100.It Va net.inet.carp.preempt 101Allow virtual hosts to preempt each other. 102When enabled, a vhid in a backup state would preempt a master that 103is announcing itself with a lower advskew. 104Disabled by default. 105.It Va net.inet.carp.log 106Value of 0 disables any logging. 107Value of 1 enables logging state changes of 108.Nm 109vhids. 110Values above 1 enable logging of bad 111.Nm 112packets. 113Default value is 1. 114.It Va net.inet.carp.demotion 115This value shows current level of CARP demotion. 116The value is added to the actual advskew sent in announcements for 117all vhids. 118At normal system operation the demotion factor is zero. 119However, problematic conditions raise its level: when 120.Nm 121experiences problem with sending announcements, when an interface 122running a vhid goes down, or while the 123.Xr pfsync 4 124interface is not synchronized. 125The demotion value is writable, so that user may alter it 126depending on some external conditions, for example on status of some 127daemon utility. 128However, altering the value should be performed with care, do 129not conflict with subsystems that adjust demotion factor 130automatically: 131.Nm 132and 133.Xr pfsync 4 . 134.It Va net.inet.carp.ifdown_demotion_factor 135Value added to 136.Va net.inet.carp.demotion 137when interface running a vhid goes down. 138Default value is 240 (maximum advskew value). 139.It Va net.inet.carp.senderr_demotion_factor 140Value added to 141.Va net.inet.carp.demotion 142when 143.Nm 144experiences errors sending its announcements. 145Default value is 240 (maximum advskew value). 146.El 147.\".Sh ARP level load balancing 148.\"The 149.\".Nm 150.\"has limited abilities for load balancing the incoming connections 151.\"between hosts in Ethernet network. 152.\"For load balancing operation, one needs several CARP interfaces that 153.\"are configured to the same IP address, but to a different vhids. 154.\"Once an ARP request is received, the CARP protocol will use a hashing 155.\"function against the source IP address in the ARP request to determine 156.\"which vhid should this request belong to. 157.\"If the corresponding CARP interface is in master state, the ARP request 158.\"will be replied, otherwise it will be ignored. 159.\"See the 160.\".Sx EXAMPLES 161.\"section for a practical example of load balancing. 162.\".Pp 163.\"The ARP load balancing has some limitations. 164.\"First, ARP balancing only works on the local network segment. 165.\"It cannot balance traffic that crosses a router, because the 166.\"router itself will always be balanced to the same virtual host. 167.\"Second, ARP load balancing can lead to asymmetric routing 168.\"of incoming and outgoing traffic, and thus combining it with 169.\".Xr pfsync 4 170.\"is dangerous, because this creates a race condition between 171.\"balanced routers and a host they are serving. 172.\"Imagine an incoming packet creating state on the first router, being 173.\"forwarded to its destination, and destination replying faster 174.\"than the state information is packed and synced with the second router. 175.\"If the reply would be load balanced to second router, it will be 176.\"dropped due to no state. 177.Sh STATE CHANGE NOTIFICATIONS 178Sometimes it is useful to get notified about 179.Nm 180status change events. 181This can be accomplished by using 182.Xr devd 8 183hooks. 184Master/slave events are signalled under system 185.Dv CARP . 186Subsystem specifies vhid and name of interface, where event occurred. 187Type of the message displays new state of vhid. 188Please see 189.Xr devd.conf 5 190and 191.Sx EXAMPLES 192section for more information. 193.Sh EXAMPLES 194For firewalls and routers with multiple interfaces, it is desirable to 195failover all of the addresses running 196.Nm 197together, when one of the physical interfaces goes down. 198This is achieved by the preempt option. 199Enable it on both host A and B: 200.Pp 201.Dl sysctl net.inet.carp.preempt=1 202.Pp 203Assume that host A is the preferred master and we are running the 204192.168.1.0/24 prefix on em0 and 192.168.2.0/24 on em1. 205This is the setup for host A: 206.Bd -literal -offset indent 207ifconfig em0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 208ifconfig em1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 209.Ed 210.Pp 211The setup for host B is identical, but it has a higher 212.Cm advskew : 213.Bd -literal -offset indent 214ifconfig em0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 215ifconfig em1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 216.Ed 217.Pp 218When one of the physical interfaces of host A fails, 219.Cm advskew 220is demoted to a configured value on all its 221.Nm 222vhids. 223Due to the preempt option, host B would start announcing itself, and thus 224preempt host A on both interfaces instead of just the failed one. 225.\".Pp 226.\"In order to set up an ARP balanced virtual host, it is necessary to configure 227.\"one virtual host for each physical host which would respond to ARP requests 228.\"and thus handle the traffic. 229.\"In the following example, two virtual hosts are configured on two hosts to 230.\"provide balancing and failover for the IP address 192.168.1.10. 231.\".Pp 232.\"First the 233.\".Nm 234.\"interfaces on host A are configured. 235.\"The 236.\".Cm advskew 237.\"of 100 on the second virtual host means that its advertisements will be sent 238.\"out slightly less frequently. 239.\".Bd -literal -offset indent 240.\"ifconfig carp0 create 241.\"ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 242.\"ifconfig carp1 create 243.\"ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 244.\".Ed 245.\".Pp 246.\"The configuration for host B is identical, except the 247.\".Cm advskew 248.\"is on virtual host 1 rather than virtual host 2. 249.\".Bd -literal -offset indent 250.\"ifconfig carp0 create 251.\"ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 252.\"ifconfig carp1 create 253.\"ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 254.\".Ed 255.\".Pp 256.\"Finally, the ARP balancing feature must be enabled on both hosts: 257.\".Pp 258.\".Dl sysctl net.inet.carp.arpbalance=1 259.\".Pp 260.\"When the hosts receive an ARP request for 192.168.1.10, the source IP address 261.\"of the request is used to compute which virtual host should answer the request. 262.\"The host which is master of the selected virtual host will reply to the 263.\"request, the other(s) will ignore it. 264.\".Pp 265.\"This way, locally connected systems will receive different ARP replies and 266.\"subsequent IP traffic will be balanced among the hosts. 267.\"If one of the hosts fails, the other will take over the virtual MAC address, 268.\"and begin answering ARP requests on its behalf. 269.Pp 270Processing of 271.Nm 272status change events can be set up by using the following devd.conf rule: 273.Bd -literal -offset indent 274notify 0 { 275 match "system" "CARP"; 276 match "subsystem" "[0-9]+@"; 277 match "type" "(MASTER|BACKUP)"; 278 action "/root/carpcontrol.sh $subsystem $type"; 279}; 280.Ed 281.Sh SEE ALSO 282.Xr inet 4 , 283.Xr pfsync 4 , 284.Xr rc.conf 5 , 285.Xr devd.conf 5 , 286.Xr ifconfig 8 , 287.Xr sysctl 8 288.Sh HISTORY 289The 290.Nm 291device first appeared in 292.Ox 3.5 . 293The 294.Nm 295device was imported into 296.Fx 5.4 . 297In 298.Fx 10.0 299the 300.Nm 301was significantly rewritten, and is no longer a pseudo-interface. 302