1.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 2.\" 3.\" Copyright (c) 2003, Ryan McBride. All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd August 15, 2011 29.Dt CARP 4 30.Os 31.Sh NAME 32.Nm carp 33.Nd Common Address Redundancy Protocol 34.Sh SYNOPSIS 35.Cd "device carp" 36.Sh DESCRIPTION 37The 38.Nm 39interface is a pseudo-device that implements and controls the 40CARP protocol. 41CARP allows multiple hosts on the same local network to share a set of IP addresses. 42Its primary purpose is to ensure that these 43addresses are always available, but in some configurations 44.Nm 45can also provide load balancing functionality. 46.Pp 47A 48.Nm 49interface can be created at runtime using the 50.Nm ifconfig Li carp Ns Ar N Cm create 51command or by configuring 52it via 53.Va cloned_interfaces 54in the 55.Pa /etc/rc.conf 56file. 57.Pp 58To use 59.Nm , 60the administrator needs to configure at minimum a common virtual host ID (VHID) 61and virtual host IP address on each machine which is to take part in the virtual 62group. 63Additional parameters can also be set on a per-interface basis: 64.Cm advbase 65and 66.Cm advskew , 67which are used to control how frequently the host sends advertisements when it 68is the master for a virtual host, and 69.Cm pass 70which is used to authenticate 71.Nm 72advertisements. 73The 74.Cm advbase 75parameter stands for 76.Dq "advertisement base" . 77It is measured in seconds and specifies the base of the advertisement interval. 78The 79.Cm advskew 80parameter stands for 81.Dq "advertisement skew" . 82It is measured in 1/256 of seconds. 83It is added to the base advertisement interval to make one host advertise 84a bit slower that the other does. 85Both 86.Cm advbase 87and 88.Cm advskew 89are put inside CARP advertisements. 90These configurations can be done using 91.Xr ifconfig 8 , 92or through the 93.Dv SIOCSVH 94.Xr ioctl 2 . 95.Pp 96Additionally, there are a number of global parameters which can be set using 97.Xr sysctl 8 : 98.Bl -tag -width ".Va net.inet.carp.arpbalance" 99.It Va net.inet.carp.allow 100Accept incoming 101.Nm 102packets. 103Enabled by default. 104.It Va net.inet.carp.preempt 105Allow virtual hosts to preempt each other. 106It is also used to failover 107.Nm 108interfaces as a group. 109When the option is enabled and one of the 110.Nm 111enabled physical interfaces 112goes down, 113.Cm advskew 114is changed to 240 on all 115.Nm 116interfaces. 117See also the first example. 118Disabled by default. 119.It Va net.inet.carp.log 120Value of 0 disables any logging. 121Value of 1 enables logging state changes of 122.Nm 123interfaces. 124Values above 1 enable logging of bad 125.Nm 126packets. 127Default value is 1. 128.It Va net.inet.carp.arpbalance 129Balance local traffic using ARP (see below). 130Disabled by default. 131.It Va net.inet.carp.suppress_preempt 132A read only value showing the status of preemption suppression. 133Preemption can be suppressed if link on an interface is down 134or when 135.Xr pfsync 4 136interface is not synchronized. 137Value of 0 means that preemption is not suppressed, since no 138problems are detected. 139Every problem increments suppression counter. 140.El 141.Sh ARP level load balancing 142The 143.Nm 144has limited abilities for load balancing the incoming connections 145between hosts in Ethernet network. 146For load balancing operation, one needs several CARP interfaces that 147are configured to the same IP address, but to a different VHIDs. 148Once an ARP request is received, the CARP protocol will use a hashing 149function against the source IP address in the ARP request to determine 150which VHID should this request belong to. 151If the corresponding CARP interface is in master state, the ARP request 152will be replied, otherwise it will be ignored. 153See the 154.Sx EXAMPLES 155section for a practical example of load balancing. 156.Pp 157The ARP load balancing has some limitations. 158First, ARP balancing only works on the local network segment. 159It cannot balance traffic that crosses a router, because the 160router itself will always be balanced to the same virtual host. 161Second, ARP load balancing can lead to asymmetric routing 162of incoming and outgoing traffic, and thus combining it with 163.Xr pfsync 4 164is dangerous, because this creates a race condition between 165balanced routers and a host they are serving. 166Imagine an incoming packet creating state on the first router, being 167forwarded to its destination, and destination replying faster 168than the state information is packed and synced with the second router. 169If the reply would be load balanced to second router, it will be 170dropped due to no state. 171.Sh STATE CHANGE NOTIFICATIONS 172Sometimes it is useful to get notified about 173.Nm 174status change events. 175This can be accomplished by using 176.Xr devd 8 177hooks. 178Master/slave events are signalled as 179.Nm 180interface 181.Dv LINK_UP 182or 183.Dv LINK_DOWN 184event. 185Please see 186.Xr devd.conf 5 187and 188.Sx EXAMPLES 189section for more information. 190.Sh EXAMPLES 191For firewalls and routers with multiple interfaces, it is desirable to 192failover all of the 193.Nm 194interfaces together, when one of the physical interfaces goes down. 195This is achieved by the preempt option. 196Enable it on both host A and B: 197.Pp 198.Dl sysctl net.inet.carp.preempt=1 199.Pp 200Assume that host A is the preferred master and 192.168.1.x/24 is 201configured on one physical interface and 192.168.2.y/24 on another. 202This is the setup for host A: 203.Bd -literal -offset indent 204ifconfig carp0 create 205ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 206ifconfig carp1 create 207ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 208.Ed 209.Pp 210The setup for host B is identical, but it has a higher 211.Cm advskew : 212.Bd -literal -offset indent 213ifconfig carp0 create 214ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 215ifconfig carp1 create 216ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 217.Ed 218.Pp 219Because of the preempt option, when one of the physical interfaces of 220host A fails, 221.Cm advskew 222is adjusted to 240 on all its 223.Nm 224interfaces. 225This will cause host B to preempt on both interfaces instead of 226just the failed one. 227.Pp 228In order to set up an ARP balanced virtual host, it is necessary to configure 229one virtual host for each physical host which would respond to ARP requests 230and thus handle the traffic. 231In the following example, two virtual hosts are configured on two hosts to 232provide balancing and failover for the IP address 192.168.1.10. 233.Pp 234First the 235.Nm 236interfaces on host A are configured. 237The 238.Cm advskew 239of 100 on the second virtual host means that its advertisements will be sent 240out slightly less frequently. 241.Bd -literal -offset indent 242ifconfig carp0 create 243ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 244ifconfig carp1 create 245ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 246.Ed 247.Pp 248The configuration for host B is identical, except the 249.Cm advskew 250is on virtual host 1 rather than virtual host 2. 251.Bd -literal -offset indent 252ifconfig carp0 create 253ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 254ifconfig carp1 create 255ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 256.Ed 257.Pp 258Finally, the ARP balancing feature must be enabled on both hosts: 259.Pp 260.Dl sysctl net.inet.carp.arpbalance=1 261.Pp 262When the hosts receive an ARP request for 192.168.1.10, the source IP address 263of the request is used to compute which virtual host should answer the request. 264The host which is master of the selected virtual host will reply to the 265request, the other(s) will ignore it. 266.Pp 267This way, locally connected systems will receive different ARP replies and 268subsequent IP traffic will be balanced among the hosts. 269If one of the hosts fails, the other will take over the virtual MAC address, 270and begin answering ARP requests on its behalf. 271.Pp 272Processing of 273.Nm 274status change events can be set up by using the following devd.conf rules: 275.Bd -literal -offset indent 276notify 0 { 277 match "system" "IFNET"; 278 match "type" "LINK_UP"; 279 match "subsystem" "carp*"; 280 action "/root/carpcontrol.sh $type $subsystem"; 281}; 282 283notify 0 { 284 match "system" "IFNET"; 285 match "type" "LINK_DOWN"; 286 match "subsystem" "carp*"; 287 action "/root/carpcontrol.sh $type $subsystem"; 288}; 289.Ed 290.Sh SEE ALSO 291.Xr inet 4 , 292.Xr pfsync 4 , 293.Xr rc.conf 5 , 294.Xr devd.conf 5 , 295.Xr ifconfig 8 , 296.Xr sysctl 8 297.Sh HISTORY 298The 299.Nm 300device first appeared in 301.Ox 3.5 . 302The 303.Nm 304device was imported into 305.Fx 5.4 . 306