1.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 2.\" 3.\" Copyright (c) 2003, Ryan McBride. All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd January 5, 2010 29.Dt CARP 4 30.Os 31.Sh NAME 32.Nm carp 33.Nd Common Address Redundancy Protocol 34.Sh SYNOPSIS 35.Cd "device carp" 36.Sh DESCRIPTION 37The 38.Nm 39interface is a pseudo-device that implements and controls the 40CARP protocol. 41CARP allows multiple hosts on the same local network to share a set of IP addresses. 42Its primary purpose is to ensure that these 43addresses are always available, but in some configurations 44.Nm 45can also provide load balancing functionality. 46.Pp 47A 48.Nm 49interface can be created at runtime using the 50.Nm ifconfig Li carp Ns Ar N Cm create 51command or by configuring 52it via 53.Va cloned_interfaces 54in the 55.Pa /etc/rc.conf 56file. 57.Pp 58To use 59.Nm , 60the administrator needs to configure at minimum a common virtual host ID (VHID) 61and virtual host IP address on each machine which is to take part in the virtual 62group. 63Additional parameters can also be set on a per-interface basis: 64.Cm advbase 65and 66.Cm advskew , 67which are used to control how frequently the host sends advertisements when it 68is the master for a virtual host, and 69.Cm pass 70which is used to authenticate 71.Nm 72advertisements. 73The 74.Cm advbase 75parameter stands for 76.Dq "advertisement base" . 77It is measured in seconds and specifies the base of the advertisement interval. 78The 79.Cm advskew 80parameter stands for 81.Dq "advertisement skew" . 82It is measured in 1/256 of seconds. 83It is added to the base advertisement interval to make one host advertise 84a bit slower that the other does. 85Both 86.Cm advbase 87and 88.Cm advskew 89are put inside CARP advertisements. 90These configurations can be done using 91.Xr ifconfig 8 , 92or through the 93.Dv SIOCSVH 94.Xr ioctl 2 . 95.Pp 96Additionally, there are a number of global parameters which can be set using 97.Xr sysctl 8 : 98.Bl -tag -width ".Va net.inet.carp.arpbalance" 99.It Va net.inet.carp.allow 100Accept incoming 101.Nm 102packets. 103Enabled by default. 104.It Va net.inet.carp.preempt 105Allow virtual hosts to preempt each other. 106It is also used to failover 107.Nm 108interfaces as a group. 109When the option is enabled and one of the 110.Nm 111enabled physical interfaces 112goes down, 113.Cm advskew 114is changed to 240 on all 115.Nm 116interfaces. 117See also the first example. 118Disabled by default. 119.It Va net.inet.carp.log 120Value of 0 disables any logging. 121Value of 1 enables logging state changes of 122.Nm 123interfaces. 124Values above 1 enable logging of bad 125.Nm 126packets. 127Default value is 1. 128.It Va net.inet.carp.arpbalance 129Balance local traffic using ARP (see below). 130Disabled by default. 131.It Va net.inet.carp.suppress_preempt 132A read only value showing the status of preemption suppression. 133Preemption can be suppressed if link on an interface is down 134or when 135.Xr pfsync 4 136interface is not synchronized. 137Value of 0 means that preemption is not suppressed, since no 138problems are detected. 139Every problem increments suppression counter. 140.El 141.Sh ARP level load balancing 142The 143.Nm 144has limited abilities for load balancing the incoming connections 145between hosts in Ethernet network. 146For load balancing operation, one needs several CARP interfaces that 147are configured to the same IP address, but to a different VHIDs. 148Once an ARP request is received, the CARP protocol will use a hashing 149function against the source IP address in the ARP request to determine 150which VHID should this request belong to. 151If the corresponding CARP interface is in master state, the ARP request 152will be replied, otherwise it will be ignored. 153See the 154.Sx EXAMPLES 155section for a practical example of load balancing. 156.Pp 157The ARP load balancing has some limitations. 158First, ARP balancing only works on the local network segment. 159It cannot balance traffic that crosses a router, because the 160router itself will always be balanced to the same virtual host. 161Second, ARP load balancing can lead to asymmetric routing 162of incoming and outgoing traffic, and thus combining it with 163.Xr pfsync 4 164is dangerous, because this creates a race condition between 165balanced routers and a host they are serving. 166Imagine an incoming packet creating state on the first router, being 167forwarded to its destination, and destination replying faster 168than the state information is packed and synced with the second router. 169If the reply would be load balanced to second router, it will be 170dropped due to no state. 171.Sh EXAMPLES 172For firewalls and routers with multiple interfaces, it is desirable to 173failover all of the 174.Nm 175interfaces together, when one of the physical interfaces goes down. 176This is achieved by the preempt option. 177Enable it on both host A and B: 178.Pp 179.Dl sysctl net.inet.carp.preempt=1 180.Pp 181Assume that host A is the preferred master and 192.168.1.x/24 is 182configured on one physical interface and 192.168.2.y/24 on another. 183This is the setup for host A: 184.Bd -literal -offset indent 185ifconfig carp0 create 186ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 187ifconfig carp1 create 188ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 189.Ed 190.Pp 191The setup for host B is identical, but it has a higher 192.Cm advskew : 193.Bd -literal -offset indent 194ifconfig carp0 create 195ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 196ifconfig carp1 create 197ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 198.Ed 199.Pp 200Because of the preempt option, when one of the physical interfaces of 201host A fails, 202.Cm advskew 203is adjusted to 240 on all its 204.Nm 205interfaces. 206This will cause host B to preempt on both interfaces instead of 207just the failed one. 208.Pp 209In order to set up an ARP balanced virtual host, it is necessary to configure 210one virtual host for each physical host which would respond to ARP requests 211and thus handle the traffic. 212In the following example, two virtual hosts are configured on two hosts to 213provide balancing and failover for the IP address 192.168.1.10. 214.Pp 215First the 216.Nm 217interfaces on host A are configured. 218The 219.Cm advskew 220of 100 on the second virtual host means that its advertisements will be sent 221out slightly less frequently. 222.Bd -literal -offset indent 223ifconfig carp0 create 224ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 225ifconfig carp1 create 226ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 227.Ed 228.Pp 229The configuration for host B is identical, except the 230.Cm advskew 231is on virtual host 1 rather than virtual host 2. 232.Bd -literal -offset indent 233ifconfig carp0 create 234ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 235ifconfig carp1 create 236ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 237.Ed 238.Pp 239Finally, the ARP balancing feature must be enabled on both hosts: 240.Pp 241.Dl sysctl net.inet.carp.arpbalance=1 242.Pp 243When the hosts receive an ARP request for 192.168.1.10, the source IP address 244of the request is used to compute which virtual host should answer the request. 245The host which is master of the selected virtual host will reply to the 246request, the other(s) will ignore it. 247.Pp 248This way, locally connected systems will receive different ARP replies and 249subsequent IP traffic will be balanced among the hosts. 250If one of the hosts fails, the other will take over the virtual MAC address, 251and begin answering ARP requests on its behalf. 252.Sh SEE ALSO 253.Xr inet 4 , 254.Xr pfsync 4 , 255.Xr rc.conf 5 , 256.Xr ifconfig 8 , 257.Xr sysctl 8 258.Sh HISTORY 259The 260.Nm 261device first appeared in 262.Ox 3.5 . 263The 264.Nm 265device was imported into 266.Fx 5.4 . 267