1007054f0SBryan Venteicher.\" Copyright (c) 2014 Bryan Venteicher 2007054f0SBryan Venteicher.\" All rights reserved. 3007054f0SBryan Venteicher.\" 4007054f0SBryan Venteicher.\" Redistribution and use in source and binary forms, with or without 5007054f0SBryan Venteicher.\" modification, are permitted provided that the following conditions 6007054f0SBryan Venteicher.\" are met: 7007054f0SBryan Venteicher.\" 1. Redistributions of source code must retain the above copyright 8007054f0SBryan Venteicher.\" notice, this list of conditions and the following disclaimer. 9007054f0SBryan Venteicher.\" 2. Redistributions in binary form must reproduce the above copyright 10007054f0SBryan Venteicher.\" notice, this list of conditions and the following disclaimer in the 11007054f0SBryan Venteicher.\" documentation and/or other materials provided with the distribution. 12007054f0SBryan Venteicher.\" 13007054f0SBryan Venteicher.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14007054f0SBryan Venteicher.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15007054f0SBryan Venteicher.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16007054f0SBryan Venteicher.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 17007054f0SBryan Venteicher.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18007054f0SBryan Venteicher.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19007054f0SBryan Venteicher.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20007054f0SBryan Venteicher.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21007054f0SBryan Venteicher.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22007054f0SBryan Venteicher.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23007054f0SBryan Venteicher.\" SUCH DAMAGE. 24007054f0SBryan Venteicher.\" 25*baacf701SKonstantin Belousov.Dd March 30, 2021 26007054f0SBryan Venteicher.Dt VXLAN 4 27007054f0SBryan Venteicher.Os 28007054f0SBryan Venteicher.Sh NAME 29007054f0SBryan Venteicher.Nm vxlan 30007054f0SBryan Venteicher.Nd "Virtual eXtensible LAN interface" 31007054f0SBryan Venteicher.Sh SYNOPSIS 32007054f0SBryan VenteicherTo compile this driver into the kernel, 33007054f0SBryan Venteicherplace the following line in your 34007054f0SBryan Venteicherkernel configuration file: 35007054f0SBryan Venteicher.Bd -ragged -offset indent 36007054f0SBryan Venteicher.Cd "device vxlan" 37007054f0SBryan Venteicher.Ed 38007054f0SBryan Venteicher.Pp 39007054f0SBryan VenteicherAlternatively, to load the driver as a 40007054f0SBryan Venteichermodule at boot time, place the following line in 41007054f0SBryan Venteicher.Xr loader.conf 5 : 42007054f0SBryan Venteicher.Bd -literal -offset indent 43007054f0SBryan Venteicherif_vxlan_load="YES" 44007054f0SBryan Venteicher.Ed 45007054f0SBryan Venteicher.Sh DESCRIPTION 46007054f0SBryan VenteicherThe 47007054f0SBryan Venteicher.Nm 48007054f0SBryan Venteicherdriver creates a virtual tunnel endpoint in a 49007054f0SBryan Venteicher.Nm 50007054f0SBryan Venteichersegment. 51007054f0SBryan VenteicherA 52007054f0SBryan Venteicher.Nm 53007054f0SBryan Venteichersegment is a virtual Layer 2 (Ethernet) network that is overlaid 54007054f0SBryan Venteicherin a Layer 3 (IP/UDP) network. 55007054f0SBryan Venteicher.Nm 56007054f0SBryan Venteicheris analogous to 57007054f0SBryan Venteicher.Xr vlan 4 58007054f0SBryan Venteicherbut is designed to be better suited for large, multiple tenant 59007054f0SBryan Venteicherdata center environments. 60007054f0SBryan Venteicher.Pp 61007054f0SBryan VenteicherEach 62007054f0SBryan Venteicher.Nm 63007054f0SBryan Venteicherinterface is created at runtime using interface cloning. 64007054f0SBryan VenteicherThis is most easily done with the 65007054f0SBryan Venteicher.Xr ifconfig 8 66007054f0SBryan Venteicher.Cm create 67007054f0SBryan Venteichercommand or using the 68007054f0SBryan Venteicher.Va cloned_interfaces 69007054f0SBryan Venteichervariable in 70007054f0SBryan Venteicher.Xr rc.conf 5 . 71007054f0SBryan VenteicherThe interface may be removed with the 72007054f0SBryan Venteicher.Xr ifconfig 8 73007054f0SBryan Venteicher.Cm destroy 74007054f0SBryan Venteichercommand. 75007054f0SBryan Venteicher.Pp 76007054f0SBryan VenteicherThe 77007054f0SBryan Venteicher.Nm 78007054f0SBryan Venteicherdriver creates a pseudo Ethernet network interface 79007054f0SBryan Venteicherthat supports the usual network 80007054f0SBryan Venteicher.Xr ioctl 2 Ns s 810fd4ac1eSAllan Judeand thus can be used with 82007054f0SBryan Venteicher.Xr ifconfig 8 83007054f0SBryan Venteicherlike any other Ethernet interface. 84007054f0SBryan VenteicherThe 85007054f0SBryan Venteicher.Nm 86007054f0SBryan Venteicherinterface encapsulates the Ethernet frame 87007054f0SBryan Venteicherby prepending IP/UDP and 88007054f0SBryan Venteicher.Nm 89007054f0SBryan Venteicherheaders. 900fd4ac1eSAllan JudeThus, the encapsulated (inner) frame is able to be transmitted 91007054f0SBryan Venteicherover a routed, Layer 3 network to the remote host. 92007054f0SBryan Venteicher.Pp 93007054f0SBryan VenteicherThe 94007054f0SBryan Venteicher.Nm 95007054f0SBryan Venteicherinterface may be configured in either unicast or multicast mode. 96007054f0SBryan VenteicherWhen in unicast mode, 97007054f0SBryan Venteicherthe interface creates a tunnel to a single remote host, 98007054f0SBryan Venteicherand all traffic is transmitted to that host. 99007054f0SBryan VenteicherWhen in multicast mode, 100007054f0SBryan Venteicherthe interface joins an IP multicast group, 101007054f0SBryan Venteicherand receives packets sent to the group address, 102007054f0SBryan Venteicherand transmits packets to either the multicast group address, 1030fd4ac1eSAllan Judeor directly to the remote host if there is an appropriate 104007054f0SBryan Venteicherforwarding table entry. 105007054f0SBryan Venteicher.Pp 106007054f0SBryan VenteicherWhen the 107007054f0SBryan Venteicher.Nm 108007054f0SBryan Venteicherinterface is brought up, a 109788cf725SJoel Dahl.Xr udp 4 110007054f0SBryan Venteicher.Xr socket 9 111007054f0SBryan Venteicheris created based on the configuration, 112007054f0SBryan Venteichersuch as the local address for unicast mode or 113007054f0SBryan Venteicherthe group address for multicast mode, 114007054f0SBryan Venteicherand the listening (local) port number. 115007054f0SBryan VenteicherSince multiple 116007054f0SBryan Venteicher.Nm 117007054f0SBryan Venteicherinterfaces may be created that either 118007054f0SBryan Venteicheruse the same local address 119007054f0SBryan Venteicheror join the same group address, 120007054f0SBryan Venteicherand use the same port, 121007054f0SBryan Venteicherthe driver may share a socket among multiple interfaces. 122007054f0SBryan VenteicherHowever, each interface within a socket must belong to 123007054f0SBryan Venteichera unique 124007054f0SBryan Venteicher.Nm 125007054f0SBryan Venteichersegment. 126007054f0SBryan VenteicherThe analogous 127007054f0SBryan Venteicher.Xr vlan 4 128007054f0SBryan Venteicherconfiguration would be a physical interface configured as 129007054f0SBryan Venteicherthe parent device for multiple VLAN interfaces, each with 130007054f0SBryan Venteichera unique VLAN tag. 131007054f0SBryan VenteicherEach 132007054f0SBryan Venteicher.Nm 133007054f0SBryan Venteichersegment is identified by a 24-bit value in the 134007054f0SBryan Venteicher.Nm 135007054f0SBryan Venteicherheader called the 136007054f0SBryan Venteicher.Dq VXLAN Network Identifier , 137007054f0SBryan Venteicheror VNI. 138007054f0SBryan Venteicher.Pp 139007054f0SBryan VenteicherWhen configured with the 140007054f0SBryan Venteicher.Xr ifconfig 8 141c6e32006SBryan Venteicher.Cm vxlanlearn 142007054f0SBryan Venteicherparameter, the interface dynamically creates forwarding table entries 143007054f0SBryan Venteicherfrom received packets. 144007054f0SBryan VenteicherAn entry in the forwarding table maps the inner source MAC address 145007054f0SBryan Venteicherto the outer remote IP address. 146007054f0SBryan VenteicherDuring transmit, the interface attempts to lookup an entry for 147007054f0SBryan Venteicherthe encapsulated destination MAC address. 148007054f0SBryan VenteicherIf an entry is found, the IP address in the entry is used to directly 149007054f0SBryan Venteichertransmit the encapsulated frame to the destination. 150007054f0SBryan VenteicherOtherwise, when configured in multicast mode, 151007054f0SBryan Venteicherthe interface must flood the frame to all hosts in the group. 152007054f0SBryan VenteicherThe maximum number of entries in the table is configurable with the 153007054f0SBryan Venteicher.Xr ifconfig 8 154c6e32006SBryan Venteicher.Cm vxlanmaxaddr 155007054f0SBryan Venteichercommand. 1560fd4ac1eSAllan JudeStale entries in the table are periodically pruned. 157007054f0SBryan VenteicherThe timeout is configurable with the 158007054f0SBryan Venteicher.Xr ifconfig 8 159c6e32006SBryan Venteicher.Cm vxlantimeout 160007054f0SBryan Venteichercommand. 161007054f0SBryan VenteicherThe table may be viewed with the 162007054f0SBryan Venteicher.Xr sysctl 8 163c6e32006SBryan Venteicher.Cm net.link.vxlan.N.ftable.dump 164007054f0SBryan Venteichercommand. 165007054f0SBryan Venteicher.Sh MTU 166007054f0SBryan VenteicherSince the 167007054f0SBryan Venteicher.Nm 168007054f0SBryan Venteicherinterface encapsulates the Ethernet frame with an IP, UDP, and 169007054f0SBryan Venteicher.Nm 170007054f0SBryan Venteicherheader, the resulting frame may be larger than the MTU of the 171007054f0SBryan Venteicherphysical network. 172007054f0SBryan VenteicherThe 173007054f0SBryan Venteicher.Nm 174007054f0SBryan Venteicherspecification recommends the physical network MTU be configured 175007054f0SBryan Venteicherto use jumbo frames to accommodate the encapsulated frame size. 176*baacf701SKonstantin Belousov.Pp 177*baacf701SKonstantin BelousovBy default, the 178*baacf701SKonstantin Belousov.Nm 179*baacf701SKonstantin Belousovdriver sets its MTU to usual ethernet MTU of 1500 bytes, reduced by 180*baacf701SKonstantin Belousovthe size of vxlan headers prepended to the encapsulated packets. 181*baacf701SKonstantin Belousov.Pp 182007054f0SBryan VenteicherAlternatively, the 183007054f0SBryan Venteicher.Xr ifconfig 8 184007054f0SBryan Venteicher.Cm mtu 185*baacf701SKonstantin Belousovcommand may be used to set the fixed MTU size on the 186007054f0SBryan Venteicher.Nm 187007054f0SBryan Venteicherinterface to allow the encapsulated frame to fit in the 188007054f0SBryan Venteichercurrent MTU of the physical network. 189*baacf701SKonstantin BelousovIf the 190*baacf701SKonstantin Belousov.Cm mtu 191*baacf701SKonstantin Belousovcommand was used, system no longer adjust the 192*baacf701SKonstantin Belousov.Nm 193*baacf701SKonstantin Belousovinterface MTU on routing or address changes. 194b092fd6cSNavdeep Parhar.Sh HARDWARE 195b092fd6cSNavdeep ParharThe 196b092fd6cSNavdeep Parhar.Nm 197b092fd6cSNavdeep Parhardriver supports hardware checksum offload (receive and transmit) and TSO on the 198b092fd6cSNavdeep Parharencapsulated traffic over physical interfaces that support these features. 199b092fd6cSNavdeep ParharThe 200b092fd6cSNavdeep Parhar.Nm 201b092fd6cSNavdeep Parharinterface examines the 202b092fd6cSNavdeep Parhar.Cm vxlandev 203b092fd6cSNavdeep Parharinterface, if one is specified, or the interface hosting the 204b092fd6cSNavdeep Parhar.Cm vxlanlocal 205b092fd6cSNavdeep Parharaddress, and configures its capabilities based on the hardware offload 206b092fd6cSNavdeep Parharcapabilities of that physical interface. 207b092fd6cSNavdeep ParharIf multiple physical interfaces will transmit or receive traffic for the 208b092fd6cSNavdeep Parhar.Nm 209b092fd6cSNavdeep Parharthen they all must have the same hardware capabilities. 210b092fd6cSNavdeep ParharThe transmit routine of a 211b092fd6cSNavdeep Parhar.Nm 212b092fd6cSNavdeep Parharinterface may fail with 213b092fd6cSNavdeep Parhar.Er ENXIO 214b092fd6cSNavdeep Parharif an outbound physical interface does not support 215b092fd6cSNavdeep Parharan offload that the 216b092fd6cSNavdeep Parhar.Nm 217b092fd6cSNavdeep Parharinterface is requesting. 218b092fd6cSNavdeep ParharThis can happen if there are multiple physical interfaces involved, with 219b092fd6cSNavdeep Parhardifferent hardware capabilities, or an interface capability was disabled after 220b092fd6cSNavdeep Parharthe 221b092fd6cSNavdeep Parhar.Nm 222b092fd6cSNavdeep Parharinterface had already started. 223b092fd6cSNavdeep Parhar.Pp 224b092fd6cSNavdeep ParharAt present, these devices are capable of generating checksums and performing TSO 225b092fd6cSNavdeep Parharon the inner frames in hardware: 226b092fd6cSNavdeep Parhar.Xr cxgbe 4 . 227007054f0SBryan Venteicher.Sh EXAMPLES 228007054f0SBryan VenteicherCreate a 229007054f0SBryan Venteicher.Nm 230007054f0SBryan Venteicherinterface in unicast mode 231007054f0SBryan Venteicherwith the 232c6e32006SBryan Venteicher.Cm vxlanlocal 233007054f0SBryan Venteichertunnel address of 192.168.100.1, 234007054f0SBryan Venteicherand the 235c6e32006SBryan Venteicher.Cm vxlanremote 236007054f0SBryan Venteichertunnel address of 192.168.100.2. 237007054f0SBryan Venteicher.Bd -literal -offset indent 238c6e32006SBryan Venteicherifconfig vxlan create vxlanid 108 vxlanlocal 192.168.100.1 vxlanremote 192.168.100.2 239007054f0SBryan Venteicher.Ed 240007054f0SBryan Venteicher.Pp 241007054f0SBryan VenteicherCreate a 242007054f0SBryan Venteicher.Nm 243007054f0SBryan Venteicherinterface in multicast mode, 244007054f0SBryan Venteicherwith the 245007054f0SBryan Venteicher.Cm local 246007054f0SBryan Venteicheraddress of 192.168.10.95, 247007054f0SBryan Venteicherand the 248007054f0SBryan Venteicher.Cm group 249007054f0SBryan Venteicheraddress of 224.0.2.6. 250007054f0SBryan VenteicherThe em0 interface will be used to transmit multicast packets. 251007054f0SBryan Venteicher.Bd -literal -offset indent 252c6e32006SBryan Venteicherifconfig vxlan create vxlanid 42 vxlanlocal 192.168.10.95 vxlangroup 224.0.2.6 vxlandev em0 253007054f0SBryan Venteicher.Ed 254007054f0SBryan Venteicher.Pp 255007054f0SBryan VenteicherOnce created, the 256007054f0SBryan Venteicher.Nm 257007054f0SBryan Venteicherinterface can be configured with 258007054f0SBryan Venteicher.Xr ifconfig 8 . 2594d5c5d3eSBryan Venteicher.Pp 2604d5c5d3eSBryan VenteicherThe following when placed in the file 2614d5c5d3eSBryan Venteicher.Pa /etc/rc.conf 2624d5c5d3eSBryan Venteicherwill cause a vxlan interface called 2634d5c5d3eSBryan Venteicher.Dq Li vxlan0 2644d5c5d3eSBryan Venteicherto be created, and will configure the interface in unicast mode. 2654d5c5d3eSBryan Venteicher.Bd -literal -offset indent 2664d5c5d3eSBryan Venteichercloned_interfaces="vxlan0" 2674d5c5d3eSBryan Venteichercreate_args_vxlan0="vxlanid 108 vxlanlocal 192.168.100.1 vxlanremote 192.168.100.2" 268788cf725SJoel Dahl.Ed 269007054f0SBryan Venteicher.Sh SEE ALSO 270007054f0SBryan Venteicher.Xr inet 4 , 2710b3504fdSChristian Brueffer.Xr inet6 4 , 272225636dcSEdward Tomasz Napierala.Xr vlan 4 , 2734d5c5d3eSBryan Venteicher.Xr rc.conf 5 , 2740b3504fdSChristian Brueffer.Xr ifconfig 8 , 275225636dcSEdward Tomasz Napierala.Xr sysctl 8 276007054f0SBryan Venteicher.Rs 277007054f0SBryan Venteicher.%A "M. Mahalingam" 278007054f0SBryan Venteicher.%A "et al" 279007054f0SBryan Venteicher.%T "Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks" 280007054f0SBryan Venteicher.%D August 2014 281007054f0SBryan Venteicher.%O "RFC 7348" 282007054f0SBryan Venteicher.Re 283788cf725SJoel Dahl.Sh AUTHORS 284007054f0SBryan Venteicher.An -nosplit 285007054f0SBryan VenteicherThe 286007054f0SBryan Venteicher.Nm 287007054f0SBryan Venteicherdriver was written by 288007054f0SBryan Venteicher.An Bryan Venteicher Aq bryanv@freebsd.org . 289b092fd6cSNavdeep ParharSupport for stateless hardware offloads was added by 290b092fd6cSNavdeep Parhar.An Navdeep Parhar Aq np@freebsd.org 291b092fd6cSNavdeep Parharin 292b092fd6cSNavdeep Parhar.Fx 13.0 . 293