1*fdc35dd8SRobert Mustacchi.\" 2*fdc35dd8SRobert Mustacchi.\" This file and its contents are supplied under the terms of the 3*fdc35dd8SRobert Mustacchi.\" Common Development and Distribution License ("CDDL"), version 1.0. 4*fdc35dd8SRobert Mustacchi.\" You may only use this file in accordance with the terms of version 5*fdc35dd8SRobert Mustacchi.\" 1.0 of the CDDL. 6*fdc35dd8SRobert Mustacchi.\" 7*fdc35dd8SRobert Mustacchi.\" A full copy of the text of the CDDL should have accompanied this 8*fdc35dd8SRobert Mustacchi.\" source. A copy of the CDDL is also available via the Internet at 9*fdc35dd8SRobert Mustacchi.\" http://www.illumos.org/license/CDDL. 10*fdc35dd8SRobert Mustacchi.\" 11*fdc35dd8SRobert Mustacchi.\" 12*fdc35dd8SRobert Mustacchi.\" Copyright 2016 Joyent, Inc. 13*fdc35dd8SRobert Mustacchi.\" 14*fdc35dd8SRobert Mustacchi.Dd March 25, 2016 15*fdc35dd8SRobert Mustacchi.Dt I40E 7D 16*fdc35dd8SRobert Mustacchi.Os 17*fdc35dd8SRobert Mustacchi.Sh NAME 18*fdc35dd8SRobert Mustacchi.Nm i40e 19*fdc35dd8SRobert Mustacchi.Nd Intel XL710 10/40 Gigabit Ethernet Device Driver 20*fdc35dd8SRobert Mustacchi.Sh SYNOPSIS 21*fdc35dd8SRobert Mustacchi.Pa /dev/i40e* 22*fdc35dd8SRobert Mustacchi.Sh DESCRIPTION 23*fdc35dd8SRobert MustacchiThe 24*fdc35dd8SRobert Mustacchi.Nm 25*fdc35dd8SRobert Mustacchidriver is a GLDv3, multi-threaded, clonable, loadable device driver that 26*fdc35dd8SRobert Mustacchisupports the Data Link Provider Interface, 27*fdc35dd8SRobert Mustacchi.Xr dlpi 7P . 28*fdc35dd8SRobert MustacchiThe 29*fdc35dd8SRobert Mustacchi.Nm 30*fdc35dd8SRobert Mustacchidriver supports the Intel XL710 Ethernet Controller family of networking 31*fdc35dd8SRobert Mustacchiinterface cards which come in both 10 GbE and 40 GbE variants. 32*fdc35dd8SRobert Mustacchi.Pp 33*fdc35dd8SRobert MustacchiIn addition to basic device initialization and the sending and receiving 34*fdc35dd8SRobert Mustacchiof frames, it supports the following features: 35*fdc35dd8SRobert Mustacchi.Bl -dash -offset indent 36*fdc35dd8SRobert Mustacchi.It 37*fdc35dd8SRobert MustacchiJumbo frames up to 9710 bytes. 38*fdc35dd8SRobert Mustacchi.It 39*fdc35dd8SRobert MustacchiPromiscuous access via 40*fdc35dd8SRobert Mustacchi.Xr snoop 1M and 41*fdc35dd8SRobert Mustacchi.Xr dlpi 7P 42*fdc35dd8SRobert Mustacchi.It 43*fdc35dd8SRobert MustacchiIPv4 Checksum Offload 44*fdc35dd8SRobert Mustacchi.It 45*fdc35dd8SRobert MustacchiTCP, UDP, and SCTP checksum offload 46*fdc35dd8SRobert Mustacchi.El 47*fdc35dd8SRobert Mustacchi.Pp 48*fdc35dd8SRobert MustacchiAt this time, the 49*fdc35dd8SRobert Mustacchi.Nm 50*fdc35dd8SRobert Mustacchidriver does not enable the use of energy efficient Ethernet (EEE) or 51*fdc35dd8SRobert Mustacchisupport the use of flow control through hardware pause frames. 52*fdc35dd8SRobert Mustacchi.Sh APPLICATION PROGRAMMING INTERFACE 53*fdc35dd8SRobert MustacchiFor each device supported by the 54*fdc35dd8SRobert Mustacchi.Nm 55*fdc35dd8SRobert Mustacchiinstalled in the system, a character-special file will be created. This 56*fdc35dd8SRobert Mustacchifile supports the Data Link Provider Interface (DLPI) which is documented 57*fdc35dd8SRobert Mustacchiin 58*fdc35dd8SRobert Mustacchi.Xr dlpi 7P . 59*fdc35dd8SRobert MustacchiFor most consumers, the use of 60*fdc35dd8SRobert Mustacchi.Xr libdlpi 3LIB , 61*fdc35dd8SRobert Mustacchiis recommended. 62*fdc35dd8SRobert Mustacchi.Pp 63*fdc35dd8SRobert MustacchiEach instance is assigned a unique ascending integer identifier. A 64*fdc35dd8SRobert Mustacchidevice which has multiple ports may appear to the system as separate 65*fdc35dd8SRobert Mustacchiinstances. The system does not provide a guarnatee on how these will be 66*fdc35dd8SRobert Mustacchipresented. Using this instance identifier, one can determine the exact 67*fdc35dd8SRobert Mustacchicharacter-special file to open. For example, the first instance 68*fdc35dd8SRobert Mustacchienumerated in the system, with id 0, would be named 69*fdc35dd8SRobert Mustacchi.Sy i40e0 . 70*fdc35dd8SRobert MustacchiIt exists in the file system at 71*fdc35dd8SRobert Mustacchi.Pa /dev/i40e0 . 72*fdc35dd8SRobert Mustacchi.Sh CONFIGURATION 73*fdc35dd8SRobert MustacchiThe 74*fdc35dd8SRobert Mustacchi.Nm i40e 75*fdc35dd8SRobert Mustacchidriver always performs auto-negotiation and depending on the model may 76*fdc35dd8SRobert Mustacchinegotiate to 40 Gbps, 10 Gbps, or 1 Gbps. At this time, there are no 77*fdc35dd8SRobert Mustacchimodels that support 1000BASE-T and thus auto-negotiation is required. 78*fdc35dd8SRobert Mustacchi.Pp 79*fdc35dd8SRobert MustacchiThe 80*fdc35dd8SRobert Mustacchi.Nm 81*fdc35dd8SRobert Mustacchidriver is managed by the 82*fdc35dd8SRobert Mustacchi.Xr dladm 1M 83*fdc35dd8SRobert Mustacchiutility. 84*fdc35dd8SRobert Mustacchi.Xr dladm 1M 85*fdc35dd8SRobert Mustacchiis the preferred interface for setting all properties. While 86*fdc35dd8SRobert Mustacchi.Xr driver.conf based configuration is possible, 87*fdc35dd8SRobert Mustacchi.Xr dladm 1M 88*fdc35dd8SRobert Mustacchiis recommended. The 89*fdc35dd8SRobert Mustacchi.Nm 90*fdc35dd8SRobert Mustacchidriver may be joined into an aggregation based on the link aggregation 91*fdc35dd8SRobert Mustacchicontrol protocol (LACP) through 92*fdc35dd8SRobert Mustacchi.Xr dladm 1M . 93*fdc35dd8SRobert Mustacchi.Sh PROPERTIES 94*fdc35dd8SRobert MustacchiThe device supports the following properties which may be tuned through 95*fdc35dd8SRobert Mustacchiits driver.conf file, 96*fdc35dd8SRobert Mustacchi.Pa /kernel/drv/i40e.conf . 97*fdc35dd8SRobert MustacchiMost of these properties cannot be changed after the device has been 98*fdc35dd8SRobert Mustacchistarted. The device is started in response to a DLPI consumer opening 99*fdc35dd8SRobert Mustacchithe device and binding to it. This happens when an IP interfaces is 100*fdc35dd8SRobert Mustacchiplumbed or another 101*fdc35dd8SRobert Mustacchi.Xr dlpi 7P 102*fdc35dd8SRobert Mustacchiconsumer such as 103*fdc35dd8SRobert Mustacchi.Xr snoop 1M 104*fdc35dd8SRobert Mustacchior an LLDP daemon is started. 105*fdc35dd8SRobert Mustacchi.Pp 106*fdc35dd8SRobert MustacchiSome properties may be tuned at runtime with the 107*fdc35dd8SRobert Mustacchi.Xr dladm 1M 108*fdc35dd8SRobert Mustacchiutility. Properties that can be will have the name of the dladm property 109*fdc35dd8SRobert Mustacchicalled out explicitly. 110*fdc35dd8SRobert Mustacchi.Pp 111*fdc35dd8SRobert MustacchiThese properties are not considered stable at this time. They may change 112*fdc35dd8SRobert Mustacchiand should not be relied on. They are considered 113*fdc35dd8SRobert Mustacchi.Sy Volatile . 114*fdc35dd8SRobert MustacchiIt is not expected that administrators of the system will have to tune 115*fdc35dd8SRobert Mustacchithese values. 116*fdc35dd8SRobert Mustacchi.Bl -hang -width Ds 117*fdc35dd8SRobert Mustacchi.It Sy default_mtu 118*fdc35dd8SRobert Mustacchi.Bd -filled -compact 119*fdc35dd8SRobert MustacchiMinimum: 120*fdc35dd8SRobert Mustacchi.Sy 1500 | 121*fdc35dd8SRobert MustacchiMaximum: 122*fdc35dd8SRobert Mustacchi.Sy 9710 | 123*fdc35dd8SRobert MustacchiRuntime Property: 124*fdc35dd8SRobert Mustacchi.Sy mtu 125*fdc35dd8SRobert Mustacchi.Ed 126*fdc35dd8SRobert Mustacchi.Bd -filled 127*fdc35dd8SRobert MustacchiThe 128*fdc35dd8SRobert Mustacchi.Sy default_mtu 129*fdc35dd8SRobert Mustacchiproperty determines the starting MTU of the various device instances. 130*fdc35dd8SRobert MustacchiNote that the device's MTU also determines the upper bound of the MTU of 131*fdc35dd8SRobert Mustacchiall VNICs created over the device. The default MTU is 132*fdc35dd8SRobert Mustacchi.Sy 1500 . 133*fdc35dd8SRobert Mustacchi.Ed 134*fdc35dd8SRobert Mustacchi.It Sy mr_enable 135*fdc35dd8SRobert Mustacchi.Bd -filled -compact 136*fdc35dd8SRobert MustacchiMinimum: 137*fdc35dd8SRobert Mustacchi.Sy 0 | 138*fdc35dd8SRobert MustacchiMaximum: 139*fdc35dd8SRobert Mustacchi.Sy 1 140*fdc35dd8SRobert Mustacchi.Ed 141*fdc35dd8SRobert Mustacchi.Bd -filled 142*fdc35dd8SRobert MustacchiThe 143*fdc35dd8SRobert Mustacchi.Sy mr_enable 144*fdc35dd8SRobert Mustacchiproeprty determines whether or not support for multiple rings is enabled 145*fdc35dd8SRobert Mustacchifor the device. The default is always to enable them. It is not 146*fdc35dd8SRobert Mustacchirecommended to to disable them. 147*fdc35dd8SRobert Mustacchi.Ed 148*fdc35dd8SRobert Mustacchi.It Sy rx_ring_size 149*fdc35dd8SRobert Mustacchi.Bd -filled -compact 150*fdc35dd8SRobert MustacchiMinimum: 151*fdc35dd8SRobert Mustacchi.Sy 64 | 152*fdc35dd8SRobert MustacchiMaximum: 153*fdc35dd8SRobert Mustacchi.Sy 4096 154*fdc35dd8SRobert Mustacchi.Ed 155*fdc35dd8SRobert Mustacchi.Bd -filled 156*fdc35dd8SRobert MustacchiThe 157*fdc35dd8SRobert Mustacchi.Sy rx_ring_size 158*fdc35dd8SRobert Mustacchiproperty determines the number of descriptors that will be used in each 159*fdc35dd8SRobert Mustacchireceive ring on the card. Administrators should not normally need to 160*fdc35dd8SRobert Mustacchitune this value. Hardware requires that the ring size be a multiple of 161*fdc35dd8SRobert Mustacchi32. The system will round up the set value to the nearest multiple of 162*fdc35dd8SRobert Mustacchi32. 163*fdc35dd8SRobert Mustacchi.Ed 164*fdc35dd8SRobert Mustacchi.It Sy tx_ring_size 165*fdc35dd8SRobert Mustacchi.Bd -filled -compact 166*fdc35dd8SRobert MustacchiMinimum: 167*fdc35dd8SRobert Mustacchi.Sy 64 | 168*fdc35dd8SRobert MustacchiMaximum: 169*fdc35dd8SRobert Mustacchi.Sy 4096 170*fdc35dd8SRobert Mustacchi.Ed 171*fdc35dd8SRobert Mustacchi.Bd -filled 172*fdc35dd8SRobert MustacchiThe 173*fdc35dd8SRobert Mustacchi.Sy tx_ring_size 174*fdc35dd8SRobert Mustacchiproperty determines the number of descriptors that will be used in each 175*fdc35dd8SRobert Mustacchitransmit ring on the card. Administrators should not normally need to 176*fdc35dd8SRobert Mustacchitune this value. Hardware requires that the ring size be a multiple of 177*fdc35dd8SRobert Mustacchi32. The system will round up the set value to the nearest multiple of 178*fdc35dd8SRobert Mustacchi32. 179*fdc35dd8SRobert Mustacchi.Ed 180*fdc35dd8SRobert Mustacchi.It Sy tx_resched_threshold 181*fdc35dd8SRobert Mustacchi.Bd -filled -compact 182*fdc35dd8SRobert MustacchiMinimum: 183*fdc35dd8SRobert Mustacchi.Sy 8 | 184*fdc35dd8SRobert MustacchiMaximum: 185*fdc35dd8SRobert Mustacchi.Sy Variable 186*fdc35dd8SRobert Mustacchi.Ed 187*fdc35dd8SRobert Mustacchi.Bd -filled 188*fdc35dd8SRobert MustacchiThe 189*fdc35dd8SRobert Mustacchi.Sy tx_resched_threshold 190*fdc35dd8SRobert Mustacchiproperty determines the number of descriptors that must be available for 191*fdc35dd8SRobert Mustacchia frame to be transmitted. The maximum is variable. It is dependent on 192*fdc35dd8SRobert Mustacchithe value of the 193*fdc35dd8SRobert Mustacchi.Sy tx_ring_size 194*fdc35dd8SRobert Mustacchiproperty. At least eight descriptors must be available for the device to 195*fdc35dd8SRobert Mustacchifunction correctly. 196*fdc35dd8SRobert Mustacchi.Ed 197*fdc35dd8SRobert Mustacchi.It Sy rx_limit_per_intr 198*fdc35dd8SRobert Mustacchi.Bd -filled -compact 199*fdc35dd8SRobert MustacchiMinimum: 200*fdc35dd8SRobert Mustacchi.Sy 16 | 201*fdc35dd8SRobert MustacchiMaximum: 202*fdc35dd8SRobert Mustacchi.Sy 4096 203*fdc35dd8SRobert Mustacchi.Ed 204*fdc35dd8SRobert Mustacchi.Bd -filled 205*fdc35dd8SRobert MustacchiThe 206*fdc35dd8SRobert Mustacchi.Sy rx_limit_per_intr 207*fdc35dd8SRobert Mustacchiproperty determines the maximum number of packets that will be processed 208*fdc35dd8SRobert Mustacchion a given ring during a single interrupt. This is done to try and 209*fdc35dd8SRobert Mustacchiguarantee some amount of liveness in the system. It is not expected 210*fdc35dd8SRobert Mustacchithat administrators will have to tune this value. 211*fdc35dd8SRobert Mustacchi.Ed 212*fdc35dd8SRobert Mustacchi.It Sy tx_hcksum_enable 213*fdc35dd8SRobert Mustacchi.Bd -filled -compact 214*fdc35dd8SRobert MustacchiMinimum: 215*fdc35dd8SRobert Mustacchi.Sy 0 | 216*fdc35dd8SRobert MustacchiMaximum: 217*fdc35dd8SRobert Mustacchi.Sy 1 218*fdc35dd8SRobert Mustacchi.Ed 219*fdc35dd8SRobert Mustacchi.Bd -filled 220*fdc35dd8SRobert MustacchiThe 221*fdc35dd8SRobert Mustacchi.Sy tx_hcksum_enable 222*fdc35dd8SRobert Mustacchiproperty controls whether or not the device enables support for hardware 223*fdc35dd8SRobert Mustacchichecksuming of outgoing packets. The default is to always enable support 224*fdc35dd8SRobert Mustacchifor this. Turning it off will increase latency and decrease throughput 225*fdc35dd8SRobert Mustacchiwhen transmitting packets, but should be done if a hardware bug is 226*fdc35dd8SRobert Mustacchisuspected. 227*fdc35dd8SRobert Mustacchi.Ed 228*fdc35dd8SRobert Mustacchi.It Sy rx_hcksum_enable 229*fdc35dd8SRobert Mustacchi.Bd -filled -compact 230*fdc35dd8SRobert MustacchiMinimum: 231*fdc35dd8SRobert Mustacchi.Sy 0 | 232*fdc35dd8SRobert MustacchiMaximum: 233*fdc35dd8SRobert Mustacchi.Sy 1 234*fdc35dd8SRobert Mustacchi.Ed 235*fdc35dd8SRobert Mustacchi.Bd -filled 236*fdc35dd8SRobert MustacchiThe 237*fdc35dd8SRobert Mustacchi.Sy rx_hcksum_enable 238*fdc35dd8SRobert Mustacchiproperty controls whether or not the device enables support for hardware 239*fdc35dd8SRobert Mustacchichecksuming of incoming packets. The default is to always enable support 240*fdc35dd8SRobert Mustacchifor this. Turning it off will increase latency and decrease throughput 241*fdc35dd8SRobert Mustacchiwhen receiving packets, but should be done if a hardware bug is 242*fdc35dd8SRobert Mustacchisuspected. 243*fdc35dd8SRobert Mustacchi.Ed 244*fdc35dd8SRobert Mustacchi.It Sy rx_dma_threshold 245*fdc35dd8SRobert Mustacchi.Bd -filled -compact 246*fdc35dd8SRobert MustacchiMinimum: 247*fdc35dd8SRobert Mustacchi.Sy 0 | 248*fdc35dd8SRobert MustacchiMaximum: 249*fdc35dd8SRobert Mustacchi.Sy INT32_MAX | 250*fdc35dd8SRobert MustacchiRuntime Property: 251*fdc35dd8SRobert Mustacchi.Sy _rx_dma_treshold 252*fdc35dd8SRobert Mustacchi.Ed 253*fdc35dd8SRobert Mustacchi.Bd -filled 254*fdc35dd8SRobert MustacchiThe 255*fdc35dd8SRobert Mustacchi.Sy rx_dma_treshold 256*fdc35dd8SRobert Mustacchiindicates the size in bytes of a received frame, including all of its 257*fdc35dd8SRobert Mustacchiheaders, at which the driver should not copy the frame but instead bind 258*fdc35dd8SRobert MustacchiDMA memory. By setting this property to its minimum, all frames will be 259*fdc35dd8SRobert Mustacchiprocessed with DMA binding. By setting this property to its maximum, all 260*fdc35dd8SRobert Mustacchiframes will be processed by copying the frame. 261*fdc35dd8SRobert Mustacchi.Ed 262*fdc35dd8SRobert Mustacchi.El 263*fdc35dd8SRobert Mustacchi.Sh ARCHITECTURE 264*fdc35dd8SRobert MustacchiThe 265*fdc35dd8SRobert Mustacchi.Nm 266*fdc35dd8SRobert Mustacchidriver is only supported on 267*fdc35dd8SRobert Mustacchi.Sy x86 268*fdc35dd8SRobert Mustacchisystems at this time. 269*fdc35dd8SRobert Mustacchi.Sh FILES 270*fdc35dd8SRobert Mustacchi.Bl -tag -width Pa 271*fdc35dd8SRobert Mustacchi.It Pa /dev/i40e* 272*fdc35dd8SRobert MustacchiPer-instance character device. 273*fdc35dd8SRobert Mustacchi.It Pa /kernel/drv/i40e 274*fdc35dd8SRobert Mustacchi32-bit device driver (x86). 275*fdc35dd8SRobert Mustacchi.It Pa /kernel/drv/amd64/i40e 276*fdc35dd8SRobert Mustacchi64-bit device driver (x86). 277*fdc35dd8SRobert Mustacchi.It Pa /kernel/drv/i40e.conf 278*fdc35dd8SRobert MustacchiDriver configuration file. 279*fdc35dd8SRobert Mustacchi.El 280*fdc35dd8SRobert Mustacchi.Sh SEE ALSO 281*fdc35dd8SRobert Mustacchi.Xr dladm 1M , 282*fdc35dd8SRobert Mustacchi.Xr snoop 1M , 283*fdc35dd8SRobert Mustacchi.Xr driver.conf 4 , 284*fdc35dd8SRobert Mustacchi.Xr dlpi 7P 285