1bbf21555SRichard Lowe.\" 2bbf21555SRichard Lowe.\" This file and its contents are supplied under the terms of the 3bbf21555SRichard Lowe.\" Common Development and Distribution License ("CDDL"), version 1.0. 4bbf21555SRichard Lowe.\" You may only use this file in accordance with the terms of version 5bbf21555SRichard Lowe.\" 1.0 of the CDDL. 6bbf21555SRichard Lowe.\" 7bbf21555SRichard Lowe.\" A full copy of the text of the CDDL should have accompanied this 8bbf21555SRichard Lowe.\" source. A copy of the CDDL is also available via the Internet at 9bbf21555SRichard Lowe.\" http://www.illumos.org/license/CDDL. 10bbf21555SRichard Lowe.\" 11bbf21555SRichard Lowe.\" 12bbf21555SRichard Lowe.\" Copyright (c) 2018 Joyent, Inc. 13bbf21555SRichard Lowe.\" Copyright 2020 Ryan Zezeski 14bbf21555SRichard Lowe.\" Copyright 2021 Oxide Computer Company 15bbf21555SRichard Lowe.\" 16*0e23ca97SAndy Fiddaman.Dd March 10, 2024 17bbf21555SRichard Lowe.Dt I40E 4D 18bbf21555SRichard Lowe.Os 19bbf21555SRichard Lowe.Sh NAME 20bbf21555SRichard Lowe.Nm i40e 21bbf21555SRichard Lowe.Nd Intel 710/722 Ethernet Device Driver 22bbf21555SRichard Lowe.Sh SYNOPSIS 23bbf21555SRichard Lowe.Pa /dev/net/i40e* 24bbf21555SRichard Lowe.Sh DESCRIPTION 25bbf21555SRichard LoweThe 26bbf21555SRichard Lowe.Nm 27bbf21555SRichard Lowedriver is a GLDv3, multi-threaded, clonable, loadable device driver that 28bbf21555SRichard Lowesupports the Data Link Provider Interface, 29bbf21555SRichard Lowe.Xr dlpi 4P . 30bbf21555SRichard LoweThe 31bbf21555SRichard Lowe.Nm 32bbf21555SRichard Lowedriver supports the Intel 710 and 722 Ethernet Controller families of 33bbf21555SRichard Lowenetworking interface cards which support speeds of 1 GbE, 2.5 GbE, 5 GbE, 10 34bbf21555SRichard LoweGbE, 25 GbE, and 40 GbE. 35bbf21555SRichard Lowe.Pp 36bbf21555SRichard LoweIn addition to basic device initialization and the sending and receiving 37bbf21555SRichard Loweof frames, it supports the following features: 38bbf21555SRichard Lowe.Bl -dash -offset indent 39bbf21555SRichard Lowe.It 40bbf21555SRichard LoweJumbo frames up to 9710 bytes. 41bbf21555SRichard Lowe.It 42bbf21555SRichard LowePromiscuous access via 43bbf21555SRichard Lowe.Xr snoop 8 and 44bbf21555SRichard Lowe.Xr dlpi 4P 45bbf21555SRichard Lowe.It 46bbf21555SRichard LoweIPv4 Checksum Offload 47bbf21555SRichard Lowe.It 48*0e23ca97SAndy FiddamanTCP, UDP, and SCTP Checksum Offload 49bbf21555SRichard Lowe.It 50*0e23ca97SAndy FiddamanIPv4 and IPv6 TCP Segmentation Offload 51bbf21555SRichard Lowe.El 52bbf21555SRichard Lowe.Pp 53bbf21555SRichard LoweAt this time, the 54bbf21555SRichard Lowe.Nm 55*0e23ca97SAndy Fiddamandriver does not enable the use of energy efficient Ethernet 56*0e23ca97SAndy Fiddaman.Pq EEE 57*0e23ca97SAndy Fiddamanor support the use of flow control through hardware pause frames. 58bbf21555SRichard Lowe.Sh APPLICATION PROGRAMMING INTERFACE 59bbf21555SRichard LoweFor each device supported by the 60bbf21555SRichard Lowe.Nm 61bbf21555SRichard Loweinstalled in the system, a character-special file will be created. 62*0e23ca97SAndy FiddamanThis file supports the Data Link Provider Interface 63*0e23ca97SAndy Fiddaman.Pq DLPI 64*0e23ca97SAndy Fiddamanwhich is documented in 65bbf21555SRichard Lowe.Xr dlpi 4P . 66bbf21555SRichard LoweFor most consumers, the use of 67bbf21555SRichard Lowe.Xr libdlpi 3LIB , 68bbf21555SRichard Loweis recommended. 69bbf21555SRichard Lowe.Pp 70bbf21555SRichard LoweEach instance is assigned a unique ascending integer identifier. 71bbf21555SRichard LoweA device which has multiple ports may appear to the system as separate 72bbf21555SRichard Loweinstances. 73bbf21555SRichard LoweThe system does not provide a guarantee on how these will be presented. 74bbf21555SRichard LoweUsing this instance identifier, one can determine the exact character-special 75bbf21555SRichard Lowefile to open. 76bbf21555SRichard LoweFor example, the first instance enumerated in the system, with id 0, would be 77bbf21555SRichard Lowenamed 78bbf21555SRichard Lowe.Sy i40e0 . 79bbf21555SRichard LoweIt exists in the file system at 80bbf21555SRichard Lowe.Pa /dev/net/i40e0 . 81bbf21555SRichard Lowe.Sh CONFIGURATION 82bbf21555SRichard LoweThe 83bbf21555SRichard Lowe.Nm i40e 84bbf21555SRichard Lowedriver always performs auto-negotiation and depending on the model may 85bbf21555SRichard Lowenegotiate to 40 Gbps, 25 Gbps, 10 Gbps, or 1 Gbps. 86bbf21555SRichard LoweAt this time, the driver requires the use of auto-negotiation. 87bbf21555SRichard Lowe.Pp 88bbf21555SRichard LoweThe 89bbf21555SRichard Lowe.Nm 90bbf21555SRichard Lowedriver is managed by the 91bbf21555SRichard Lowe.Xr dladm 8 92bbf21555SRichard Loweutility. 93bbf21555SRichard Lowe.Xr dladm 8 94bbf21555SRichard Loweis the preferred interface for setting all properties. 95bbf21555SRichard LoweWhile 96bbf21555SRichard Lowe.Xr driver.conf 5 97bbf21555SRichard Lowebased configuration is possible, 98bbf21555SRichard Lowe.Xr dladm 8 99bbf21555SRichard Loweis recommended. 100bbf21555SRichard LoweThe 101bbf21555SRichard Lowe.Nm 102bbf21555SRichard Lowedriver may be joined into an aggregation based on the link aggregation 103*0e23ca97SAndy Fiddamancontrol protocol 104*0e23ca97SAndy Fiddaman.Pq LACP 105*0e23ca97SAndy Fiddamanthrough 106bbf21555SRichard Lowe.Xr dladm 8 . 107bbf21555SRichard Lowe.Sh PROPERTIES 108bbf21555SRichard LoweThe device supports the following properties which may be tuned through 109bbf21555SRichard Loweits driver.conf file, 110bbf21555SRichard Lowe.Pa /kernel/drv/i40e.conf . 111bbf21555SRichard LoweMost of these properties cannot be changed after the device has been started. 112bbf21555SRichard LoweThe device is started in response to a DLPI consumer opening the device and 113bbf21555SRichard Lowebinding to it. 114bbf21555SRichard LoweThis happens when an IP interfaces is plumbed or another 115bbf21555SRichard Lowe.Xr dlpi 4P 116bbf21555SRichard Loweconsumer such as 117bbf21555SRichard Lowe.Xr snoop 8 118bbf21555SRichard Loweor an LLDP daemon is started. 119bbf21555SRichard Lowe.Pp 120bbf21555SRichard LoweSome properties may be tuned at runtime with the 121bbf21555SRichard Lowe.Xr dladm 8 122bbf21555SRichard Loweutility. 123bbf21555SRichard LoweProperties that can be will have the name of the dladm property called out 124bbf21555SRichard Loweexplicitly. 125bbf21555SRichard Lowe.Pp 126bbf21555SRichard LoweThese properties are not considered stable at this time. 127bbf21555SRichard LoweThey may change and should not be relied on. 128bbf21555SRichard LoweThey are considered 129bbf21555SRichard Lowe.Sy Volatile . 130bbf21555SRichard LoweIt is not expected that administrators of the system will have to tune 131bbf21555SRichard Lowethese values. 132bbf21555SRichard Lowe.Bl -hang -width Ds 133bbf21555SRichard Lowe.It Sy default_mtu 134bbf21555SRichard Lowe.Bd -filled -compact 135bbf21555SRichard LoweMinimum: 136bbf21555SRichard Lowe.Sy 1500 | 137bbf21555SRichard LoweMaximum: 138bbf21555SRichard Lowe.Sy 9710 | 139bbf21555SRichard LoweRuntime Property: 140bbf21555SRichard Lowe.Sy mtu 141bbf21555SRichard Lowe.Ed 142bbf21555SRichard Lowe.Bd -filled 143bbf21555SRichard LoweThe 144bbf21555SRichard Lowe.Sy default_mtu 145bbf21555SRichard Loweproperty determines the starting MTU of the various device instances. 146bbf21555SRichard LoweNote that the device's MTU also determines the upper bound of the MTU of 147bbf21555SRichard Loweall VNICs created over the device. 148bbf21555SRichard LoweThe default MTU is 149bbf21555SRichard Lowe.Sy 1500 . 150bbf21555SRichard Lowe.Ed 151bbf21555SRichard Lowe.It Sy mr_enable 152bbf21555SRichard Lowe.Bd -filled -compact 153bbf21555SRichard LoweMinimum: 154bbf21555SRichard Lowe.Sy 0 | 155bbf21555SRichard LoweMaximum: 156bbf21555SRichard Lowe.Sy 1 157bbf21555SRichard Lowe.Ed 158bbf21555SRichard Lowe.Bd -filled 159bbf21555SRichard LoweThe 160bbf21555SRichard Lowe.Sy mr_enable 161bbf21555SRichard Loweproperty determines whether or not support for multiple rings is enabled 162bbf21555SRichard Lowefor the device. 163bbf21555SRichard LoweThe default is always to enable them. 164bbf21555SRichard LoweIt is not recommended to to disable them. 165bbf21555SRichard Lowe.Ed 166bbf21555SRichard Lowe.It Sy rx_num_groups 167bbf21555SRichard Lowe.Bd -filled -compact 168bbf21555SRichard LoweMinimum: 169bbf21555SRichard Lowe.Sy 1 | 170bbf21555SRichard LoweMaximum: 171bbf21555SRichard Lowe.Sy 32 172bbf21555SRichard Lowe.Ed 173bbf21555SRichard Lowe.Bd -filled 174bbf21555SRichard LoweThe 175bbf21555SRichard Lowe.Sy rx_num_groups 176bbf21555SRichard Loweproperty determines the number of receive mac groups provided by the driver. 177bbf21555SRichard LoweEach group can handle all unicast traffic for a single MAC address, more groups 178bbf21555SRichard Lowemeans more unicast traffic that can be steered by hardware. 179bbf21555SRichard LoweHowever, more groups also means more demand for kernel memory. 180bbf21555SRichard LoweIf you are not making heavy use of VNICs, or do not need the efficiency gains 181bbf21555SRichard Loweof hardware steering, then reducing this number can reduce kernel memory 182bbf21555SRichard Lowetaken by 183bbf21555SRichard Lowe.Nm i40e. 184bbf21555SRichard Lowe.Ed 185bbf21555SRichard Lowe.It Sy rx_ring_size 186bbf21555SRichard Lowe.Bd -filled -compact 187bbf21555SRichard LoweMinimum: 188bbf21555SRichard Lowe.Sy 64 | 189bbf21555SRichard LoweMaximum: 190bbf21555SRichard Lowe.Sy 4096 191bbf21555SRichard Lowe.Ed 192bbf21555SRichard Lowe.Bd -filled 193bbf21555SRichard LoweThe 194bbf21555SRichard Lowe.Sy rx_ring_size 195bbf21555SRichard Loweproperty determines the number of descriptors that will be used in each 196bbf21555SRichard Lowereceive ring on the card. 197bbf21555SRichard LoweAdministrators should not normally need to tune this value. 198bbf21555SRichard LoweHardware requires that the ring size be a multiple of 32. 199bbf21555SRichard LoweThe system will round up the set value to the nearest multiple of 32. 200bbf21555SRichard Lowe.Ed 201bbf21555SRichard Lowe.It Sy tx_ring_size 202bbf21555SRichard Lowe.Bd -filled -compact 203bbf21555SRichard LoweMinimum: 204bbf21555SRichard Lowe.Sy 64 | 205bbf21555SRichard LoweMaximum: 206bbf21555SRichard Lowe.Sy 4096 207bbf21555SRichard Lowe.Ed 208bbf21555SRichard Lowe.Bd -filled 209bbf21555SRichard LoweThe 210bbf21555SRichard Lowe.Sy tx_ring_size 211bbf21555SRichard Loweproperty determines the number of descriptors that will be used in each 212bbf21555SRichard Lowetransmit ring on the card. 213bbf21555SRichard LoweAdministrators should not normally need to tune this value. 214bbf21555SRichard LoweHardware requires that the ring size be a multiple of 32. 215bbf21555SRichard LoweThe system will round up the set value to the nearest multiple of 32. 216bbf21555SRichard Lowe.Ed 217bbf21555SRichard Lowe.It Sy tx_resched_threshold 218bbf21555SRichard Lowe.Bd -filled -compact 219bbf21555SRichard LoweMinimum: 220bbf21555SRichard Lowe.Sy 8 | 221bbf21555SRichard LoweMaximum: 222bbf21555SRichard Lowe.Sy Variable 223bbf21555SRichard Lowe.Ed 224bbf21555SRichard Lowe.Bd -filled 225bbf21555SRichard LoweThe 226bbf21555SRichard Lowe.Sy tx_resched_threshold 227bbf21555SRichard Loweproperty determines the number of descriptors that must be available for 228bbf21555SRichard Lowea frame to be transmitted. 229bbf21555SRichard LoweThe maximum is variable. 230bbf21555SRichard LoweIt is dependent on the value of the 231bbf21555SRichard Lowe.Sy tx_ring_size 232bbf21555SRichard Loweproperty. 233bbf21555SRichard LoweAt least eight descriptors must be available for the device to function 234bbf21555SRichard Lowecorrectly. 235bbf21555SRichard Lowe.Ed 236bbf21555SRichard Lowe.It Sy rx_limit_per_intr 237bbf21555SRichard Lowe.Bd -filled -compact 238bbf21555SRichard LoweMinimum: 239bbf21555SRichard Lowe.Sy 16 | 240bbf21555SRichard LoweMaximum: 241bbf21555SRichard Lowe.Sy 4096 242bbf21555SRichard Lowe.Ed 243bbf21555SRichard Lowe.Bd -filled 244bbf21555SRichard LoweThe 245bbf21555SRichard Lowe.Sy rx_limit_per_intr 246bbf21555SRichard Loweproperty determines the maximum number of packets that will be processed 247bbf21555SRichard Loweon a given ring during a single interrupt. 248bbf21555SRichard LoweThis is done to try and guarantee some amount of liveness in the system. 249bbf21555SRichard LoweIt is not expected that administrators will have to tune this value. 250bbf21555SRichard Lowe.Ed 251bbf21555SRichard Lowe.It Sy tx_hcksum_enable 252bbf21555SRichard Lowe.Bd -filled -compact 253bbf21555SRichard LoweMinimum: 254bbf21555SRichard Lowe.Sy 0 | 255bbf21555SRichard LoweMaximum: 256bbf21555SRichard Lowe.Sy 1 257bbf21555SRichard Lowe.Ed 258bbf21555SRichard Lowe.Bd -filled 259bbf21555SRichard LoweThe 260bbf21555SRichard Lowe.Sy tx_hcksum_enable 261bbf21555SRichard Loweproperty controls whether or not the device enables support for hardware 262bbf21555SRichard Lowechecksumming of outgoing packets. 263bbf21555SRichard LoweThe default is to always enable support for this. 264bbf21555SRichard LoweTurning it off will increase latency and decrease throughput when transmitting 265bbf21555SRichard Lowepackets, but should be done if a hardware bug is suspected. 266bbf21555SRichard Lowe.Ed 267bbf21555SRichard Lowe.It Sy rx_hcksum_enable 268bbf21555SRichard Lowe.Bd -filled -compact 269bbf21555SRichard LoweMinimum: 270bbf21555SRichard Lowe.Sy 0 | 271bbf21555SRichard LoweMaximum: 272bbf21555SRichard Lowe.Sy 1 273bbf21555SRichard Lowe.Ed 274bbf21555SRichard Lowe.Bd -filled 275bbf21555SRichard LoweThe 276bbf21555SRichard Lowe.Sy rx_hcksum_enable 277bbf21555SRichard Loweproperty controls whether or not the device enables support for hardware 278bbf21555SRichard Lowechecksumming of incoming packets. 279bbf21555SRichard LoweThe default is to always enable support for this. 280bbf21555SRichard LoweTurning it off will increase latency and decrease throughput when receiving 281bbf21555SRichard Lowepackets, but should be done if a hardware bug is suspected. 282bbf21555SRichard Lowe.Ed 283bbf21555SRichard Lowe.It Sy rx_dma_threshold 284bbf21555SRichard Lowe.Bd -filled -compact 285bbf21555SRichard LoweMinimum: 286bbf21555SRichard Lowe.Sy 0 | 287bbf21555SRichard LoweMaximum: 288bbf21555SRichard Lowe.Sy INT32_MAX | 289bbf21555SRichard LoweRuntime Property: 290bbf21555SRichard Lowe.Sy _rx_dma_threshold 291bbf21555SRichard Lowe.Ed 292bbf21555SRichard Lowe.Bd -filled 293bbf21555SRichard LoweThe 294bbf21555SRichard Lowe.Sy rx_dma_threshold 295bbf21555SRichard Loweindicates the size in bytes of a received frame, including all of its 296bbf21555SRichard Loweheaders, at which the driver should not copy the frame but instead bind 297bbf21555SRichard LoweDMA memory. 298bbf21555SRichard LoweBy setting this property to its minimum, all frames will be processed with DMA 299bbf21555SRichard Lowebinding. 300bbf21555SRichard LoweBy setting this property to its maximum, all frames will be processed by copying 301bbf21555SRichard Lowethe frame. 302bbf21555SRichard Lowe.Ed 303bbf21555SRichard Lowe.It Sy tx_lso_enable 304bbf21555SRichard Lowe.Bd -filled -compact 305bbf21555SRichard LoweMinimum: 306bbf21555SRichard Lowe.Sy 0 | 307bbf21555SRichard LoweMaximum: 308bbf21555SRichard Lowe.Sy 1 309bbf21555SRichard Lowe.Ed 310bbf21555SRichard Lowe.Bd -filled 311bbf21555SRichard LoweThe 312bbf21555SRichard Lowe.Sy tx_lso_enable 313bbf21555SRichard Loweproperty controls whether or not the device enables support for Large Segment 314*0e23ca97SAndy FiddamanOffload 315*0e23ca97SAndy Fiddaman.Pq LSO 316*0e23ca97SAndy Fiddamanwhen transmitting packets. 317bbf21555SRichard LoweThe default is to always enable support for this. 318bbf21555SRichard LoweTurning it off will decrease throughput when transmitting packets, but should 319bbf21555SRichard Lowebe done if a hardware bug is suspected. 320bbf21555SRichard Lowe.Ed 321bbf21555SRichard Lowe.El 322bbf21555SRichard Lowe.Sh ARCHITECTURE 323bbf21555SRichard LoweThe 324bbf21555SRichard Lowe.Nm 325bbf21555SRichard Lowedriver is only supported on 326bbf21555SRichard Lowe.Sy x86 327bbf21555SRichard Lowesystems at this time. 328bbf21555SRichard Lowe.Sh FILES 329bbf21555SRichard Lowe.Bl -tag -width Pa 330bbf21555SRichard Lowe.It Pa /dev/net/i40e* 331bbf21555SRichard LowePer-instance character device. 332bbf21555SRichard Lowe.It Pa /kernel/drv/amd64/i40e 333*0e23ca97SAndy FiddamanDevice driver 334*0e23ca97SAndy Fiddaman.Pq x86 335bbf21555SRichard Lowe.It Pa /kernel/drv/i40e.conf 336bbf21555SRichard LoweDriver configuration file 337bbf21555SRichard Lowe.El 338bbf21555SRichard Lowe.Sh SEE ALSO 339bbf21555SRichard Lowe.Xr dlpi 4P , 340bbf21555SRichard Lowe.Xr driver.conf 5 , 341bbf21555SRichard Lowe.Xr dladm 8 , 342bbf21555SRichard Lowe.Xr snoop 8 343