1*ad016cb8SNavdeep Parhar.\" Copyright (c) 2011-2013, Chelsio Inc 254e4ee71SNavdeep Parhar.\" All rights reserved. 354e4ee71SNavdeep Parhar.\" 454e4ee71SNavdeep Parhar.\" Redistribution and use in source and binary forms, with or without 554e4ee71SNavdeep Parhar.\" modification, are permitted provided that the following conditions are met: 654e4ee71SNavdeep Parhar.\" 754e4ee71SNavdeep Parhar.\" 1. Redistributions of source code must retain the above copyright notice, 854e4ee71SNavdeep Parhar.\" this list of conditions and the following disclaimer. 954e4ee71SNavdeep Parhar.\" 1054e4ee71SNavdeep Parhar.\" 2. Redistributions in binary form must reproduce the above copyright 1154e4ee71SNavdeep Parhar.\" notice, this list of conditions and the following disclaimer in the 1254e4ee71SNavdeep Parhar.\" documentation and/or other materials provided with the distribution. 1354e4ee71SNavdeep Parhar.\" 1454e4ee71SNavdeep Parhar.\" 3. Neither the name of the Chelsio Inc nor the names of its 1554e4ee71SNavdeep Parhar.\" contributors may be used to endorse or promote products derived from 1654e4ee71SNavdeep Parhar.\" this software without specific prior written permission. 1754e4ee71SNavdeep Parhar.\" 1854e4ee71SNavdeep Parhar.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 1954e4ee71SNavdeep Parhar.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 2054e4ee71SNavdeep Parhar.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 2154e4ee71SNavdeep Parhar.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 2254e4ee71SNavdeep Parhar.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 2354e4ee71SNavdeep Parhar.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 2454e4ee71SNavdeep Parhar.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 2554e4ee71SNavdeep Parhar.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 2654e4ee71SNavdeep Parhar.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 2754e4ee71SNavdeep Parhar.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 2854e4ee71SNavdeep Parhar.\" POSSIBILITY OF SUCH DAMAGE. 2954e4ee71SNavdeep Parhar.\" 3054e4ee71SNavdeep Parhar.\" * Other names and brands may be claimed as the property of others. 3154e4ee71SNavdeep Parhar.\" 3254e4ee71SNavdeep Parhar.\" $FreeBSD$ 3354e4ee71SNavdeep Parhar.\" 34*ad016cb8SNavdeep Parhar.Dd December 18, 2013 3554e4ee71SNavdeep Parhar.Dt CXGBE 4 3654e4ee71SNavdeep Parhar.Os 3754e4ee71SNavdeep Parhar.Sh NAME 3854e4ee71SNavdeep Parhar.Nm cxgbe 39*ad016cb8SNavdeep Parhar.Nd "Chelsio T4 and T5 based 40Gb, 10Gb, and 1Gb Ethernet adapter driver" 4054e4ee71SNavdeep Parhar.Sh SYNOPSIS 4154e4ee71SNavdeep ParharTo compile this driver into the kernel, 4254e4ee71SNavdeep Parharplace the following lines in your 4354e4ee71SNavdeep Parharkernel configuration file: 4454e4ee71SNavdeep Parhar.Bd -ragged -offset indent 4554e4ee71SNavdeep Parhar.Cd "device cxgbe" 4654e4ee71SNavdeep Parhar.Ed 4754e4ee71SNavdeep Parhar.Pp 4854e4ee71SNavdeep ParharTo load the driver as a 497bbe4d1fSNavdeep Parharmodule at boot time, place the following lines in 5054e4ee71SNavdeep Parhar.Xr loader.conf 5 : 5154e4ee71SNavdeep Parhar.Bd -literal -offset indent 527bbe4d1fSNavdeep Parhart4fw_cfg_load="YES" 53*ad016cb8SNavdeep Parhart5fw_cfg_load="YES" 5454e4ee71SNavdeep Parharif_cxgbe_load="YES" 5554e4ee71SNavdeep Parhar.Ed 5654e4ee71SNavdeep Parhar.Sh DESCRIPTION 5754e4ee71SNavdeep ParharThe 5854e4ee71SNavdeep Parhar.Nm 5954e4ee71SNavdeep Parhardriver provides support for PCI Express Ethernet adapters based on 60*ad016cb8SNavdeep Parharthe Chelsio Terminator 4 and Terminator 5 ASICs (T4 and T5). 613102cfe2SGlen BarberThe driver supports Jumbo Frames, Transmit/Receive checksum offload, 6254e4ee71SNavdeep ParharTCP segmentation offload (TSO), Large Receive Offload (LRO), VLAN 6354e4ee71SNavdeep Parhartag insertion/extraction, VLAN checksum offload, VLAN TSO, and 6454e4ee71SNavdeep ParharReceive Side Steering (RSS). 6554e4ee71SNavdeep ParharFor further hardware information and questions related to hardware 6654e4ee71SNavdeep Parharrequirements, see 6754e4ee71SNavdeep Parhar.Pa http://www.chelsio.com/ . 6854e4ee71SNavdeep Parhar.Pp 69*ad016cb8SNavdeep ParharNote that ports of T5 cards are named cxl and attach to a t5nex parent device 70*ad016cb8SNavdeep Parhar(in contrast to ports named cxgbe that attach to a t4nex parent for a T4 card). 71*ad016cb8SNavdeep ParharLoader tunables with the hw.cxgbe prefix apply to both T4 and T5 cards. 72*ad016cb8SNavdeep ParharThe sysctl MIBs are at dev.t5nex and dev.cxl for T5 cards and at dev.t4nex and 73*ad016cb8SNavdeep Parhardev.cxgbe for T4 cards. 74*ad016cb8SNavdeep Parhar 75*ad016cb8SNavdeep Parhar.Pp 7654e4ee71SNavdeep ParharFor more information on configuring this device, see 7754e4ee71SNavdeep Parhar.Xr ifconfig 8 . 7854e4ee71SNavdeep Parhar.Sh HARDWARE 7954e4ee71SNavdeep ParharThe 8054e4ee71SNavdeep Parhar.Nm 81*ad016cb8SNavdeep Parhardriver supports 40Gb, 10Gb and 1Gb Ethernet adapters based on the T5 ASIC 82*ad016cb8SNavdeep Parhar(ports will be named cxl): 83*ad016cb8SNavdeep Parhar.Pp 84*ad016cb8SNavdeep Parhar.Bl -bullet -compact 85*ad016cb8SNavdeep Parhar.It 86*ad016cb8SNavdeep ParharChelsio T580-CR 87*ad016cb8SNavdeep Parhar.It 88*ad016cb8SNavdeep ParharChelsio T580-LP-CR 89*ad016cb8SNavdeep Parhar.It 90*ad016cb8SNavdeep ParharChelsio T580-LP-SO-CR 91*ad016cb8SNavdeep Parhar.It 92*ad016cb8SNavdeep ParharChelsio T560-CR 93*ad016cb8SNavdeep Parhar.It 94*ad016cb8SNavdeep ParharChelsio T540-CR 95*ad016cb8SNavdeep Parhar.It 96*ad016cb8SNavdeep ParharChelsio T540-LP-CR 97*ad016cb8SNavdeep Parhar.It 98*ad016cb8SNavdeep ParharChelsio T522-CR 99*ad016cb8SNavdeep Parhar.It 100*ad016cb8SNavdeep ParharChelsio T520-LL-CR 101*ad016cb8SNavdeep Parhar.It 102*ad016cb8SNavdeep ParharChelsio T520-CR 103*ad016cb8SNavdeep Parhar.It 104*ad016cb8SNavdeep ParharChelsio T520-SO 105*ad016cb8SNavdeep Parhar.It 106*ad016cb8SNavdeep ParharChelsio T520-BT 107*ad016cb8SNavdeep Parhar.It 108*ad016cb8SNavdeep ParharChelsio T504-BT 109*ad016cb8SNavdeep Parhar.El 110*ad016cb8SNavdeep Parhar.Pp 111*ad016cb8SNavdeep ParharThe 112*ad016cb8SNavdeep Parhar.Nm 11354e4ee71SNavdeep Parhardriver supports 10Gb and 1Gb Ethernet adapters based on the T4 ASIC: 11454e4ee71SNavdeep Parhar.Pp 11554e4ee71SNavdeep Parhar.Bl -bullet -compact 11654e4ee71SNavdeep Parhar.It 11754e4ee71SNavdeep ParharChelsio T420-CR 11854e4ee71SNavdeep Parhar.It 11954e4ee71SNavdeep ParharChelsio T422-CR 12054e4ee71SNavdeep Parhar.It 12154e4ee71SNavdeep ParharChelsio T440-CR 12254e4ee71SNavdeep Parhar.It 12354e4ee71SNavdeep ParharChelsio T420-BCH 12454e4ee71SNavdeep Parhar.It 12554e4ee71SNavdeep ParharChelsio T440-BCH 12654e4ee71SNavdeep Parhar.It 12754e4ee71SNavdeep ParharChelsio T440-CH 12854e4ee71SNavdeep Parhar.It 12954e4ee71SNavdeep ParharChelsio T420-SO 13054e4ee71SNavdeep Parhar.It 13154e4ee71SNavdeep ParharChelsio T420-CX 13254e4ee71SNavdeep Parhar.It 13354e4ee71SNavdeep ParharChelsio T420-BT 13454e4ee71SNavdeep Parhar.It 13554e4ee71SNavdeep ParharChelsio T404-BT 13654e4ee71SNavdeep Parhar.El 13754e4ee71SNavdeep Parhar.Sh LOADER TUNABLES 13854e4ee71SNavdeep ParharTunables can be set at the 13954e4ee71SNavdeep Parhar.Xr loader 8 14054e4ee71SNavdeep Parharprompt before booting the kernel or stored in 14154e4ee71SNavdeep Parhar.Xr loader.conf 5 . 14254e4ee71SNavdeep Parhar.Bl -tag -width indent 143733b9277SNavdeep Parhar.It Va hw.cxgbe.ntxq10g 1449d496f5aSGlen BarberThe number of tx queues to use for a 10Gb port. 1459d496f5aSGlen BarberThe default is 16 or the number 146733b9277SNavdeep Parharof CPU cores in the system, whichever is less. 147733b9277SNavdeep Parhar.It Va hw.cxgbe.nrxq10g 1489d496f5aSGlen BarberThe number of rx queues to use for a 10Gb port. 1499d496f5aSGlen BarberThe default is 8 or the number 150733b9277SNavdeep Parharof CPU cores in the system, whichever is less. 151733b9277SNavdeep Parhar.It Va hw.cxgbe.ntxq1g 1529d496f5aSGlen BarberThe number of tx queues to use for a 1Gb port. 1539d496f5aSGlen BarberThe default is 4 or the number 154733b9277SNavdeep Parharof CPU cores in the system, whichever is less. 155733b9277SNavdeep Parhar.It Va hw.cxgbe.nrxq1g 1569d496f5aSGlen BarberThe number of rx queues to use for a 1Gb port. 1579d496f5aSGlen BarberThe default is 2 or the number 158733b9277SNavdeep Parharof CPU cores in the system, whichever is less. 159733b9277SNavdeep Parhar.It Va hw.cxgbe.nofldtxq10g 1609d496f5aSGlen BarberThe number of TOE tx queues to use for a 10Gb port. 1619d496f5aSGlen BarberThe default is 8 or the 162733b9277SNavdeep Parharnumber of CPU cores in the system, whichever is less. 163733b9277SNavdeep Parhar.It Va hw.cxgbe.nofldrxq10g 1649d496f5aSGlen BarberThe number of TOE rx queues to use for a 10Gb port. 1659d496f5aSGlen BarberThe default is 2 or the 166733b9277SNavdeep Parharnumber of CPU cores in the system, whichever is less. 167733b9277SNavdeep Parhar.It Va hw.cxgbe.nofldtxq1g 1689d496f5aSGlen BarberThe number of TOE tx queues to use for a 1Gb port. 1699d496f5aSGlen BarberThe default is 2 or the 170733b9277SNavdeep Parharnumber of CPU cores in the system, whichever is less. 171733b9277SNavdeep Parhar.It Va hw.cxgbe.nofldrxq1g 1729d496f5aSGlen BarberThe number of TOE rx queues to use for a 1Gb port. 1739d496f5aSGlen BarberThe default is 1. 17454e4ee71SNavdeep Parhar.It Va hw.cxgbe.holdoff_timer_idx_10G 17554e4ee71SNavdeep Parhar.It Va hw.cxgbe.holdoff_timer_idx_1G 17654e4ee71SNavdeep ParharThe timer index value to use to delay interrupts. 17754e4ee71SNavdeep ParharThe holdoff timer list has the values 1, 5, 10, 50, 100, and 200 17854e4ee71SNavdeep Parharby default (all values are in microseconds) and the index selects a 17954e4ee71SNavdeep Parharvalue from this list. 18054e4ee71SNavdeep ParharThe default value is 1 for both 10Gb and 1Gb ports, which means the 18154e4ee71SNavdeep Parhartimer value is 5us. 182*ad016cb8SNavdeep ParharDifferent interfaces can be assigned different values at any time via the 183*ad016cb8SNavdeep Parhardev.cxgbe.X.holdoff_tmr_idx or dev.cxl.X.holdoff_tmr_idx sysctl. 18454e4ee71SNavdeep Parhar.It Va hw.cxgbe.holdoff_pktc_idx_10G 18554e4ee71SNavdeep Parhar.It Va hw.cxgbe.holdoff_pktc_idx_1G 18654e4ee71SNavdeep ParharThe packet-count index value to use to delay interrupts. 18754e4ee71SNavdeep ParharThe packet-count list has the values 1, 8, 16, and 32 by default 18854e4ee71SNavdeep Parharand the index selects a value from this list. 1897a32954cSNavdeep ParharThe default value is -1 for both 10Gb and 1Gb ports, which means packet 1907a32954cSNavdeep Parharcounting is disabled and interrupts are generated based solely on the 1917a32954cSNavdeep Parharholdoff timer value. 192*ad016cb8SNavdeep ParharDifferent interfaces can be assigned different values via the 193*ad016cb8SNavdeep Parhardev.cxgbe.X.holdoff_pktc_idx or dev.cxl.X.holdoff_pktc_idx sysctl. 194733b9277SNavdeep ParharThis sysctl works only when the interface has never been marked up (as done by 195733b9277SNavdeep Parharifconfig up). 19654e4ee71SNavdeep Parhar.It Va hw.cxgbe.qsize_txq 19754e4ee71SNavdeep ParharThe size, in number of entries, of the descriptor ring used for a tx 19854e4ee71SNavdeep Parharqueue. 19954e4ee71SNavdeep ParharA buf_ring of the same size is also allocated for additional 2009d496f5aSGlen Barbersoftware queuing. 2019d496f5aSGlen BarberSee 20254e4ee71SNavdeep Parhar.Xr ifnet 9 . 20354e4ee71SNavdeep ParharThe default value is 1024. 204*ad016cb8SNavdeep ParharDifferent interfaces can be assigned different values via the 205*ad016cb8SNavdeep Parhardev.cxgbe.X.qsize_txq sysctl or dev.cxl.X.qsize_txq sysctl. 206733b9277SNavdeep ParharThis sysctl works only when the interface has never been marked up (as done by 207733b9277SNavdeep Parharifconfig up). 20854e4ee71SNavdeep Parhar.It Va hw.cxgbe.qsize_rxq 20954e4ee71SNavdeep ParharThe size, in number of entries, of the descriptor ring used for an 21054e4ee71SNavdeep Parharrx queue. 21154e4ee71SNavdeep ParharThe default value is 1024. 212*ad016cb8SNavdeep ParharDifferent interfaces can be assigned different values via the 213*ad016cb8SNavdeep Parhardev.cxgbe.X.qsize_rxq or dev.cxl.X.qsize_rxq sysctl. 214733b9277SNavdeep ParharThis sysctl works only when the interface has never been marked up (as done by 215733b9277SNavdeep Parharifconfig up). 216733b9277SNavdeep Parhar.It Va hw.cxgbe.interrupt_types 217733b9277SNavdeep ParharThe interrupt types that the driver is allowed to use. 218733b9277SNavdeep ParharBit 0 represents INTx (line interrupts), bit 1 MSI, bit 2 MSI-X. 219733b9277SNavdeep ParharThe default is 7 (all allowed). 220733b9277SNavdeep ParharThe driver will select the best possible type out of the allowed types by 221733b9277SNavdeep Parharitself. 222d78bd33fSNavdeep Parhar.It Va hw.cxgbe.fw_install 223d78bd33fSNavdeep Parhar0 prohibits the driver from installing a firmware on the card. 224d78bd33fSNavdeep Parhar1 allows the driver to install a new firmware if internal driver 225d78bd33fSNavdeep Parharheuristics indicate that the new firmware is preferable to the one 226d78bd33fSNavdeep Parharalready on the card. 227d78bd33fSNavdeep Parhar2 instructs the driver to always install the new firmware on the card as 228d78bd33fSNavdeep Parharlong as it is compatible with the driver and is a different version than 229d78bd33fSNavdeep Parharthe one already on the card. 230d78bd33fSNavdeep ParharThe default is 1. 231733b9277SNavdeep Parhar.It Va hw.cxgbe.config_file 232733b9277SNavdeep ParharSelect a pre-packaged device configuration file. 233733b9277SNavdeep ParharA configuration file contains a recipe for partitioning and configuring the 234733b9277SNavdeep Parharhardware resources on the card. 235733b9277SNavdeep ParharThis tunable is for specialized applications only and should not be used in 236733b9277SNavdeep Parharnormal operation. 237733b9277SNavdeep ParharThe configuration profile currently in use is available in the dev.t4nex.X.cf 238733b9277SNavdeep Parharand dev.t4nex.X.cfcsum sysctls. 239733b9277SNavdeep Parhar.It Va hw.cxgbe.linkcaps_allowed 240733b9277SNavdeep Parhar.It Va hw.cxgbe.niccaps_allowed 241733b9277SNavdeep Parhar.It Va hw.cxgbe.toecaps_allowed 242733b9277SNavdeep Parhar.It Va hw.cxgbe.rdmacaps_allowed 243733b9277SNavdeep Parhar.It Va hw.cxgbe.iscsicaps_allowed 244733b9277SNavdeep Parhar.It Va hw.cxgbe.fcoecaps_allowed 245733b9277SNavdeep ParharDisallowing capabilities provides a hint to the driver and firmware to not 246733b9277SNavdeep Parharreserve hardware resources for that feature. 247733b9277SNavdeep ParharEach of these is a bit field with a bit for each sub-capability within the 248733b9277SNavdeep Parharcapability. 249733b9277SNavdeep ParharThis tunable is for specialized applications only and should not be used in 250733b9277SNavdeep Parharnormal operation. 251733b9277SNavdeep ParharThe capabilities for which hardware resources have been reserved are listed in 252733b9277SNavdeep Parhardev.t4nex.X.*caps sysctls. 253cb379161SGlen Barber.El 25454e4ee71SNavdeep Parhar.Sh SUPPORT 25554e4ee71SNavdeep ParharFor general information and support, 25654e4ee71SNavdeep Parhargo to the Chelsio support website at: 25754e4ee71SNavdeep Parhar.Pa http://www.chelsio.com/ . 25854e4ee71SNavdeep Parhar.Pp 25954e4ee71SNavdeep ParharIf an issue is identified with this driver with a supported adapter, 26054e4ee71SNavdeep Parharemail all the specific information related to the issue to 26154e4ee71SNavdeep Parhar.Aq support@chelsio.com . 26254e4ee71SNavdeep Parhar.Sh SEE ALSO 26354e4ee71SNavdeep Parhar.Xr altq 4 , 26454e4ee71SNavdeep Parhar.Xr arp 4 , 26554e4ee71SNavdeep Parhar.Xr cxgb 4 , 26654e4ee71SNavdeep Parhar.Xr netintro 4 , 26754e4ee71SNavdeep Parhar.Xr ng_ether 4 , 26854e4ee71SNavdeep Parhar.Xr ifconfig 8 26954e4ee71SNavdeep Parhar.Sh HISTORY 27054e4ee71SNavdeep ParharThe 27154e4ee71SNavdeep Parhar.Nm 27254e4ee71SNavdeep Parhardevice driver first appeared in 27350d922a0SGlen Barber.Fx 9.0 . 274*ad016cb8SNavdeep ParharSupport for T5 cards first appeared in 275*ad016cb8SNavdeep Parhar.Fx 9.2 276*ad016cb8SNavdeep Parharand 277*ad016cb8SNavdeep Parhar.Fx 10.0 . 27854e4ee71SNavdeep Parhar.Sh AUTHORS 27954e4ee71SNavdeep Parhar.An -nosplit 28054e4ee71SNavdeep ParharThe 28154e4ee71SNavdeep Parhar.Nm 28254e4ee71SNavdeep Parhardriver was written by 28354e4ee71SNavdeep Parhar.An Navdeep Parhar Aq np@FreeBSD.org . 284