1132db935SJakub Kicinski.. SPDX-License-Identifier: GPL-2.0 2132db935SJakub Kicinski 3132db935SJakub Kicinski========================================================= 4132db935SJakub KicinskiNeterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver 5132db935SJakub Kicinski========================================================= 6132db935SJakub Kicinski 7132db935SJakub KicinskiRelease notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver. 8132db935SJakub Kicinski 9132db935SJakub Kicinski.. Contents 10132db935SJakub Kicinski - 1. Introduction 11132db935SJakub Kicinski - 2. Identifying the adapter/interface 12132db935SJakub Kicinski - 3. Features supported 13132db935SJakub Kicinski - 4. Command line parameters 14132db935SJakub Kicinski - 5. Performance suggestions 15132db935SJakub Kicinski - 6. Available Downloads 16132db935SJakub Kicinski 17132db935SJakub Kicinski 18132db935SJakub Kicinski1. Introduction 19132db935SJakub Kicinski=============== 20132db935SJakub KicinskiThis Linux driver supports Neterion's Xframe I PCI-X 1.0 and 21132db935SJakub KicinskiXframe II PCI-X 2.0 adapters. It supports several features 22132db935SJakub Kicinskisuch as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on. 23132db935SJakub KicinskiSee below for complete list of features. 24132db935SJakub Kicinski 25132db935SJakub KicinskiAll features are supported for both IPv4 and IPv6. 26132db935SJakub Kicinski 27132db935SJakub Kicinski2. Identifying the adapter/interface 28132db935SJakub Kicinski==================================== 29132db935SJakub Kicinski 30132db935SJakub Kicinskia. Insert the adapter(s) in your system. 31132db935SJakub Kicinskib. Build and load driver:: 32132db935SJakub Kicinski 33132db935SJakub Kicinski # insmod s2io.ko 34132db935SJakub Kicinski 35132db935SJakub Kicinskic. View log messages:: 36132db935SJakub Kicinski 37132db935SJakub Kicinski # dmesg | tail -40 38132db935SJakub Kicinski 39132db935SJakub KicinskiYou will see messages similar to:: 40132db935SJakub Kicinski 41132db935SJakub Kicinski eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA 42132db935SJakub Kicinski eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA 43132db935SJakub Kicinski eth4: Device is on 64 bit 133MHz PCIX(M1) bus 44132db935SJakub Kicinski 45132db935SJakub KicinskiThe above messages identify the adapter type(Xframe I/II), adapter revision, 46132db935SJakub Kicinskidriver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X). 47132db935SJakub KicinskiIn case of Xframe II, the PCI/PCI-X bus width and frequency are displayed 48132db935SJakub Kicinskias well. 49132db935SJakub Kicinski 50132db935SJakub KicinskiTo associate an interface with a physical adapter use "ethtool -p <ethX>". 51132db935SJakub KicinskiThe corresponding adapter's LED will blink multiple times. 52132db935SJakub Kicinski 53132db935SJakub Kicinski3. Features supported 54132db935SJakub Kicinski===================== 55132db935SJakub Kicinskia. Jumbo frames. Xframe I/II supports MTU up to 9600 bytes, 56132db935SJakub Kicinski modifiable using ip command. 57132db935SJakub Kicinski 58132db935SJakub Kicinskib. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit 59132db935SJakub Kicinski and receive, TSO. 60132db935SJakub Kicinski 61132db935SJakub Kicinskic. Multi-buffer receive mode. Scattering of packet across multiple 62132db935SJakub Kicinski buffers. Currently driver supports 2-buffer mode which yields 63132db935SJakub Kicinski significant performance improvement on certain platforms(SGI Altix, 64132db935SJakub Kicinski IBM xSeries). 65132db935SJakub Kicinski 66132db935SJakub Kicinskid. MSI/MSI-X. Can be enabled on platforms which support this feature 67*94483490SArd Biesheuvel resulting in noticeable performance improvement (up to 7% on certain 68*94483490SArd Biesheuvel platforms). 69132db935SJakub Kicinski 70132db935SJakub Kicinskie. Statistics. Comprehensive MAC-level and software statistics displayed 71132db935SJakub Kicinski using "ethtool -S" option. 72132db935SJakub Kicinski 73132db935SJakub Kicinskif. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, 74132db935SJakub Kicinski with multiple steering options. 75132db935SJakub Kicinski 76132db935SJakub Kicinski4. Command line parameters 77132db935SJakub Kicinski========================== 78132db935SJakub Kicinski 79132db935SJakub Kicinskia. tx_fifo_num 80132db935SJakub Kicinski Number of transmit queues 81132db935SJakub Kicinski 82132db935SJakub KicinskiValid range: 1-8 83132db935SJakub Kicinski 84132db935SJakub KicinskiDefault: 1 85132db935SJakub Kicinski 86132db935SJakub Kicinskib. rx_ring_num 87132db935SJakub Kicinski Number of receive rings 88132db935SJakub Kicinski 89132db935SJakub KicinskiValid range: 1-8 90132db935SJakub Kicinski 91132db935SJakub KicinskiDefault: 1 92132db935SJakub Kicinski 93132db935SJakub Kicinskic. tx_fifo_len 94132db935SJakub Kicinski Size of each transmit queue 95132db935SJakub Kicinski 96132db935SJakub KicinskiValid range: Total length of all queues should not exceed 8192 97132db935SJakub Kicinski 98132db935SJakub KicinskiDefault: 4096 99132db935SJakub Kicinski 100132db935SJakub Kicinskid. rx_ring_sz 101132db935SJakub Kicinski Size of each receive ring(in 4K blocks) 102132db935SJakub Kicinski 103132db935SJakub KicinskiValid range: Limited by memory on system 104132db935SJakub Kicinski 105132db935SJakub KicinskiDefault: 30 106132db935SJakub Kicinski 107132db935SJakub Kicinskie. intr_type 108132db935SJakub Kicinski Specifies interrupt type. Possible values 0(INTA), 2(MSI-X) 109132db935SJakub Kicinski 110132db935SJakub KicinskiValid values: 0, 2 111132db935SJakub Kicinski 112132db935SJakub KicinskiDefault: 2 113132db935SJakub Kicinski 114132db935SJakub Kicinski5. Performance suggestions 115132db935SJakub Kicinski========================== 116132db935SJakub Kicinski 117132db935SJakub KicinskiGeneral: 118132db935SJakub Kicinski 119132db935SJakub Kicinskia. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration) 120132db935SJakub Kicinskib. Set TCP windows size to optimal value. 121132db935SJakub Kicinski 122132db935SJakub KicinskiFor instance, for MTU=1500 a value of 210K has been observed to result in 123132db935SJakub Kicinskigood performance:: 124132db935SJakub Kicinski 125132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000" 126132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000" 127132db935SJakub Kicinski 128132db935SJakub KicinskiFor MTU=9000, TCP window size of 10 MB is recommended:: 129132db935SJakub Kicinski 130132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000" 131132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000" 132132db935SJakub Kicinski 133132db935SJakub KicinskiTransmit performance: 134132db935SJakub Kicinski 135132db935SJakub Kicinskia. By default, the driver respects BIOS settings for PCI bus parameters. 136132db935SJakub Kicinski However, you may want to experiment with PCI bus parameters 137132db935SJakub Kicinski max-split-transactions(MOST) and MMRBC (use setpci command). 138132db935SJakub Kicinski 139132db935SJakub Kicinski A MOST value of 2 has been found optimal for Opterons and 3 for Itanium. 140132db935SJakub Kicinski 141132db935SJakub Kicinski It could be different for your hardware. 142132db935SJakub Kicinski 143132db935SJakub Kicinski Set MMRBC to 4K**. 144132db935SJakub Kicinski 145132db935SJakub Kicinski For example you can set 146132db935SJakub Kicinski 147132db935SJakub Kicinski For opteron:: 148132db935SJakub Kicinski 149132db935SJakub Kicinski #setpci -d 17d5:* 62=1d 150132db935SJakub Kicinski 151132db935SJakub Kicinski For Itanium:: 152132db935SJakub Kicinski 153132db935SJakub Kicinski #setpci -d 17d5:* 62=3d 154132db935SJakub Kicinski 155132db935SJakub Kicinski For detailed description of the PCI registers, please see Xframe User Guide. 156132db935SJakub Kicinski 157132db935SJakub Kicinskib. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this 158132db935SJakub Kicinski parameter. 159132db935SJakub Kicinski 160132db935SJakub Kicinskic. Turn on TSO(using "ethtool -K"):: 161132db935SJakub Kicinski 162132db935SJakub Kicinski # ethtool -K <ethX> tso on 163132db935SJakub Kicinski 164132db935SJakub KicinskiReceive performance: 165132db935SJakub Kicinski 166132db935SJakub Kicinskia. By default, the driver respects BIOS settings for PCI bus parameters. 167132db935SJakub Kicinski However, you may want to set PCI latency timer to 248:: 168132db935SJakub Kicinski 169132db935SJakub Kicinski #setpci -d 17d5:* LATENCY_TIMER=f8 170132db935SJakub Kicinski 171132db935SJakub Kicinski For detailed description of the PCI registers, please see Xframe User Guide. 172132db935SJakub Kicinski 173132db935SJakub Kicinskib. Use 2-buffer mode. This results in large performance boost on 174132db935SJakub Kicinski certain platforms(eg. SGI Altix, IBM xSeries). 175132db935SJakub Kicinski 176132db935SJakub Kicinskic. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to 177132db935SJakub Kicinski set/verify this option. 178132db935SJakub Kicinski 179132db935SJakub Kicinskid. Enable NAPI feature(in kernel configuration Device Drivers ---> Network 180132db935SJakub Kicinski device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to 181132db935SJakub Kicinski bring down CPU utilization. 182132db935SJakub Kicinski 183132db935SJakub Kicinski.. note:: 184132db935SJakub Kicinski 185132db935SJakub Kicinski For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are 186132db935SJakub Kicinski recommended as safe parameters. 187132db935SJakub Kicinski 188132db935SJakub KicinskiFor more information, please review the AMD8131 errata at 189132db935SJakub Kicinskihttp://vip.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/ 190132db935SJakub Kicinski26310_AMD-8131_HyperTransport_PCI-X_Tunnel_Revision_Guide_rev_3_18.pdf 191132db935SJakub Kicinski 192132db935SJakub Kicinski6. Support 193132db935SJakub Kicinski========== 194132db935SJakub Kicinski 195132db935SJakub KicinskiFor further support please contact either your 10GbE Xframe NIC vendor (IBM, 196132db935SJakub KicinskiHP, SGI etc.) 197