1.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) 2 3============================================= 4Netronome Flow Processor (NFP) Kernel Drivers 5============================================= 6 7Copyright (c) 2019, Netronome Systems, Inc. 8 9Contents 10======== 11 12- `Overview`_ 13- `Acquiring Firmware`_ 14 15Overview 16======== 17 18This driver supports Netronome's line of Flow Processor devices, 19including the NFP4000, NFP5000, and NFP6000 models, which are also 20incorporated in the company's family of Agilio SmartNICs. The SR-IOV 21physical and virtual functions for these devices are supported by 22the driver. 23 24Acquiring Firmware 25================== 26 27The NFP4000 and NFP6000 devices require application specific firmware 28to function. Application firmware can be located either on the host file system 29or in the device flash (if supported by management firmware). 30 31Firmware files on the host filesystem contain card type (`AMDA-*` string), media 32config etc. They should be placed in `/lib/firmware/netronome` directory to 33load firmware from the host file system. 34 35Firmware for basic NIC operation is available in the upstream 36`linux-firmware.git` repository. 37 38Firmware in NVRAM 39----------------- 40 41Recent versions of management firmware supports loading application 42firmware from flash when the host driver gets probed. The firmware loading 43policy configuration may be used to configure this feature appropriately. 44 45Devlink or ethtool can be used to update the application firmware on the device 46flash by providing the appropriate `nic_AMDA*.nffw` file to the respective 47command. Users need to take care to write the correct firmware image for the 48card and media configuration to flash. 49 50Available storage space in flash depends on the card being used. 51 52Dealing with multiple projects 53------------------------------ 54 55NFP hardware is fully programmable therefore there can be different 56firmware images targeting different applications. 57 58When using application firmware from host, we recommend placing 59actual firmware files in application-named subdirectories in 60`/lib/firmware/netronome` and linking the desired files, e.g.:: 61 62 $ tree /lib/firmware/netronome/ 63 /lib/firmware/netronome/ 64 ├── bpf 65 │ ├── nic_AMDA0081-0001_1x40.nffw 66 │ └── nic_AMDA0081-0001_4x10.nffw 67 ├── flower 68 │ ├── nic_AMDA0081-0001_1x40.nffw 69 │ └── nic_AMDA0081-0001_4x10.nffw 70 ├── nic 71 │ ├── nic_AMDA0081-0001_1x40.nffw 72 │ └── nic_AMDA0081-0001_4x10.nffw 73 ├── nic_AMDA0081-0001_1x40.nffw -> bpf/nic_AMDA0081-0001_1x40.nffw 74 └── nic_AMDA0081-0001_4x10.nffw -> bpf/nic_AMDA0081-0001_4x10.nffw 75 76 3 directories, 8 files 77 78You may need to use hard instead of symbolic links on distributions 79which use old `mkinitrd` command instead of `dracut` (e.g. Ubuntu). 80 81After changing firmware files you may need to regenerate the initramfs 82image. Initramfs contains drivers and firmware files your system may 83need to boot. Refer to the documentation of your distribution to find 84out how to update initramfs. Good indication of stale initramfs 85is system loading wrong driver or firmware on boot, but when driver is 86later reloaded manually everything works correctly. 87 88Selecting firmware per device 89----------------------------- 90 91Most commonly all cards on the system use the same type of firmware. 92If you want to load specific firmware image for a specific card, you 93can use either the PCI bus address or serial number. Driver will print 94which files it's looking for when it recognizes a NFP device:: 95 96 nfp: Looking for firmware file in order of priority: 97 nfp: netronome/serial-00-12-34-aa-bb-cc-10-ff.nffw: not found 98 nfp: netronome/pci-0000:02:00.0.nffw: not found 99 nfp: netronome/nic_AMDA0081-0001_1x40.nffw: found, loading... 100 101In this case if file (or link) called *serial-00-12-34-aa-bb-5d-10-ff.nffw* 102or *pci-0000:02:00.0.nffw* is present in `/lib/firmware/netronome` this 103firmware file will take precedence over `nic_AMDA*` files. 104 105Note that `serial-*` and `pci-*` files are **not** automatically included 106in initramfs, you will have to refer to documentation of appropriate tools 107to find out how to include them. 108 109Firmware loading policy 110----------------------- 111 112Firmware loading policy is controlled via three HWinfo parameters 113stored as key value pairs in the device flash: 114 115app_fw_from_flash 116 Defines which firmware should take precedence, 'Disk' (0), 'Flash' (1) or 117 the 'Preferred' (2) firmware. When 'Preferred' is selected, the management 118 firmware makes the decision over which firmware will be loaded by comparing 119 versions of the flash firmware and the host supplied firmware. 120 This variable is configurable using the 'fw_load_policy' 121 devlink parameter. 122 123abi_drv_reset 124 Defines if the driver should reset the firmware when 125 the driver is probed, either 'Disk' (0) if firmware was found on disk, 126 'Always' (1) reset or 'Never' (2) reset. Note that the device is always 127 reset on driver unload if firmware was loaded when the driver was probed. 128 This variable is configurable using the 'reset_dev_on_drv_probe' 129 devlink parameter. 130 131abi_drv_load_ifc 132 Defines a list of PF devices allowed to load FW on the device. 133 This variable is not currently user configurable. 134 135Statistics 136========== 137 138Following device statistics are available through the ``ethtool -S`` interface: 139 140.. flat-table:: NFP device statistics 141 :header-rows: 1 142 :widths: 3 1 11 143 144 * - Name 145 - ID 146 - Meaning 147 148 * - dev_rx_discards 149 - 1 150 - Packet can be discarded on the RX path for one of the following reasons: 151 152 * The NIC is not in promisc mode, and the destination MAC address 153 doesn't match the interfaces' MAC address. 154 * The received packet is larger than the max buffer size on the host. 155 I.e. it exceeds the Layer 3 MRU. 156 * There is no freelist descriptor available on the host for the packet. 157 It is likely that the NIC couldn't cache one in time. 158 * A BPF program discarded the packet. 159 * The datapath drop action was executed. 160 * The MAC discarded the packet due to lack of ingress buffer space 161 on the NIC. 162 163 * - dev_rx_errors 164 - 2 165 - A packet can be counted (and dropped) as RX error for the following 166 reasons: 167 168 * A problem with the VEB lookup (only when SR-IOV is used). 169 * A physical layer problem that causes Ethernet errors, like FCS or 170 alignment errors. The cause is usually faulty cables or SFPs. 171 172 * - dev_rx_bytes 173 - 3 174 - Total number of bytes received. 175 176 * - dev_rx_uc_bytes 177 - 4 178 - Unicast bytes received. 179 180 * - dev_rx_mc_bytes 181 - 5 182 - Multicast bytes received. 183 184 * - dev_rx_bc_bytes 185 - 6 186 - Broadcast bytes received. 187 188 * - dev_rx_pkts 189 - 7 190 - Total number of packets received. 191 192 * - dev_rx_mc_pkts 193 - 8 194 - Multicast packets received. 195 196 * - dev_rx_bc_pkts 197 - 9 198 - Broadcast packets received. 199 200 * - dev_tx_discards 201 - 10 202 - A packet can be discarded in the TX direction if the MAC is 203 being flow controlled and the NIC runs out of TX queue space. 204 205 * - dev_tx_errors 206 - 11 207 - A packet can be counted as TX error (and dropped) for one for the 208 following reasons: 209 210 * The packet is an LSO segment, but the Layer 3 or Layer 4 offset 211 could not be determined. Therefore LSO could not continue. 212 * An invalid packet descriptor was received over PCIe. 213 * The packet Layer 3 length exceeds the device MTU. 214 * An error on the MAC/physical layer. Usually due to faulty cables or 215 SFPs. 216 * A CTM buffer could not be allocated. 217 * The packet offset was incorrect and could not be fixed by the NIC. 218 219 * - dev_tx_bytes 220 - 12 221 - Total number of bytes transmitted. 222 223 * - dev_tx_uc_bytes 224 - 13 225 - Unicast bytes transmitted. 226 227 * - dev_tx_mc_bytes 228 - 14 229 - Multicast bytes transmitted. 230 231 * - dev_tx_bc_bytes 232 - 15 233 - Broadcast bytes transmitted. 234 235 * - dev_tx_pkts 236 - 16 237 - Total number of packets transmitted. 238 239 * - dev_tx_mc_pkts 240 - 17 241 - Multicast packets transmitted. 242 243 * - dev_tx_bc_pkts 244 - 18 245 - Broadcast packets transmitted. 246 247Note that statistics unknown to the driver will be displayed as 248``dev_unknown_stat$ID``, where ``$ID`` refers to the second column 249above. 250