1.. SPDX-License-Identifier: GPL-2.0 2 3==================== 4Interface statistics 5==================== 6 7Overview 8======== 9 10This document is a guide to Linux network interface statistics. 11 12There are three main sources of interface statistics in Linux: 13 14 - standard interface statistics based on 15 :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`; 16 - protocol-specific statistics; and 17 - driver-defined statistics available via ethtool. 18 19Standard interface statistics 20----------------------------- 21 22There are multiple interfaces to reach the standard statistics. 23Most commonly used is the `ip` command from `iproute2`:: 24 25 $ ip -s -s link show dev ens4u1u1 26 6: ens4u1u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 27 link/ether 48:2a:e3:4c:b1:d1 brd ff:ff:ff:ff:ff:ff 28 RX: bytes packets errors dropped overrun mcast 29 74327665117 69016965 0 0 0 0 30 RX errors: length crc frame fifo missed 31 0 0 0 0 0 32 TX: bytes packets errors dropped carrier collsns 33 21405556176 44608960 0 0 0 0 34 TX errors: aborted fifo window heartbeat transns 35 0 0 0 0 128 36 altname enp58s0u1u1 37 38Note that `-s` has been specified twice to see all members of 39:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`. 40If `-s` is specified once the detailed errors won't be shown. 41 42`ip` supports JSON formatting via the `-j` option. 43 44Queue statistics 45~~~~~~~~~~~~~~~~ 46 47Queue statistics are accessible via the netdev netlink family. 48 49Currently no widely distributed CLI exists to access those statistics. 50Kernel development tools (ynl) can be used to experiment with them, 51see `Documentation/userspace-api/netlink/intro-specs.rst`. 52 53Protocol-specific statistics 54---------------------------- 55 56Protocol-specific statistics are exposed via relevant interfaces, 57the same interfaces as are used to configure them. 58 59ethtool 60~~~~~~~ 61 62Ethtool exposes common low-level statistics. 63All the standard statistics are expected to be maintained 64by the device, not the driver (as opposed to driver-defined stats 65described in the next section which mix software and hardware stats). 66For devices which contain unmanaged 67switches (e.g. legacy SR-IOV or multi-host NICs) the events counted 68may not pertain exclusively to the packets destined to 69the local host interface. In other words the events may 70be counted at the network port (MAC/PHY blocks) without separation 71for different host side (PCIe) devices. Such ambiguity must not 72be present when internal switch is managed by Linux (so called 73switchdev mode for NICs). 74 75Standard ethtool statistics can be accessed via the interfaces used 76for configuration. For example ethtool interface used 77to configure pause frames can report corresponding hardware counters:: 78 79 $ ethtool --include-statistics -a eth0 80 Pause parameters for eth0: 81 Autonegotiate: on 82 RX: on 83 TX: on 84 Statistics: 85 tx_pause_frames: 1 86 rx_pause_frames: 1 87 88General Ethernet statistics not associated with any particular 89functionality are exposed via ``ethtool -S $ifc`` by specifying 90the ``--groups`` parameter:: 91 92 $ ethtool -S eth0 --groups eth-phy eth-mac eth-ctrl rmon 93 Stats for eth0: 94 eth-phy-SymbolErrorDuringCarrier: 0 95 eth-mac-FramesTransmittedOK: 1 96 eth-mac-FrameTooLongErrors: 1 97 eth-ctrl-MACControlFramesTransmitted: 1 98 eth-ctrl-MACControlFramesReceived: 0 99 eth-ctrl-UnsupportedOpcodesReceived: 1 100 rmon-etherStatsUndersizePkts: 1 101 rmon-etherStatsJabbers: 0 102 rmon-rx-etherStatsPkts64Octets: 1 103 rmon-rx-etherStatsPkts65to127Octets: 0 104 rmon-rx-etherStatsPkts128to255Octets: 0 105 rmon-tx-etherStatsPkts64Octets: 2 106 rmon-tx-etherStatsPkts65to127Octets: 3 107 rmon-tx-etherStatsPkts128to255Octets: 0 108 109Driver-defined statistics 110------------------------- 111 112Driver-defined ethtool statistics can be dumped using `ethtool -S $ifc`, e.g.:: 113 114 $ ethtool -S ens4u1u1 115 NIC statistics: 116 tx_single_collisions: 0 117 tx_multi_collisions: 0 118 119uAPIs 120===== 121 122procfs 123------ 124 125The historical `/proc/net/dev` text interface gives access to the list 126of interfaces as well as their statistics. 127 128Note that even though this interface is using 129:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` 130internally it combines some of the fields. 131 132sysfs 133----- 134 135Each device directory in sysfs contains a `statistics` directory (e.g. 136`/sys/class/net/lo/statistics/`) with files corresponding to 137members of :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`. 138 139This simple interface is convenient especially in constrained/embedded 140environments without access to tools. However, it's inefficient when 141reading multiple stats as it internally performs a full dump of 142:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` 143and reports only the stat corresponding to the accessed file. 144 145Sysfs files are documented in 146Documentation/ABI/testing/sysfs-class-net-statistics. 147 148 149netlink 150------- 151 152`rtnetlink` (`NETLINK_ROUTE`) is the preferred method of accessing 153:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` stats. 154 155Statistics are reported both in the responses to link information 156requests (`RTM_GETLINK`) and statistic requests (`RTM_GETSTATS`, 157when `IFLA_STATS_LINK_64` bit is set in the `.filter_mask` of the request). 158 159netdev (netlink) 160~~~~~~~~~~~~~~~~ 161 162`netdev` generic netlink family allows accessing page pool and per queue 163statistics. 164 165ethtool 166------- 167 168Ethtool IOCTL interface allows drivers to report implementation 169specific statistics. Historically it has also been used to report 170statistics for which other APIs did not exist, like per-device-queue 171statistics, or standard-based statistics (e.g. RFC 2863). 172 173Statistics and their string identifiers are retrieved separately. 174Identifiers via `ETHTOOL_GSTRINGS` with `string_set` set to `ETH_SS_STATS`, 175and values via `ETHTOOL_GSTATS`. User space should use `ETHTOOL_GDRVINFO` 176to retrieve the number of statistics (`.n_stats`). 177 178ethtool-netlink 179--------------- 180 181Ethtool netlink is a replacement for the older IOCTL interface. 182 183Protocol-related statistics can be requested in get commands by setting 184the `ETHTOOL_FLAG_STATS` flag in `ETHTOOL_A_HEADER_FLAGS`. Currently 185statistics are supported in the following commands: 186 187 - `ETHTOOL_MSG_FEC_GET` 188 - `ETHTOOL_MSG_LINKSTATE_GET` 189 - `ETHTOOL_MSG_MM_GET` 190 - `ETHTOOL_MSG_PAUSE_GET` 191 - `ETHTOOL_MSG_TSINFO_GET` 192 193debugfs 194------- 195 196Some drivers expose extra statistics via `debugfs`. 197 198struct rtnl_link_stats64 199======================== 200 201.. kernel-doc:: include/uapi/linux/if_link.h 202 :identifiers: rtnl_link_stats64 203 204Notes for driver authors 205======================== 206 207Drivers should report all statistics which have a matching member in 208:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` exclusively 209via `.ndo_get_stats64`. Reporting such standard stats via ethtool 210or debugfs will not be accepted. 211 212Drivers must ensure best possible compliance with 213:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`. 214Please note for example that detailed error statistics must be 215added into the general `rx_error` / `tx_error` counters. 216 217The `.ndo_get_stats64` callback can not sleep because of accesses 218via `/proc/net/dev`. If driver may sleep when retrieving the statistics 219from the device it should do so periodically asynchronously and only return 220a recent copy from `.ndo_get_stats64`. Ethtool interrupt coalescing interface 221allows setting the frequency of refreshing statistics, if needed. 222 223Retrieving ethtool statistics is a multi-syscall process, drivers are advised 224to keep the number of statistics constant to avoid race conditions with 225user space trying to read them. 226 227Statistics must persist across routine operations like bringing the interface 228down and up. 229 230Kernel-internal data structures 231------------------------------- 232 233The following structures are internal to the kernel, their members are 234translated to netlink attributes when dumped. Drivers must not overwrite 235the statistics they don't report with 0. 236 237- ethtool_pause_stats() 238- ethtool_fec_stats() 239