1.. SPDX-License-Identifier: GPL-2.0 2 3Running driver tests 4==================== 5 6Networking driver tests are executed within kselftest framework like any 7other tests. They support testing both real device drivers and emulated / 8software drivers (latter mostly to test the core parts of the stack). 9 10SW mode 11~~~~~~~ 12 13By default, when no extra parameters are set or exported, tests execute 14against software drivers such as netdevsim. No extra preparation is required 15the software devices are created and destroyed as part of the test. 16In this mode the tests are indistinguishable from other selftests and 17(for example) can be run under ``virtme-ng`` like the core networking selftests. 18 19HW mode 20~~~~~~~ 21 22Executing tests against a real device requires external preparation. 23The netdevice against which tests will be run must exist, be running 24(in UP state) and be configured with an IP address. 25 26Refer to list of :ref:`Variables` later in this file to set up running 27the tests against a real device. 28 29Both modes required 30~~~~~~~~~~~~~~~~~~~ 31 32All tests in drivers/net must support running both against a software device 33and a real device. SW-only tests should instead be placed in net/ or 34drivers/net/netdevsim, HW-only tests in drivers/net/hw. 35 36Variables 37========= 38 39The variables can be set in the environment or by creating a net.config 40file in the same directory as this README file. Example:: 41 42 $ NETIF=eth0 ./some_test.sh 43 44or:: 45 46 $ cat tools/testing/selftests/drivers/net/net.config 47 # Variable set in a file 48 NETIF=eth0 49 50Please note that the config parser is very simple, if there are 51any non-alphanumeric characters in the value it needs to be in 52double quotes. 53 54Local test (which don't require endpoint for sending / receiving traffic) 55need only the ``NETIF`` variable. Remaining variables define the endpoint 56and communication method. 57 58NETIF 59~~~~~ 60 61Name of the netdevice against which the test should be executed. 62When empty or not set software devices will be used. 63 64LOCAL_V4, LOCAL_V6, REMOTE_V4, REMOTE_V6 65~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 66 67Local and remote endpoint IP addresses. 68 69LOCAL_PREFIX_V6 70~~~~~~~~~~~~~~~ 71 72Local IP prefix/subnet which can be used to allocate extra IP addresses (for 73network name spaces behind macvlan, veth, netkit devices). DUT must be 74reachable using these addresses from the endpoint. 75 76LOCAL_PREFIX_V6 must NOT match LOCAL_V6. 77 78Example: 79 NETIF = "eth0" 80 LOCAL_V6 = "2001:db8:1::1" 81 REMOTE_V6 = "2001:db8:1::2" 82 LOCAL_PREFIX_V6 = "2001:db8:2::0/64" 83 84 +-----------------------------+ +------------------------------+ 85 dst | INIT NS | | TEST NS | 86 2001: | +---------------+ | | | 87 db8:2::2| | NETIF | | bpf | | 88 +---|>| 2001:db8:1::1 | |redirect| +-------------------------+ | 89 | | | |-----------|--------|>| Netkit | | 90 | | +---------------+ | _peer | | nk_guest | | 91 | | +-------------+ Netkit pair | | | fe80::2/64 | | 92 | | | Netkit |.............|........|>| 2001:db8:2::2/64 | | 93 | | | nk_host | | | +-------------------------+ | 94 | | | fe80::1/64 | | | | 95 | | +-------------+ | | route: | 96 | | | | default | 97 | | route: | | via fe80::1 dev nk_guest | 98 | | 2001:db8:2::2/128 | +------------------------------+ 99 | | via fe80::2 dev nk_host | 100 | +-----------------------------+ 101 | 102 | +---------------+ 103 | | REMOTE | 104 +---| 2001:db8:1::2 | 105 +---------------+ 106 107REMOTE_TYPE 108~~~~~~~~~~~ 109 110Communication method used to run commands on the remote endpoint. 111Test framework has built-in support for ``netns`` and ``ssh`` channels. 112``netns`` assumes the "remote" interface is part of the same 113host, just moved to the specified netns. 114``ssh`` communicates with remote endpoint over ``ssh`` and ``scp``. 115Using persistent SSH connections is strongly encouraged to avoid 116the latency of SSH connection setup on every command. 117 118Communication methods are defined by classes in ``lib/py/remote_{name}.py``. 119It should be possible to add a new method without modifying any of 120the framework, by simply adding an appropriately named file to ``lib/py``. 121 122REMOTE_ARGS 123~~~~~~~~~~~ 124 125Arguments used to construct the communication channel. 126Communication channel dependent:: 127 128 for netns - name of the "remote" namespace 129 for ssh - name/address of the remote host 130 131Example 132======= 133 134Build the selftests:: 135 136 # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" 137 138"Install" the tests and copy them over to the target machine:: 139 140 # make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" \ 141 install INSTALL_PATH=/tmp/ksft-net-drv 142 143 # rsync -ra --delete /tmp/ksft-net-drv root@192.168.1.1:/root/ 144 145On the target machine, running the tests will use netdevsim by default:: 146 147 [/root] # ./ksft-net-drv/run_kselftest.sh -t drivers/net:ping.py 148 TAP version 13 149 1..1 150 # timeout set to 45 151 # selftests: drivers/net: ping.py 152 # KTAP version 1 153 # 1..3 154 # ok 1 ping.test_v4 155 # ok 2 ping.test_v6 156 # ok 3 ping.test_tcp 157 # # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0 158 ok 1 selftests: drivers/net: ping.py 159 160Create a config with remote info:: 161 162 [/root] # cat > ./ksft-net-drv/drivers/net/net.config <<EOF 163 NETIF=eth0 164 LOCAL_V4=192.168.1.1 165 REMOTE_V4=192.168.1.2 166 REMOTE_TYPE=ssh 167 REMOTE_ARGS=root@192.168.1.2 168 EOF 169 170Run the test:: 171 172 [/root] # ./ksft-net-drv/drivers/net/ping.py 173 KTAP version 1 174 1..3 175 ok 1 ping.test_v4 176 ok 2 ping.test_v6 # SKIP Test requires IPv6 connectivity 177 ok 3 ping.test_tcp 178 # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:1 error:0 179 180Dependencies 181~~~~~~~~~~~~ 182 183The tests have a handful of dependencies. For Fedora / CentOS:: 184 185 dnf -y install netsniff-ng python-yaml socat iperf3 186 187Guidance for test authors 188========================= 189 190This section mostly applies to Python tests but some of the guidance 191may be more broadly applicable. 192 193Kernel config 194~~~~~~~~~~~~~ 195 196Each test directory has a ``config`` file listing which kernel 197configuration options the tests depend on. This file must be kept 198up to date, the CIs build minimal kernels for each test group. 199 200Adding checks inside the tests to validate that the necessary kernel 201configs are enabled is discouraged. The test author may include such 202checks, but standalone patches to make tests compatible e.g. with 203distro kernel configs are unlikely to be accepted. 204 205Avoid libraries and frameworks 206~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 207 208Test files should be relatively self contained. The libraries should 209only include very core or non-trivial code. 210It may be tempting to "factor out" the common code, but fight that urge. 211Library code increases the barrier of entry, and complexity in general. 212 213Avoid mixing test code and boilerplate 214~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 215 216In Python, try to avoid adding code in the ``main()`` function which 217instantiates ``NetDrvEnv()`` and calls ``ksft_run()``. It's okay to 218set up global resources (e.g. open an RtNetlink socket used by multiple 219tests), but any complex logic, test-specific environment configuration 220and validation should be done in the tests (even if it means it has to 221be repeated). 222 223Local host is the DUT 224~~~~~~~~~~~~~~~~~~~~~ 225 226Dual-host tests (tests with an endpoint) should be written from the DUT 227perspective. IOW the local machine should be the one tested, remote is 228just for traffic generation. 229 230Avoid modifying remote 231~~~~~~~~~~~~~~~~~~~~~~ 232 233Avoid making configuration changes to the remote system as much as possible. 234Remote system may be used concurrently by multiple DUTs. 235 236defer() 237~~~~~~~ 238 239The env must be clean after test exits. Register a ``defer()`` for any 240action that needs an "undo" as soon as possible. If you need to run 241the cancel action as part of the test - ``defer()`` returns an object 242you can ``.exec()``-ute. 243 244ksft_pr() 245~~~~~~~~~ 246 247Use ``ksft_pr()`` instead of ``print()`` to avoid breaking TAP format. 248 249ksft_disruptive 250~~~~~~~~~~~~~~~ 251 252By default the tests are expected to be able to run on 253single-interface systems. All tests which may disconnect ``NETIF`` 254must be annotated with ``@ksft_disruptive``. 255 256Running tests CI-style 257====================== 258 259See https://github.com/linux-netdev/nipa/wiki for instructions on how 260to easily run the tests using ``virtme-ng``. 261