1====================== 2Linux Kernel Selftests 3====================== 4 5The kernel contains a set of "self tests" under the tools/testing/selftests/ 6directory. These are intended to be small tests to exercise individual code 7paths in the kernel. Tests are intended to be run after building, installing 8and booting a kernel. 9 10Kselftest from mainline can be run on older stable kernels. Running tests 11from mainline offers the best coverage. Several test rings run mainline 12kselftest suite on stable releases. The reason is that when a new test 13gets added to test existing code to regression test a bug, we should be 14able to run that test on an older kernel. Hence, it is important to keep 15code that can still test an older kernel and make sure it skips the test 16gracefully on newer releases. 17 18You can find additional information on Kselftest framework, how to 19write new tests using the framework on Kselftest wiki: 20 21https://kselftest.wiki.kernel.org/ 22 23On some systems, hot-plug tests could hang forever waiting for cpu and 24memory to be ready to be offlined. A special hot-plug target is created 25to run the full range of hot-plug tests. In default mode, hot-plug tests run 26in safe mode with a limited scope. In limited mode, cpu-hotplug test is 27run on a single cpu as opposed to all hotplug capable cpus, and memory 28hotplug test is run on 2% of hotplug capable memory instead of 10%. 29 30kselftest runs as a userspace process. Tests that can be written/run in 31userspace may wish to use the `Test Harness`_. Tests that need to be 32run in kernel space may wish to use a `Test Module`_. 33 34Running the selftests (hotplug tests are run in limited mode) 35============================================================= 36 37To build the tests:: 38 39 $ make -C tools/testing/selftests 40 41To run the tests:: 42 43 $ make -C tools/testing/selftests run_tests 44 45To build and run the tests with a single command, use:: 46 47 $ make kselftest 48 49Note that some tests will require root privileges. 50 51Kselftest supports saving output files in a separate directory and then 52running tests. To locate output files in a separate directory two syntaxes 53are supported. In both cases the working directory must be the root of the 54kernel src. This is applicable to "Running a subset of selftests" section 55below. 56 57To build, save output files in a separate directory with O= :: 58 59 $ make O=/tmp/kselftest kselftest 60 61To build, save output files in a separate directory with KBUILD_OUTPUT :: 62 63 $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest 64 65The O= assignment takes precedence over the KBUILD_OUTPUT environment 66variable. 67 68The above commands by default run the tests and print full pass/fail report. 69Kselftest supports "summary" option to make it easier to understand the test 70results. Please find the detailed individual test results for each test in 71/tmp/testname file(s) when summary option is specified. This is applicable 72to "Running a subset of selftests" section below. 73 74To run kselftest with summary option enabled :: 75 76 $ make summary=1 kselftest 77 78Running a subset of selftests 79============================= 80 81You can use the "TARGETS" variable on the make command line to specify 82single test to run, or a list of tests to run. 83 84To run only tests targeted for a single subsystem:: 85 86 $ make -C tools/testing/selftests TARGETS=ptrace run_tests 87 88You can specify multiple tests to build and run:: 89 90 $ make TARGETS="size timers" kselftest 91 92To build, save output files in a separate directory with O= :: 93 94 $ make O=/tmp/kselftest TARGETS="size timers" kselftest 95 96To build, save output files in a separate directory with KBUILD_OUTPUT :: 97 98 $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest 99 100Additionally you can use the "SKIP_TARGETS" variable on the make command 101line to specify one or more targets to exclude from the TARGETS list. 102 103To run all tests but a single subsystem:: 104 105 $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests 106 107You can specify multiple tests to skip:: 108 109 $ make SKIP_TARGETS="size timers" kselftest 110 111You can also specify a restricted list of tests to run together with a 112dedicated skiplist:: 113 114 $ make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest 115 116See the top-level tools/testing/selftests/Makefile for the list of all 117possible targets. 118 119Running the full range hotplug selftests 120======================================== 121 122To build the hotplug tests:: 123 124 $ make -C tools/testing/selftests hotplug 125 126To run the hotplug tests:: 127 128 $ make -C tools/testing/selftests run_hotplug 129 130Note that some tests will require root privileges. 131 132 133Install selftests 134================= 135 136You can use the "install" target of "make" (which calls the `kselftest_install.sh` 137tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`), 138or in a user specified location via the `INSTALL_PATH` "make" variable. 139 140To install selftests in default location:: 141 142 $ make -C tools/testing/selftests install 143 144To install selftests in a user specified location:: 145 146 $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path 147 148Running installed selftests 149=========================== 150 151Found in the install directory, as well as in the Kselftest tarball, 152is a script named `run_kselftest.sh` to run the tests. 153 154You can simply do the following to run the installed Kselftests. Please 155note some tests will require root privileges:: 156 157 $ cd kselftest_install 158 $ ./run_kselftest.sh 159 160To see the list of available tests, the `-l` option can be used:: 161 162 $ ./run_kselftest.sh -l 163 164The `-c` option can be used to run all the tests from a test collection, or 165the `-t` option for specific single tests. Either can be used multiple times:: 166 167 $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep 168 169For other features see the script usage output, seen with the `-h` option. 170 171Packaging selftests 172=================== 173 174In some cases packaging is desired, such as when tests need to run on a 175different system. To package selftests, run:: 176 177 $ make -C tools/testing/selftests gen_tar 178 179This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By 180default, `.gz` format is used. The tar compression format can be overridden by 181specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ 182option is supported, such as:: 183 184 $ make -C tools/testing/selftests gen_tar FORMAT=.xz 185 186`make gen_tar` invokes `make install` so you can use it to package a subset of 187tests by using variables specified in `Running a subset of selftests`_ 188section:: 189 190 $ make -C tools/testing/selftests gen_tar TARGETS="bpf" FORMAT=.xz 191 192.. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress 193 194Contributing new tests 195====================== 196 197In general, the rules for selftests are 198 199 * Do as much as you can if you're not root; 200 201 * Don't take too long; 202 203 * Don't break the build on any architecture, and 204 205 * Don't cause the top-level "make run_tests" to fail if your feature is 206 unconfigured. 207 208Contributing new tests (details) 209================================ 210 211 * In your Makefile, use facilities from lib.mk by including it instead of 212 reinventing the wheel. Specify flags and binaries generation flags on 213 need basis before including lib.mk. :: 214 215 CFLAGS = $(KHDR_INCLUDES) 216 TEST_GEN_PROGS := close_range_test 217 include ../lib.mk 218 219 * Use TEST_GEN_XXX if such binaries or files are generated during 220 compiling. 221 222 TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by 223 default. 224 225 TEST_CUSTOM_PROGS should be used by tests that require custom build 226 rules and prevent common build rule use. 227 228 TEST_PROGS are for test shell scripts. Please ensure shell script has 229 its exec bit set. Otherwise, lib.mk run_tests will generate a warning. 230 231 TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests. 232 233 TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the 234 executable which is not tested by default. 235 TEST_FILES, TEST_GEN_FILES mean it is the file which is used by 236 test. 237 238 * First use the headers inside the kernel source and/or git repo, and then the 239 system headers. Headers for the kernel release as opposed to headers 240 installed by the distro on the system should be the primary focus to be able 241 to find regressions. Use KHDR_INCLUDES in Makefile to include headers from 242 the kernel source. 243 244 * If a test needs specific kernel config options enabled, add a config file in 245 the test directory to enable them. 246 247 e.g: tools/testing/selftests/android/config 248 249 * Create a .gitignore file inside test directory and add all generated objects 250 in it. 251 252 * Add new test name in TARGETS in selftests/Makefile:: 253 254 TARGETS += android 255 256 * All changes should pass:: 257 258 kselftest-{all,install,clean,gen_tar} 259 kselftest-{all,install,clean,gen_tar} O=abo_path 260 kselftest-{all,install,clean,gen_tar} O=rel_path 261 make -C tools/testing/selftests {all,install,clean,gen_tar} 262 make -C tools/testing/selftests {all,install,clean,gen_tar} O=abs_path 263 make -C tools/testing/selftests {all,install,clean,gen_tar} O=rel_path 264 265Test Module 266=========== 267 268Kselftest tests the kernel from userspace. Sometimes things need 269testing from within the kernel, one method of doing this is to create a 270test module. We can tie the module into the kselftest framework by 271using a shell script test runner. ``kselftest/module.sh`` is designed 272to facilitate this process. There is also a header file provided to 273assist writing kernel modules that are for use with kselftest: 274 275- ``tools/testing/selftests/kselftest_module.h`` 276- ``tools/testing/selftests/kselftest/module.sh`` 277 278Note that test modules should taint the kernel with TAINT_TEST. This will 279happen automatically for modules which are in the ``tools/testing/`` 280directory, or for modules which use the ``kselftest_module.h`` header above. 281Otherwise, you'll need to add ``MODULE_INFO(test, "Y")`` to your module 282source. selftests which do not load modules typically should not taint the 283kernel, but in cases where a non-test module is loaded, TEST_TAINT can be 284applied from userspace by writing to ``/proc/sys/kernel/tainted``. 285 286How to use 287---------- 288 289Here we show the typical steps to create a test module and tie it into 290kselftest. We use kselftests for lib/ as an example. 291 2921. Create the test module 293 2942. Create the test script that will run (load/unload) the module 295 e.g. ``tools/testing/selftests/lib/printf.sh`` 296 2973. Add line to config file e.g. ``tools/testing/selftests/lib/config`` 298 2994. Add test script to makefile e.g. ``tools/testing/selftests/lib/Makefile`` 300 3015. Verify it works: 302 303.. code-block:: sh 304 305 # Assumes you have booted a fresh build of this kernel tree 306 cd /path/to/linux/tree 307 make kselftest-merge 308 make modules 309 sudo make modules_install 310 make TARGETS=lib kselftest 311 312Example Module 313-------------- 314 315A bare bones test module might look like this: 316 317.. code-block:: c 318 319 // SPDX-License-Identifier: GPL-2.0+ 320 321 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt 322 323 #include "../tools/testing/selftests/kselftest_module.h" 324 325 KSTM_MODULE_GLOBALS(); 326 327 /* 328 * Kernel module for testing the foobinator 329 */ 330 331 static int __init test_function() 332 { 333 ... 334 } 335 336 static void __init selftest(void) 337 { 338 KSTM_CHECK_ZERO(do_test_case("", 0)); 339 } 340 341 KSTM_MODULE_LOADERS(test_foo); 342 MODULE_AUTHOR("John Developer <jd@fooman.org>"); 343 MODULE_LICENSE("GPL"); 344 MODULE_INFO(test, "Y"); 345 346Example test script 347------------------- 348 349.. code-block:: sh 350 351 #!/bin/bash 352 # SPDX-License-Identifier: GPL-2.0+ 353 $(dirname $0)/../kselftest/module.sh "foo" test_foo 354 355 356Test Harness 357============ 358 359The kselftest_harness.h file contains useful helpers to build tests. The 360test harness is for userspace testing, for kernel space testing see `Test 361Module`_ above. 362 363The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as 364example. 365 366Example 367------- 368 369.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 370 :doc: example 371 372 373Helpers 374------- 375 376.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 377 :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP 378 FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT 379 FIXTURE_VARIANT_ADD 380 381Operators 382--------- 383 384.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 385 :doc: operators 386 387.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 388 :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE 389 ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE 390 ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT 391 EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE 392 EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE 393