1.. SPDX-License-Identifier: GPL-2.0 2 3========================== 4Frequently Asked Questions 5========================== 6 7How is this different from Autotest, kselftest, etc? 8==================================================== 9KUnit is a unit testing framework. Autotest, kselftest (and some others) are 10not. 11 12A `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is supposed to 13test a single unit of code in isolation, hence the name. A unit test should be 14the finest granularity of testing and as such should allow all possible code 15paths to be tested in the code under test; this is only possible if the code 16under test is very small and does not have any external dependencies outside of 17the test's control like hardware. 18 19There are no testing frameworks currently available for the kernel that do not 20require installing the kernel on a test machine or in a VM and all require 21tests to be written in userspace and run on the kernel under test; this is true 22for Autotest, kselftest, and some others, disqualifying any of them from being 23considered unit testing frameworks. 24 25Does KUnit support running on architectures other than UML? 26=========================================================== 27 28Yes, well, mostly. 29 30For the most part, the KUnit core framework (what you use to write the tests) 31can compile to any architecture; it compiles like just another part of the 32kernel and runs when the kernel boots, or when built as a module, when the 33module is loaded. However, there is some infrastructure, 34like the KUnit Wrapper (``tools/testing/kunit/kunit.py``) that does not support 35other architectures. 36 37In short, this means that, yes, you can run KUnit on other architectures, but 38it might require more work than using KUnit on UML. 39 40For more information, see :ref:`kunit-on-non-uml`. 41 42What is the difference between a unit test and these other kinds of tests? 43========================================================================== 44Most existing tests for the Linux kernel would be categorized as an integration 45test, or an end-to-end test. 46 47- A unit test is supposed to test a single unit of code in isolation, hence the 48 name. A unit test should be the finest granularity of testing and as such 49 should allow all possible code paths to be tested in the code under test; this 50 is only possible if the code under test is very small and does not have any 51 external dependencies outside of the test's control like hardware. 52- An integration test tests the interaction between a minimal set of components, 53 usually just two or three. For example, someone might write an integration 54 test to test the interaction between a driver and a piece of hardware, or to 55 test the interaction between the userspace libraries the kernel provides and 56 the kernel itself; however, one of these tests would probably not test the 57 entire kernel along with hardware interactions and interactions with the 58 userspace. 59- An end-to-end test usually tests the entire system from the perspective of the 60 code under test. For example, someone might write an end-to-end test for the 61 kernel by installing a production configuration of the kernel on production 62 hardware with a production userspace and then trying to exercise some behavior 63 that depends on interactions between the hardware, the kernel, and userspace. 64 65KUnit isn't working, what should I do? 66====================================== 67 68Unfortunately, there are a number of things which can break, but here are some 69things to try. 70 711. Try running ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output`` 72 parameter. This might show details or error messages hidden by the kunit_tool 73 parser. 742. Instead of running ``kunit.py run``, try running ``kunit.py config``, 75 ``kunit.py build``, and ``kunit.py exec`` independently. This can help track 76 down where an issue is occurring. (If you think the parser is at fault, you 77 can run it manually against stdin or a file with ``kunit.py parse``.) 783. Running the UML kernel directly can often reveal issues or error messages 79 kunit_tool ignores. This should be as simple as running ``./vmlinux`` after 80 building the UML kernel (e.g., by using ``kunit.py build``). Note that UML 81 has some unusual requirements (such as the host having a tmpfs filesystem 82 mounted), and has had issues in the past when built statically and the host 83 has KASLR enabled. (On older host kernels, you may need to run ``setarch 84 `uname -m` -R ./vmlinux`` to disable KASLR.) 854. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test 86 (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config 87 around, so you can see what config was used after running ``kunit.py run``. 88 It also preserves any config changes you might make, so you can 89 enable/disable things with ``make ARCH=um menuconfig`` or similar, and then 90 re-run kunit_tool. 915. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This 92 may help clean up any residual config items which could be causing problems. 936. Finally, try running KUnit outside UML. KUnit and KUnit tests can be 94 built into any kernel, or can be built as a module and loaded at runtime. 95 Doing so should allow you to determine if UML is causing the issue you're 96 seeing. When tests are built-in, they will execute when the kernel boots, and 97 modules will automatically execute associated tests when loaded. Test results 98 can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and 99 can be parsed with ``kunit.py parse``. For more details, see "KUnit on 100 non-UML architectures" in :doc:`usage`. 101 102If none of the above tricks help, you are always welcome to email any issues to 103kunit-dev@googlegroups.com. 104