1.. SPDX-License-Identifier: GPL-2.0 2 3===================================== 4Writing a test for objtool's klp code 5===================================== 6 7Instructions for adding a test to tools/objtool/tests/. Read 8klp-test-design.txt first if you need to know how the harness works; this 9document is the procedure and the rules. 10 11Two kinds of request bring you here: 12 13* *"write a test for commit <sha>"* -- a fix went in without one. 14* *"port the test at <location>, written against another harness"* -- a case 15 exists elsewhere and should live in tree. 16 17Both follow the same procedure. 18 19 20The one rule that matters 21========================= 22 23**A test is not finished until you have watched it fail.** 24 25Break the thing it guards -- revert the fix, or sabotage the exact line -- and 26confirm the test fails. Then restore and confirm it passes. A test that has 27never failed is not known to test anything, and this suite has produced 28several that passed against deliberately broken code: 29 30* an alternatives fixture whose empty entry pointed at its own end label rather 31 than the neighbour's replacement, so the bug it guarded made no difference; 32* a sympos fixture where symbol-table order and address order agreed, so 33 counting and reading the linked image gave the same answer; 34* a string fixture using a named ``char[]``, which never reached the 35 contents-hashing path because that keys on ``SHF_STRINGS``; 36* a static array the compiler proved constant, folded to zero, and emitted no 37 relocation for -- so the two builds were byte-identical. 38 39Every one looked correct. Say in the commit message how you verified, and if 40you could not isolate the behaviour to a single line, **say that too** rather 41than implying otherwise. 42 43 44Procedure 45========= 46 471. **Read the fix.** What input reaches the broken line? What is observable 48 in the output object when it misbehaves -- a missing section, a relocation 49 naming the wrong symbol, an unchanged checksum, a rejected build? If 50 nothing is observable, stop and say so; see `When to give up`_. 51 522. **Decide where it lives.** ``generic/`` unless the fixture needs 53 architecture-specific assembly or the behaviour is architecture-specific, 54 in which case ``x86/`` (or a new directory named for the architecture). 55 563. **Write the fixture** in the same directory's ``fixtures/``. Reuse an 57 existing one if it already produces the shape; add a ``-D`` knob rather 58 than copying a fixture to change one line. 59 604. **Write the test.** Assert the *premise* before the result -- see 61 `State the premise`_. 62 635. **Verify by breaking the code.** Then restore. 64 656. **Run the whole suite under both compilers**:: 66 67 make -C tools/objtool tests 68 CC=clang make -C tools/objtool tests 69 707. **Commit** the test and its fixture together, alone. One test per commit. 71 72 73Writing the fixture 74=================== 75 76Fixtures are freestanding C. No kernel headers -- write out the kernel 77structure by hand if you need one, as the existing special-section fixtures do. 78 79Every fixture needs a ``.modinfo`` name, because klp diff reads the object's 80module name from it:: 81 82 static const char __modinfo[] 83 __attribute__((section(".modinfo"), used, aligned(1))) = "\0name=vmlinux"; 84 85Use ``MODNAME`` if the test needs to vary it. Note that most fixtures hardcode 86``vmlinux``: passing ``-DMODNAME`` to one that does silently does nothing and 87the test quietly becomes a vmlinux test. 88 89The patched build is selected with ``-DPATCHED``. For a fixture with several 90variants, gate each on both, so the original is always the baseline:: 91 92 #if defined(PATCHED) && defined(WHICH_CALL) 93 r = callee_b(x); 94 #else 95 r = callee_a(x); 96 #endif 97 98and select one per build: ``build_pair foo.c -DWHICH_CALL``. Without the 99``defined(PATCHED)`` the flag applies to *both* builds and nothing differs. 100 101Traps that have bitten before 102----------------------------- 103 104* **The compiler optimises your fixture away.** A static never written is 105 proved constant, its reads folded, and no relocation emitted. Add a writer 106 the compiler cannot see through. 107* **String literals versus named arrays.** The contents-hashing path keys on 108 ``SHF_STRINGS``, which the compiler sets on the mergeable section a *literal* 109 lands in, not on a ``char[]`` given a section of its own. 110* **Per-function sections hide movement.** With the default 111 ``-ffunction-sections`` every function sits at offset 0 of its own section, 112 so nothing ever moves. A test about position needs 113 ``build_pair foo.c -fno-function-sections``. 114* **Special sections need boundaries.** Either an entsize on the section or an 115 ``ANNOTATE_DATA_SPECIAL`` annotation, or klp diff reports "missing special 116 section entsize or annotations". Their targets need real (global) symbols, 117 or it reports "failed to convert reloc sym". 118* **Prefer letting objtool generate what objtool generates.** 119 ``.static_call_sites``, ``.mcount_loc``, ``.ibt_endbr_seal`` and ORC come 120 from its check pass. Call ``run_objtool_check --mcount`` and let it build 121 them; a hand-written copy tests your reading of the format, not the format. 122 123 Write one by hand only when the test needs a shape objtool will not produce, 124 and say so in the fixture. generic/fixtures/static_call.c does: objtool 125 emits the site but not the ``ANNOTATE_DATA_SPECIAL`` that describes its 126 boundaries -- those come from the kernel's macros -- so a fixture which has 127 to vary whether the annotation is there writes both itself. 128 129 130Writing the test 131================ 132 133Start from the shortest existing test, generic/test-basic.sh. 134 135State the premise 136----------------- 137 138A test which asserts only on the output passes when the compiler never emitted 139the construct in the first place, and reads as coverage it does not have. Say 140what the input must contain:: 141 142 assert_input_section __jump_table # the fixture must produce it -> fail 143 require_input_section .kcfi_traps # this compiler may not -> skip 144 145Prefer ``assert_*``. Reach for ``require_*`` only where absence genuinely 146depends on compiler version or flags, and follow it with something 147unconditional so the test can never be entirely vacuous. 148 149Where a test would otherwise duplicate a sibling, assert what makes it 150different. ``test-jump-label-module-static-key`` checks that the key really is 151reached through its section symbol -- without that it is a second copy of 152``test-jump-label-module-key``. 153 154Assert both directions 155---------------------- 156 157Check that the right thing happened *and* that the wrong thing did not. A klp 158diff which clones everything is as wrong as one which clones nothing:: 159 160 assert_patched changed 161 assert_not_patched untouched 162 163Skips 164----- 165 166* ``gcc_only``/``clang_only`` -- a settled fact about the compiler. Declared, 167 so it reads as expected forever. 168* ``probe_skip`` -- this toolchain did not produce the construct. Include what 169 to do about it if there is anything:: 170 171 probe_skip "no matching clang/lld pair for a ThinLTO link; set THIN_CC and THIN_LD to one" 172 173* A bare ``skip`` is **counted as a failure**. Never use it. 174 175Standing in for a kernel configuration 176-------------------------------------- 177 178``FIXTURE_CFLAGS`` is what a fixture is built with. Since objtool reads no 179``.config``, changing these flags is how a test covers a configuration without 180building a kernel. Two ways: 181 182* trailing arguments to ``build_pair``/``build_one``, which win, and cover 183 anything expressible as a negation:: 184 185 build_pair foo.c -fno-function-sections 186 187* otherwise assign ``FIXTURE_CFLAGS`` before building. 188 189Either way **say in a comment which kernel configuration the change stands in 190for**. A flag with no stated motive is indistinguishable from a mistake. 191 192 193What the test's comment must say 194================================ 195 196The comment at the top is the test's justification. It should let a reader 197decide, without archaeology, whether a skip or a failure matters. Include: 198 199* **what breaks** in the running kernel if the behaviour regresses -- not the 200 mechanism, the consequence; 201* **why it is not caught otherwise**, which is usually "nothing fails at build 202 time"; 203* **the fix commit** it guards, if there is one; 204* **anything load-bearing about the fixture** that is not obvious, especially 205 anything you got wrong first. 206 207That last point is the one people skip. If the fixture has to be built without 208per-function sections, or the static must not be named ``__warned``, or the key 209must be file-local -- write it down, or the next person will simplify it away. 210 211 212Porting a test from another harness 213=================================== 214 215Read the original's *case*, not its code. The other harness probably builds a 216real kernel module; here you write freestanding C. A transliteration will 217usually test something else. 218 219* Work out which objtool behaviour the case exercises, then produce that shape 220 the cheapest way here. 221* Verify by breaking the code, exactly as for a new test -- a port is not 222 correct because the original was. 223* If the original names a fix commit, cite it. 224* Credit the source in the commit message with the trailers the original 225 carried, followed by your own. 226 227Sometimes the port shows the case is already covered, and sometimes it shows 228the case cannot be reproduced here. Both are results; report them rather than 229committing something that passes vacuously. 230 231 232When to give up 233=============== 234 235Some behaviour cannot be reached from a compiled fixture. Say so, with what 236you tried, instead of committing a test that passes either way. Examples that 237were genuinely abandoned: 238 239* a memory leak -- needs valgrind, not an assertion on ELF; 240* ``mkstemp`` with long paths, and other I/O edge cases; 241* a NULL dereference reachable only through a debug path; 242* changes made redundant by a fallback: removing the code changes no output 243 because something else already handles the case; 244* a fix whose code has since been rewritten, so there is nothing left to 245 revert. 246 247Also stop when the behaviour depends on something outside the fixture's 248control -- an ELF library's handling of empty sections, or a compiler version's 249naming of anonymous data. A test which passes for you and skips for everyone 250else is worse than none. 251 252 253Checklist 254========= 255 256Before committing: 257 258* the test fails with the code broken, and passes with it fixed 259* the whole suite passes under **both** gcc and clang 260* the premise is asserted, not assumed 261* both directions are asserted where that applies 262* no bare ``skip`` 263* the fixture is in the same directory as the test 264* the comment names the consequence, the fix commit, and anything load-bearing 265* the commit contains one test and its fixtures, and nothing else 266* the commit message says how you verified it 267