1.. SPDX-License-Identifier: (GPL-2.0+ OR CC-BY-4.0) 2.. [see the bottom of this file for redistribution information] 3 4====================== 5Bisecting a regression 6====================== 7 8This document describes how to use a ``git bisect`` to find the source code 9change that broke something -- for example when some functionality stopped 10working after upgrading from Linux 6.0 to 6.1. 11 12The text focuses on the gist of the process. If you are new to bisecting the 13kernel, better follow Documentation/admin-guide/verify-bugs-and-bisect-regressions.rst 14instead: it depicts everything from start to finish while covering multiple 15aspects even kernel developers occasionally forget. This includes detecting 16situations early where a bisection would be a waste of time, as nobody would 17care about the result -- for example, because the problem happens after the 18kernel marked itself as 'tainted', occurs in an abandoned version, was already 19fixed, or is caused by a .config change you or your Linux distributor performed. 20 21Finding the change causing a kernel issue using a bisection 22=========================================================== 23 24*Note: the following process assumes you prepared everything for a bisection. 25This includes having a Git clone with the appropriate sources, installing the 26software required to build and install kernels, as well as a .config file stored 27in a safe place (the following example assumes '~/prepared_kernel_.config') to 28use as pristine base at each bisection step; ideally, you have also worked out 29a fully reliable and straight-forward way to reproduce the regression, too.* 30 31* Preparation: start the bisection and tell Git about the points in the history 32 you consider to be working and broken, which Git calls 'good' and 'bad':: 33 34 git bisect start 35 git bisect good v6.0 36 git bisect bad v6.1 37 38 Instead of Git tags like 'v6.0' and 'v6.1' you can specify commit-ids, too. 39 401. Copy your prepared .config into the build directory and adjust it to the 41 needs of the codebase Git checked out for testing:: 42 43 cp ~/prepared_kernel_.config .config 44 make olddefconfig 45 462. Now build, install, and boot a kernel. This might fail for unrelated reasons, 47 for example, when a compile error happens at the current stage of the 48 bisection a later change resolves. In such cases run ``git bisect skip`` and 49 go back to step 1. 50 513. Check if the functionality that regressed works in the kernel you just built. 52 53 If it works, execute:: 54 55 git bisect good 56 57 If it is broken, run:: 58 59 git bisect bad 60 61 Note, getting this wrong just once will send the rest of the bisection 62 totally off course. To prevent having to start anew later you thus want to 63 ensure what you tell Git is correct; it is thus often wise to spend a few 64 minutes more on testing in case your reproducer is unreliable. 65 66 After issuing one of these two commands, Git will usually check out another 67 bisection point and print something like 'Bisecting: 675 revisions left to 68 test after this (roughly 10 steps)'. In that case go back to step 1. 69 70 If Git instead prints something like 'cafecaca0c0dacafecaca0c0dacafecaca0c0da 71 is the first bad commit', then you have finished the bisection. In that case 72 move to the next point below. Note, right after displaying that line Git will 73 show some details about the culprit including its patch description; this can 74 easily fill your terminal, so you might need to scroll up to see the message 75 mentioning the culprit's commit-id. 76 77 In case you missed Git's output, you can always run ``git bisect log`` to 78 print the status: it will show how many steps remain or mention the result of 79 the bisection. 80 81* Recommended complementary task: put the bisection log and the current .config 82 file aside for the bug report; furthermore tell Git to reset the sources to 83 the state before the bisection:: 84 85 git bisect log > ~/bisection-log 86 cp .config ~/bisection-config-culprit 87 git bisect reset 88 89* Recommended optional task: try reverting the culprit on top of the latest 90 codebase and check if that fixes your bug; if that is the case, it validates 91 the bisection and enables developers to resolve the regression through a 92 revert. 93 94 To try this, update your clone and check out latest mainline. Then tell Git 95 to revert the change by specifying its commit-id:: 96 97 git revert --no-edit cafec0cacaca0 98 99 Git might reject this, for example when the bisection landed on a merge 100 commit. In that case, abandon the attempt. Do the same, if Git fails to revert 101 the culprit on its own because later changes depend on it -- at least unless 102 you bisected a stable or longterm kernel series, in which case you want to 103 check out its latest codebase and try a revert there. 104 105 If a revert succeeds, build and test another kernel to check if reverting 106 resolved your regression. 107 108With that the process is complete. Now report the regression as described by 109Documentation/admin-guide/reporting-issues.rst. 110 111Bisecting linux-next 112-------------------- 113 114If you face a problem only happening in linux-next, bisect between the 115linux-next branches 'stable' and 'master'. The following commands will start 116the process for a linux-next tree you added as a remote called 'next':: 117 118 git bisect start 119 git bisect good next/stable 120 git bisect bad next/master 121 122The 'stable' branch refers to the state of linux-mainline that the current 123linux-next release (found in the 'master' branch) is based on -- the former 124thus should be free of any problems that show up in -next, but not in Linus' 125tree. 126 127This will bisect across a wide range of changes, some of which you might have 128used in earlier linux-next releases without problems. Sadly there is no simple 129way to avoid checking them: bisecting from one linux-next release to a later 130one (say between 'next-20241020' and 'next-20241021') is impossible, as they 131share no common history. 132 133Additional reading material 134--------------------------- 135 136* The `man page for 'git bisect' <https://git-scm.com/docs/git-bisect>`_ and 137 `fighting regressions with 'git bisect' <https://git-scm.com/docs/git-bisect-lk2009.html>`_ 138 in the Git documentation. 139* `Working with git bisect <https://nathanchance.dev/posts/working-with-git-bisect/>`_ 140 from kernel developer Nathan Chancellor. 141* `Using Git bisect to figure out when brokenness was introduced <http://webchick.net/node/99>`_. 142* `Fully automated bisecting with 'git bisect run' <https://lwn.net/Articles/317154>`_. 143 144.. 145 end-of-content 146.. 147 This document is maintained by Thorsten Leemhuis <linux@leemhuis.info>. If 148 you spot a typo or small mistake, feel free to let him know directly and 149 he'll fix it. You are free to do the same in a mostly informal way if you 150 want to contribute changes to the text -- but for copyright reasons please CC 151 linux-doc@vger.kernel.org and 'sign-off' your contribution as 152 Documentation/process/submitting-patches.rst explains in the section 'Sign 153 your work - the Developer's Certificate of Origin'. 154.. 155 This text is available under GPL-2.0+ or CC-BY-4.0, as stated at the top 156 of the file. If you want to distribute this text under CC-BY-4.0 only, 157 please use 'The Linux kernel development community' for author attribution 158 and link this as source: 159 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/admin-guide/bug-bisect.rst 160 161.. 162 Note: Only the content of this RST file as found in the Linux kernel sources 163 is available under CC-BY-4.0, as versions of this text that were processed 164 (for example by the kernel's build system) might contain content taken from 165 files which use a more restrictive license. 166