1.. SPDX-License-Identifier: GPL-2.0-or-later 2 3.. _kho-concepts: 4 5======================== 6Kexec Handover Subsystem 7======================== 8 9Overview 10======== 11 12Kexec HandOver (KHO) is a mechanism that allows Linux to preserve memory 13regions, which could contain serialized system states, across kexec. 14 15KHO uses :ref:`flattened device tree (FDT) <kho_fdt>` to pass information about 16the preserved state from pre-exec kernel to post-kexec kernel and :ref:`scratch 17memory regions <kho_scratch>` to ensure integrity of the preserved memory. 18 19.. _kho_fdt: 20 21KHO FDT 22======= 23Every KHO kexec carries a KHO specific flattened device tree (FDT) blob that 24describes the preserved state. The FDT includes properties describing preserved 25memory regions and nodes that hold subsystem specific state. 26 27The preserved memory regions contain either serialized subsystem states, or 28in-memory data that shall not be touched across kexec. After KHO, subsystems 29can retrieve and restore the preserved state from KHO FDT. 30 31Subsystems participating in KHO can define their own format for state 32serialization and preservation. 33 34KHO FDT and structures defined by the subsystems form an ABI between pre-kexec 35and post-kexec kernels. This ABI is defined by header files in 36``include/linux/kho/abi`` directory. 37 38.. toctree:: 39 :maxdepth: 1 40 41 abi.rst 42 43.. _kho_scratch: 44 45Scratch Regions 46=============== 47 48To boot into kexec, we need to have a physically contiguous memory range that 49contains no handed over memory. Kexec then places the target kernel and initrd 50into that region. The new kernel exclusively uses this region for memory 51allocations before during boot up to the initialization of the page allocator. 52 53We guarantee that we always have such regions through the scratch regions: On 54first boot KHO allocates several physically contiguous memory regions. Since 55after kexec these regions will be used by early memory allocations, there is a 56scratch region per NUMA node plus a scratch region to satisfy allocations 57requests that do not require particular NUMA node assignment. 58By default, size of the scratch region is calculated based on amount of memory 59allocated during boot. The ``kho_scratch`` kernel command line option may be 60used to explicitly define size of the scratch regions. 61The scratch regions are declared as CMA when page allocator is initialized so 62that their memory can be used during system lifetime. CMA gives us the 63guarantee that no handover pages land in that region, because handover pages 64must be at a static physical memory location and CMA enforces that only 65movable pages can be located inside. 66 67After KHO kexec, we ignore the ``kho_scratch`` kernel command line option and 68instead reuse the exact same region that was originally allocated. This allows 69us to recursively execute any amount of KHO kexecs. Because we used this region 70for boot memory allocations and as target memory for kexec blobs, some parts 71of that memory region may be reserved. These reservations are irrelevant for 72the next KHO, because kexec can overwrite even the original kernel. 73 74.. _kho-finalization-phase: 75 76KHO finalization phase 77====================== 78 79To enable user space based kexec file loader, the kernel needs to be able to 80provide the FDT that describes the current kernel's state before 81performing the actual kexec. The process of generating that FDT is 82called serialization. When the FDT is generated, some properties 83of the system may become immutable because they are already written down 84in the FDT. That state is called the KHO finalization phase. 85 86See Also 87======== 88 89- :doc:`/admin-guide/mm/kho` 90