1.. SPDX-License-Identifier: GPL-2.0 2.. _cpuhp_index: 3 4==================== 5CPU Hotplug and ACPI 6==================== 7 8CPU hotplug in the arm64 world is commonly used to describe the kernel taking 9CPUs online/offline using PSCI. This document is about ACPI firmware allowing 10CPUs that were not available during boot to be added to the system later. 11 12``possible`` and ``present`` refer to the state of the CPU as seen by linux. 13 14 15CPU Hotplug on physical systems - CPUs not present at boot 16---------------------------------------------------------- 17 18Physical systems need to mark a CPU that is ``possible`` but not ``present`` as 19being ``present``. An example would be a dual socket machine, where the package 20in one of the sockets can be replaced while the system is running. 21 22This is not supported. 23 24In the arm64 world CPUs are not a single device but a slice of the system. 25There are no systems that support the physical addition (or removal) of CPUs 26while the system is running, and ACPI is not able to sufficiently describe 27them. 28 29e.g. New CPUs come with new caches, but the platform's cache topology is 30described in a static table, the PPTT. How caches are shared between CPUs is 31not discoverable, and must be described by firmware. 32 33e.g. The GIC redistributor for each CPU must be accessed by the driver during 34boot to discover the system wide supported features. ACPI's MADT GICC 35structures can describe a redistributor associated with a disabled CPU, but 36can't describe whether the redistributor is accessible, only that it is not 37'always on'. 38 39arm64's ACPI tables assume that everything described is ``present``. 40 41 42CPU Hotplug on virtual systems - CPUs not enabled at boot 43--------------------------------------------------------- 44 45Virtual systems have the advantage that all the properties the system will 46ever have can be described at boot. There are no power-domain considerations 47as such devices are emulated. 48 49CPU Hotplug on virtual systems is supported. It is distinct from physical 50CPU Hotplug as all vCPU resources are statically described in the firmware 51configuration tables (e.g. MADT), meaning their maximum possible count is 52known at boot. However, vCPUs that are not enabled at boot are not marked 53as ``present`` by the kernel until they are hotplugged. An example is where 54a virtual machine boots with a single CPU, and additional CPUs are added 55once a cloud orchestrator deploys the workload. 56 57For a virtual machine, the VMM (e.g. Qemu) plays the part of firmware. 58 59Virtual hotplug is implemented as a firmware policy affecting which CPUs can be 60brought online. Firmware can enforce its policy via PSCI's return codes. e.g. 61``DENIED``. 62 63The ACPI tables must describe all the resources of the virtual machine. CPUs 64that are hot-pluggable must have the ``online capable`` bit set and the 65``enabled`` bit cleared in the MADT GICC structures to indicate they can be 66enabled later. The boot CPU must be marked as ``enabled`` with its 67``online capable`` bit cleared. The 'always on' GICR structure must be used 68to describe the redistributors. 69 70CPUs described as ``online capable`` but not ``enabled`` can be set to enabled 71by the DSDT's Processor object's _STA method. On virtual systems the _STA method 72must always set the ``ACPI_STA_DEVICE_PRESENT`` bit, while toggling the 73``ACPI_STA_DEVICE_ENABLED`` bit to reflect its plug status. The kernel will 74then dynamically mark the vCPU as ``present`` within the OS when the 75``ACPI_STA_DEVICE_ENABLED`` bit becomes set during hot-add. Changes to the 76firmware policy can be notified to the OS via device-check or eject-request. 77 78CPUs described as ``enabled`` in the static table, should not have their _STA 79modified dynamically by firmware. Soft-restart features such as kexec will 80re-read the static properties of the system from these static tables, and 81may malfunction if these no longer describe the running system. Linux will 82re-discover the dynamic properties of the system from the _STA method later 83during boot. 84