Lines Matching +full:4 +full:- +full:cpu
2 CPU hotplug in the Kernel
19 insertion and removal require support for CPU hotplug.
22 provisioning reasons, or for RAS purposes to keep an offending CPU off
23 system execution path. Hence the need for CPU hotplug support in the
26 A more novel use of CPU-hotplug support is its use today in suspend resume
27 support for SMP. Dual-core and HT support makes even a laptop run SMP kernels
53 CPU maps
66 after a CPU is available for kernel scheduling and ready to receive
67 interrupts from devices. Its cleared when a CPU is brought down using
69 migrated to another target CPU.
75 from the map depending on the event is hot-add/hot-remove. There are currently
79 You really don't need to manipulate any of the system CPU maps. They should
80 be read-only for most use. When setting up per-cpu resources almost always use
82 ``for_each_cpu()`` can be used to iterate over a custom CPU mask.
87 Using CPU hotplug
94 $ ls -lh /sys/devices/system/cpu
96 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu0
97 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu1
98 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu2
99 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu3
100 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu4
101 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu5
102 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu6
103 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu7
104 drwxr-xr-x 2 root root 0 Dec 21 16:33 hotplug
105 -r--r--r-- 1 root root 4.0K Dec 21 16:33 offline
106 -r--r--r-- 1 root root 4.0K Dec 21 16:33 online
107 -r--r--r-- 1 root root 4.0K Dec 21 16:33 possible
108 -r--r--r-- 1 root root 4.0K Dec 21 16:33 present
110 The files *offline*, *online*, *possible*, *present* represent the CPU masks.
111 Each CPU folder contains an *online* file which controls the logical on (1) and
114 $ echo 0 > /sys/devices/system/cpu/cpu4/online
115 smpboot: CPU 4 is now offline
117 Once the CPU is shutdown, it will be removed from */proc/interrupts*,
121 $ echo 1 > /sys/devices/system/cpu/cpu4/online
122 smpboot: Booting Node 0 Processor 4 APIC 0x1
124 The CPU is usable again. This should work on all CPUs, but CPU0 is often special
125 and excluded from CPU hotplug.
127 The CPU hotplug coordination
131 ----------------
133 Once a CPU has been logically shutdown the teardown callbacks of registered
139 * All processes are migrated away from this outgoing CPU to new CPUs.
140 The new CPU is chosen from each process' current cpuset, which may be
142 * All interrupts targeted to this CPU are migrated to a new CPU
143 * timers are also migrated to a new CPU
148 The CPU hotplug API
151 CPU hotplug state machine
152 -------------------------
154 CPU hotplug uses a trivial state machine with a linear state space from
158 When a CPU is onlined, the startup callbacks are invoked sequentially until
160 callbacks of a state are set up or an instance is added to a multi-instance
163 When a CPU is offlined the teardown callbacks are invoked in the reverse
166 removed from a multi-instance state.
169 operations (CPU online or CPU offline) then the other not-required callback
179 The startup callbacks in this section are invoked before the CPU is
180 started during a CPU online operation. The teardown callbacks are invoked
181 after the CPU has become dysfunctional during a CPU offline operation.
183 The callbacks are invoked on a control CPU as they can't obviously run on
184 the hotplugged CPU which is either not yet started or has become
188 bring a CPU successfully online. The teardown callbacks are used to free
189 resources or to move pending work to an online CPU after the hotplugged
190 CPU became dysfunctional.
192 The startup callbacks are allowed to fail. If a callback fails, the CPU
193 online operation is aborted and the CPU is brought down to the previous
203 The startup callbacks in this section are invoked on the hotplugged CPU
204 with interrupts disabled during a CPU online operation in the early CPU
206 on the hotplugged CPU during a CPU offline operation shortly before the
207 CPU is completely shut down.
219 The startup callbacks in this section are invoked on the hotplugged CPU
220 during a CPU online operation. The teardown callbacks are invoked on the
221 hotplugged CPU during a CPU offline operation.
223 The callbacks are invoked in the context of the per CPU hotplug thread,
224 which is pinned on the hotplugged CPU. The callbacks are invoked with
228 operation is aborted and the CPU is brought back to the previous state.
230 CPU online/offline operations
231 -----------------------------
236 [CPUHP_OFFLINE + 1]->startup() -> success
237 [CPUHP_OFFLINE + 2]->startup() -> success
238 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
240 [CPUHP_BRINGUP_CPU]->startup() -> success
242 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
244 [CPUHP_AP_ONLINE]->startup() -> success
246 [CPUHP_AP_ONLINE + 1]->startup() -> success
248 [CPUHP_ONLINE - 1]->startup() -> success
254 [CPUHP_ONLINE - 1]->teardown() -> success
256 [CPUHP_AP_ONLINE + 1]->teardown() -> success
258 [CPUHP_AP_ONLINE]->teardown() -> success
260 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
263 [CPUHP_BRINGUP_CPU]->teardown()
264 [CPUHP_OFFLINE + 3]->teardown()
265 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
266 [CPUHP_OFFLINE + 1]->teardown()
272 [CPUHP_OFFLINE + 1]->startup() -> success
273 [CPUHP_OFFLINE + 2]->startup() -> success
274 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
276 [CPUHP_BRINGUP_CPU]->startup() -> success
278 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
280 [CPUHP_AP_ONLINE]->startup() -> success
282 [CPUHP_AP_ONLINE + 1]->startup() -> success
283 ---
284 [CPUHP_AP_ONLINE + N]->startup() -> fail
285 [CPUHP_AP_ONLINE + (N - 1)]->teardown()
287 [CPUHP_AP_ONLINE + 1]->teardown()
289 [CPUHP_AP_ONLINE]->teardown()
291 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
294 [CPUHP_BRINGUP_CPU]->teardown()
295 [CPUHP_OFFLINE + 3]->teardown()
296 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
297 [CPUHP_OFFLINE + 1]->teardown()
303 [CPUHP_ONLINE - 1]->teardown() -> success
305 [CPUHP_ONLINE - N]->teardown() -> fail
306 [CPUHP_ONLINE - (N - 1)]->startup()
308 [CPUHP_ONLINE - 1]->startup()
315 [CPUHP_ONLINE - 1]->teardown() -> success
317 [CPUHP_ONLINE - N]->teardown() -> fail
318 [CPUHP_ONLINE - (N - 1)]->startup() -> success
319 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
321 The CPU hotplug state machine stops right here and does not try to go back
324 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
325 [CPUHP_ONLINE - N]->teardown() -> fail
326 [CPUHP_ONLINE - (N - 1)]->startup() -> success
327 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
328 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
329 [CPUHP_ONLINE - N]->teardown() -> fail
331 Lather, rinse and repeat. In this case the CPU left in state::
333 [CPUHP_ONLINE - (N - 1)]
339 ------------------
341 There are two ways to allocate a CPU hotplug state:
346 ordering requirements versus other CPU hotplug states. E.g. the PERF core
348 callbacks during a CPU online operation. During a CPU offline operation
358 up at runtime and are part of the initializer of the CPU hotplug state
359 array in kernel/cpu.c.
371 Setup of a CPU hotplug state
372 ----------------------------
382 CPU hotplug state callbacks need to be invoked for each instance, the CPU
383 hotplug core provides multi-instance support. The advantage over driver
385 serialized against CPU hotplug operations and provide the automatic
387 multi-instance state the following function is available:
392 constants for dynamically allocated states - CPUHP_BP_PREPARE_DYN,
393 CPUHP_AP_ONLINE_DYN - depending on the state section (PREPARE, ONLINE) for
425 invoked during a CPU online operation. If the usage site does not require a
429 be invoked during a CPU offline operation. If the usage site does not
441 the current CPU (PREPARE section) or on each online CPU (ONLINE
442 section) in the context of the CPU's hotplug thread.
444 If a callback fails for CPU N then the teardown callback for CPU
445 0 .. N-1 is invoked to rollback the operation. The state setup fails,
449 The state setup and the callback invocations are serialized against CPU
450 hotplug operations. If the setup function has to be called from a CPU
452 used. These functions cannot be used from within CPU hotplug callbacks.
464 multi-instance states the dynamically allocated state number is
471 Removal of a CPU hotplug state
472 ------------------------------
494 callback is either invoked on the current CPU (PREPARE section) or on
495 each online CPU (ONLINE section) in the context of the CPU's hotplug
500 The state removal and the callback invocations are serialized against CPU
501 hotplug operations. If the remove function has to be called from a CPU
503 used. These functions cannot be used from within CPU hotplug callbacks.
505 If a multi-instance state is removed then the caller has to remove all
508 Multi-Instance state instance management
509 ----------------------------------------
511 Once the multi-instance state is set up, instances can be added to the
521 instance's data structure. The pointer is handed to the multi-instance
528 multi-instance state's node list.
534 the callback is either invoked on the current CPU (PREPARE section) or
535 on each online CPU (ONLINE section) in the context of the CPU's hotplug
538 If a callback fails for CPU N then the teardown callback for CPU
539 0 .. N-1 is invoked to rollback the operation, the function fails and
540 the instance is not added to the node list of the multi-instance state.
560 section the callback is either invoked on the current CPU (PREPARE
561 section) or on each online CPU (ONLINE section) in the context of the
562 CPU's hotplug thread.
567 serialized against CPU hotplug operations. These functions cannot be used
568 from within CPU hotplug callbacks and CPU hotplug read locked regions.
571 --------
600 Setup, use and teardown a dynamically allocated multi-instance state in the
607 ret = cpuhp_state_add_instance(state, &inst1->node);
611 ret = cpuhp_state_add_instance(state, &inst2->node);
615 cpuhp_remove_instance(state, &inst1->node);
617 cpuhp_remove_instance(state, &inst2->node);
626 shutdown a CPU and then put it online again. It is also possible to put the CPU
631 All registered states are enumerated in ``/sys/devices/system/cpu/hotplug/states`` ::
633 $ tail /sys/devices/system/cpu/hotplug/states
637 141: acpi/cpu-drv:online
647 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
649 $ echo 140 > /sys/devices/system/cpu/cpu4/hotplug/target
650 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
656 $ echo 169 > /sys/devices/system/cpu/cpu4/hotplug/target
657 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
662 # TASK-PID CPU# TIMESTAMP FUNCTION
664 bash-394 [001] 22.976: cpuhp_enter: cpu: 0004 target: 140 step: 169 (cpuhp_kick_ap_work)
665 cpuhp/4-31 [004] 22.977: cpuhp_enter: cpu: 0004 target: 140 step: 168 (sched_cpu_deactivate)
666 cpuhp/4-31 [004] 22.990: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
667 cpuhp/4-31 [004] 22.991: cpuhp_enter: cpu: 0004 target: 140 step: 144 (mce_cpu_pre_down)
668 cpuhp/4-31 [004] 22.992: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
669 …cpuhp/4-31 [004] 22.993: cpuhp_multi_enter: cpu: 0004 target: 140 step: 143 (virtnet_cpu_down_p…
670 cpuhp/4-31 [004] 22.994: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
671 cpuhp/4-31 [004] 22.995: cpuhp_enter: cpu: 0004 target: 140 step: 142 (cacheinfo_cpu_pre_down)
672 cpuhp/4-31 [004] 22.996: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
673 bash-394 [001] 22.997: cpuhp_exit: cpu: 0004 state: 140 step: 169 ret: 0
674 bash-394 [005] 95.540: cpuhp_enter: cpu: 0004 target: 169 step: 140 (cpuhp_kick_ap_work)
675 cpuhp/4-31 [004] 95.541: cpuhp_enter: cpu: 0004 target: 169 step: 141 (acpi_soft_cpu_online)
676 cpuhp/4-31 [004] 95.542: cpuhp_exit: cpu: 0004 state: 141 step: 141 ret: 0
677 cpuhp/4-31 [004] 95.543: cpuhp_enter: cpu: 0004 target: 169 step: 142 (cacheinfo_cpu_online)
678 cpuhp/4-31 [004] 95.544: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
679 …cpuhp/4-31 [004] 95.545: cpuhp_multi_enter: cpu: 0004 target: 169 step: 143 (virtnet_cpu_online)
680 cpuhp/4-31 [004] 95.546: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
681 cpuhp/4-31 [004] 95.547: cpuhp_enter: cpu: 0004 target: 169 step: 144 (mce_cpu_online)
682 cpuhp/4-31 [004] 95.548: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
683 cpuhp/4-31 [004] 95.549: cpuhp_enter: cpu: 0004 target: 169 step: 145 (console_cpu_notify)
684 cpuhp/4-31 [004] 95.550: cpuhp_exit: cpu: 0004 state: 145 step: 145 ret: 0
685 cpuhp/4-31 [004] 95.551: cpuhp_enter: cpu: 0004 target: 169 step: 168 (sched_cpu_activate)
686 cpuhp/4-31 [004] 95.552: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
687 bash-394 [005] 95.553: cpuhp_exit: cpu: 0004 state: 169 step: 140 ret: 0
702 Arch interface to bring up a CPU
705 Arch interface to shutdown a CPU, no more interrupts can be handled by the
709 This actually supposed to ensure death of the CPU. Actually look at some
710 example code in other arch that implement CPU hotplug. The processor is taken
718 After CPU successfully onlined or offline udev events are sent. A udev rule like::
720 …SUBSYSTEM=="cpu", DRIVERS=="processor", DEVPATH=="/devices/system/cpu/*", RUN+="the_hotplug_receiv…
728 echo "CPU ${DEVPATH##*/} offline"
732 echo "CPU ${DEVPATH##*/} online"
739 /sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
747 To skip userspace processing of CPU hot un/plug events for kdump
748 (i.e. the unload-then-reload to obtain a current list of CPUs), this sysfs
751 SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
753 For a CPU hot un/plug event, if the architecture supports kernel updates
755 kexec segments, then the rule skips the unload-then-reload of the kdump
761 .. kernel-doc:: include/linux/cpuhotplug.h