1.. SPDX-License-Identifier: GPL-2.0 2.. _sched-paravirt: 3 4Preferred CPUs 5============== 6 7In paravirtualized environments CPU overcommit is a common scenario. 8i.e. the sum of virtual CPUs (vCPUs) of all VMs is greater than number of 9physical CPUs (pCPUs). Under such conditions when all or many VMs have 10high utilization, hypervisor won't be able to satisfy the CPU requirement 11and has to context switch within or across VMs. The hypervisor needs to 12preempt one vCPU to run another. This is called vCPU preemption. 13This is more expensive compared to task context switch within a vCPU, since 14hypervisor lacks vCPU context and could preempt a critical section which 15slows forward progress. 16 17In such cases it is better that combined vCPU demand from all VMs is reduced 18by not using some of the vCPUs in each VM. vCPUs where workload can be safely 19scheduled which won't increase any contention for pCPU are called 20"Preferred CPUs". 21 22One of the main design constructs is that preferred CPUs are always 23a subset of active CPUs. In most cases preferred CPUs will be same as 24active CPUs. When there is pCPU contention, Preferred CPUs will reduce 25based on the steal time. When the pCPU contention goes away as indicated 26by steal time, Preferred CPUs could become same as active CPUs again. 27The policy decisions are to be taken by driver. 28For example, steal_governor. Look at its documentation for more 29details. (``drivers/virt/steal_governor.c``) 30 31Scheduling decisions such as wakeup, pushing the task etc, need this 32CPU state info. This is maintained in ``cpu_preferred_mask``. 33vCPUs which are not in ``cpu_preferred_mask`` should be treated as vCPUs which 34should not be used at this moment provided it doesn't break user affinity. 35 36This is achieved by: 37 381. Selecting a preferred CPU at wakeup using fallback mechanism. 392. Pushing the task away from non-preferred CPU at tick. 403. Selecting only preferred CPUs for load balance. 41 42``/sys/devices/system/cpu/preferred`` prints the current ``cpu_preferred_mask`` 43in cpulist format. 44 45Notes: 46 471. This feature is available under ``CONFIG_PREFERRED_CPU``. Driver which 48 makes decisions should enable it. For example, steal_governor driver 49 (``CONFIG_STEAL_GOVERNOR``). On enabling the driver, CPU preferred state 50 can change based on steal time. Without the driver, preferred CPUs is 51 same as active CPUs. 52 532. This feature works for the FAIR class only. 54 553. A pinned task, which can't be moved to preferred CPUs will continue 56 to run based on its affinity. But no load balancing happens if it is affined 57 only on non-preferred CPUs. 58 594. Decision to change the preferred CPU state is driven by the kernel. 60 Hence it shouldn't break user affinities. One of the main reasons why 61 CPU hotplug or Isolated cpuset partitions was not a solution. 62 635. This feature works best only when all the Guest VMs enable the feature as 64 it is a co-operative scheme. If a specific VM doesn't enable this feature 65 it may end up with more CPUs than others, still should lead to better 66 performance when seen from system view. 67 Users who enable this driver must ensure it is enabled in all Guest VMs. 68