1.. SPDX-License-Identifier: GPL-2.0 2.. include:: <isonum.txt> 3 4=============================================== 5``intel_pstate`` CPU Performance Scaling Driver 6=============================================== 7 8:Copyright: |copy| 2017 Intel Corporation 9 10:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com> 11 12 13General Information 14=================== 15 16``intel_pstate`` is a part of the 17:doc:`CPU performance scaling subsystem <cpufreq>` in the Linux kernel 18(``CPUFreq``). It is a scaling driver for the Sandy Bridge and later 19generations of Intel processors. Note, however, that some of those processors 20may not be supported. [To understand ``intel_pstate`` it is necessary to know 21how ``CPUFreq`` works in general, so this is the time to read 22Documentation/admin-guide/pm/cpufreq.rst if you have not done that yet.] 23 24For the processors supported by ``intel_pstate``, the P-state concept is broader 25than just an operating frequency or an operating performance point (see the 26LinuxCon Europe 2015 presentation by Kristen Accardi [1]_ for more 27information about that). For this reason, the representation of P-states used 28by ``intel_pstate`` internally follows the hardware specification (for details 29refer to Intel Software Developer’s Manual [2]_). However, the ``CPUFreq`` core 30uses frequencies for identifying operating performance points of CPUs and 31frequencies are involved in the user space interface exposed by it, so 32``intel_pstate`` maps its internal representation of P-states to frequencies too 33(fortunately, that mapping is unambiguous). At the same time, it would not be 34practical for ``intel_pstate`` to supply the ``CPUFreq`` core with a table of 35available frequencies due to the possible size of it, so the driver does not do 36that. Some functionality of the core is limited by that. 37 38Since the hardware P-state selection interface used by ``intel_pstate`` is 39available at the logical CPU level, the driver always works with individual 40CPUs. Consequently, if ``intel_pstate`` is in use, every ``CPUFreq`` policy 41object corresponds to one logical CPU and ``CPUFreq`` policies are effectively 42equivalent to CPUs. In particular, this means that they become "inactive" every 43time the corresponding CPU is taken offline and need to be re-initialized when 44it goes back online. 45 46``intel_pstate`` is not modular, so it cannot be unloaded, which means that the 47only way to pass early-configuration-time parameters to it is via the kernel 48command line. However, its configuration can be adjusted via ``sysfs`` to a 49great extent. In some configurations it even is possible to unregister it via 50``sysfs`` which allows another ``CPUFreq`` scaling driver to be loaded and 51registered (see :ref:`below <status_attr>`). 52 53.. _operation_modes: 54 55Operation Modes 56=============== 57 58``intel_pstate`` can operate in two different modes, active or passive. In the 59active mode, it uses its own internal performance scaling governor algorithm or 60allows the hardware to do performance scaling by itself, while in the passive 61mode it responds to requests made by a generic ``CPUFreq`` governor implementing 62a certain performance scaling algorithm. Which of them will be in effect 63depends on what kernel command line options are used and on the capabilities of 64the processor. 65 66.. _active_mode: 67 68Active Mode 69----------- 70 71This is the default operation mode of ``intel_pstate`` for processors with 72hardware-managed P-states (HWP) support. If it works in this mode, the 73``scaling_driver`` policy attribute in ``sysfs`` for all ``CPUFreq`` policies 74contains the string "intel_pstate". 75 76In this mode the driver bypasses the scaling governors layer of ``CPUFreq`` and 77provides its own scaling algorithms for P-state selection. Those algorithms 78can be applied to ``CPUFreq`` policies in the same way as generic scaling 79governors (that is, through the ``scaling_governor`` policy attribute in 80``sysfs``). [Note that different P-state selection algorithms may be chosen for 81different policies, but that is not recommended.] 82 83They are not generic scaling governors, but their names are the same as the 84names of some of those governors. Moreover, confusingly enough, they generally 85do not work in the same way as the generic governors they share the names with. 86For example, the ``powersave`` P-state selection algorithm provided by 87``intel_pstate`` is not a counterpart of the generic ``powersave`` governor 88(roughly, it corresponds to the ``schedutil`` and ``ondemand`` governors). 89 90There are two P-state selection algorithms provided by ``intel_pstate`` in the 91active mode: ``powersave`` and ``performance``. The way they both operate 92depends on whether or not the hardware-managed P-states (HWP) feature has been 93enabled in the processor and possibly on the processor model. 94 95Which of the P-state selection algorithms is used by default depends on the 96:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option. 97Namely, if that option is set, the ``performance`` algorithm will be used by 98default, and the other one will be used by default if it is not set. 99 100.. _active_mode_hwp: 101 102Active Mode With HWP 103~~~~~~~~~~~~~~~~~~~~ 104 105If the processor supports the HWP feature, it will be enabled during the 106processor initialization and cannot be disabled after that. It is possible 107to avoid enabling it by passing the ``intel_pstate=no_hwp`` argument to the 108kernel in the command line. 109 110If the HWP feature has been enabled, ``intel_pstate`` relies on the processor to 111select P-states by itself, but still it can give hints to the processor's 112internal P-state selection logic. What those hints are depends on which P-state 113selection algorithm has been applied to the given policy (or to the CPU it 114corresponds to). 115 116Even though the P-state selection is carried out by the processor automatically, 117``intel_pstate`` registers utilization update callbacks with the CPU scheduler 118in this mode. However, they are not used for running a P-state selection 119algorithm, but for periodic updates of the current CPU frequency information to 120be made available from the ``scaling_cur_freq`` policy attribute in ``sysfs``. 121 122HWP + ``performance`` 123..................... 124 125In this configuration ``intel_pstate`` will write 0 to the processor's 126Energy-Performance Preference (EPP) knob (if supported) or its 127Energy-Performance Bias (EPB) knob (otherwise), which means that the processor's 128internal P-state selection logic is expected to focus entirely on performance. 129 130This will override the EPP/EPB setting coming from the ``sysfs`` interface 131(see :ref:`energy_performance_hints` below). Moreover, any attempts to change 132the EPP/EPB to a value different from 0 ("performance") via ``sysfs`` in this 133configuration will be rejected. 134 135Also, in this configuration the range of P-states available to the processor's 136internal P-state selection logic is always restricted to the upper boundary 137(that is, the maximum P-state that the driver is allowed to use). 138 139HWP + ``powersave`` 140................... 141 142In this configuration ``intel_pstate`` will set the processor's 143Energy-Performance Preference (EPP) knob (if supported) or its 144Energy-Performance Bias (EPB) knob (otherwise) to whatever value it was 145previously set to via ``sysfs`` (or whatever default value it was 146set to by the platform firmware). This usually causes the processor's 147internal P-state selection logic to be less performance-focused. 148 149Active Mode Without HWP 150~~~~~~~~~~~~~~~~~~~~~~~ 151 152This operation mode is optional for processors that do not support the HWP 153feature or when the ``intel_pstate=no_hwp`` argument is passed to the kernel in 154the command line. The active mode is used in those cases if the 155``intel_pstate=active`` argument is passed to the kernel in the command line. 156In this mode ``intel_pstate`` may refuse to work with processors that are not 157recognized by it. [Note that ``intel_pstate`` will never refuse to work with 158any processor with the HWP feature enabled.] 159 160In this mode ``intel_pstate`` registers utilization update callbacks with the 161CPU scheduler in order to run a P-state selection algorithm, either 162``powersave`` or ``performance``, depending on the ``scaling_governor`` policy 163setting in ``sysfs``. The current CPU frequency information to be made 164available from the ``scaling_cur_freq`` policy attribute in ``sysfs`` is 165periodically updated by those utilization update callbacks too. 166 167``performance`` 168............... 169 170Without HWP, this P-state selection algorithm is always the same regardless of 171the processor model and platform configuration. 172 173It selects the maximum P-state it is allowed to use, subject to limits set via 174``sysfs``, every time the driver configuration for the given CPU is updated 175(e.g. via ``sysfs``). 176 177This is the default P-state selection algorithm if the 178:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option 179is set. 180 181``powersave`` 182............. 183 184Without HWP, this P-state selection algorithm is similar to the algorithm 185implemented by the generic ``schedutil`` scaling governor except that the 186utilization metric used by it is based on numbers coming from feedback 187registers of the CPU. It generally selects P-states proportional to the 188current CPU utilization. 189 190This algorithm is run by the driver's utilization update callback for the 191given CPU when it is invoked by the CPU scheduler, but not more often than 192every 10 ms. Like in the ``performance`` case, the hardware configuration 193is not touched if the new P-state turns out to be the same as the current 194one. 195 196This is the default P-state selection algorithm if the 197:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option 198is not set. 199 200.. _passive_mode: 201 202Passive Mode 203------------ 204 205This is the default operation mode of ``intel_pstate`` for processors without 206hardware-managed P-states (HWP) support. It is always used if the 207``intel_pstate=passive`` argument is passed to the kernel in the command line 208regardless of whether or not the given processor supports HWP. [Note that the 209``intel_pstate=no_hwp`` setting causes the driver to start in the passive mode 210if it is not combined with ``intel_pstate=active``.] Like in the active mode 211without HWP support, in this mode ``intel_pstate`` may refuse to work with 212processors that are not recognized by it if HWP is prevented from being enabled 213through the kernel command line. 214 215If the driver works in this mode, the ``scaling_driver`` policy attribute in 216``sysfs`` for all ``CPUFreq`` policies contains the string "intel_cpufreq". 217Then, the driver behaves like a regular ``CPUFreq`` scaling driver. That is, 218it is invoked by generic scaling governors when necessary to talk to the 219hardware in order to change the P-state of a CPU (in particular, the 220``schedutil`` governor can invoke it directly from scheduler context). 221 222While in this mode, ``intel_pstate`` can be used with all of the (generic) 223scaling governors listed by the ``scaling_available_governors`` policy attribute 224in ``sysfs`` (and the P-state selection algorithms described above are not 225used). Then, it is responsible for the configuration of policy objects 226corresponding to CPUs and provides the ``CPUFreq`` core (and the scaling 227governors attached to the policy objects) with accurate information on the 228maximum and minimum operating frequencies supported by the hardware (including 229the so-called "turbo" frequency ranges). In other words, in the passive mode 230the entire range of available P-states is exposed by ``intel_pstate`` to the 231``CPUFreq`` core. However, in this mode the driver does not register 232utilization update callbacks with the CPU scheduler and the ``scaling_cur_freq`` 233information comes from the ``CPUFreq`` core (and is the last frequency selected 234by the current scaling governor for the given policy). 235 236 237.. _turbo: 238 239Turbo P-states Support 240====================== 241 242In the majority of cases, the entire range of P-states available to 243``intel_pstate`` can be divided into two sub-ranges that correspond to 244different types of processor behavior, above and below a boundary that 245will be referred to as the "turbo threshold" in what follows. 246 247The P-states above the turbo threshold are referred to as "turbo P-states" and 248the whole sub-range of P-states they belong to is referred to as the "turbo 249range". These names are related to the Turbo Boost technology allowing a 250multicore processor to opportunistically increase the P-state of one or more 251cores if there is enough power to do that and if that is not going to cause the 252thermal envelope of the processor package to be exceeded. 253 254Specifically, if software sets the P-state of a CPU core within the turbo range 255(that is, above the turbo threshold), the processor is permitted to take over 256performance scaling control for that core and put it into turbo P-states of its 257choice going forward. However, that permission is interpreted differently by 258different processor generations. Namely, the Sandy Bridge generation of 259processors will never use any P-states above the last one set by software for 260the given core, even if it is within the turbo range, whereas all of the later 261processor generations will take it as a license to use any P-states from the 262turbo range, even above the one set by software. In other words, on those 263processors setting any P-state from the turbo range will enable the processor 264to put the given core into all turbo P-states up to and including the maximum 265supported one as it sees fit. 266 267One important property of turbo P-states is that they are not sustainable. More 268precisely, there is no guarantee that any CPUs will be able to stay in any of 269those states indefinitely, because the power distribution within the processor 270package may change over time or the thermal envelope it was designed for might 271be exceeded if a turbo P-state was used for too long. 272 273In turn, the P-states below the turbo threshold generally are sustainable. In 274fact, if one of them is set by software, the processor is not expected to change 275it to a lower one unless in a thermal stress or a power limit violation 276situation (a higher P-state may still be used if it is set for another CPU in 277the same package at the same time, for example). 278 279Some processors allow multiple cores to be in turbo P-states at the same time, 280but the maximum P-state that can be set for them generally depends on the number 281of cores running concurrently. The maximum turbo P-state that can be set for 3 282cores at the same time usually is lower than the analogous maximum P-state for 2832 cores, which in turn usually is lower than the maximum turbo P-state that can 284be set for 1 core. The one-core maximum turbo P-state is thus the maximum 285supported one overall. 286 287The maximum supported turbo P-state, the turbo threshold (the maximum supported 288non-turbo P-state) and the minimum supported P-state are specific to the 289processor model and can be determined by reading the processor's model-specific 290registers (MSRs). Moreover, some processors support the Configurable TDP 291(Thermal Design Power) feature and, when that feature is enabled, the turbo 292threshold effectively becomes a configurable value that can be set by the 293platform firmware. 294 295Unlike ``_PSS`` objects in the ACPI tables, ``intel_pstate`` always exposes 296the entire range of available P-states, including the whole turbo range, to the 297``CPUFreq`` core and (in the passive mode) to generic scaling governors. This 298generally causes turbo P-states to be set more often when ``intel_pstate`` is 299used relative to ACPI-based CPU performance scaling (see 300:ref:`below <acpi-cpufreq>` for more information). 301 302Moreover, since ``intel_pstate`` always knows what the real turbo threshold is 303(even if the Configurable TDP feature is enabled in the processor), its 304``no_turbo`` attribute in ``sysfs`` (described :ref:`below <no_turbo_attr>`) should 305work as expected in all cases (that is, if set to disable turbo P-states, it 306always should prevent ``intel_pstate`` from using them). 307 308 309Processor Support 310================= 311 312To handle a given processor ``intel_pstate`` requires a number of different 313pieces of information on it to be known, including: 314 315 * The minimum supported P-state. 316 317 * The maximum supported :ref:`non-turbo P-state <turbo>`. 318 319 * Whether or not turbo P-states are supported at all. 320 321 * The maximum supported :ref:`one-core turbo P-state <turbo>` (if turbo 322 P-states are supported). 323 324 * The scaling formula to translate the driver's internal representation 325 of P-states into frequencies and the other way around. 326 327Generally, ways to obtain that information are specific to the processor model 328or family. Although it often is possible to obtain all of it from the processor 329itself (using model-specific registers), there are cases in which hardware 330manuals need to be consulted to get to it too. 331 332For this reason, there is a list of supported processors in ``intel_pstate`` and 333the driver initialization will fail if the detected processor is not in that 334list, unless it supports the HWP feature. [The interface to obtain all of the 335information listed above is the same for all of the processors supporting the 336HWP feature, which is why ``intel_pstate`` works with all of them.] 337 338 339Support for Hybrid Processors 340============================= 341 342Some processors supported by ``intel_pstate`` contain two or more types of CPU 343cores differing by the maximum turbo P-state, performance vs power characteristics, 344cache sizes, and possibly other properties. They are commonly referred to as 345hybrid processors. To support them, ``intel_pstate`` requires HWP to be enabled 346and it assumes the HWP performance units to be the same for all CPUs in the 347system, so a given HWP performance level always represents approximately the 348same physical performance regardless of the core (CPU) type. 349 350Hybrid Processors with SMT 351-------------------------- 352 353On systems where SMT (Simultaneous Multithreading), also referred to as 354HyperThreading (HT) in the context of Intel processors, is enabled on at least 355one core, ``intel_pstate`` assigns performance-based priorities to CPUs. Namely, 356the priority of a given CPU reflects its highest HWP performance level which 357causes the CPU scheduler to generally prefer more performant CPUs, so the less 358performant CPUs are used when the other ones are fully loaded. SMT siblings 359(that is, logical CPUs sharing one physical core) are given the same priority. 360The scheduler can pull tasks from lower-priority cores and place them on any 361sibling. Since the scheduler spreads tasks among physical cores, tasks will be 362placed on the SMT siblings of physical cores only after all physical cores are 363busy. 364 365This approach maximizes performance in the majority of cases, but unfortunately 366it also leads to excessive energy usage in some important scenarios, like video 367playback, which is not generally desirable. While there is no other viable 368choice with SMT enabled because the effective capacity and utilization of SMT 369siblings are hard to determine, hybrid processors without SMT can be handled in 370more energy-efficient ways. 371 372.. _CAS: 373 374Capacity-Aware Scheduling Support 375--------------------------------- 376 377The capacity-aware scheduling (CAS) support in the CPU scheduler is enabled by 378``intel_pstate`` by default on hybrid processors without SMT. CAS generally 379causes the scheduler to put tasks on a CPU so long as there is a sufficient 380amount of spare capacity on it, and if the utilization of a given task is too 381high for it, the task will need to go somewhere else. 382 383Since CAS takes CPU capacities into account, it does not require CPU 384prioritization and it allows tasks to be distributed more symmetrically among 385the more performant and less performant CPUs. Once placed on a CPU with enough 386capacity to accommodate it, a task may just continue to run there regardless of 387whether or not the other CPUs are fully loaded, so on average CAS reduces the 388utilization of the more performant CPUs which causes the energy usage to be more 389balanced because the more performant CPUs are generally less energy-efficient 390than the less performant ones. 391 392In order to use CAS, the scheduler needs to know the capacity of each CPU in 393the system and it needs to be able to compute scale-invariant utilization of 394CPUs, so ``intel_pstate`` provides it with the requisite information. 395 396First of all, the capacity of each CPU is represented by the ratio of its highest 397HWP performance level, multiplied by 1024, to the highest HWP performance level 398of the most performant CPU in the system, which works because the HWP performance 399units are the same for all CPUs. Second, the frequency-invariance computations, 400carried out by the scheduler to always express CPU utilization in the same units 401regardless of the frequency it is currently running at, are adjusted to take the 402CPU capacity into account. All of this happens when ``intel_pstate`` has 403registered itself with the ``CPUFreq`` core and it has figured out that it is 404running on a hybrid processor without SMT. 405 406Energy-Aware Scheduling Support 407------------------------------- 408 409If ``CONFIG_ENERGY_MODEL`` has been set during kernel configuration and 410``intel_pstate`` runs on a hybrid processor without SMT, in addition to enabling 411:ref:`CAS` it registers an Energy Model for the processor. This allows the 412Energy-Aware Scheduling (EAS) support to be enabled in the CPU scheduler if 413``schedutil`` is used as the ``CPUFreq`` governor which requires ``intel_pstate`` 414to operate in the :ref:`passive mode <passive_mode>`. 415 416The Energy Model registered by ``intel_pstate`` is artificial (that is, it is 417based on abstract cost values and it does not include any real power numbers) 418and it is relatively simple to avoid unnecessary computations in the scheduler. 419There is a performance domain in it for every CPU in the system and the cost 420values for these performance domains have been chosen so that running a task on 421a less performant (small) CPU appears to be always cheaper than running that 422task on a more performant (big) CPU. However, for two CPUs of the same type, 423the cost difference depends on their current utilization, and the CPU whose 424current utilization is higher generally appears to be a more expensive 425destination for a given task. This helps to balance the load among CPUs of the 426same type. 427 428Since EAS works on top of CAS, high-utilization tasks are always migrated to 429CPUs with enough capacity to accommodate them, but thanks to EAS, low-utilization 430tasks tend to be placed on the CPUs that look less expensive to the scheduler. 431Effectively, this causes the less performant and less loaded CPUs to be 432preferred as long as they have enough spare capacity to run the given task 433which generally leads to reduced energy usage. 434 435The Energy Model created by ``intel_pstate`` can be inspected by looking at 436the ``energy_model`` directory in ``debugfs`` (typlically mounted on 437``/sys/kernel/debug/``). 438 439 440User Space Interface in ``sysfs`` 441================================= 442 443.. _global_attributes: 444 445Global Attributes 446----------------- 447 448``intel_pstate`` exposes several global attributes (files) in ``sysfs`` to 449control its functionality at the system level. They are located in the 450``/sys/devices/system/cpu/intel_pstate/`` directory and affect all CPUs. 451 452Some of them are not present if the ``intel_pstate=per_cpu_perf_limits`` 453argument is passed to the kernel in the command line. 454 455``max_perf_pct`` 456 Maximum P-state the driver is allowed to set in percent of the 457 maximum supported performance level (the highest supported :ref:`turbo 458 P-state <turbo>`). 459 460 This attribute will not be exposed if the 461 ``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel 462 command line. 463 464``min_perf_pct`` 465 Minimum P-state the driver is allowed to set in percent of the 466 maximum supported performance level (the highest supported :ref:`turbo 467 P-state <turbo>`). 468 469 This attribute will not be exposed if the 470 ``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel 471 command line. 472 473``num_pstates`` 474 Number of P-states supported by the processor (between 0 and 255 475 inclusive) including both turbo and non-turbo P-states (see 476 :ref:`turbo`). 477 478 This attribute is present only if the value exposed by it is the same 479 for all of the CPUs in the system. 480 481 The value of this attribute is not affected by the ``no_turbo`` 482 setting described :ref:`below <no_turbo_attr>`. 483 484 This attribute is read-only. 485 486``turbo_pct`` 487 Ratio of the :ref:`turbo range <turbo>` size to the size of the entire 488 range of supported P-states, in percent. 489 490 This attribute is present only if the value exposed by it is the same 491 for all of the CPUs in the system. 492 493 This attribute is read-only. 494 495.. _no_turbo_attr: 496 497``no_turbo`` 498 If set (equal to 1), the driver is not allowed to set any turbo P-states 499 (see :ref:`turbo`). If unset (equal to 0, which is the 500 default), turbo P-states can be set by the driver. 501 [Note that ``intel_pstate`` does not support the general ``boost`` 502 attribute (supported by some other scaling drivers) which is replaced 503 by this one.] 504 505 This attribute does not affect the maximum supported frequency value 506 supplied to the ``CPUFreq`` core and exposed via the policy interface, 507 but it affects the maximum possible value of per-policy P-state limits 508 (see :ref:`policy_attributes_interpretation` below for details). 509 510``hwp_dynamic_boost`` 511 This attribute is only present if ``intel_pstate`` works in the 512 :ref:`active mode with the HWP feature enabled <active_mode_hwp>` in 513 the processor. If set (equal to 1), it causes the minimum P-state limit 514 to be increased dynamically for a short time whenever a task previously 515 waiting on I/O is selected to run on a given logical CPU (the purpose 516 of this mechanism is to improve performance). 517 518 This setting has no effect on logical CPUs whose minimum P-state limit 519 is directly set to the highest non-turbo P-state or above it. 520 521.. _status_attr: 522 523``status`` 524 Operation mode of the driver: "active", "passive" or "off". 525 526 "active" 527 The driver is functional and in the :ref:`active mode 528 <active_mode>`. 529 530 "passive" 531 The driver is functional and in the :ref:`passive mode 532 <passive_mode>`. 533 534 "off" 535 The driver is not functional (it is not registered as a scaling 536 driver with the ``CPUFreq`` core). 537 538 This attribute can be written to in order to change the driver's 539 operation mode or to unregister it. The string written to it must be 540 one of the possible values of it and, if successful, the write will 541 cause the driver to switch over to the operation mode represented by 542 that string - or to be unregistered in the "off" case. [Actually, 543 switching over from the active mode to the passive mode or the other 544 way around causes the driver to be unregistered and registered again 545 with a different set of callbacks, so all of its settings (the global 546 as well as the per-policy ones) are then reset to their default 547 values, possibly depending on the target operation mode.] 548 549``energy_efficiency`` 550 This attribute is only present on platforms with CPUs matching the Kaby 551 Lake or Coffee Lake desktop CPU model. By default, energy-efficiency 552 optimizations are disabled on these CPU models if HWP is enabled. 553 Enabling energy-efficiency optimizations may limit maximum operating 554 frequency with or without the HWP feature. With HWP enabled, the 555 optimizations are done only in the turbo frequency range. Without it, 556 they are done in the entire available frequency range. Setting this 557 attribute to "1" enables the energy-efficiency optimizations and setting 558 to "0" disables them. 559 560.. _policy_attributes_interpretation: 561 562Interpretation of Policy Attributes 563----------------------------------- 564 565The interpretation of some ``CPUFreq`` policy attributes described in 566Documentation/admin-guide/pm/cpufreq.rst is special with ``intel_pstate`` 567as the current scaling driver and it generally depends on the driver's 568:ref:`operation mode <operation_modes>`. 569 570First of all, the values of the ``cpuinfo_max_freq``, ``cpuinfo_min_freq`` and 571``scaling_cur_freq`` attributes are produced by applying a processor-specific 572multiplier to the internal P-state representation used by ``intel_pstate``. 573Also, the values of the ``scaling_max_freq`` and ``scaling_min_freq`` 574attributes are capped by the frequency corresponding to the maximum P-state that 575the driver is allowed to set. 576 577If the ``no_turbo`` :ref:`global attribute <no_turbo_attr>` is set, the driver 578is not allowed to use turbo P-states, so the maximum value of 579``scaling_max_freq`` and ``scaling_min_freq`` is limited to the maximum 580non-turbo P-state frequency. 581Accordingly, setting ``no_turbo`` causes ``scaling_max_freq`` and 582``scaling_min_freq`` to go down to that value if they were above it before. 583However, the old values of ``scaling_max_freq`` and ``scaling_min_freq`` will be 584restored after unsetting ``no_turbo``, unless these attributes have been written 585to after ``no_turbo`` was set. 586 587If ``no_turbo`` is not set, the maximum possible value of ``scaling_max_freq`` 588and ``scaling_min_freq`` corresponds to the maximum supported turbo P-state, 589which also is the value of ``cpuinfo_max_freq`` in either case. 590 591Next, the following policy attributes have special meaning if 592``intel_pstate`` works in the :ref:`active mode <active_mode>`: 593 594``scaling_available_governors`` 595 List of P-state selection algorithms provided by ``intel_pstate``. 596 597``scaling_governor`` 598 P-state selection algorithm provided by ``intel_pstate`` currently in 599 use with the given policy. 600 601``scaling_cur_freq`` 602 Frequency of the average P-state of the CPU represented by the given 603 policy for the time interval between the last two invocations of the 604 driver's utilization update callback by the CPU scheduler for that CPU. 605 606One more policy attribute is present if the HWP feature is enabled in the 607processor: 608 609``base_frequency`` 610 Shows the base frequency of the CPU. Any frequency above this will be 611 in the turbo frequency range. 612 613The meaning of these attributes in the :ref:`passive mode <passive_mode>` is the 614same as for other scaling drivers. 615 616Additionally, the value of the ``scaling_driver`` attribute for ``intel_pstate`` 617depends on the operation mode of the driver. Namely, it is either 618"intel_pstate" (in the :ref:`active mode <active_mode>`) or "intel_cpufreq" 619(in the :ref:`passive mode <passive_mode>`). 620 621.. _pstate_limits_coordination: 622 623Coordination of P-State Limits 624------------------------------ 625 626``intel_pstate`` allows P-state limits to be set in two ways: with the help of 627the ``max_perf_pct`` and ``min_perf_pct`` :ref:`global attributes 628<global_attributes>` or via the ``scaling_max_freq`` and ``scaling_min_freq`` 629``CPUFreq`` policy attributes. The coordination between those limits is based 630on the following rules, regardless of the current operation mode of the driver: 631 632 1. All CPUs are affected by the global limits (that is, none of them can be 633 requested to run faster than the global maximum and none of them can be 634 requested to run slower than the global minimum). 635 636 2. Each individual CPU is affected by its own per-policy limits (that is, it 637 cannot be requested to run faster than its own per-policy maximum and it 638 cannot be requested to run slower than its own per-policy minimum). The 639 effective performance depends on whether the platform supports per core 640 P-states, hyper-threading is enabled and on current performance requests 641 from other CPUs. When platform doesn't support per core P-states, the 642 effective performance can be more than the policy limits set on a CPU, if 643 other CPUs are requesting higher performance at that moment. Even with per 644 core P-states support, when hyper-threading is enabled, if the sibling CPU 645 is requesting higher performance, the other siblings will get higher 646 performance than their policy limits. 647 648 3. The global and per-policy limits can be set independently. 649 650In the :ref:`active mode with the HWP feature enabled <active_mode_hwp>`, the 651resulting effective values are written into hardware registers whenever the 652limits change in order to request its internal P-state selection logic to always 653set P-states within these limits. Otherwise, the limits are taken into account 654by scaling governors (in the :ref:`passive mode <passive_mode>`) and by the 655driver every time before setting a new P-state for a CPU. 656 657Additionally, if the ``intel_pstate=per_cpu_perf_limits`` command line argument 658is passed to the kernel, ``max_perf_pct`` and ``min_perf_pct`` are not exposed 659at all and the only way to set the limits is by using the policy attributes. 660 661.. _energy_performance_hints: 662 663Energy vs Performance Hints 664--------------------------- 665 666If the hardware-managed P-states (HWP) is enabled in the processor, additional 667attributes, intended to allow user space to help ``intel_pstate`` to adjust the 668processor's internal P-state selection logic by focusing it on performance or on 669energy-efficiency, or somewhere between the two extremes, are present in every 670``CPUFreq`` policy directory in ``sysfs``. They are : 671 672``energy_performance_preference`` 673 Current value of the energy vs performance hint for the given policy 674 (or the CPU represented by it). 675 676 The hint can be changed by writing to this attribute. 677 678``energy_performance_available_preferences`` 679 List of strings that can be written to the 680 ``energy_performance_preference`` attribute. 681 682 They represent different energy vs performance hints and should be 683 self-explanatory, except that ``default`` represents whatever hint 684 value was set by the platform firmware. 685 686Strings written to the ``energy_performance_preference`` attribute are 687internally translated to integer values written to the processor's 688Energy-Performance Preference (EPP) knob (if supported) or its 689Energy-Performance Bias (EPB) knob. It is also possible to write a positive 690integer value between 0 to 255, if the EPP feature is present. If the EPP 691feature is not present, writing integer value to this attribute is not 692supported. In this case, user can use the 693"/sys/devices/system/cpu/cpu*/power/energy_perf_bias" interface. 694 695[Note that tasks may by migrated from one CPU to another by the scheduler's 696load-balancing algorithm and if different energy vs performance hints are 697set for those CPUs, that may lead to undesirable outcomes. To avoid such 698issues it is better to set the same energy vs performance hint for all CPUs 699or to pin every task potentially sensitive to them to a specific CPU.] 700 701.. _acpi-cpufreq: 702 703``intel_pstate`` vs ``acpi-cpufreq`` 704==================================== 705 706On the majority of systems supported by ``intel_pstate``, the ACPI tables 707provided by the platform firmware contain ``_PSS`` objects returning information 708that can be used for CPU performance scaling (refer to the ACPI specification 709[3]_ for details on the ``_PSS`` objects and the format of the information 710returned by them). 711 712The information returned by the ACPI ``_PSS`` objects is used by the 713``acpi-cpufreq`` scaling driver. On systems supported by ``intel_pstate`` 714the ``acpi-cpufreq`` driver uses the same hardware CPU performance scaling 715interface, but the set of P-states it can use is limited by the ``_PSS`` 716output. 717 718On those systems each ``_PSS`` object returns a list of P-states supported by 719the corresponding CPU which basically is a subset of the P-states range that can 720be used by ``intel_pstate`` on the same system, with one exception: the whole 721:ref:`turbo range <turbo>` is represented by one item in it (the topmost one). 722By convention, the frequency returned by ``_PSS`` for that item is greater by 7231 MHz than the frequency of the highest non-turbo P-state listed by it, but the 724corresponding P-state representation (following the hardware specification) 725returned for it matches the maximum supported turbo P-state (or is the 726special value 255 meaning essentially "go as high as you can get"). 727 728The list of P-states returned by ``_PSS`` is reflected by the table of 729available frequencies supplied by ``acpi-cpufreq`` to the ``CPUFreq`` core and 730scaling governors and the minimum and maximum supported frequencies reported by 731it come from that list as well. In particular, given the special representation 732of the turbo range described above, this means that the maximum supported 733frequency reported by ``acpi-cpufreq`` is higher by 1 MHz than the frequency 734of the highest supported non-turbo P-state listed by ``_PSS`` which, of course, 735affects decisions made by the scaling governors, except for ``powersave`` and 736``performance``. 737 738For example, if a given governor attempts to select a frequency proportional to 739estimated CPU load and maps the load of 100% to the maximum supported frequency 740(possibly multiplied by a constant), then it will tend to choose P-states below 741the turbo threshold if ``acpi-cpufreq`` is used as the scaling driver, because 742in that case the turbo range corresponds to a small fraction of the frequency 743band it can use (1 MHz vs 1 GHz or more). In consequence, it will only go to 744the turbo range for the highest loads and the other loads above 50% that might 745benefit from running at turbo frequencies will be given non-turbo P-states 746instead. 747 748One more issue related to that may appear on systems supporting the 749:ref:`Configurable TDP feature <turbo>` allowing the platform firmware to set 750the turbo threshold. Namely, if that is not coordinated with the lists of 751P-states returned by ``_PSS`` properly, there may be more than one item 752corresponding to a turbo P-state in those lists and there may be a problem with 753avoiding the turbo range (if desirable or necessary). Usually, to avoid using 754turbo P-states overall, ``acpi-cpufreq`` simply avoids using the topmost state 755listed by ``_PSS``, but that is not sufficient when there are other turbo 756P-states in the list returned by it. 757 758Apart from the above, ``acpi-cpufreq`` works like ``intel_pstate`` in the 759:ref:`passive mode <passive_mode>`, except that the number of P-states it can 760set is limited to the ones listed by the ACPI ``_PSS`` objects. 761 762 763Kernel Command Line Options for ``intel_pstate`` 764================================================ 765 766Several kernel command line options can be used to pass early-configuration-time 767parameters to ``intel_pstate`` in order to enforce specific behavior of it. All 768of them have to be prepended with the ``intel_pstate=`` prefix. 769 770``disable`` 771 Do not register ``intel_pstate`` as the scaling driver even if the 772 processor is supported by it. 773 774``active`` 775 Register ``intel_pstate`` in the :ref:`active mode <active_mode>` to 776 start with. 777 778``passive`` 779 Register ``intel_pstate`` in the :ref:`passive mode <passive_mode>` to 780 start with. 781 782``force`` 783 Register ``intel_pstate`` as the scaling driver instead of 784 ``acpi-cpufreq`` even if the latter is preferred on the given system. 785 786 This may prevent some platform features (such as thermal controls and 787 power capping) that rely on the availability of ACPI P-states 788 information from functioning as expected, so it should be used with 789 caution. 790 791 This option does not work with processors that are not supported by 792 ``intel_pstate`` and on platforms where the ``pcc-cpufreq`` scaling 793 driver is used instead of ``acpi-cpufreq``. 794 795``no_hwp`` 796 Do not enable the hardware-managed P-states (HWP) feature even if it is 797 supported by the processor. 798 799``hwp_only`` 800 Register ``intel_pstate`` as the scaling driver only if the 801 hardware-managed P-states (HWP) feature is supported by the processor. 802 803``support_acpi_ppc`` 804 Take ACPI ``_PPC`` performance limits into account. 805 806 If the preferred power management profile in the FADT (Fixed ACPI 807 Description Table) is set to "Enterprise Server" or "Performance 808 Server", the ACPI ``_PPC`` limits are taken into account by default 809 and this option has no effect. 810 811``per_cpu_perf_limits`` 812 Use per-logical-CPU P-State limits (see 813 :ref:`pstate_limits_coordination` for details). 814 815``no_cas`` 816 Do not enable :ref:`capacity-aware scheduling <CAS>` which is enabled 817 by default on hybrid systems without SMT. 818 819Diagnostics and Tuning 820====================== 821 822Trace Events 823------------ 824 825There are two static trace events that can be used for ``intel_pstate`` 826diagnostics. One of them is the ``cpu_frequency`` trace event generally used 827by ``CPUFreq``, and the other one is the ``pstate_sample`` trace event specific 828to ``intel_pstate``. Both of them are triggered by ``intel_pstate`` only if 829it works in the :ref:`active mode <active_mode>`. 830 831The following sequence of shell commands can be used to enable them and see 832their output (if the kernel is generally configured to support event tracing):: 833 834 # cd /sys/kernel/tracing/ 835 # echo 1 > events/power/pstate_sample/enable 836 # echo 1 > events/power/cpu_frequency/enable 837 # cat trace 838 gnome-terminal--4510 [001] ..s. 1177.680733: pstate_sample: core_busy=107 scaled=94 from=26 to=26 mperf=1143818 aperf=1230607 tsc=29838618 freq=2474476 839 cat-5235 [002] ..s. 1177.681723: cpu_frequency: state=2900000 cpu_id=2 840 841If ``intel_pstate`` works in the :ref:`passive mode <passive_mode>`, the 842``cpu_frequency`` trace event will be triggered either by the ``schedutil`` 843scaling governor (for the policies it is attached to), or by the ``CPUFreq`` 844core (for the policies with other scaling governors). 845 846``ftrace`` 847---------- 848 849The ``ftrace`` interface can be used for low-level diagnostics of 850``intel_pstate``. For example, to check how often the function to set a 851P-state is called, the ``ftrace`` filter can be set to 852:c:func:`intel_pstate_set_pstate`:: 853 854 # cd /sys/kernel/tracing/ 855 # cat available_filter_functions | grep -i pstate 856 intel_pstate_set_pstate 857 intel_pstate_cpu_init 858 ... 859 # echo intel_pstate_set_pstate > set_ftrace_filter 860 # echo function > current_tracer 861 # cat trace | head -15 862 # tracer: function 863 # 864 # entries-in-buffer/entries-written: 80/80 #P:4 865 # 866 # _-----=> irqs-off 867 # / _----=> need-resched 868 # | / _---=> hardirq/softirq 869 # || / _--=> preempt-depth 870 # ||| / delay 871 # TASK-PID CPU# |||| TIMESTAMP FUNCTION 872 # | | | |||| | | 873 Xorg-3129 [000] ..s. 2537.644844: intel_pstate_set_pstate <-intel_pstate_timer_func 874 gnome-terminal--4510 [002] ..s. 2537.649844: intel_pstate_set_pstate <-intel_pstate_timer_func 875 gnome-shell-3409 [001] ..s. 2537.650850: intel_pstate_set_pstate <-intel_pstate_timer_func 876 <idle>-0 [000] ..s. 2537.654843: intel_pstate_set_pstate <-intel_pstate_timer_func 877 878 879References 880========== 881 882.. [1] Kristen Accardi, *Balancing Power and Performance in the Linux Kernel*, 883 https://events.static.linuxfound.org/sites/events/files/slides/LinuxConEurope_2015.pdf 884 885.. [2] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3: System Programming Guide*, 886 https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-system-programming-manual-325384.html 887 888.. [3] *Advanced Configuration and Power Interface Specification*, 889 https://uefi.org/sites/default/files/resources/ACPI_6_3_final_Jan30.pdf 890