1.. SPDX-License-Identifier: GPL-2.0 2.. include:: <isonum.txt> 3 4=============================================== 5``intel_pstate`` CPU Performance Scaling Driver 6=============================================== 7 8:Copyright: |copy| 2017 Intel Corporation 9 10:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com> 11 12 13General Information 14=================== 15 16``intel_pstate`` is a part of the 17:doc:`CPU performance scaling subsystem <cpufreq>` in the Linux kernel 18(``CPUFreq``). It is a scaling driver for the Sandy Bridge and later 19generations of Intel processors. Note, however, that some of those processors 20may not be supported. [To understand ``intel_pstate`` it is necessary to know 21how ``CPUFreq`` works in general, so this is the time to read 22Documentation/admin-guide/pm/cpufreq.rst if you have not done that yet.] 23 24For the processors supported by ``intel_pstate``, the P-state concept is broader 25than just an operating frequency or an operating performance point (see the 26LinuxCon Europe 2015 presentation by Kristen Accardi [1]_ for more 27information about that). For this reason, the representation of P-states used 28by ``intel_pstate`` internally follows the hardware specification (for details 29refer to Intel Software Developer’s Manual [2]_). However, the ``CPUFreq`` core 30uses frequencies for identifying operating performance points of CPUs and 31frequencies are involved in the user space interface exposed by it, so 32``intel_pstate`` maps its internal representation of P-states to frequencies too 33(fortunately, that mapping is unambiguous). At the same time, it would not be 34practical for ``intel_pstate`` to supply the ``CPUFreq`` core with a table of 35available frequencies due to the possible size of it, so the driver does not do 36that. Some functionality of the core is limited by that. 37 38Since the hardware P-state selection interface used by ``intel_pstate`` is 39available at the logical CPU level, the driver always works with individual 40CPUs. Consequently, if ``intel_pstate`` is in use, every ``CPUFreq`` policy 41object corresponds to one logical CPU and ``CPUFreq`` policies are effectively 42equivalent to CPUs. In particular, this means that they become "inactive" every 43time the corresponding CPU is taken offline and need to be re-initialized when 44it goes back online. 45 46``intel_pstate`` is not modular, so it cannot be unloaded, which means that the 47only way to pass early-configuration-time parameters to it is via the kernel 48command line. However, its configuration can be adjusted via ``sysfs`` to a 49great extent. In some configurations it even is possible to unregister it via 50``sysfs`` which allows another ``CPUFreq`` scaling driver to be loaded and 51registered (see :ref:`below <status_attr>`). 52 53.. _operation_modes: 54 55Operation Modes 56=============== 57 58``intel_pstate`` can operate in two different modes, active or passive. In the 59active mode, it uses its own internal performance scaling governor algorithm or 60allows the hardware to do performance scaling by itself, while in the passive 61mode it responds to requests made by a generic ``CPUFreq`` governor implementing 62a certain performance scaling algorithm. Which of them will be in effect 63depends on what kernel command line options are used and on the capabilities of 64the processor. 65 66.. _active_mode: 67 68Active Mode 69----------- 70 71This is the default operation mode of ``intel_pstate`` for processors with 72hardware-managed P-states (HWP) support. If it works in this mode, the 73``scaling_driver`` policy attribute in ``sysfs`` for all ``CPUFreq`` policies 74contains the string "intel_pstate". 75 76In this mode the driver bypasses the scaling governors layer of ``CPUFreq`` and 77provides its own scaling algorithms for P-state selection. Those algorithms 78can be applied to ``CPUFreq`` policies in the same way as generic scaling 79governors (that is, through the ``scaling_governor`` policy attribute in 80``sysfs``). [Note that different P-state selection algorithms may be chosen for 81different policies, but that is not recommended.] 82 83They are not generic scaling governors, but their names are the same as the 84names of some of those governors. Moreover, confusingly enough, they generally 85do not work in the same way as the generic governors they share the names with. 86For example, the ``powersave`` P-state selection algorithm provided by 87``intel_pstate`` is not a counterpart of the generic ``powersave`` governor 88(roughly, it corresponds to the ``schedutil`` and ``ondemand`` governors). 89 90There are two P-state selection algorithms provided by ``intel_pstate`` in the 91active mode: ``powersave`` and ``performance``. The way they both operate 92depends on whether or not the hardware-managed P-states (HWP) feature has been 93enabled in the processor and possibly on the processor model. 94 95Which of the P-state selection algorithms is used by default depends on the 96:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option. 97Namely, if that option is set, the ``performance`` algorithm will be used by 98default, and the other one will be used by default if it is not set. 99 100.. _active_mode_hwp: 101 102Active Mode With HWP 103~~~~~~~~~~~~~~~~~~~~ 104 105If the processor supports the HWP feature, it will be enabled during the 106processor initialization and cannot be disabled after that. It is possible 107to avoid enabling it by passing the ``intel_pstate=no_hwp`` argument to the 108kernel in the command line. 109 110If the HWP feature has been enabled, ``intel_pstate`` relies on the processor to 111select P-states by itself, but still it can give hints to the processor's 112internal P-state selection logic. What those hints are depends on which P-state 113selection algorithm has been applied to the given policy (or to the CPU it 114corresponds to). 115 116Even though the P-state selection is carried out by the processor automatically, 117``intel_pstate`` registers utilization update callbacks with the CPU scheduler 118in this mode. However, they are not used for running a P-state selection 119algorithm, but for periodic updates of the current CPU frequency information to 120be made available from the ``scaling_cur_freq`` policy attribute in ``sysfs``. 121 122HWP + ``performance`` 123..................... 124 125In this configuration ``intel_pstate`` will write 0 to the processor's 126Energy-Performance Preference (EPP) knob (if supported) or its 127Energy-Performance Bias (EPB) knob (otherwise), which means that the processor's 128internal P-state selection logic is expected to focus entirely on performance. 129 130This will override the EPP/EPB setting coming from the ``sysfs`` interface 131(see :ref:`energy_performance_hints` below). Moreover, any attempts to change 132the EPP/EPB to a value different from 0 ("performance") via ``sysfs`` in this 133configuration will be rejected. 134 135Also, in this configuration the range of P-states available to the processor's 136internal P-state selection logic is always restricted to the upper boundary 137(that is, the maximum P-state that the driver is allowed to use). 138 139HWP + ``powersave`` 140................... 141 142In this configuration ``intel_pstate`` will set the processor's 143Energy-Performance Preference (EPP) knob (if supported) or its 144Energy-Performance Bias (EPB) knob (otherwise) to whatever value it was 145previously set to via ``sysfs`` (or whatever default value it was 146set to by the platform firmware). This usually causes the processor's 147internal P-state selection logic to be less performance-focused. 148 149Active Mode Without HWP 150~~~~~~~~~~~~~~~~~~~~~~~ 151 152This operation mode is optional for processors that do not support the HWP 153feature or when the ``intel_pstate=no_hwp`` argument is passed to the kernel in 154the command line. The active mode is used in those cases if the 155``intel_pstate=active`` argument is passed to the kernel in the command line. 156In this mode ``intel_pstate`` may refuse to work with processors that are not 157recognized by it. [Note that ``intel_pstate`` will never refuse to work with 158any processor with the HWP feature enabled.] 159 160In this mode ``intel_pstate`` registers utilization update callbacks with the 161CPU scheduler in order to run a P-state selection algorithm, either 162``powersave`` or ``performance``, depending on the ``scaling_governor`` policy 163setting in ``sysfs``. The current CPU frequency information to be made 164available from the ``scaling_cur_freq`` policy attribute in ``sysfs`` is 165periodically updated by those utilization update callbacks too. 166 167``performance`` 168............... 169 170Without HWP, this P-state selection algorithm is always the same regardless of 171the processor model and platform configuration. 172 173It selects the maximum P-state it is allowed to use, subject to limits set via 174``sysfs``, every time the driver configuration for the given CPU is updated 175(e.g. via ``sysfs``). 176 177This is the default P-state selection algorithm if the 178:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option 179is set. 180 181``powersave`` 182............. 183 184Without HWP, this P-state selection algorithm is similar to the algorithm 185implemented by the generic ``schedutil`` scaling governor except that the 186utilization metric used by it is based on numbers coming from feedback 187registers of the CPU. It generally selects P-states proportional to the 188current CPU utilization. 189 190This algorithm is run by the driver's utilization update callback for the 191given CPU when it is invoked by the CPU scheduler, but not more often than 192every 10 ms. Like in the ``performance`` case, the hardware configuration 193is not touched if the new P-state turns out to be the same as the current 194one. 195 196This is the default P-state selection algorithm if the 197:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option 198is not set. 199 200.. _passive_mode: 201 202Passive Mode 203------------ 204 205This is the default operation mode of ``intel_pstate`` for processors without 206hardware-managed P-states (HWP) support. It is always used if the 207``intel_pstate=passive`` argument is passed to the kernel in the command line 208regardless of whether or not the given processor supports HWP. [Note that the 209``intel_pstate=no_hwp`` setting causes the driver to start in the passive mode 210if it is not combined with ``intel_pstate=active``.] Like in the active mode 211without HWP support, in this mode ``intel_pstate`` may refuse to work with 212processors that are not recognized by it if HWP is prevented from being enabled 213through the kernel command line. 214 215If the driver works in this mode, the ``scaling_driver`` policy attribute in 216``sysfs`` for all ``CPUFreq`` policies contains the string "intel_cpufreq". 217Then, the driver behaves like a regular ``CPUFreq`` scaling driver. That is, 218it is invoked by generic scaling governors when necessary to talk to the 219hardware in order to change the P-state of a CPU (in particular, the 220``schedutil`` governor can invoke it directly from scheduler context). 221 222While in this mode, ``intel_pstate`` can be used with all of the (generic) 223scaling governors listed by the ``scaling_available_governors`` policy attribute 224in ``sysfs`` (and the P-state selection algorithms described above are not 225used). Then, it is responsible for the configuration of policy objects 226corresponding to CPUs and provides the ``CPUFreq`` core (and the scaling 227governors attached to the policy objects) with accurate information on the 228maximum and minimum operating frequencies supported by the hardware (including 229the so-called "turbo" frequency ranges). In other words, in the passive mode 230the entire range of available P-states is exposed by ``intel_pstate`` to the 231``CPUFreq`` core. However, in this mode the driver does not register 232utilization update callbacks with the CPU scheduler and the ``scaling_cur_freq`` 233information comes from the ``CPUFreq`` core (and is the last frequency selected 234by the current scaling governor for the given policy). 235 236 237.. _turbo: 238 239Turbo P-states Support 240====================== 241 242In the majority of cases, the entire range of P-states available to 243``intel_pstate`` can be divided into two sub-ranges that correspond to 244different types of processor behavior, above and below a boundary that 245will be referred to as the "turbo threshold" in what follows. 246 247The P-states above the turbo threshold are referred to as "turbo P-states" and 248the whole sub-range of P-states they belong to is referred to as the "turbo 249range". These names are related to the Turbo Boost technology allowing a 250multicore processor to opportunistically increase the P-state of one or more 251cores if there is enough power to do that and if that is not going to cause the 252thermal envelope of the processor package to be exceeded. 253 254Specifically, if software sets the P-state of a CPU core within the turbo range 255(that is, above the turbo threshold), the processor is permitted to take over 256performance scaling control for that core and put it into turbo P-states of its 257choice going forward. However, that permission is interpreted differently by 258different processor generations. Namely, the Sandy Bridge generation of 259processors will never use any P-states above the last one set by software for 260the given core, even if it is within the turbo range, whereas all of the later 261processor generations will take it as a license to use any P-states from the 262turbo range, even above the one set by software. In other words, on those 263processors setting any P-state from the turbo range will enable the processor 264to put the given core into all turbo P-states up to and including the maximum 265supported one as it sees fit. 266 267One important property of turbo P-states is that they are not sustainable. More 268precisely, there is no guarantee that any CPUs will be able to stay in any of 269those states indefinitely, because the power distribution within the processor 270package may change over time or the thermal envelope it was designed for might 271be exceeded if a turbo P-state was used for too long. 272 273In turn, the P-states below the turbo threshold generally are sustainable. In 274fact, if one of them is set by software, the processor is not expected to change 275it to a lower one unless in a thermal stress or a power limit violation 276situation (a higher P-state may still be used if it is set for another CPU in 277the same package at the same time, for example). 278 279Some processors allow multiple cores to be in turbo P-states at the same time, 280but the maximum P-state that can be set for them generally depends on the number 281of cores running concurrently. The maximum turbo P-state that can be set for 3 282cores at the same time usually is lower than the analogous maximum P-state for 2832 cores, which in turn usually is lower than the maximum turbo P-state that can 284be set for 1 core. The one-core maximum turbo P-state is thus the maximum 285supported one overall. 286 287The maximum supported turbo P-state, the turbo threshold (the maximum supported 288non-turbo P-state) and the minimum supported P-state are specific to the 289processor model and can be determined by reading the processor's model-specific 290registers (MSRs). Moreover, some processors support the Configurable TDP 291(Thermal Design Power) feature and, when that feature is enabled, the turbo 292threshold effectively becomes a configurable value that can be set by the 293platform firmware. 294 295Unlike ``_PSS`` objects in the ACPI tables, ``intel_pstate`` always exposes 296the entire range of available P-states, including the whole turbo range, to the 297``CPUFreq`` core and (in the passive mode) to generic scaling governors. This 298generally causes turbo P-states to be set more often when ``intel_pstate`` is 299used relative to ACPI-based CPU performance scaling (see 300:ref:`below <acpi-cpufreq>` for more information). 301 302Moreover, since ``intel_pstate`` always knows what the real turbo threshold is 303(even if the Configurable TDP feature is enabled in the processor), its 304``no_turbo`` attribute in ``sysfs`` (described :ref:`below <no_turbo_attr>`) should 305work as expected in all cases (that is, if set to disable turbo P-states, it 306always should prevent ``intel_pstate`` from using them). 307 308 309Processor Support 310================= 311 312To handle a given processor ``intel_pstate`` requires a number of different 313pieces of information on it to be known, including: 314 315 * The minimum supported P-state. 316 317 * The maximum supported :ref:`non-turbo P-state <turbo>`. 318 319 * Whether or not turbo P-states are supported at all. 320 321 * The maximum supported :ref:`one-core turbo P-state <turbo>` (if turbo 322 P-states are supported). 323 324 * The scaling formula to translate the driver's internal representation 325 of P-states into frequencies and the other way around. 326 327Generally, ways to obtain that information are specific to the processor model 328or family. Although it often is possible to obtain all of it from the processor 329itself (using model-specific registers), there are cases in which hardware 330manuals need to be consulted to get to it too. 331 332For this reason, there is a list of supported processors in ``intel_pstate`` and 333the driver initialization will fail if the detected processor is not in that 334list, unless it supports the HWP feature. [The interface to obtain all of the 335information listed above is the same for all of the processors supporting the 336HWP feature, which is why ``intel_pstate`` works with all of them.] 337 338 339Support for Hybrid Processors 340============================= 341 342Some processors supported by ``intel_pstate`` contain two or more types of CPU 343cores differing by the maximum turbo P-state, performance vs power characteristics, 344cache sizes, and possibly other properties. They are commonly referred to as 345hybrid processors. To support them, ``intel_pstate`` requires HWP to be enabled 346and it assumes the HWP performance units to be the same for all CPUs in the 347system, so a given HWP performance level always represents approximately the 348same physical performance regardless of the core (CPU) type. 349 350Hybrid Processors with SMT 351-------------------------- 352 353On systems where SMT (Simultaneous Multithreading), also referred to as 354HyperThreading (HT) in the context of Intel processors, is enabled on at least 355one core, ``intel_pstate`` assigns performance-based priorities to CPUs. Namely, 356the priority of a given CPU reflects its highest HWP performance level which 357causes the CPU scheduler to generally prefer more performant CPUs, so the less 358performant CPUs are used when the other ones are fully loaded. However, SMT 359siblings (that is, logical CPUs sharing one physical core) are treated in a 360special way such that if one of them is in use, the effective priority of the 361other ones is lowered below the priorities of the CPUs located in the other 362physical cores. 363 364This approach maximizes performance in the majority of cases, but unfortunately 365it also leads to excessive energy usage in some important scenarios, like video 366playback, which is not generally desirable. While there is no other viable 367choice with SMT enabled because the effective capacity and utilization of SMT 368siblings are hard to determine, hybrid processors without SMT can be handled in 369more energy-efficient ways. 370 371.. _CAS: 372 373Capacity-Aware Scheduling Support 374--------------------------------- 375 376The capacity-aware scheduling (CAS) support in the CPU scheduler is enabled by 377``intel_pstate`` by default on hybrid processors without SMT. CAS generally 378causes the scheduler to put tasks on a CPU so long as there is a sufficient 379amount of spare capacity on it, and if the utilization of a given task is too 380high for it, the task will need to go somewhere else. 381 382Since CAS takes CPU capacities into account, it does not require CPU 383prioritization and it allows tasks to be distributed more symmetrically among 384the more performant and less performant CPUs. Once placed on a CPU with enough 385capacity to accommodate it, a task may just continue to run there regardless of 386whether or not the other CPUs are fully loaded, so on average CAS reduces the 387utilization of the more performant CPUs which causes the energy usage to be more 388balanced because the more performant CPUs are generally less energy-efficient 389than the less performant ones. 390 391In order to use CAS, the scheduler needs to know the capacity of each CPU in 392the system and it needs to be able to compute scale-invariant utilization of 393CPUs, so ``intel_pstate`` provides it with the requisite information. 394 395First of all, the capacity of each CPU is represented by the ratio of its highest 396HWP performance level, multiplied by 1024, to the highest HWP performance level 397of the most performant CPU in the system, which works because the HWP performance 398units are the same for all CPUs. Second, the frequency-invariance computations, 399carried out by the scheduler to always express CPU utilization in the same units 400regardless of the frequency it is currently running at, are adjusted to take the 401CPU capacity into account. All of this happens when ``intel_pstate`` has 402registered itself with the ``CPUFreq`` core and it has figured out that it is 403running on a hybrid processor without SMT. 404 405Energy-Aware Scheduling Support 406------------------------------- 407 408If ``CONFIG_ENERGY_MODEL`` has been set during kernel configuration and 409``intel_pstate`` runs on a hybrid processor without SMT, in addition to enabling 410:ref:`CAS` it registers an Energy Model for the processor. This allows the 411Energy-Aware Scheduling (EAS) support to be enabled in the CPU scheduler if 412``schedutil`` is used as the ``CPUFreq`` governor which requires ``intel_pstate`` 413to operate in the :ref:`passive mode <passive_mode>`. 414 415The Energy Model registered by ``intel_pstate`` is artificial (that is, it is 416based on abstract cost values and it does not include any real power numbers) 417and it is relatively simple to avoid unnecessary computations in the scheduler. 418There is a performance domain in it for every CPU in the system and the cost 419values for these performance domains have been chosen so that running a task on 420a less performant (small) CPU appears to be always cheaper than running that 421task on a more performant (big) CPU. However, for two CPUs of the same type, 422the cost difference depends on their current utilization, and the CPU whose 423current utilization is higher generally appears to be a more expensive 424destination for a given task. This helps to balance the load among CPUs of the 425same type. 426 427Since EAS works on top of CAS, high-utilization tasks are always migrated to 428CPUs with enough capacity to accommodate them, but thanks to EAS, low-utilization 429tasks tend to be placed on the CPUs that look less expensive to the scheduler. 430Effectively, this causes the less performant and less loaded CPUs to be 431preferred as long as they have enough spare capacity to run the given task 432which generally leads to reduced energy usage. 433 434The Energy Model created by ``intel_pstate`` can be inspected by looking at 435the ``energy_model`` directory in ``debugfs`` (typlically mounted on 436``/sys/kernel/debug/``). 437 438 439User Space Interface in ``sysfs`` 440================================= 441 442.. _global_attributes: 443 444Global Attributes 445----------------- 446 447``intel_pstate`` exposes several global attributes (files) in ``sysfs`` to 448control its functionality at the system level. They are located in the 449``/sys/devices/system/cpu/intel_pstate/`` directory and affect all CPUs. 450 451Some of them are not present if the ``intel_pstate=per_cpu_perf_limits`` 452argument is passed to the kernel in the command line. 453 454``max_perf_pct`` 455 Maximum P-state the driver is allowed to set in percent of the 456 maximum supported performance level (the highest supported :ref:`turbo 457 P-state <turbo>`). 458 459 This attribute will not be exposed if the 460 ``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel 461 command line. 462 463``min_perf_pct`` 464 Minimum P-state the driver is allowed to set in percent of the 465 maximum supported performance level (the highest supported :ref:`turbo 466 P-state <turbo>`). 467 468 This attribute will not be exposed if the 469 ``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel 470 command line. 471 472``num_pstates`` 473 Number of P-states supported by the processor (between 0 and 255 474 inclusive) including both turbo and non-turbo P-states (see 475 :ref:`turbo`). 476 477 This attribute is present only if the value exposed by it is the same 478 for all of the CPUs in the system. 479 480 The value of this attribute is not affected by the ``no_turbo`` 481 setting described :ref:`below <no_turbo_attr>`. 482 483 This attribute is read-only. 484 485``turbo_pct`` 486 Ratio of the :ref:`turbo range <turbo>` size to the size of the entire 487 range of supported P-states, in percent. 488 489 This attribute is present only if the value exposed by it is the same 490 for all of the CPUs in the system. 491 492 This attribute is read-only. 493 494.. _no_turbo_attr: 495 496``no_turbo`` 497 If set (equal to 1), the driver is not allowed to set any turbo P-states 498 (see :ref:`turbo`). If unset (equal to 0, which is the 499 default), turbo P-states can be set by the driver. 500 [Note that ``intel_pstate`` does not support the general ``boost`` 501 attribute (supported by some other scaling drivers) which is replaced 502 by this one.] 503 504 This attribute does not affect the maximum supported frequency value 505 supplied to the ``CPUFreq`` core and exposed via the policy interface, 506 but it affects the maximum possible value of per-policy P-state limits 507 (see :ref:`policy_attributes_interpretation` below for details). 508 509``hwp_dynamic_boost`` 510 This attribute is only present if ``intel_pstate`` works in the 511 :ref:`active mode with the HWP feature enabled <active_mode_hwp>` in 512 the processor. If set (equal to 1), it causes the minimum P-state limit 513 to be increased dynamically for a short time whenever a task previously 514 waiting on I/O is selected to run on a given logical CPU (the purpose 515 of this mechanism is to improve performance). 516 517 This setting has no effect on logical CPUs whose minimum P-state limit 518 is directly set to the highest non-turbo P-state or above it. 519 520.. _status_attr: 521 522``status`` 523 Operation mode of the driver: "active", "passive" or "off". 524 525 "active" 526 The driver is functional and in the :ref:`active mode 527 <active_mode>`. 528 529 "passive" 530 The driver is functional and in the :ref:`passive mode 531 <passive_mode>`. 532 533 "off" 534 The driver is not functional (it is not registered as a scaling 535 driver with the ``CPUFreq`` core). 536 537 This attribute can be written to in order to change the driver's 538 operation mode or to unregister it. The string written to it must be 539 one of the possible values of it and, if successful, the write will 540 cause the driver to switch over to the operation mode represented by 541 that string - or to be unregistered in the "off" case. [Actually, 542 switching over from the active mode to the passive mode or the other 543 way around causes the driver to be unregistered and registered again 544 with a different set of callbacks, so all of its settings (the global 545 as well as the per-policy ones) are then reset to their default 546 values, possibly depending on the target operation mode.] 547 548``energy_efficiency`` 549 This attribute is only present on platforms with CPUs matching the Kaby 550 Lake or Coffee Lake desktop CPU model. By default, energy-efficiency 551 optimizations are disabled on these CPU models if HWP is enabled. 552 Enabling energy-efficiency optimizations may limit maximum operating 553 frequency with or without the HWP feature. With HWP enabled, the 554 optimizations are done only in the turbo frequency range. Without it, 555 they are done in the entire available frequency range. Setting this 556 attribute to "1" enables the energy-efficiency optimizations and setting 557 to "0" disables them. 558 559.. _policy_attributes_interpretation: 560 561Interpretation of Policy Attributes 562----------------------------------- 563 564The interpretation of some ``CPUFreq`` policy attributes described in 565Documentation/admin-guide/pm/cpufreq.rst is special with ``intel_pstate`` 566as the current scaling driver and it generally depends on the driver's 567:ref:`operation mode <operation_modes>`. 568 569First of all, the values of the ``cpuinfo_max_freq``, ``cpuinfo_min_freq`` and 570``scaling_cur_freq`` attributes are produced by applying a processor-specific 571multiplier to the internal P-state representation used by ``intel_pstate``. 572Also, the values of the ``scaling_max_freq`` and ``scaling_min_freq`` 573attributes are capped by the frequency corresponding to the maximum P-state that 574the driver is allowed to set. 575 576If the ``no_turbo`` :ref:`global attribute <no_turbo_attr>` is set, the driver 577is not allowed to use turbo P-states, so the maximum value of 578``scaling_max_freq`` and ``scaling_min_freq`` is limited to the maximum 579non-turbo P-state frequency. 580Accordingly, setting ``no_turbo`` causes ``scaling_max_freq`` and 581``scaling_min_freq`` to go down to that value if they were above it before. 582However, the old values of ``scaling_max_freq`` and ``scaling_min_freq`` will be 583restored after unsetting ``no_turbo``, unless these attributes have been written 584to after ``no_turbo`` was set. 585 586If ``no_turbo`` is not set, the maximum possible value of ``scaling_max_freq`` 587and ``scaling_min_freq`` corresponds to the maximum supported turbo P-state, 588which also is the value of ``cpuinfo_max_freq`` in either case. 589 590Next, the following policy attributes have special meaning if 591``intel_pstate`` works in the :ref:`active mode <active_mode>`: 592 593``scaling_available_governors`` 594 List of P-state selection algorithms provided by ``intel_pstate``. 595 596``scaling_governor`` 597 P-state selection algorithm provided by ``intel_pstate`` currently in 598 use with the given policy. 599 600``scaling_cur_freq`` 601 Frequency of the average P-state of the CPU represented by the given 602 policy for the time interval between the last two invocations of the 603 driver's utilization update callback by the CPU scheduler for that CPU. 604 605One more policy attribute is present if the HWP feature is enabled in the 606processor: 607 608``base_frequency`` 609 Shows the base frequency of the CPU. Any frequency above this will be 610 in the turbo frequency range. 611 612The meaning of these attributes in the :ref:`passive mode <passive_mode>` is the 613same as for other scaling drivers. 614 615Additionally, the value of the ``scaling_driver`` attribute for ``intel_pstate`` 616depends on the operation mode of the driver. Namely, it is either 617"intel_pstate" (in the :ref:`active mode <active_mode>`) or "intel_cpufreq" 618(in the :ref:`passive mode <passive_mode>`). 619 620.. _pstate_limits_coordination: 621 622Coordination of P-State Limits 623------------------------------ 624 625``intel_pstate`` allows P-state limits to be set in two ways: with the help of 626the ``max_perf_pct`` and ``min_perf_pct`` :ref:`global attributes 627<global_attributes>` or via the ``scaling_max_freq`` and ``scaling_min_freq`` 628``CPUFreq`` policy attributes. The coordination between those limits is based 629on the following rules, regardless of the current operation mode of the driver: 630 631 1. All CPUs are affected by the global limits (that is, none of them can be 632 requested to run faster than the global maximum and none of them can be 633 requested to run slower than the global minimum). 634 635 2. Each individual CPU is affected by its own per-policy limits (that is, it 636 cannot be requested to run faster than its own per-policy maximum and it 637 cannot be requested to run slower than its own per-policy minimum). The 638 effective performance depends on whether the platform supports per core 639 P-states, hyper-threading is enabled and on current performance requests 640 from other CPUs. When platform doesn't support per core P-states, the 641 effective performance can be more than the policy limits set on a CPU, if 642 other CPUs are requesting higher performance at that moment. Even with per 643 core P-states support, when hyper-threading is enabled, if the sibling CPU 644 is requesting higher performance, the other siblings will get higher 645 performance than their policy limits. 646 647 3. The global and per-policy limits can be set independently. 648 649In the :ref:`active mode with the HWP feature enabled <active_mode_hwp>`, the 650resulting effective values are written into hardware registers whenever the 651limits change in order to request its internal P-state selection logic to always 652set P-states within these limits. Otherwise, the limits are taken into account 653by scaling governors (in the :ref:`passive mode <passive_mode>`) and by the 654driver every time before setting a new P-state for a CPU. 655 656Additionally, if the ``intel_pstate=per_cpu_perf_limits`` command line argument 657is passed to the kernel, ``max_perf_pct`` and ``min_perf_pct`` are not exposed 658at all and the only way to set the limits is by using the policy attributes. 659 660.. _energy_performance_hints: 661 662Energy vs Performance Hints 663--------------------------- 664 665If the hardware-managed P-states (HWP) is enabled in the processor, additional 666attributes, intended to allow user space to help ``intel_pstate`` to adjust the 667processor's internal P-state selection logic by focusing it on performance or on 668energy-efficiency, or somewhere between the two extremes, are present in every 669``CPUFreq`` policy directory in ``sysfs``. They are : 670 671``energy_performance_preference`` 672 Current value of the energy vs performance hint for the given policy 673 (or the CPU represented by it). 674 675 The hint can be changed by writing to this attribute. 676 677``energy_performance_available_preferences`` 678 List of strings that can be written to the 679 ``energy_performance_preference`` attribute. 680 681 They represent different energy vs performance hints and should be 682 self-explanatory, except that ``default`` represents whatever hint 683 value was set by the platform firmware. 684 685Strings written to the ``energy_performance_preference`` attribute are 686internally translated to integer values written to the processor's 687Energy-Performance Preference (EPP) knob (if supported) or its 688Energy-Performance Bias (EPB) knob. It is also possible to write a positive 689integer value between 0 to 255, if the EPP feature is present. If the EPP 690feature is not present, writing integer value to this attribute is not 691supported. In this case, user can use the 692"/sys/devices/system/cpu/cpu*/power/energy_perf_bias" interface. 693 694[Note that tasks may by migrated from one CPU to another by the scheduler's 695load-balancing algorithm and if different energy vs performance hints are 696set for those CPUs, that may lead to undesirable outcomes. To avoid such 697issues it is better to set the same energy vs performance hint for all CPUs 698or to pin every task potentially sensitive to them to a specific CPU.] 699 700.. _acpi-cpufreq: 701 702``intel_pstate`` vs ``acpi-cpufreq`` 703==================================== 704 705On the majority of systems supported by ``intel_pstate``, the ACPI tables 706provided by the platform firmware contain ``_PSS`` objects returning information 707that can be used for CPU performance scaling (refer to the ACPI specification 708[3]_ for details on the ``_PSS`` objects and the format of the information 709returned by them). 710 711The information returned by the ACPI ``_PSS`` objects is used by the 712``acpi-cpufreq`` scaling driver. On systems supported by ``intel_pstate`` 713the ``acpi-cpufreq`` driver uses the same hardware CPU performance scaling 714interface, but the set of P-states it can use is limited by the ``_PSS`` 715output. 716 717On those systems each ``_PSS`` object returns a list of P-states supported by 718the corresponding CPU which basically is a subset of the P-states range that can 719be used by ``intel_pstate`` on the same system, with one exception: the whole 720:ref:`turbo range <turbo>` is represented by one item in it (the topmost one). 721By convention, the frequency returned by ``_PSS`` for that item is greater by 7221 MHz than the frequency of the highest non-turbo P-state listed by it, but the 723corresponding P-state representation (following the hardware specification) 724returned for it matches the maximum supported turbo P-state (or is the 725special value 255 meaning essentially "go as high as you can get"). 726 727The list of P-states returned by ``_PSS`` is reflected by the table of 728available frequencies supplied by ``acpi-cpufreq`` to the ``CPUFreq`` core and 729scaling governors and the minimum and maximum supported frequencies reported by 730it come from that list as well. In particular, given the special representation 731of the turbo range described above, this means that the maximum supported 732frequency reported by ``acpi-cpufreq`` is higher by 1 MHz than the frequency 733of the highest supported non-turbo P-state listed by ``_PSS`` which, of course, 734affects decisions made by the scaling governors, except for ``powersave`` and 735``performance``. 736 737For example, if a given governor attempts to select a frequency proportional to 738estimated CPU load and maps the load of 100% to the maximum supported frequency 739(possibly multiplied by a constant), then it will tend to choose P-states below 740the turbo threshold if ``acpi-cpufreq`` is used as the scaling driver, because 741in that case the turbo range corresponds to a small fraction of the frequency 742band it can use (1 MHz vs 1 GHz or more). In consequence, it will only go to 743the turbo range for the highest loads and the other loads above 50% that might 744benefit from running at turbo frequencies will be given non-turbo P-states 745instead. 746 747One more issue related to that may appear on systems supporting the 748:ref:`Configurable TDP feature <turbo>` allowing the platform firmware to set 749the turbo threshold. Namely, if that is not coordinated with the lists of 750P-states returned by ``_PSS`` properly, there may be more than one item 751corresponding to a turbo P-state in those lists and there may be a problem with 752avoiding the turbo range (if desirable or necessary). Usually, to avoid using 753turbo P-states overall, ``acpi-cpufreq`` simply avoids using the topmost state 754listed by ``_PSS``, but that is not sufficient when there are other turbo 755P-states in the list returned by it. 756 757Apart from the above, ``acpi-cpufreq`` works like ``intel_pstate`` in the 758:ref:`passive mode <passive_mode>`, except that the number of P-states it can 759set is limited to the ones listed by the ACPI ``_PSS`` objects. 760 761 762Kernel Command Line Options for ``intel_pstate`` 763================================================ 764 765Several kernel command line options can be used to pass early-configuration-time 766parameters to ``intel_pstate`` in order to enforce specific behavior of it. All 767of them have to be prepended with the ``intel_pstate=`` prefix. 768 769``disable`` 770 Do not register ``intel_pstate`` as the scaling driver even if the 771 processor is supported by it. 772 773``active`` 774 Register ``intel_pstate`` in the :ref:`active mode <active_mode>` to 775 start with. 776 777``passive`` 778 Register ``intel_pstate`` in the :ref:`passive mode <passive_mode>` to 779 start with. 780 781``force`` 782 Register ``intel_pstate`` as the scaling driver instead of 783 ``acpi-cpufreq`` even if the latter is preferred on the given system. 784 785 This may prevent some platform features (such as thermal controls and 786 power capping) that rely on the availability of ACPI P-states 787 information from functioning as expected, so it should be used with 788 caution. 789 790 This option does not work with processors that are not supported by 791 ``intel_pstate`` and on platforms where the ``pcc-cpufreq`` scaling 792 driver is used instead of ``acpi-cpufreq``. 793 794``no_hwp`` 795 Do not enable the hardware-managed P-states (HWP) feature even if it is 796 supported by the processor. 797 798``hwp_only`` 799 Register ``intel_pstate`` as the scaling driver only if the 800 hardware-managed P-states (HWP) feature is supported by the processor. 801 802``support_acpi_ppc`` 803 Take ACPI ``_PPC`` performance limits into account. 804 805 If the preferred power management profile in the FADT (Fixed ACPI 806 Description Table) is set to "Enterprise Server" or "Performance 807 Server", the ACPI ``_PPC`` limits are taken into account by default 808 and this option has no effect. 809 810``per_cpu_perf_limits`` 811 Use per-logical-CPU P-State limits (see 812 :ref:`pstate_limits_coordination` for details). 813 814``no_cas`` 815 Do not enable :ref:`capacity-aware scheduling <CAS>` which is enabled 816 by default on hybrid systems without SMT. 817 818Diagnostics and Tuning 819====================== 820 821Trace Events 822------------ 823 824There are two static trace events that can be used for ``intel_pstate`` 825diagnostics. One of them is the ``cpu_frequency`` trace event generally used 826by ``CPUFreq``, and the other one is the ``pstate_sample`` trace event specific 827to ``intel_pstate``. Both of them are triggered by ``intel_pstate`` only if 828it works in the :ref:`active mode <active_mode>`. 829 830The following sequence of shell commands can be used to enable them and see 831their output (if the kernel is generally configured to support event tracing):: 832 833 # cd /sys/kernel/tracing/ 834 # echo 1 > events/power/pstate_sample/enable 835 # echo 1 > events/power/cpu_frequency/enable 836 # cat trace 837 gnome-terminal--4510 [001] ..s. 1177.680733: pstate_sample: core_busy=107 scaled=94 from=26 to=26 mperf=1143818 aperf=1230607 tsc=29838618 freq=2474476 838 cat-5235 [002] ..s. 1177.681723: cpu_frequency: state=2900000 cpu_id=2 839 840If ``intel_pstate`` works in the :ref:`passive mode <passive_mode>`, the 841``cpu_frequency`` trace event will be triggered either by the ``schedutil`` 842scaling governor (for the policies it is attached to), or by the ``CPUFreq`` 843core (for the policies with other scaling governors). 844 845``ftrace`` 846---------- 847 848The ``ftrace`` interface can be used for low-level diagnostics of 849``intel_pstate``. For example, to check how often the function to set a 850P-state is called, the ``ftrace`` filter can be set to 851:c:func:`intel_pstate_set_pstate`:: 852 853 # cd /sys/kernel/tracing/ 854 # cat available_filter_functions | grep -i pstate 855 intel_pstate_set_pstate 856 intel_pstate_cpu_init 857 ... 858 # echo intel_pstate_set_pstate > set_ftrace_filter 859 # echo function > current_tracer 860 # cat trace | head -15 861 # tracer: function 862 # 863 # entries-in-buffer/entries-written: 80/80 #P:4 864 # 865 # _-----=> irqs-off 866 # / _----=> need-resched 867 # | / _---=> hardirq/softirq 868 # || / _--=> preempt-depth 869 # ||| / delay 870 # TASK-PID CPU# |||| TIMESTAMP FUNCTION 871 # | | | |||| | | 872 Xorg-3129 [000] ..s. 2537.644844: intel_pstate_set_pstate <-intel_pstate_timer_func 873 gnome-terminal--4510 [002] ..s. 2537.649844: intel_pstate_set_pstate <-intel_pstate_timer_func 874 gnome-shell-3409 [001] ..s. 2537.650850: intel_pstate_set_pstate <-intel_pstate_timer_func 875 <idle>-0 [000] ..s. 2537.654843: intel_pstate_set_pstate <-intel_pstate_timer_func 876 877 878References 879========== 880 881.. [1] Kristen Accardi, *Balancing Power and Performance in the Linux Kernel*, 882 https://events.static.linuxfound.org/sites/events/files/slides/LinuxConEurope_2015.pdf 883 884.. [2] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3: System Programming Guide*, 885 https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-system-programming-manual-325384.html 886 887.. [3] *Advanced Configuration and Power Interface Specification*, 888 https://uefi.org/sites/default/files/resources/ACPI_6_3_final_Jan30.pdf 889