1.. SPDX-License-Identifier: GPL-2.0 2 3============================== 4Real-Time Kernel configuration 5============================== 6 7.. contents:: Table of Contents 8 :depth: 3 9 :local: 10 11Introduction 12============ 13 14This document lists the kernel configuration options that might affect a 15real-time kernel's worst-case latency. It is intended for system integrators. 16 17Configuration options 18===================== 19 20.. Please keep the configuration listings alphabetically ordered 21 22CPU frequency governors 23----------------------- 24 25``CONFIG_CPU_FREQ`` 26^^^^^^^^^^^^^^^^^^^ 27 28:Expectation: enabled 29:Severity: *high* 30 31The CPU frequency scaling subsystem ensures that the processor can operate at 32its maximum supported frequency. While, in general, bootloaders are tasked 33with setting the CPU clock to the highest speed on boot, some do not. It is 34thus desirable to keep this option enabled. 35 36.. caution:: 37 38 A real-time kernel is not about being "as fast as possible", however 39 real-time requirements may demand that the CPU is clocked at a particular 40 speed. 41 42``CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE`` 43^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 44 45:Expectation: enabled 46:Severity: *high* 47 48Real-Time workloads expect a fixed CPU frequency during execution. Using the 49performance governor is an easy way to achieve that purely from kernel 50configuration. 51 52This is not an absolute rule. Some setups might prefer to clock the CPU to 53lower speeds due to thermal packaging or other requirements. The key is that 54the CPU frequency remains constant once set. 55 56Non-performance CPU frequency governors 57^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 58 59:Expectation: disabled 60:Severity: *medium* 61 62To ensure reproducible system latency measurements, disable the 63non-``PERFORMANCE`` CPU frequency governors whenever possible. This avoids 64the risk of unknown userspace tasks implicitly or explicitly setting a 65different CPU frequency governor, and thereby changing latency behavior while 66the system is running. 67 68If disabling other frequency governors is not an option, use a governor that 69keeps the CPU frequency fixed. For example, 70``CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE`` can be enabled when userspace is 71responsible for setting a *stable* frequency during system initialization. 72 73If a low CPU frequency is desired, then 74``CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE`` can be set. 75 76The ``ONDEMAND`` governor should not be enabled on a real-time system. Its 77frequency changes depend on workload behavior and can significantly harm 78determinism. 79 80For more information, see Documentation/admin-guide/pm/cpufreq.rst 81 82``CONFIG_CPU_IDLE`` 83------------------- 84 85:Expectation: enabled 86:Severity: *info* 87 88CPU idle states (C-states) allow the processor to enter low-power modes during 89periods of inactivity. Very-low CPU idle states may require flushing the CPU 90caches and lowering or disabling the clocking. This can lower power 91consumption, but it also increases the entry and exit latency from such 92states. 93 94While disabling this option eliminates cpuidle-related latencies, doing so can 95significantly impact hardware longevity, warranty, and thermal behavior. 96Users should cap the maximum C-state to C1 instead. For ACPI platforms, this 97can be achieved by using the boot parameter [1]_:: 98 99 processor.max_cstate=1 100 101Higher C-states can be acceptable depending on the user workload's latency 102requirements. For ACPI-based platforms, use the ``cpupower idle-info`` 103command to inspect the available idle states. 104 105For more information, please see: 106 107- ``linux/tools/power/cpupower`` 108- Documentation/admin-guide/pm/cpuidle.rst 109- Documentation/admin-guide/pm/index.rst 110 111``CONFIG_DRM`` 112-------------- 113 114:Expectation: disabled 115:Severity: *info* 116 117GPU-accelerated workloads can share system resources with the CPU, including 118last-level cache (LLC) and memory bandwidth. Modern integrated GPUs optimize 119graphics performance at the expense of CPU determinism. 120 121Examples of affected platforms: 122 123- Intel processors with integrated graphics (Gen9 and later) 124- AMD APUs with Radeon Graphics 125- Xilinx Zynq UltraScale+ MPSoC EG/EV series 126 127If graphics workloads must run alongside real-time tasks, users must conduct 128thorough stress testing using tools like ``glmark2`` while measuring the 129overall system latency. 130 131For more information, please check: 132 133- Documentation/core-api/real-time/hardware.rst ("Regarding hardware" section) 134- Documentation/filesystems/resctrl.rst 135- `Real-Time and Graphics: A Contradiction? <https://web.archive.org/web/20221025085614/https://linutronix.de/PDF/Realtime_and_graphics-acontradiction2021.pdf>`_ 136 137``CONFIG_EFI_DISABLE_RUNTIME`` 138------------------------------ 139 140:Expectation: enabled 141:Severity: *medium* 142 143EFI is the standard boot and firmware interface for multiple architectures. 144EFI runtime services provide callback functions to be called from the kernel; 145e.g., as utilized by (``CONFIG_EFI_VARS*``) or (``CONFIG_RTC_DRV_EFI``). For 146the former, the kernel calls into EFI to update the EFI variables. 147 148Calling into EFI means invoking firmware callbacks. During such invocations, 149the system might not be able to react to interrupts and will thus not be able 150to perform a context switch. This can cause significant latency spikes for 151the real-time system. 152 153``CONFIG_PREEMPT_RT`` enables this option by default. If this option is 154manually disabled at build time, the following boot parameter [1]_ may be used 155to disable EFI runtime at boot up:: 156 157 efi=noruntime 158 159Alternatively, confine EFI runtime service calls to a housekeeping CPU by 160restricting the ``efi_runtime`` workqueue CPU affinity. For example, set that 161workqueue's affinity to CPU #0 and pin your RT tasks to a different CPU range. 162See Documentation/core-api/workqueue.rst 163 164``CONFIG_NO_HZ`` / ``CONFIG_NO_HZ_FULL`` 165---------------------------------------- 166 167:Expectation: disabled 168:Severity: *medium* 169 170Tickless operation can increase kernel-to-userspace transition latency due to 171the extra accounting and state book-keeping. 172 173*Guidance by real-time workload type:* 174 175- For periodic workloads; e.g., control loops executing every 100 µs, avoid 176 ``NO_HZ`` modes. Consistent kernel ticks are preferable. 177 178- For computation-intensive workloads; e.g. extended userspace execution, 179 ``NO_HZ_FULL`` may be beneficial. In such cases, users should offload the 180 kernel housekeeping to dedicated CPUs and isolate compute cores. 181 182See also Documentation/timers/no_hz.rst 183 184``CONFIG_PREEMPT_RT`` 185--------------------- 186 187:Expectation: enabled 188:Severity: **fatal** 189 190This option must be enabled, or the resulting kernel will not be fully 191preemptible and real-time capable. 192 193``CONFIG_TRACING`` (and tracing options) 194---------------------------------------- 195 196:Expectation: enabled 197:Severity: *info* 198 199Shipping kernels with tracing support enabled (but not actively running) is 200highly recommended. This will allow the users to extract more information if 201latency problems arise. Nonetheless, some tracers do incur latency overhead 202just by being enabled. 203 204.. caution:: 205 206 Users should *not* make use of tracers or trace events during production 207 real-time kernel operation as they can add considerable overhead and degrade 208 the system's latency. 209 210``CONFIG_IRQSOFF_TRACER`` and ``CONFIG_PREEMPT_TRACER`` 211^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 212 213:Expectation: disabled 214:Severity: *high* 215 216These tracers do incur measurable latency overhead even when tracing is not 217currently active. 218 219Kernel Debug Options 220==================== 221 222Most kernel debug options add runtime overhead that increases the worst-case 223latency. 224 225.. caution:: 226 227 During development and early testing, users are encouraged to run their 228 real-time workloads and peripherals with lockdep (:ref:`lockdep`) and other 229 kernel debug options enabled, for a considerable amount of time. Such 230 workloads might trigger kernel code paths that were not triggered during the 231 internal Linux real-time kernel development, thus helping to uncover locking 232 and other types of kernel bugs. 233 234``CONFIG_DEBUG_ATOMIC_SLEEP`` 235----------------------------- 236 237:Expectation: allowed 238 239This sanity check catches common kernel programming errors with a tolerable 240latency cost. It also increases overall scheduling as each ``might_sleep()`` 241can lead to a context switch. 242 243``CONFIG_DEBUG_BUGVERBOSE`` and ``CONFIG_DEBUG_INFO*`` 244------------------------------------------------------ 245 246:Expectation: allowed 247 248These options increase the kernel image size but have no latency impact. They 249are also essential for meaningful BUG logs, crash dumps, and profiling. 250 251``CONFIG_DEBUG_FS`` 252------------------- 253 254:Expectation: allowed 255 256This is safe to include in real-time kernels, *provided that debugfs is not 257accessed during production runtime*. 258 259``CONFIG_DEBUG_KERNEL`` 260----------------------- 261 262:Expectation: allowed 263 264Meta-option which allows debug features to be enabled. It has no runtime 265impact, but beware of any debug features that it may have implicitly enabled. 266 267``CONFIG_LOCKUP_DETECTOR`` 268-------------------------- 269 270:Expectation: disabled 271:Severity: *high* 272 273The lockup detector creates kernel timer callbacks that execute every few 274seconds, in hard-IRQ context, even on real-time kernels. These periodic 275interrupts can cause latency spikes. 276 277Users should use hardware watchdogs instead, which will provide a similar 278functionality without the software-induced latency. 279 280.. _lockdep: 281 282``CONFIG_PROVE_LOCKING`` 283------------------------ 284 285:Expectation: disabled 286:Severity: *high* 287 288Proving the correctness of all kernel locking adds substantial overhead and 289significantly increases worst-case latency. 290 291Summary 292======= 293 294There is no "one size fits all" solution for configuring a real-time Linux 295system. Beginning with the system real-time requirements, integrators must 296consider the features and functions of the system's hardware, kernel, and 297userspace. All such components must be properly configured in order to 298establish and constrain the system's maximum latency. 299 300With that in mind, any incorrect real-time kernel configuration could cause a 301new maximum latency that shows up at the wrong time and is catastrophic for 302the real-time system's latency. 303 304References 305========== 306 307.. [1] See Documentation/admin-guide/kernel-parameters.rst 308