1.. SPDX-License-Identifier: GPL-2.0 2 3============================= 4DAMON-based LRU-lists Sorting 5============================= 6 7DAMON-based LRU-lists Sorting (DAMON_LRU_SORT) is a static kernel module that 8aimed to be used for proactive and lightweight data access pattern based 9(de)prioritization of pages on their LRU-lists for making LRU-lists a more 10trusworthy data access pattern source. 11 12Where Proactive LRU-lists Sorting is Required? 13============================================== 14 15As page-granularity access checking overhead could be significant on huge 16systems, LRU lists are normally not proactively sorted but partially and 17reactively sorted for special events including specific user requests, system 18calls and memory pressure. As a result, LRU lists are sometimes not so 19perfectly prepared to be used as a trustworthy access pattern source for some 20situations including reclamation target pages selection under sudden memory 21pressure. 22 23Because DAMON can identify access patterns of best-effort accuracy while 24inducing only user-specified range of overhead, proactively running 25DAMON_LRU_SORT could be helpful for making LRU lists more trustworthy access 26pattern source with low and controlled overhead. 27 28How It Works? 29============= 30 31DAMON_LRU_SORT finds hot pages (pages of memory regions that showing access 32rates that higher than a user-specified threshold) and cold pages (pages of 33memory regions that showing no access for a time that longer than a 34user-specified threshold) using DAMON, and prioritizes hot pages while 35deprioritizing cold pages on their LRU-lists. To avoid it consuming too much 36CPU for the prioritizations, a CPU time usage limit can be configured. Under 37the limit, it prioritizes and deprioritizes more hot and cold pages first, 38respectively. System administrators can also configure under what situation 39this scheme should automatically activated and deactivated with three memory 40pressure watermarks. 41 42Its default parameters for hotness/coldness thresholds and CPU quota limit are 43conservatively chosen. That is, the module under its default parameters could 44be widely used without harm for common situations while providing a level of 45benefits for systems having clear hot/cold access patterns under memory 46pressure while consuming only a limited small portion of CPU time. 47 48Interface: Module Parameters 49============================ 50 51To use this feature, you should first ensure your system is running on a kernel 52that is built with ``CONFIG_DAMON_LRU_SORT=y``. 53 54To let sysadmins enable or disable it and tune for the given system, 55DAMON_LRU_SORT utilizes module parameters. That is, you can put 56``damon_lru_sort.<parameter>=<value>`` on the kernel boot command line or write 57proper values to ``/sys/module/damon_lru_sort/parameters/<parameter>`` files. 58 59Below are the description of each parameter. 60 61enabled 62------- 63 64Enable or disable DAMON_LRU_SORT. 65 66You can enable DAMON_LRU_SORT by setting the value of this parameter as ``Y``. 67Setting it as ``N`` disables DAMON_LRU_SORT. Note that DAMON_LRU_SORT could do 68no real monitoring and LRU-lists sorting due to the watermarks-based activation 69condition. Refer to below descriptions for the watermarks parameter for this. 70 71commit_inputs 72------------- 73 74Make DAMON_LRU_SORT reads the input parameters again, except ``enabled``. 75 76Input parameters that updated while DAMON_LRU_SORT is running are not applied 77by default. Once this parameter is set as ``Y``, DAMON_LRU_SORT reads values 78of parameters except ``enabled`` again. Once the re-reading is done, this 79parameter is set as ``N``. If invalid parameters are found while the 80re-reading, DAMON_LRU_SORT will be disabled. 81 82Once ``Y`` is written to this parameter, the user must not write to any 83parameters until reading ``commit_inputs`` again returns ``N``. If users 84violate this rule, the kernel may exhibit undefined behavior. 85 86active_mem_bp 87------------- 88 89Desired active to [in]active memory ratio in bp (1/10,000). 90 91While keeping the caps that set by other quotas, DAMON_LRU_SORT automatically 92increases and decreases the effective level of the quota aiming the LRU 93[de]prioritizations of the hot and cold memory resulting in this active to 94[in]active memory ratio. Value zero means disabling this auto-tuning feature. 95 96Disabled by default. 97 98autotune_monitoring_intervals 99----------------------------- 100 101If this parameter is set as ``Y``, DAMON_LRU_SORT automatically tunes DAMON's 102sampling and aggregation intervals. The auto-tuning aims to capture meaningful 103amount of access events in each DAMON-snapshot, while keeping the sampling 104interval 5 milliseconds in minimum, and 10 seconds in maximum. Setting this as 105``N`` disables the auto-tuning. 106 107Disabled by default. 108 109filter_young_pages 110------------------ 111 112Filter [non-]young pages accordingly for LRU [de]prioritizations. 113 114If this is set, check page level access (youngness) once again before each 115LRU [de]prioritization operation. LRU prioritization operation is skipped 116if the page has not accessed since the last check (not young). LRU 117deprioritization operation is skipped if the page has accessed since the 118last check (young). The feature is enabled or disabled if this parameter is 119set as ``Y`` or ``N``, respectively. 120 121Disabled by default. 122 123hot_thres_access_freq 124--------------------- 125 126Access frequency threshold for hot memory regions identification in permil. 127 128If a memory region is accessed in frequency of this or higher, DAMON_LRU_SORT 129identifies the region as hot, and mark it as accessed on the LRU list, so that 130it could not be reclaimed under memory pressure. 50% by default. 131 132cold_min_age 133------------ 134 135Time threshold for cold memory regions identification in microseconds. 136 137If a memory region is not accessed for this or longer time, DAMON_LRU_SORT 138identifies the region as cold, and mark it as unaccessed on the LRU list, so 139that it could be reclaimed first under memory pressure. 120 seconds by 140default. 141 142quota_ms 143-------- 144 145Limit of time for trying the LRU lists sorting in milliseconds. 146 147DAMON_LRU_SORT tries to use only up to this time within a time window 148(quota_reset_interval_ms) for trying LRU lists sorting. This can be used 149for limiting CPU consumption of DAMON_LRU_SORT. If the value is zero, the 150limit is disabled. 151 15210 ms by default. 153 154quota_reset_interval_ms 155----------------------- 156 157The time quota charge reset interval in milliseconds. 158 159The charge reset interval for the quota of time (quota_ms). That is, 160DAMON_LRU_SORT does not try LRU-lists sorting for more than quota_ms 161milliseconds or quota_sz bytes within quota_reset_interval_ms milliseconds. 162 1631 second by default. 164 165wmarks_interval 166--------------- 167 168The watermarks check time interval in microseconds. 169 170Minimal time to wait before checking the watermarks, when DAMON_LRU_SORT is 171enabled but inactive due to its watermarks rule. 5 seconds by default. 172 173wmarks_high 174----------- 175 176Free memory rate (per thousand) for the high watermark. 177 178If free memory of the system in bytes per thousand bytes is higher than this, 179DAMON_LRU_SORT becomes inactive, so it does nothing but periodically checks the 180watermarks. 200 (20%) by default. 181 182wmarks_mid 183---------- 184 185Free memory rate (per thousand) for the middle watermark. 186 187If free memory of the system in bytes per thousand bytes is between this and 188the low watermark, DAMON_LRU_SORT becomes active, so starts the monitoring and 189the LRU-lists sorting. 150 (15%) by default. 190 191wmarks_low 192---------- 193 194Free memory rate (per thousand) for the low watermark. 195 196If free memory of the system in bytes per thousand bytes is lower than this, 197DAMON_LRU_SORT becomes inactive, so it does nothing but periodically checks the 198watermarks. 50 (5%) by default. 199 200sample_interval 201--------------- 202 203Sampling interval for the monitoring in microseconds. 204 205The sampling interval of DAMON for the cold memory monitoring. Please refer to 206the DAMON documentation (:doc:`usage`) for more detail. 5ms by default. 207 208aggr_interval 209------------- 210 211Aggregation interval for the monitoring in microseconds. 212 213The aggregation interval of DAMON for the cold memory monitoring. Please 214refer to the DAMON documentation (:doc:`usage`) for more detail. 100ms by 215default. 216 217min_nr_regions 218-------------- 219 220Minimum number of monitoring regions. 221 222The minimal number of monitoring regions of DAMON for the cold memory 223monitoring. This can be used to set lower-bound of the monitoring quality. 224But, setting this too high could result in increased monitoring overhead. 225Please refer to the DAMON documentation (:doc:`usage`) for more detail. 10 by 226default. 227 228Note that this must be 3 or higher. Please refer to the :ref:`Monitoring 229<damon_design_monitoring>` section of the design document for the rationale 230behind this lower bound. 231 232max_nr_regions 233-------------- 234 235Maximum number of monitoring regions. 236 237The maximum number of monitoring regions of DAMON for the cold memory 238monitoring. This can be used to set upper-bound of the monitoring overhead. 239However, setting this too low could result in bad monitoring quality. Please 240refer to the DAMON documentation (:doc:`usage`) for more detail. 1000 by 241defaults. 242 243monitor_region_start 244-------------------- 245 246Start of target memory region in physical address. 247 248The start physical address of memory region that DAMON_LRU_SORT will do work 249against. By default, the system's entire physical memory is used as the 250region. 251 252monitor_region_end 253------------------ 254 255End of target memory region in physical address. 256 257The end physical address of memory region that DAMON_LRU_SORT will do work 258against. By default, the system's entire physical memory is used as the 259region. 260 261addr_unit 262--------- 263 264A scale factor for memory addresses and bytes. 265 266This parameter is for setting and getting the :ref:`address unit 267<damon_design_addr_unit>` parameter of the DAMON instance for DAMON_RECLAIM. 268 269``monitor_region_start`` and ``monitor_region_end`` should be provided in this 270unit. For example, let's suppose ``addr_unit``, ``monitor_region_start`` and 271``monitor_region_end`` are set as ``1024``, ``0`` and ``10``, respectively. 272Then DAMON_LRU_SORT will work for 10 KiB length of physical address range that 273starts from address zero (``[0 * 1024, 10 * 1024)`` in bytes). 274 275Stat parameters having ``bytes_`` prefix are also in this unit. For example, 276let's suppose values of ``addr_unit``, ``bytes_lru_sort_tried_hot_regions`` and 277``bytes_lru_sorted_hot_regions`` are ``1024``, ``42``, and ``32``, 278respectively. Then it means DAMON_LRU_SORT tried to LRU-sort 42 KiB of hot 279memory and successfully LRU-sorted 32 KiB of the memory in total. 280 281If unsure, use only the default value (``1``) and forget about this. 282 283kdamond_pid 284----------- 285 286PID of the DAMON thread. 287 288If DAMON_LRU_SORT is enabled, this becomes the PID of the worker thread. Else, 289-1. 290 291nr_lru_sort_tried_hot_regions 292----------------------------- 293 294Number of hot memory regions that tried to be LRU-sorted. 295 296bytes_lru_sort_tried_hot_regions 297-------------------------------- 298 299Total bytes of hot memory regions that tried to be LRU-sorted. 300 301nr_lru_sorted_hot_regions 302------------------------- 303 304Number of hot memory regions that successfully be LRU-sorted. 305 306bytes_lru_sorted_hot_regions 307---------------------------- 308 309Total bytes of hot memory regions that successfully be LRU-sorted. 310 311nr_hot_quota_exceeds 312-------------------- 313 314Number of times that the time quota limit for hot regions have exceeded. 315 316nr_lru_sort_tried_cold_regions 317------------------------------ 318 319Number of cold memory regions that tried to be LRU-sorted. 320 321bytes_lru_sort_tried_cold_regions 322--------------------------------- 323 324Total bytes of cold memory regions that tried to be LRU-sorted. 325 326nr_lru_sorted_cold_regions 327-------------------------- 328 329Number of cold memory regions that successfully be LRU-sorted. 330 331bytes_lru_sorted_cold_regions 332----------------------------- 333 334Total bytes of cold memory regions that successfully be LRU-sorted. 335 336nr_cold_quota_exceeds 337--------------------- 338 339Number of times that the time quota limit for cold regions have exceeded. 340 341Example 342======= 343 344Below runtime example commands make DAMON_LRU_SORT to find memory regions 345having >=50% access frequency and LRU-prioritize while LRU-deprioritizing 346memory regions that not accessed for 120 seconds. The prioritization and 347deprioritization is limited to be done using only up to 1% CPU time to avoid 348DAMON_LRU_SORT consuming too much CPU time for the (de)prioritization. It also 349asks DAMON_LRU_SORT to do nothing if the system's free memory rate is more than 35050%, but start the real works if it becomes lower than 40%. If DAMON_RECLAIM 351doesn't make progress and therefore the free memory rate becomes lower than 35220%, it asks DAMON_LRU_SORT to do nothing again, so that we can fall back to 353the LRU-list based page granularity reclamation. :: 354 355 # cd /sys/module/damon_lru_sort/parameters 356 # echo 500 > hot_thres_access_freq 357 # echo 120000000 > cold_min_age 358 # echo 10 > quota_ms 359 # echo 1000 > quota_reset_interval_ms 360 # echo 500 > wmarks_high 361 # echo 400 > wmarks_mid 362 # echo 200 > wmarks_low 363 # echo Y > enabled 364 365Note that this module (damon_lru_sort) cannot run simultaneously with other 366DAMON-based special-purpose modules. Refer to :ref:`DAMON design special 367purpose modules exclusivity <damon_design_special_purpose_modules_exclusivity>` 368for more details. 369