1.. SPDX-License-Identifier: GPL-2.0 2 3======================= 4DAMON-based Reclamation 5======================= 6 7DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to 8be used for proactive and lightweight reclamation under light memory pressure. 9It doesn't aim to replace the LRU-list based page_granularity reclamation, but 10to be selectively used for different level of memory pressure and requirements. 11 12Where Proactive Reclamation is Required? 13======================================== 14 15On general memory over-committed systems, proactively reclaiming cold pages 16helps saving memory and reducing latency spikes that incurred by the direct 17reclaim of the process or CPU consumption of kswapd, while incurring only 18minimal performance degradation [1]_ [2]_ . 19 20Free Pages Reporting [3]_ based memory over-commit virtualization systems are 21good example of the cases. In such systems, the guest VMs reports their free 22memory to host, and the host reallocates the reported memory to other guests. 23As a result, the memory of the systems are fully utilized. However, the 24guests could be not so memory-frugal, mainly because some kernel subsystems and 25user-space applications are designed to use as much memory as available. Then, 26guests could report only small amount of memory as free to host, results in 27memory utilization drop of the systems. Running the proactive reclamation in 28guests could mitigate this problem. 29 30How It Works? 31============= 32 33DAMON_RECLAIM finds memory regions that didn't accessed for specific time 34duration and page out. To avoid it consuming too much CPU for the paging out 35operation, a speed limit can be configured. Under the speed limit, it pages 36out memory regions that didn't accessed longer time first. System 37administrators can also configure under what situation this scheme should 38automatically activated and deactivated with three memory pressure watermarks. 39 40Interface: Module Parameters 41============================ 42 43To use this feature, you should first ensure your system is running on a kernel 44that is built with ``CONFIG_DAMON_RECLAIM=y``. 45 46To let sysadmins enable or disable it and tune for the given system, 47DAMON_RECLAIM utilizes module parameters. That is, you can put 48``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write 49proper values to ``/sys/module/damon_reclaim/parameters/<parameter>`` files. 50 51Below are the description of each parameter. 52 53enabled 54------- 55 56Enable or disable DAMON_RECLAIM. 57 58You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. 59Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do 60no real monitoring and reclamation due to the watermarks-based activation 61condition. Refer to below descriptions for the watermarks parameter for this. 62 63commit_inputs 64------------- 65 66Make DAMON_RECLAIM reads the input parameters again, except ``enabled``. 67 68Input parameters that updated while DAMON_RECLAIM is running are not applied 69by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values 70of parametrs except ``enabled`` again. Once the re-reading is done, this 71parameter is set as ``N``. If invalid parameters are found while the 72re-reading, DAMON_RECLAIM will be disabled. 73 74min_age 75------- 76 77Time threshold for cold memory regions identification in microseconds. 78 79If a memory region is not accessed for this or longer time, DAMON_RECLAIM 80identifies the region as cold, and reclaims it. 81 82120 seconds by default. 83 84quota_ms 85-------- 86 87Limit of time for the reclamation in milliseconds. 88 89DAMON_RECLAIM tries to use only up to this time within a time window 90(quota_reset_interval_ms) for trying reclamation of cold pages. This can be 91used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the 92limit is disabled. 93 9410 ms by default. 95 96quota_sz 97-------- 98 99Limit of size of memory for the reclamation in bytes. 100 101DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time 102window (quota_reset_interval_ms) and makes no more than this limit is tried. 103This can be used for limiting consumption of CPU and IO. If this value is 104zero, the limit is disabled. 105 106128 MiB by default. 107 108quota_reset_interval_ms 109----------------------- 110 111The time/size quota charge reset interval in milliseconds. 112 113The charget reset interval for the quota of time (quota_ms) and size 114(quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than 115quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms 116milliseconds. 117 1181 second by default. 119 120quota_mem_pressure_us 121--------------------- 122 123Desired level of memory pressure-stall time in microseconds. 124 125While keeping the caps that set by other quotas, DAMON_RECLAIM automatically 126increases and decreases the effective level of the quota aiming this level of 127memory pressure is incurred. System-wide ``some`` memory PSI in microseconds 128per quota reset interval (``quota_reset_interval_ms``) is collected and 129compared to this value to see if the aim is satisfied. Value zero means 130disabling this auto-tuning feature. 131 132Disabled by default. 133 134quota_autotune_feedback 135----------------------- 136 137User-specifiable feedback for auto-tuning of the effective quota. 138 139While keeping the caps that set by other quotas, DAMON_RECLAIM automatically 140increases and decreases the effective level of the quota aiming receiving this 141feedback of value ``10,000`` from the user. DAMON_RECLAIM assumes the feedback 142value and the quota are positively proportional. Value zero means disabling 143this auto-tuning feature. 144 145Disabled by default. 146 147wmarks_interval 148--------------- 149 150Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is 151enabled but inactive due to its watermarks rule. 152 153wmarks_high 154----------- 155 156Free memory rate (per thousand) for the high watermark. 157 158If free memory of the system in bytes per thousand bytes is higher than this, 159DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks 160the watermarks. 161 162wmarks_mid 163---------- 164 165Free memory rate (per thousand) for the middle watermark. 166 167If free memory of the system in bytes per thousand bytes is between this and 168the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and 169the reclaiming. 170 171wmarks_low 172---------- 173 174Free memory rate (per thousand) for the low watermark. 175 176If free memory of the system in bytes per thousand bytes is lower than this, 177DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the 178watermarks. In the case, the system falls back to the LRU-list based page 179granularity reclamation logic. 180 181sample_interval 182--------------- 183 184Sampling interval for the monitoring in microseconds. 185 186The sampling interval of DAMON for the cold memory monitoring. Please refer to 187the DAMON documentation (:doc:`usage`) for more detail. 188 189aggr_interval 190------------- 191 192Aggregation interval for the monitoring in microseconds. 193 194The aggregation interval of DAMON for the cold memory monitoring. Please 195refer to the DAMON documentation (:doc:`usage`) for more detail. 196 197min_nr_regions 198-------------- 199 200Minimum number of monitoring regions. 201 202The minimal number of monitoring regions of DAMON for the cold memory 203monitoring. This can be used to set lower-bound of the monitoring quality. 204But, setting this too high could result in increased monitoring overhead. 205Please refer to the DAMON documentation (:doc:`usage`) for more detail. 206 207max_nr_regions 208-------------- 209 210Maximum number of monitoring regions. 211 212The maximum number of monitoring regions of DAMON for the cold memory 213monitoring. This can be used to set upper-bound of the monitoring overhead. 214However, setting this too low could result in bad monitoring quality. Please 215refer to the DAMON documentation (:doc:`usage`) for more detail. 216 217monitor_region_start 218-------------------- 219 220Start of target memory region in physical address. 221 222The start physical address of memory region that DAMON_RECLAIM will do work 223against. That is, DAMON_RECLAIM will find cold memory regions in this region 224and reclaims. By default, biggest System RAM is used as the region. 225 226monitor_region_end 227------------------ 228 229End of target memory region in physical address. 230 231The end physical address of memory region that DAMON_RECLAIM will do work 232against. That is, DAMON_RECLAIM will find cold memory regions in this region 233and reclaims. By default, biggest System RAM is used as the region. 234 235skip_anon 236--------- 237 238Skip anonymous pages reclamation. 239 240If this parameter is set as ``Y``, DAMON_RECLAIM does not reclaim anonymous 241pages. By default, ``N``. 242 243 244kdamond_pid 245----------- 246 247PID of the DAMON thread. 248 249If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else, 250-1. 251 252nr_reclaim_tried_regions 253------------------------ 254 255Number of memory regions that tried to be reclaimed by DAMON_RECLAIM. 256 257bytes_reclaim_tried_regions 258--------------------------- 259 260Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM. 261 262nr_reclaimed_regions 263-------------------- 264 265Number of memory regions that successfully be reclaimed by DAMON_RECLAIM. 266 267bytes_reclaimed_regions 268----------------------- 269 270Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM. 271 272nr_quota_exceeds 273---------------- 274 275Number of times that the time/space quota limits have exceeded. 276 277Example 278======= 279 280Below runtime example commands make DAMON_RECLAIM to find memory regions that 281not accessed for 30 seconds or more and pages out. The reclamation is limited 282to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too 283much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do 284nothing if the system's free memory rate is more than 50%, but start the real 285works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and 286therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to 287do nothing again, so that we can fall back to the LRU-list based page 288granularity reclamation. :: 289 290 # cd /sys/module/damon_reclaim/parameters 291 # echo 30000000 > min_age 292 # echo $((1 * 1024 * 1024 * 1024)) > quota_sz 293 # echo 1000 > quota_reset_interval_ms 294 # echo 500 > wmarks_high 295 # echo 400 > wmarks_mid 296 # echo 200 > wmarks_low 297 # echo Y > enabled 298 299.. [1] https://research.google/pubs/pub48551/ 300.. [2] https://lwn.net/Articles/787611/ 301.. [3] https://www.kernel.org/doc/html/latest/mm/free_page_reporting.html 302