1.. SPDX-License-Identifier: GPL-2.0 2 3======================= 4DAMON-based Reclamation 5======================= 6 7DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to 8be used for proactive and lightweight reclamation under light memory pressure. 9It doesn't aim to replace the LRU-list based page_granularity reclamation, but 10to be selectively used for different level of memory pressure and requirements. 11 12Where Proactive Reclamation is Required? 13======================================== 14 15On general memory over-committed systems, proactively reclaiming cold pages 16helps saving memory and reducing latency spikes that incurred by the direct 17reclaim of the process or CPU consumption of kswapd, while incurring only 18minimal performance degradation [1]_ [2]_ . 19 20Free Pages Reporting [3]_ based memory over-commit virtualization systems are 21good example of the cases. In such systems, the guest VMs reports their free 22memory to host, and the host reallocates the reported memory to other guests. 23As a result, the memory of the systems are fully utilized. However, the 24guests could be not so memory-frugal, mainly because some kernel subsystems and 25user-space applications are designed to use as much memory as available. Then, 26guests could report only small amount of memory as free to host, results in 27memory utilization drop of the systems. Running the proactive reclamation in 28guests could mitigate this problem. 29 30How It Works? 31============= 32 33DAMON_RECLAIM finds memory regions that didn't accessed for specific time 34duration and page out. To avoid it consuming too much CPU for the paging out 35operation, a speed limit can be configured. Under the speed limit, it pages 36out memory regions that didn't accessed longer time first. System 37administrators can also configure under what situation this scheme should 38automatically activated and deactivated with three memory pressure watermarks. 39 40Interface: Module Parameters 41============================ 42 43To use this feature, you should first ensure your system is running on a kernel 44that is built with ``CONFIG_DAMON_RECLAIM=y``. 45 46To let sysadmins enable or disable it and tune for the given system, 47DAMON_RECLAIM utilizes module parameters. That is, you can put 48``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write 49proper values to ``/sys/module/damon_reclaim/parameters/<parameter>`` files. 50 51Below are the description of each parameter. 52 53enabled 54------- 55 56Enable or disable DAMON_RECLAIM. 57 58You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. 59Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do 60no real monitoring and reclamation due to the watermarks-based activation 61condition. Refer to below descriptions for the watermarks parameter for this. 62 63commit_inputs 64------------- 65 66Make DAMON_RECLAIM reads the input parameters again, except ``enabled``. 67 68Input parameters that updated while DAMON_RECLAIM is running are not applied 69by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values 70of parametrs except ``enabled`` again. Once the re-reading is done, this 71parameter is set as ``N``. If invalid parameters are found while the 72re-reading, DAMON_RECLAIM will be disabled. 73 74min_age 75------- 76 77Time threshold for cold memory regions identification in microseconds. 78 79If a memory region is not accessed for this or longer time, DAMON_RECLAIM 80identifies the region as cold, and reclaims it. 81 82120 seconds by default. 83 84quota_ms 85-------- 86 87Limit of time for the reclamation in milliseconds. 88 89DAMON_RECLAIM tries to use only up to this time within a time window 90(quota_reset_interval_ms) for trying reclamation of cold pages. This can be 91used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the 92limit is disabled. 93 9410 ms by default. 95 96quota_sz 97-------- 98 99Limit of size of memory for the reclamation in bytes. 100 101DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time 102window (quota_reset_interval_ms) and makes no more than this limit is tried. 103This can be used for limiting consumption of CPU and IO. If this value is 104zero, the limit is disabled. 105 106128 MiB by default. 107 108quota_reset_interval_ms 109----------------------- 110 111The time/size quota charge reset interval in milliseconds. 112 113The charget reset interval for the quota of time (quota_ms) and size 114(quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than 115quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms 116milliseconds. 117 1181 second by default. 119 120wmarks_interval 121--------------- 122 123Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is 124enabled but inactive due to its watermarks rule. 125 126wmarks_high 127----------- 128 129Free memory rate (per thousand) for the high watermark. 130 131If free memory of the system in bytes per thousand bytes is higher than this, 132DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks 133the watermarks. 134 135wmarks_mid 136---------- 137 138Free memory rate (per thousand) for the middle watermark. 139 140If free memory of the system in bytes per thousand bytes is between this and 141the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and 142the reclaiming. 143 144wmarks_low 145---------- 146 147Free memory rate (per thousand) for the low watermark. 148 149If free memory of the system in bytes per thousand bytes is lower than this, 150DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the 151watermarks. In the case, the system falls back to the LRU-list based page 152granularity reclamation logic. 153 154sample_interval 155--------------- 156 157Sampling interval for the monitoring in microseconds. 158 159The sampling interval of DAMON for the cold memory monitoring. Please refer to 160the DAMON documentation (:doc:`usage`) for more detail. 161 162aggr_interval 163------------- 164 165Aggregation interval for the monitoring in microseconds. 166 167The aggregation interval of DAMON for the cold memory monitoring. Please 168refer to the DAMON documentation (:doc:`usage`) for more detail. 169 170min_nr_regions 171-------------- 172 173Minimum number of monitoring regions. 174 175The minimal number of monitoring regions of DAMON for the cold memory 176monitoring. This can be used to set lower-bound of the monitoring quality. 177But, setting this too high could result in increased monitoring overhead. 178Please refer to the DAMON documentation (:doc:`usage`) for more detail. 179 180max_nr_regions 181-------------- 182 183Maximum number of monitoring regions. 184 185The maximum number of monitoring regions of DAMON for the cold memory 186monitoring. This can be used to set upper-bound of the monitoring overhead. 187However, setting this too low could result in bad monitoring quality. Please 188refer to the DAMON documentation (:doc:`usage`) for more detail. 189 190monitor_region_start 191-------------------- 192 193Start of target memory region in physical address. 194 195The start physical address of memory region that DAMON_RECLAIM will do work 196against. That is, DAMON_RECLAIM will find cold memory regions in this region 197and reclaims. By default, biggest System RAM is used as the region. 198 199monitor_region_end 200------------------ 201 202End of target memory region in physical address. 203 204The end physical address of memory region that DAMON_RECLAIM will do work 205against. That is, DAMON_RECLAIM will find cold memory regions in this region 206and reclaims. By default, biggest System RAM is used as the region. 207 208skip_anon 209--------- 210 211Skip anonymous pages reclamation. 212 213If this parameter is set as ``Y``, DAMON_RECLAIM does not reclaim anonymous 214pages. By default, ``N``. 215 216 217kdamond_pid 218----------- 219 220PID of the DAMON thread. 221 222If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else, 223-1. 224 225nr_reclaim_tried_regions 226------------------------ 227 228Number of memory regions that tried to be reclaimed by DAMON_RECLAIM. 229 230bytes_reclaim_tried_regions 231--------------------------- 232 233Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM. 234 235nr_reclaimed_regions 236-------------------- 237 238Number of memory regions that successfully be reclaimed by DAMON_RECLAIM. 239 240bytes_reclaimed_regions 241----------------------- 242 243Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM. 244 245nr_quota_exceeds 246---------------- 247 248Number of times that the time/space quota limits have exceeded. 249 250Example 251======= 252 253Below runtime example commands make DAMON_RECLAIM to find memory regions that 254not accessed for 30 seconds or more and pages out. The reclamation is limited 255to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too 256much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do 257nothing if the system's free memory rate is more than 50%, but start the real 258works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and 259therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to 260do nothing again, so that we can fall back to the LRU-list based page 261granularity reclamation. :: 262 263 # cd /sys/module/damon_reclaim/parameters 264 # echo 30000000 > min_age 265 # echo $((1 * 1024 * 1024 * 1024)) > quota_sz 266 # echo 1000 > quota_reset_interval_ms 267 # echo 500 > wmarks_high 268 # echo 400 > wmarks_mid 269 # echo 200 > wmarks_low 270 # echo Y > enabled 271 272.. [1] https://research.google/pubs/pub48551/ 273.. [2] https://lwn.net/Articles/787611/ 274.. [3] https://www.kernel.org/doc/html/latest/mm/free_page_reporting.html 275