1.. SPDX-License-Identifier: GPL-2.0 2 3======================= 4DAMON-based Reclamation 5======================= 6 7DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to 8be used for proactive and lightweight reclamation under light memory pressure. 9It doesn't aim to replace the LRU-list based page_granularity reclamation, but 10to be selectively used for different level of memory pressure and requirements. 11 12Where Proactive Reclamation is Required? 13======================================== 14 15On general memory over-committed systems, proactively reclaiming cold pages 16helps saving memory and reducing latency spikes that incurred by the direct 17reclaim of the process or CPU consumption of kswapd, while incurring only 18minimal performance degradation [1]_ [2]_ . 19 20Free Pages Reporting [3]_ based memory over-commit virtualization systems are 21good example of the cases. In such systems, the guest VMs reports their free 22memory to host, and the host reallocates the reported memory to other guests. 23As a result, the memory of the systems are fully utilized. However, the 24guests could be not so memory-frugal, mainly because some kernel subsystems and 25user-space applications are designed to use as much memory as available. Then, 26guests could report only small amount of memory as free to host, results in 27memory utilization drop of the systems. Running the proactive reclamation in 28guests could mitigate this problem. 29 30How It Works? 31============= 32 33DAMON_RECLAIM finds memory regions that didn't accessed for specific time 34duration and page out. To avoid it consuming too much CPU for the paging out 35operation, a speed limit can be configured. Under the speed limit, it pages 36out memory regions that didn't accessed longer time first. System 37administrators can also configure under what situation this scheme should 38automatically activated and deactivated with three memory pressure watermarks. 39 40Interface: Module Parameters 41============================ 42 43To use this feature, you should first ensure your system is running on a kernel 44that is built with ``CONFIG_DAMON_RECLAIM=y``. 45 46To let sysadmins enable or disable it and tune for the given system, 47DAMON_RECLAIM utilizes module parameters. That is, you can put 48``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write 49proper values to ``/sys/modules/damon_reclaim/parameters/<parameter>`` files. 50 51Note that the parameter values except ``enabled`` are applied only when 52DAMON_RECLAIM starts. Therefore, if you want to apply new parameter values in 53runtime and DAMON_RECLAIM is already enabled, you should disable and re-enable 54it via ``enabled`` parameter file. Writing of the new values to proper 55parameter values should be done before the re-enablement. 56 57Below are the description of each parameter. 58 59enabled 60------- 61 62Enable or disable DAMON_RECLAIM. 63 64You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. 65Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do 66no real monitoring and reclamation due to the watermarks-based activation 67condition. Refer to below descriptions for the watermarks parameter for this. 68 69min_age 70------- 71 72Time threshold for cold memory regions identification in microseconds. 73 74If a memory region is not accessed for this or longer time, DAMON_RECLAIM 75identifies the region as cold, and reclaims it. 76 77120 seconds by default. 78 79quota_ms 80-------- 81 82Limit of time for the reclamation in milliseconds. 83 84DAMON_RECLAIM tries to use only up to this time within a time window 85(quota_reset_interval_ms) for trying reclamation of cold pages. This can be 86used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the 87limit is disabled. 88 8910 ms by default. 90 91quota_sz 92-------- 93 94Limit of size of memory for the reclamation in bytes. 95 96DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time 97window (quota_reset_interval_ms) and makes no more than this limit is tried. 98This can be used for limiting consumption of CPU and IO. If this value is 99zero, the limit is disabled. 100 101128 MiB by default. 102 103quota_reset_interval_ms 104----------------------- 105 106The time/size quota charge reset interval in milliseconds. 107 108The charget reset interval for the quota of time (quota_ms) and size 109(quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than 110quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms 111milliseconds. 112 1131 second by default. 114 115wmarks_interval 116--------------- 117 118Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is 119enabled but inactive due to its watermarks rule. 120 121wmarks_high 122----------- 123 124Free memory rate (per thousand) for the high watermark. 125 126If free memory of the system in bytes per thousand bytes is higher than this, 127DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks 128the watermarks. 129 130wmarks_mid 131---------- 132 133Free memory rate (per thousand) for the middle watermark. 134 135If free memory of the system in bytes per thousand bytes is between this and 136the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and 137the reclaiming. 138 139wmarks_low 140---------- 141 142Free memory rate (per thousand) for the low watermark. 143 144If free memory of the system in bytes per thousand bytes is lower than this, 145DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the 146watermarks. In the case, the system falls back to the LRU-list based page 147granularity reclamation logic. 148 149sample_interval 150--------------- 151 152Sampling interval for the monitoring in microseconds. 153 154The sampling interval of DAMON for the cold memory monitoring. Please refer to 155the DAMON documentation (:doc:`usage`) for more detail. 156 157aggr_interval 158------------- 159 160Aggregation interval for the monitoring in microseconds. 161 162The aggregation interval of DAMON for the cold memory monitoring. Please 163refer to the DAMON documentation (:doc:`usage`) for more detail. 164 165min_nr_regions 166-------------- 167 168Minimum number of monitoring regions. 169 170The minimal number of monitoring regions of DAMON for the cold memory 171monitoring. This can be used to set lower-bound of the monitoring quality. 172But, setting this too high could result in increased monitoring overhead. 173Please refer to the DAMON documentation (:doc:`usage`) for more detail. 174 175max_nr_regions 176-------------- 177 178Maximum number of monitoring regions. 179 180The maximum number of monitoring regions of DAMON for the cold memory 181monitoring. This can be used to set upper-bound of the monitoring overhead. 182However, setting this too low could result in bad monitoring quality. Please 183refer to the DAMON documentation (:doc:`usage`) for more detail. 184 185monitor_region_start 186-------------------- 187 188Start of target memory region in physical address. 189 190The start physical address of memory region that DAMON_RECLAIM will do work 191against. That is, DAMON_RECLAIM will find cold memory regions in this region 192and reclaims. By default, biggest System RAM is used as the region. 193 194monitor_region_end 195------------------ 196 197End of target memory region in physical address. 198 199The end physical address of memory region that DAMON_RECLAIM will do work 200against. That is, DAMON_RECLAIM will find cold memory regions in this region 201and reclaims. By default, biggest System RAM is used as the region. 202 203kdamond_pid 204----------- 205 206PID of the DAMON thread. 207 208If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else, 209-1. 210 211nr_reclaim_tried_regions 212------------------------ 213 214Number of memory regions that tried to be reclaimed by DAMON_RECLAIM. 215 216bytes_reclaim_tried_regions 217--------------------------- 218 219Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM. 220 221nr_reclaimed_regions 222-------------------- 223 224Number of memory regions that successfully be reclaimed by DAMON_RECLAIM. 225 226bytes_reclaimed_regions 227----------------------- 228 229Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM. 230 231nr_quota_exceeds 232---------------- 233 234Number of times that the time/space quota limits have exceeded. 235 236Example 237======= 238 239Below runtime example commands make DAMON_RECLAIM to find memory regions that 240not accessed for 30 seconds or more and pages out. The reclamation is limited 241to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too 242much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do 243nothing if the system's free memory rate is more than 50%, but start the real 244works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and 245therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to 246do nothing again, so that we can fall back to the LRU-list based page 247granularity reclamation. :: 248 249 # cd /sys/modules/damon_reclaim/parameters 250 # echo 30000000 > min_age 251 # echo $((1 * 1024 * 1024 * 1024)) > quota_sz 252 # echo 1000 > quota_reset_interval_ms 253 # echo 500 > wmarks_high 254 # echo 400 > wmarks_mid 255 # echo 200 > wmarks_low 256 # echo Y > enabled 257 258.. [1] https://research.google/pubs/pub48551/ 259.. [2] https://lwn.net/Articles/787611/ 260.. [3] https://www.kernel.org/doc/html/latest/vm/free_page_reporting.html 261