1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. For more detail, 14 please refer to its `usage document 15 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 16- *sysfs interface.* 17 :ref:`This <sysfs_interface>` is for privileged user space programmers who 18 want more optimized use of DAMON. Using this, users can use DAMON’s major 19 features by reading from and writing to special sysfs files. Therefore, 20 you can write and use your personalized DAMON sysfs wrapper programs that 21 reads/writes the sysfs files instead of you. The `DAMON user space tool 22 <https://github.com/awslabs/damo>`_ is one example of such programs. 23- *Kernel Space Programming Interface.* 24 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 25 users can utilize every feature of DAMON most flexibly and efficiently by 26 writing kernel space DAMON application programs for you. You can even extend 27 DAMON for various address spaces. For detail, please refer to the interface 28 :doc:`document </mm/damon/api>`. 29- *debugfs interface. (DEPRECATED!)* 30 :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface 31 <sysfs_interface>`. This is deprecated, so users should move to the 32 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 33 move, please report your usecase to damon@lists.linux.dev and 34 linux-mm@kvack.org. 35 36.. _sysfs_interface: 37 38sysfs Interface 39=============== 40 41DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 42creates multiple directories and files under its sysfs directory, 43``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 44from the files under the directory. 45 46For a short example, users can monitor the virtual address space of a given 47workload as below. :: 48 49 # cd /sys/kernel/mm/damon/admin/ 50 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 51 # echo vaddr > kdamonds/0/contexts/0/operations 52 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 53 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 54 # echo on > kdamonds/0/state 55 56Files Hierarchy 57--------------- 58 59The files hierarchy of DAMON sysfs interface is shown below. In the below 60figure, parents-children relations are represented with indentations, each 61directory is having ``/`` suffix, and files in each directory are separated by 62comma (","). 63 64.. parsed-literal:: 65 66 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 67 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 68 │ │ :ref:`0 <sysfs_kdamond>`/state,pid 69 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 70 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations 71 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 72 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 73 │ │ │ │ │ │ nr_regions/min,max 74 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 75 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target 76 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 77 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 78 │ │ │ │ │ │ │ │ ... 79 │ │ │ │ │ │ ... 80 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 81 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,apply_interval_us 82 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 83 │ │ │ │ │ │ │ │ sz/min,max 84 │ │ │ │ │ │ │ │ nr_accesses/min,max 85 │ │ │ │ │ │ │ │ age/min,max 86 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms 87 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 88 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 89 │ │ │ │ │ │ │ │ │ 0/target_value,current_value 90 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 91 │ │ │ │ │ │ │ :ref:`filters <sysfs_filters>`/nr_filters 92 │ │ │ │ │ │ │ │ 0/type,matching,memcg_id 93 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds 94 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 95 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age 96 │ │ │ │ │ │ │ │ ... 97 │ │ │ │ │ │ ... 98 │ │ │ │ ... 99 │ │ ... 100 101.. _sysfs_root: 102 103Root 104---- 105 106The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 107has one directory named ``admin``. The directory contains the files for 108privileged user space programs' control of DAMON. User space tools or daemons 109having the root permission could use this directory. 110 111.. _sysfs_kdamonds: 112 113kdamonds/ 114--------- 115 116Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 117controlling the kdamonds (refer to 118:ref:`design <damon_design_execution_model_and_data_structures>` for more 119details) exists. In the beginning, this directory has only one file, 120``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 121child directories named ``0`` to ``N-1``. Each directory represents each 122kdamond. 123 124.. _sysfs_kdamond: 125 126kdamonds/<N>/ 127------------- 128 129In each kdamond directory, two files (``state`` and ``pid``) and one directory 130(``contexts``) exist. 131 132Reading ``state`` returns ``on`` if the kdamond is currently running, or 133``off`` if it is not running. 134 135Users can write below commands for the kdamond to the ``state`` file. 136 137- ``on``: Start running. 138- ``off``: Stop running. 139- ``commit``: Read the user inputs in the sysfs files except ``state`` file 140 again. 141- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 142 :ref:`quota goals <sysfs_schemes_quota_goals>`. 143- ``update_schemes_stats``: Update the contents of stats files for each 144 DAMON-based operation scheme of the kdamond. For details of the stats, 145 please refer to :ref:`stats section <sysfs_schemes_stats>`. 146- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 147 action tried regions directory for each DAMON-based operation scheme of the 148 kdamond. For details of the DAMON-based operation scheme action tried 149 regions directory, please refer to 150 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 151- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 152 files. 153- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 154 action tried regions directory for each DAMON-based operation scheme of the 155 kdamond. 156 157If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 158 159``contexts`` directory contains files for controlling the monitoring contexts 160that this kdamond will execute. 161 162.. _sysfs_contexts: 163 164kdamonds/<N>/contexts/ 165---------------------- 166 167In the beginning, this directory has only one file, ``nr_contexts``. Writing a 168number (``N``) to the file creates the number of child directories named as 169``0`` to ``N-1``. Each directory represents each monitoring context (refer to 170:ref:`design <damon_design_execution_model_and_data_structures>` for more 171details). At the moment, only one context per kdamond is supported, so only 172``0`` or ``1`` can be written to the file. 173 174.. _sysfs_context: 175 176contexts/<N>/ 177------------- 178 179In each context directory, two files (``avail_operations`` and ``operations``) 180and three directories (``monitoring_attrs``, ``targets``, and ``schemes``) 181exist. 182 183DAMON supports multiple types of monitoring operations, including those for 184virtual address space and the physical address space. You can get the list of 185available monitoring operations set on the currently running kernel by reading 186``avail_operations`` file. Based on the kernel configuration, the file will 187list some or all of below keywords. 188 189 - vaddr: Monitor virtual address spaces of specific processes 190 - fvaddr: Monitor fixed virtual address ranges 191 - paddr: Monitor the physical address space of the system 192 193Please refer to :ref:`regions sysfs directory <sysfs_regions>` for detailed 194differences between the operations sets in terms of the monitoring target 195regions. 196 197You can set and get what type of monitoring operations DAMON will use for the 198context by writing one of the keywords listed in ``avail_operations`` file and 199reading from the ``operations`` file. 200 201.. _sysfs_monitoring_attrs: 202 203contexts/<N>/monitoring_attrs/ 204------------------------------ 205 206Files for specifying attributes of the monitoring including required quality 207and efficiency of the monitoring are in ``monitoring_attrs`` directory. 208Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 209directory. 210 211Under ``intervals`` directory, three files for DAMON's sampling interval 212(``sample_us``), aggregation interval (``aggr_us``), and update interval 213(``update_us``) exist. You can set and get the values in micro-seconds by 214writing to and reading from the files. 215 216Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 217of DAMON's monitoring regions (``min`` and ``max``, respectively), which 218controls the monitoring overhead, exist. You can set and get the values by 219writing to and rading from the files. 220 221For more details about the intervals and monitoring regions range, please refer 222to the Design document (:doc:`/mm/damon/design`). 223 224.. _sysfs_targets: 225 226contexts/<N>/targets/ 227--------------------- 228 229In the beginning, this directory has only one file, ``nr_targets``. Writing a 230number (``N``) to the file creates the number of child directories named ``0`` 231to ``N-1``. Each directory represents each monitoring target. 232 233.. _sysfs_target: 234 235targets/<N>/ 236------------ 237 238In each target directory, one file (``pid_target``) and one directory 239(``regions``) exist. 240 241If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 242be a process. You can specify the process to DAMON by writing the pid of the 243process to the ``pid_target`` file. 244 245.. _sysfs_regions: 246 247targets/<N>/regions 248------------------- 249 250When ``vaddr`` monitoring operations set is being used (``vaddr`` is written to 251the ``contexts/<N>/operations`` file), DAMON automatically sets and updates the 252monitoring target regions so that entire memory mappings of target processes 253can be covered. However, users could want to set the initial monitoring region 254to specific address ranges. 255 256In contrast, DAMON do not automatically sets and updates the monitoring target 257regions when ``fvaddr`` or ``paddr`` monitoring operations sets are being used 258(``fvaddr`` or ``paddr`` have written to the ``contexts/<N>/operations``). 259Therefore, users should set the monitoring target regions by themselves in the 260cases. 261 262For such cases, users can explicitly set the initial monitoring target regions 263as they want, by writing proper values to the files under this directory. 264 265In the beginning, this directory has only one file, ``nr_regions``. Writing a 266number (``N``) to the file creates the number of child directories named ``0`` 267to ``N-1``. Each directory represents each initial monitoring target region. 268 269.. _sysfs_region: 270 271regions/<N>/ 272------------ 273 274In each region directory, you will find two files (``start`` and ``end``). You 275can set and get the start and end addresses of the initial monitoring target 276region by writing to and reading from the files, respectively. 277 278Each region should not overlap with others. ``end`` of directory ``N`` should 279be equal or smaller than ``start`` of directory ``N+1``. 280 281.. _sysfs_schemes: 282 283contexts/<N>/schemes/ 284--------------------- 285 286The directory for DAMON-based Operation Schemes (:ref:`DAMOS 287<damon_design_damos>`). Users can get and set the schemes by reading from and 288writing to files under this directory. 289 290In the beginning, this directory has only one file, ``nr_schemes``. Writing a 291number (``N``) to the file creates the number of child directories named ``0`` 292to ``N-1``. Each directory represents each DAMON-based operation scheme. 293 294.. _sysfs_scheme: 295 296schemes/<N>/ 297------------ 298 299In each scheme directory, five directories (``access_pattern``, ``quotas``, 300``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and two files 301(``action`` and ``apply_interval``) exist. 302 303The ``action`` file is for setting and getting the scheme's :ref:`action 304<damon_design_damos_action>`. The keywords that can be written to and read 305from the file and their meaning are as below. 306 307Note that support of each action depends on the running DAMON operations set 308:ref:`implementation <sysfs_context>`. 309 310 - ``willneed``: Call ``madvise()`` for the region with ``MADV_WILLNEED``. 311 Supported by ``vaddr`` and ``fvaddr`` operations set. 312 - ``cold``: Call ``madvise()`` for the region with ``MADV_COLD``. 313 Supported by ``vaddr`` and ``fvaddr`` operations set. 314 - ``pageout``: Call ``madvise()`` for the region with ``MADV_PAGEOUT``. 315 Supported by ``vaddr``, ``fvaddr`` and ``paddr`` operations set. 316 - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``. 317 Supported by ``vaddr`` and ``fvaddr`` operations set. 318 - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``. 319 Supported by ``vaddr`` and ``fvaddr`` operations set. 320 - ``lru_prio``: Prioritize the region on its LRU lists. 321 Supported by ``paddr`` operations set. 322 - ``lru_deprio``: Deprioritize the region on its LRU lists. 323 Supported by ``paddr`` operations set. 324 - ``stat``: Do nothing but count the statistics. 325 Supported by all operations sets. 326 327The ``apply_interval_us`` file is for setting and getting the scheme's 328:ref:`apply_interval <damon_design_damos>` in microseconds. 329 330.. _sysfs_access_pattern: 331 332schemes/<N>/access_pattern/ 333--------------------------- 334 335The directory for the target access :ref:`pattern 336<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 337 338Under the ``access_pattern`` directory, three directories (``sz``, 339``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 340exist. You can set and get the access pattern for the given scheme by writing 341to and reading from the ``min`` and ``max`` files under ``sz``, 342``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 343and the ``max`` form a closed interval. 344 345.. _sysfs_quotas: 346 347schemes/<N>/quotas/ 348------------------- 349 350The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 351DAMON-based operation scheme. 352 353Under ``quotas`` directory, three files (``ms``, ``bytes``, 354``reset_interval_ms``) and two directores (``weights`` and ``goals``) exist. 355 356You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 357``reset interval`` in milliseconds by writing the values to the three files, 358respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 359for applying the ``action`` to memory regions of the ``access_pattern``, and to 360apply the action to only up to ``bytes`` bytes of memory regions within the 361``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 362quota limits. 363 364Under ``weights`` directory, three files (``sz_permil``, 365``nr_accesses_permil``, and ``age_permil``) exist. 366You can set the :ref:`prioritization weights 367<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 368in per-thousand unit by writing the values to the three files under the 369``weights`` directory. 370 371.. _sysfs_schemes_quota_goals: 372 373schemes/<N>/quotas/goals/ 374------------------------- 375 376The directory for the :ref:`automatic quota tuning goals 377<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 378scheme. 379 380In the beginning, this directory has only one file, ``nr_goals``. Writing a 381number (``N``) to the file creates the number of child directories named ``0`` 382to ``N-1``. Each directory represents each goal and current achievement. 383Among the multiple feedback, the best one is used. 384 385Each goal directory contains two files, namely ``target_value`` and 386``current_value``. Users can set and get any number to those files to set the 387feedback. User space main workload's latency or throughput, system metrics 388like free memory ratio or memory pressure stall time (PSI) could be example 389metrics for the values. Note that users should write 390``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 391directory <sysfs_kdamond>` to pass the feedback to DAMON. 392 393.. _sysfs_watermarks: 394 395schemes/<N>/watermarks/ 396----------------------- 397 398The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 399given DAMON-based operation scheme. 400 401Under the watermarks directory, five files (``metric``, ``interval_us``, 402``high``, ``mid``, and ``low``) for setting the metric, the time interval 403between check of the metric, and the three watermarks exist. You can set and 404get the five values by writing to the files, respectively. 405 406Keywords and meanings of those that can be written to the ``metric`` file are 407as below. 408 409 - none: Ignore the watermarks 410 - free_mem_rate: System's free memory rate (per thousand) 411 412The ``interval`` should written in microseconds unit. 413 414.. _sysfs_filters: 415 416schemes/<N>/filters/ 417-------------------- 418 419The directory for the :ref:`filters <damon_design_damos_filters>` of the given 420DAMON-based operation scheme. 421 422In the beginning, this directory has only one file, ``nr_filters``. Writing a 423number (``N``) to the file creates the number of child directories named ``0`` 424to ``N-1``. Each directory represents each filter. The filters are evaluated 425in the numeric order. 426 427Each filter directory contains six files, namely ``type``, ``matcing``, 428``memcg_path``, ``addr_start``, ``addr_end``, and ``target_idx``. To ``type`` 429file, you can write one of four special keywords: ``anon`` for anonymous pages, 430``memcg`` for specific memory cgroup, ``addr`` for specific address range (an 431open-ended interval), or ``target`` for specific DAMON monitoring target 432filtering. In case of the memory cgroup filtering, you can specify the memory 433cgroup of the interest by writing the path of the memory cgroup from the 434cgroups mount point to ``memcg_path`` file. In case of the address range 435filtering, you can specify the start and end address of the range to 436``addr_start`` and ``addr_end`` files, respectively. For the DAMON monitoring 437target filtering, you can specify the index of the target between the list of 438the DAMON context's monitoring targets list to ``target_idx`` file. You can 439write ``Y`` or ``N`` to ``matching`` file to filter out pages that does or does 440not match to the type, respectively. Then, the scheme's action will not be 441applied to the pages that specified to be filtered out. 442 443For example, below restricts a DAMOS action to be applied to only non-anonymous 444pages of all memory cgroups except ``/having_care_already``.:: 445 446 # echo 2 > nr_filters 447 # # filter out anonymous pages 448 echo anon > 0/type 449 echo Y > 0/matching 450 # # further filter out all cgroups except one at '/having_care_already' 451 echo memcg > 1/type 452 echo /having_care_already > 1/memcg_path 453 echo N > 1/matching 454 455Note that ``anon`` and ``memcg`` filters are currently supported only when 456``paddr`` :ref:`implementation <sysfs_context>` is being used. 457 458Also, memory regions that are filtered out by ``addr`` or ``target`` filters 459are not counted as the scheme has tried to those, while regions that filtered 460out by other type filters are counted as the scheme has tried to. The 461difference is applied to :ref:`stats <damos_stats>` and 462:ref:`tried regions <sysfs_schemes_tried_regions>`. 463 464.. _sysfs_schemes_stats: 465 466schemes/<N>/stats/ 467------------------ 468 469DAMON counts the total number and bytes of regions that each scheme is tried to 470be applied, the two numbers for the regions that each scheme is successfully 471applied, and the total number of the quota limit exceeds. This statistics can 472be used for online analysis or tuning of the schemes. 473 474The statistics can be retrieved by reading the files under ``stats`` directory 475(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and 476``qt_exceeds``), respectively. The files are not updated in real time, so you 477should ask DAMON sysfs interface to update the content of the files for the 478stats by writing a special keyword, ``update_schemes_stats`` to the relevant 479``kdamonds/<N>/state`` file. 480 481.. _sysfs_schemes_tried_regions: 482 483schemes/<N>/tried_regions/ 484-------------------------- 485 486This directory initially has one file, ``total_bytes``. 487 488When a special keyword, ``update_schemes_tried_regions``, is written to the 489relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 490that reading it returns the total size of the scheme tried regions, and creates 491directories named integer starting from ``0`` under this directory. Each 492directory contains files exposing detailed information about each of the memory 493region that the corresponding scheme's ``action`` has tried to be applied under 494this directory, during next :ref:`apply interval <damon_design_damos>` of the 495corresponding scheme. The information includes address range, ``nr_accesses``, 496and ``age`` of the region. 497 498Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 499file will only update the ``total_bytes`` file, and will not create the 500subdirectories. 501 502The directories will be removed when another special keyword, 503``clear_schemes_tried_regions``, is written to the relevant 504``kdamonds/<N>/state`` file. 505 506The expected usage of this directory is investigations of schemes' behaviors, 507and query-like efficient data access monitoring results retrievals. For the 508latter use case, in particular, users can set the ``action`` as ``stat`` and 509set the ``access pattern`` as their interested pattern that they want to query. 510 511.. _sysfs_schemes_tried_region: 512 513tried_regions/<N>/ 514------------------ 515 516In each region directory, you will find four files (``start``, ``end``, 517``nr_accesses``, and ``age``). Reading the files will show the start and end 518addresses, ``nr_accesses``, and ``age`` of the region that corresponding 519DAMON-based operation scheme ``action`` has tried to be applied. 520 521Example 522~~~~~~~ 523 524Below commands applies a scheme saying "If a memory region of size in [4KiB, 5258KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 526interval in [10, 20], page out the region. For the paging out, use only up to 52710ms per second, and also don't page out more than 1GiB per second. Under the 528limitation, page out memory regions having longer age first. Also, check the 529free memory rate of the system every 5 seconds, start the monitoring and paging 530out when the free memory rate becomes lower than 50%, but stop it if the free 531memory rate becomes larger than 60%, or lower than 30%". :: 532 533 # cd <sysfs>/kernel/mm/damon/admin 534 # # populate directories 535 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 536 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 537 # cd kdamonds/0/contexts/0/schemes/0 538 # # set the basic access pattern and the action 539 # echo 4096 > access_pattern/sz/min 540 # echo 8192 > access_pattern/sz/max 541 # echo 0 > access_pattern/nr_accesses/min 542 # echo 5 > access_pattern/nr_accesses/max 543 # echo 10 > access_pattern/age/min 544 # echo 20 > access_pattern/age/max 545 # echo pageout > action 546 # # set quotas 547 # echo 10 > quotas/ms 548 # echo $((1024*1024*1024)) > quotas/bytes 549 # echo 1000 > quotas/reset_interval_ms 550 # # set watermark 551 # echo free_mem_rate > watermarks/metric 552 # echo 5000000 > watermarks/interval_us 553 # echo 600 > watermarks/high 554 # echo 500 > watermarks/mid 555 # echo 300 > watermarks/low 556 557Please note that it's highly recommended to use user space tools like `damo 558<https://github.com/awslabs/damo>`_ rather than manually reading and writing 559the files as above. Above is only for an example. 560 561.. _tracepoint: 562 563Tracepoints for Monitoring Results 564================================== 565 566Users can get the monitoring results via the :ref:`tried_regions 567<sysfs_schemes_tried_regions>`. The interface is useful for getting a 568snapshot, but it could be inefficient for fully recording all the monitoring 569results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 570and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 571provides the whole monitoring results, while ``damon:damos_before_apply`` 572provides the monitoring results for regions that each DAMON-based Operation 573Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 574``damon:damos_before_apply`` is more useful for recording internal behavior of 575DAMOS, or DAMOS target access 576:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 577monitoring results recording. 578 579While the monitoring is turned on, you could record the tracepoint events and 580show results using tracepoint supporting tools like ``perf``. For example:: 581 582 # echo on > kdamonds/0/state 583 # perf record -e damon:damon_aggregated & 584 # sleep 5 585 # kill 9 $(pidof perf) 586 # echo off > kdamonds/0/state 587 # perf script 588 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 589 [...] 590 591Each line of the perf script output represents each monitoring region. The 592first five fields are as usual other tracepoint outputs. The sixth field 593(``target_id=X``) shows the ide of the monitoring target of the region. The 594seventh field (``nr_regions=X``) shows the total number of monitoring regions 595for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 596(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 597``nr_accesses`` of the region (refer to 598:ref:`design <damon_design_region_based_sampling>` for more details of the 599counter). Finally the tenth field (``X``) shows the ``age`` of the region 600(refer to :ref:`design <damon_design_age_tracking>` for more details of the 601counter). 602 603If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would 604be somewhat like below:: 605 606 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 607 [...] 608 609Each line of the output represents each monitoring region that each DAMON-based 610Operation Scheme was about to be applied at the traced time. The first five 611fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 612of the scheme in the list of the contexts of the context's kdamond, the index 613of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 614addition to the output of ``damon_aggregated`` tracepoint. 615 616 617.. _debugfs_interface: 618 619debugfs Interface (DEPRECATED!) 620=============================== 621 622.. note:: 623 624 THIS IS DEPRECATED! 625 626 DAMON debugfs interface is deprecated, so users should move to the 627 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 628 move, please report your usecase to damon@lists.linux.dev and 629 linux-mm@kvack.org. 630 631DAMON exports nine files, ``DEPRECATED``, ``attrs``, ``target_ids``, 632``init_regions``, ``schemes``, ``monitor_on_DEPRECATED``, ``kdamond_pid``, 633``mk_contexts`` and ``rm_contexts`` under its debugfs directory, 634``<debugfs>/damon/``. 635 636 637``DEPRECATED`` is a read-only file for the DAMON debugfs interface deprecation 638notice. Reading it returns the deprecation notice, as below:: 639 640 # cat DEPRECATED 641 DAMON debugfs interface is deprecated, so users should move to DAMON_SYSFS. If you cannot, please report your usecase to damon@lists.linux.dev and linux-mm@kvack.org. 642 643 644Attributes 645---------- 646 647Users can get and set the ``sampling interval``, ``aggregation interval``, 648``update interval``, and min/max number of monitoring target regions by 649reading from and writing to the ``attrs`` file. To know about the monitoring 650attributes in detail, please refer to the :doc:`/mm/damon/design`. For 651example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 6521000, and then check it again:: 653 654 # cd <debugfs>/damon 655 # echo 5000 100000 1000000 10 1000 > attrs 656 # cat attrs 657 5000 100000 1000000 10 1000 658 659 660Target IDs 661---------- 662 663Some types of address spaces supports multiple monitoring target. For example, 664the virtual memory address spaces monitoring can have multiple processes as the 665monitoring targets. Users can set the targets by writing relevant id values of 666the targets to, and get the ids of the current targets by reading from the 667``target_ids`` file. In case of the virtual address spaces monitoring, the 668values should be pids of the monitoring target processes. For example, below 669commands set processes having pids 42 and 4242 as the monitoring targets and 670check it again:: 671 672 # cd <debugfs>/damon 673 # echo 42 4242 > target_ids 674 # cat target_ids 675 42 4242 676 677Users can also monitor the physical memory address space of the system by 678writing a special keyword, "``paddr\n``" to the file. Because physical address 679space monitoring doesn't support multiple targets, reading the file will show a 680fake value, ``42``, as below:: 681 682 # cd <debugfs>/damon 683 # echo paddr > target_ids 684 # cat target_ids 685 42 686 687Note that setting the target ids doesn't start the monitoring. 688 689 690Initial Monitoring Target Regions 691--------------------------------- 692 693In case of the virtual address space monitoring, DAMON automatically sets and 694updates the monitoring target regions so that entire memory mappings of target 695processes can be covered. However, users can want to limit the monitoring 696region to specific address ranges, such as the heap, the stack, or specific 697file-mapped area. Or, some users can know the initial access pattern of their 698workloads and therefore want to set optimal initial regions for the 'adaptive 699regions adjustment'. 700 701In contrast, DAMON do not automatically sets and updates the monitoring target 702regions in case of physical memory monitoring. Therefore, users should set the 703monitoring target regions by themselves. 704 705In such cases, users can explicitly set the initial monitoring target regions 706as they want, by writing proper values to the ``init_regions`` file. The input 707should be a sequence of three integers separated by white spaces that represent 708one region in below form.:: 709 710 <target idx> <start address> <end address> 711 712The ``target idx`` should be the index of the target in ``target_ids`` file, 713starting from ``0``, and the regions should be passed in address order. For 714example, below commands will set a couple of address ranges, ``1-100`` and 715``100-200`` as the initial monitoring target region of pid 42, which is the 716first one (index ``0``) in ``target_ids``, and another couple of address 717ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one 718(index ``1``) in ``target_ids``.:: 719 720 # cd <debugfs>/damon 721 # cat target_ids 722 42 4242 723 # echo "0 1 100 \ 724 0 100 200 \ 725 1 20 40 \ 726 1 50 100" > init_regions 727 728Note that this sets the initial monitoring target regions only. In case of 729virtual memory monitoring, DAMON will automatically updates the boundary of the 730regions after one ``update interval``. Therefore, users should set the 731``update interval`` large enough in this case, if they don't want the 732update. 733 734 735Schemes 736------- 737 738Users can get and set the DAMON-based operation :ref:`schemes 739<damon_design_damos>` by reading from and writing to ``schemes`` debugfs file. 740Reading the file also shows the statistics of each scheme. To the file, each 741of the schemes should be represented in each line in below form:: 742 743 <target access pattern> <action> <quota> <watermarks> 744 745You can disable schemes by simply writing an empty string to the file. 746 747Target Access Pattern 748~~~~~~~~~~~~~~~~~~~~~ 749 750The target access :ref:`pattern <damon_design_damos_access_pattern>` of the 751scheme. The ``<target access pattern>`` is constructed with three ranges in 752below form:: 753 754 min-size max-size min-acc max-acc min-age max-age 755 756Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 757number of monitored accesses per aggregate interval for access frequency 758(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 759regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 760closed interval. 761 762Action 763~~~~~~ 764 765The ``<action>`` is a predefined integer for memory management :ref:`actions 766<damon_design_damos_action>`. The supported numbers and their meanings are as 767below. 768 769 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``. Ignored if 770 ``target`` is ``paddr``. 771 - 1: Call ``madvise()`` for the region with ``MADV_COLD``. Ignored if 772 ``target`` is ``paddr``. 773 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT``. 774 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``. Ignored if 775 ``target`` is ``paddr``. 776 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``. Ignored if 777 ``target`` is ``paddr``. 778 - 5: Do nothing but count the statistics 779 780Quota 781~~~~~ 782 783Users can set the :ref:`quotas <damon_design_damos_quotas>` of the given scheme 784via the ``<quota>`` in below form:: 785 786 <ms> <sz> <reset interval> <priority weights> 787 788This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 789the action to memory regions of the ``target access pattern`` within the 790``<reset interval>`` milliseconds, and to apply the action to only up to 791``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 792``<ms>`` and ``<sz>`` zero disables the quota limits. 793 794For the :ref:`prioritization <damon_design_damos_quotas_prioritization>`, users 795can set the weights for the three properties in ``<priority weights>`` in below 796form:: 797 798 <size weight> <access frequency weight> <age weight> 799 800Watermarks 801~~~~~~~~~~ 802 803Users can specify :ref:`watermarks <damon_design_damos_watermarks>` of the 804given scheme via ``<watermarks>`` in below form:: 805 806 <metric> <check interval> <high mark> <middle mark> <low mark> 807 808``<metric>`` is a predefined integer for the metric to be checked. The 809supported numbers and their meanings are as below. 810 811 - 0: Ignore the watermarks 812 - 1: System's free memory rate (per thousand) 813 814The value of the metric is checked every ``<check interval>`` microseconds. 815 816If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 817scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 818is activated. 819 820.. _damos_stats: 821 822Statistics 823~~~~~~~~~~ 824 825It also counts the total number and bytes of regions that each scheme is tried 826to be applied, the two numbers for the regions that each scheme is successfully 827applied, and the total number of the quota limit exceeds. This statistics can 828be used for online analysis or tuning of the schemes. 829 830The statistics can be shown by reading the ``schemes`` file. Reading the file 831will show each scheme you entered in each line, and the five numbers for the 832statistics will be added at the end of each line. 833 834Example 835~~~~~~~ 836 837Below commands applies a scheme saying "If a memory region of size in [4KiB, 8388KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 839interval in [10, 20], page out the region. For the paging out, use only up to 84010ms per second, and also don't page out more than 1GiB per second. Under the 841limitation, page out memory regions having longer age first. Also, check the 842free memory rate of the system every 5 seconds, start the monitoring and paging 843out when the free memory rate becomes lower than 50%, but stop it if the free 844memory rate becomes larger than 60%, or lower than 30%".:: 845 846 # cd <debugfs>/damon 847 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 848 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 849 # scheme+=" 0 0 100" # prioritization weights 850 # scheme+=" 1 5000000 600 500 300" # watermarks 851 # echo "$scheme" > schemes 852 853 854Turning On/Off 855-------------- 856 857Setting the files as described above doesn't incur effect unless you explicitly 858start the monitoring. You can start, stop, and check the current status of the 859monitoring by writing to and reading from the ``monitor_on_DEPRECATED`` file. 860Writing ``on`` to the file starts the monitoring of the targets with the 861attributes. Writing ``off`` to the file stops those. DAMON also stops if 862every target process is terminated. Below example commands turn on, off, and 863check the status of DAMON:: 864 865 # cd <debugfs>/damon 866 # echo on > monitor_on_DEPRECATED 867 # echo off > monitor_on_DEPRECATED 868 # cat monitor_on_DEPRECATED 869 off 870 871Please note that you cannot write to the above-mentioned debugfs files while 872the monitoring is turned on. If you write to the files while DAMON is running, 873an error code such as ``-EBUSY`` will be returned. 874 875 876Monitoring Thread PID 877--------------------- 878 879DAMON does requested monitoring with a kernel thread called ``kdamond``. You 880can get the pid of the thread by reading the ``kdamond_pid`` file. When the 881monitoring is turned off, reading the file returns ``none``. :: 882 883 # cd <debugfs>/damon 884 # cat monitor_on_DEPRECATED 885 off 886 # cat kdamond_pid 887 none 888 # echo on > monitor_on_DEPRECATED 889 # cat kdamond_pid 890 18594 891 892 893Using Multiple Monitoring Threads 894--------------------------------- 895 896One ``kdamond`` thread is created for each monitoring context. You can create 897and remove monitoring contexts for multiple ``kdamond`` required use case using 898the ``mk_contexts`` and ``rm_contexts`` files. 899 900Writing the name of the new context to the ``mk_contexts`` file creates a 901directory of the name on the DAMON debugfs directory. The directory will have 902DAMON debugfs files for the context. :: 903 904 # cd <debugfs>/damon 905 # ls foo 906 # ls: cannot access 'foo': No such file or directory 907 # echo foo > mk_contexts 908 # ls foo 909 # attrs init_regions kdamond_pid schemes target_ids 910 911If the context is not needed anymore, you can remove it and the corresponding 912directory by putting the name of the context to the ``rm_contexts`` file. :: 913 914 # echo foo > rm_contexts 915 # ls foo 916 # ls: cannot access 'foo': No such file or directory 917 918Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on_DEPRECATED`` files 919are in the root directory only. 920