1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. For more detail, 14 please refer to its `usage document 15 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 16- *sysfs interface.* 17 :ref:`This <sysfs_interface>` is for privileged user space programmers who 18 want more optimized use of DAMON. Using this, users can use DAMON’s major 19 features by reading from and writing to special sysfs files. Therefore, 20 you can write and use your personalized DAMON sysfs wrapper programs that 21 reads/writes the sysfs files instead of you. The `DAMON user space tool 22 <https://github.com/awslabs/damo>`_ is one example of such programs. 23- *Kernel Space Programming Interface.* 24 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 25 users can utilize every feature of DAMON most flexibly and efficiently by 26 writing kernel space DAMON application programs for you. You can even extend 27 DAMON for various address spaces. For detail, please refer to the interface 28 :doc:`document </mm/damon/api>`. 29- *debugfs interface. (DEPRECATED!)* 30 :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface 31 <sysfs_interface>`. This is deprecated, so users should move to the 32 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 33 move, please report your usecase to damon@lists.linux.dev and 34 linux-mm@kvack.org. 35 36.. _sysfs_interface: 37 38sysfs Interface 39=============== 40 41DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 42creates multiple directories and files under its sysfs directory, 43``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 44from the files under the directory. 45 46For a short example, users can monitor the virtual address space of a given 47workload as below. :: 48 49 # cd /sys/kernel/mm/damon/admin/ 50 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 51 # echo vaddr > kdamonds/0/contexts/0/operations 52 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 53 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 54 # echo on > kdamonds/0/state 55 56Files Hierarchy 57--------------- 58 59The files hierarchy of DAMON sysfs interface is shown below. In the below 60figure, parents-children relations are represented with indentations, each 61directory is having ``/`` suffix, and files in each directory are separated by 62comma (","). 63 64.. parsed-literal:: 65 66 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 67 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 68 │ │ :ref:`0 <sysfs_kdamond>`/state,pid 69 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 70 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations 71 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 72 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 73 │ │ │ │ │ │ nr_regions/min,max 74 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 75 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target 76 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 77 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 78 │ │ │ │ │ │ │ │ ... 79 │ │ │ │ │ │ ... 80 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 81 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,apply_interval_us 82 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 83 │ │ │ │ │ │ │ │ sz/min,max 84 │ │ │ │ │ │ │ │ nr_accesses/min,max 85 │ │ │ │ │ │ │ │ age/min,max 86 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,effective_bytes 87 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 88 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 89 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value 90 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 91 │ │ │ │ │ │ │ :ref:`filters <sysfs_filters>`/nr_filters 92 │ │ │ │ │ │ │ │ 0/type,matching,memcg_id 93 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds 94 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 95 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age 96 │ │ │ │ │ │ │ │ ... 97 │ │ │ │ │ │ ... 98 │ │ │ │ ... 99 │ │ ... 100 101.. _sysfs_root: 102 103Root 104---- 105 106The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 107has one directory named ``admin``. The directory contains the files for 108privileged user space programs' control of DAMON. User space tools or daemons 109having the root permission could use this directory. 110 111.. _sysfs_kdamonds: 112 113kdamonds/ 114--------- 115 116Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 117controlling the kdamonds (refer to 118:ref:`design <damon_design_execution_model_and_data_structures>` for more 119details) exists. In the beginning, this directory has only one file, 120``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 121child directories named ``0`` to ``N-1``. Each directory represents each 122kdamond. 123 124.. _sysfs_kdamond: 125 126kdamonds/<N>/ 127------------- 128 129In each kdamond directory, two files (``state`` and ``pid``) and one directory 130(``contexts``) exist. 131 132Reading ``state`` returns ``on`` if the kdamond is currently running, or 133``off`` if it is not running. 134 135Users can write below commands for the kdamond to the ``state`` file. 136 137- ``on``: Start running. 138- ``off``: Stop running. 139- ``commit``: Read the user inputs in the sysfs files except ``state`` file 140 again. 141- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 142 :ref:`quota goals <sysfs_schemes_quota_goals>`. 143- ``update_schemes_stats``: Update the contents of stats files for each 144 DAMON-based operation scheme of the kdamond. For details of the stats, 145 please refer to :ref:`stats section <sysfs_schemes_stats>`. 146- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 147 action tried regions directory for each DAMON-based operation scheme of the 148 kdamond. For details of the DAMON-based operation scheme action tried 149 regions directory, please refer to 150 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 151- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 152 files. 153- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 154 action tried regions directory for each DAMON-based operation scheme of the 155 kdamond. 156- ``update_schemes_effective_bytes``: Update the contents of 157 ``effective_bytes`` files for each DAMON-based operation scheme of the 158 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`. 159 160If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 161 162``contexts`` directory contains files for controlling the monitoring contexts 163that this kdamond will execute. 164 165.. _sysfs_contexts: 166 167kdamonds/<N>/contexts/ 168---------------------- 169 170In the beginning, this directory has only one file, ``nr_contexts``. Writing a 171number (``N``) to the file creates the number of child directories named as 172``0`` to ``N-1``. Each directory represents each monitoring context (refer to 173:ref:`design <damon_design_execution_model_and_data_structures>` for more 174details). At the moment, only one context per kdamond is supported, so only 175``0`` or ``1`` can be written to the file. 176 177.. _sysfs_context: 178 179contexts/<N>/ 180------------- 181 182In each context directory, two files (``avail_operations`` and ``operations``) 183and three directories (``monitoring_attrs``, ``targets``, and ``schemes``) 184exist. 185 186DAMON supports multiple types of :ref:`monitoring operations 187<damon_design_configurable_operations_set>`, including those for virtual address 188space and the physical address space. You can get the list of available 189monitoring operations set on the currently running kernel by reading 190``avail_operations`` file. Based on the kernel configuration, the file will 191list different available operation sets. Please refer to the :ref:`design 192<damon_operations_set>` for the list of all available operation sets and their 193brief explanations. 194 195You can set and get what type of monitoring operations DAMON will use for the 196context by writing one of the keywords listed in ``avail_operations`` file and 197reading from the ``operations`` file. 198 199.. _sysfs_monitoring_attrs: 200 201contexts/<N>/monitoring_attrs/ 202------------------------------ 203 204Files for specifying attributes of the monitoring including required quality 205and efficiency of the monitoring are in ``monitoring_attrs`` directory. 206Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 207directory. 208 209Under ``intervals`` directory, three files for DAMON's sampling interval 210(``sample_us``), aggregation interval (``aggr_us``), and update interval 211(``update_us``) exist. You can set and get the values in micro-seconds by 212writing to and reading from the files. 213 214Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 215of DAMON's monitoring regions (``min`` and ``max``, respectively), which 216controls the monitoring overhead, exist. You can set and get the values by 217writing to and rading from the files. 218 219For more details about the intervals and monitoring regions range, please refer 220to the Design document (:doc:`/mm/damon/design`). 221 222.. _sysfs_targets: 223 224contexts/<N>/targets/ 225--------------------- 226 227In the beginning, this directory has only one file, ``nr_targets``. Writing a 228number (``N``) to the file creates the number of child directories named ``0`` 229to ``N-1``. Each directory represents each monitoring target. 230 231.. _sysfs_target: 232 233targets/<N>/ 234------------ 235 236In each target directory, one file (``pid_target``) and one directory 237(``regions``) exist. 238 239If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 240be a process. You can specify the process to DAMON by writing the pid of the 241process to the ``pid_target`` file. 242 243.. _sysfs_regions: 244 245targets/<N>/regions 246------------------- 247 248In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are 249required to set the monitoring target address ranges. In case of ``vaddr`` 250operations set, it is not mandatory, but users can optionally set the initial 251monitoring region to specific address ranges. Please refer to the :ref:`design 252<damon_design_vaddr_target_regions_construction>` for more details. 253 254For such cases, users can explicitly set the initial monitoring target regions 255as they want, by writing proper values to the files under this directory. 256 257In the beginning, this directory has only one file, ``nr_regions``. Writing a 258number (``N``) to the file creates the number of child directories named ``0`` 259to ``N-1``. Each directory represents each initial monitoring target region. 260 261.. _sysfs_region: 262 263regions/<N>/ 264------------ 265 266In each region directory, you will find two files (``start`` and ``end``). You 267can set and get the start and end addresses of the initial monitoring target 268region by writing to and reading from the files, respectively. 269 270Each region should not overlap with others. ``end`` of directory ``N`` should 271be equal or smaller than ``start`` of directory ``N+1``. 272 273.. _sysfs_schemes: 274 275contexts/<N>/schemes/ 276--------------------- 277 278The directory for DAMON-based Operation Schemes (:ref:`DAMOS 279<damon_design_damos>`). Users can get and set the schemes by reading from and 280writing to files under this directory. 281 282In the beginning, this directory has only one file, ``nr_schemes``. Writing a 283number (``N``) to the file creates the number of child directories named ``0`` 284to ``N-1``. Each directory represents each DAMON-based operation scheme. 285 286.. _sysfs_scheme: 287 288schemes/<N>/ 289------------ 290 291In each scheme directory, five directories (``access_pattern``, ``quotas``, 292``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and two files 293(``action`` and ``apply_interval``) exist. 294 295The ``action`` file is for setting and getting the scheme's :ref:`action 296<damon_design_damos_action>`. The keywords that can be written to and read 297from the file and their meaning are same to those of the list on 298:ref:`design doc <damon_design_damos_action>`. 299 300The ``apply_interval_us`` file is for setting and getting the scheme's 301:ref:`apply_interval <damon_design_damos>` in microseconds. 302 303.. _sysfs_access_pattern: 304 305schemes/<N>/access_pattern/ 306--------------------------- 307 308The directory for the target access :ref:`pattern 309<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 310 311Under the ``access_pattern`` directory, three directories (``sz``, 312``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 313exist. You can set and get the access pattern for the given scheme by writing 314to and reading from the ``min`` and ``max`` files under ``sz``, 315``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 316and the ``max`` form a closed interval. 317 318.. _sysfs_quotas: 319 320schemes/<N>/quotas/ 321------------------- 322 323The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 324DAMON-based operation scheme. 325 326Under ``quotas`` directory, four files (``ms``, ``bytes``, 327``reset_interval_ms``, ``effective_bytes``) and two directores (``weights`` and 328``goals``) exist. 329 330You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 331``reset interval`` in milliseconds by writing the values to the three files, 332respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 333for applying the ``action`` to memory regions of the ``access_pattern``, and to 334apply the action to only up to ``bytes`` bytes of memory regions within the 335``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 336quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is 337set. 338 339The time quota is internally transformed to a size quota. Between the 340transformed size quota and user-specified size quota, smaller one is applied. 341Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the 342effective size quota is further adjusted. Reading ``effective_bytes`` returns 343the current effective size quota. The file is not updated in real time, so 344users should ask DAMON sysfs interface to update the content of the file for 345the stats by writing a special keyword, ``update_schemes_effective_bytes`` to 346the relevant ``kdamonds/<N>/state`` file. 347 348Under ``weights`` directory, three files (``sz_permil``, 349``nr_accesses_permil``, and ``age_permil``) exist. 350You can set the :ref:`prioritization weights 351<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 352in per-thousand unit by writing the values to the three files under the 353``weights`` directory. 354 355.. _sysfs_schemes_quota_goals: 356 357schemes/<N>/quotas/goals/ 358------------------------- 359 360The directory for the :ref:`automatic quota tuning goals 361<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 362scheme. 363 364In the beginning, this directory has only one file, ``nr_goals``. Writing a 365number (``N``) to the file creates the number of child directories named ``0`` 366to ``N-1``. Each directory represents each goal and current achievement. 367Among the multiple feedback, the best one is used. 368 369Each goal directory contains three files, namely ``target_metric``, 370``target_value`` and ``current_value``. Users can set and get the three 371parameters for the quota auto-tuning goals that specified on the :ref:`design 372doc <damon_design_damos_quotas_auto_tuning>` by writing to and reading from each 373of the files. Note that users should further write 374``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 375directory <sysfs_kdamond>` to pass the feedback to DAMON. 376 377.. _sysfs_watermarks: 378 379schemes/<N>/watermarks/ 380----------------------- 381 382The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 383given DAMON-based operation scheme. 384 385Under the watermarks directory, five files (``metric``, ``interval_us``, 386``high``, ``mid``, and ``low``) for setting the metric, the time interval 387between check of the metric, and the three watermarks exist. You can set and 388get the five values by writing to the files, respectively. 389 390Keywords and meanings of those that can be written to the ``metric`` file are 391as below. 392 393 - none: Ignore the watermarks 394 - free_mem_rate: System's free memory rate (per thousand) 395 396The ``interval`` should written in microseconds unit. 397 398.. _sysfs_filters: 399 400schemes/<N>/filters/ 401-------------------- 402 403The directory for the :ref:`filters <damon_design_damos_filters>` of the given 404DAMON-based operation scheme. 405 406In the beginning, this directory has only one file, ``nr_filters``. Writing a 407number (``N``) to the file creates the number of child directories named ``0`` 408to ``N-1``. Each directory represents each filter. The filters are evaluated 409in the numeric order. 410 411Each filter directory contains six files, namely ``type``, ``matcing``, 412``memcg_path``, ``addr_start``, ``addr_end``, and ``target_idx``. To ``type`` 413file, you can write one of four special keywords: ``anon`` for anonymous pages, 414``memcg`` for specific memory cgroup, ``addr`` for specific address range (an 415open-ended interval), or ``target`` for specific DAMON monitoring target 416filtering. In case of the memory cgroup filtering, you can specify the memory 417cgroup of the interest by writing the path of the memory cgroup from the 418cgroups mount point to ``memcg_path`` file. In case of the address range 419filtering, you can specify the start and end address of the range to 420``addr_start`` and ``addr_end`` files, respectively. For the DAMON monitoring 421target filtering, you can specify the index of the target between the list of 422the DAMON context's monitoring targets list to ``target_idx`` file. You can 423write ``Y`` or ``N`` to ``matching`` file to filter out pages that does or does 424not match to the type, respectively. Then, the scheme's action will not be 425applied to the pages that specified to be filtered out. 426 427For example, below restricts a DAMOS action to be applied to only non-anonymous 428pages of all memory cgroups except ``/having_care_already``.:: 429 430 # echo 2 > nr_filters 431 # # filter out anonymous pages 432 echo anon > 0/type 433 echo Y > 0/matching 434 # # further filter out all cgroups except one at '/having_care_already' 435 echo memcg > 1/type 436 echo /having_care_already > 1/memcg_path 437 echo N > 1/matching 438 439Note that ``anon`` and ``memcg`` filters are currently supported only when 440``paddr`` :ref:`implementation <sysfs_context>` is being used. 441 442Also, memory regions that are filtered out by ``addr`` or ``target`` filters 443are not counted as the scheme has tried to those, while regions that filtered 444out by other type filters are counted as the scheme has tried to. The 445difference is applied to :ref:`stats <damos_stats>` and 446:ref:`tried regions <sysfs_schemes_tried_regions>`. 447 448.. _sysfs_schemes_stats: 449 450schemes/<N>/stats/ 451------------------ 452 453DAMON counts the total number and bytes of regions that each scheme is tried to 454be applied, the two numbers for the regions that each scheme is successfully 455applied, and the total number of the quota limit exceeds. This statistics can 456be used for online analysis or tuning of the schemes. 457 458The statistics can be retrieved by reading the files under ``stats`` directory 459(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and 460``qt_exceeds``), respectively. The files are not updated in real time, so you 461should ask DAMON sysfs interface to update the content of the files for the 462stats by writing a special keyword, ``update_schemes_stats`` to the relevant 463``kdamonds/<N>/state`` file. 464 465.. _sysfs_schemes_tried_regions: 466 467schemes/<N>/tried_regions/ 468-------------------------- 469 470This directory initially has one file, ``total_bytes``. 471 472When a special keyword, ``update_schemes_tried_regions``, is written to the 473relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 474that reading it returns the total size of the scheme tried regions, and creates 475directories named integer starting from ``0`` under this directory. Each 476directory contains files exposing detailed information about each of the memory 477region that the corresponding scheme's ``action`` has tried to be applied under 478this directory, during next :ref:`apply interval <damon_design_damos>` of the 479corresponding scheme. The information includes address range, ``nr_accesses``, 480and ``age`` of the region. 481 482Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 483file will only update the ``total_bytes`` file, and will not create the 484subdirectories. 485 486The directories will be removed when another special keyword, 487``clear_schemes_tried_regions``, is written to the relevant 488``kdamonds/<N>/state`` file. 489 490The expected usage of this directory is investigations of schemes' behaviors, 491and query-like efficient data access monitoring results retrievals. For the 492latter use case, in particular, users can set the ``action`` as ``stat`` and 493set the ``access pattern`` as their interested pattern that they want to query. 494 495.. _sysfs_schemes_tried_region: 496 497tried_regions/<N>/ 498------------------ 499 500In each region directory, you will find four files (``start``, ``end``, 501``nr_accesses``, and ``age``). Reading the files will show the start and end 502addresses, ``nr_accesses``, and ``age`` of the region that corresponding 503DAMON-based operation scheme ``action`` has tried to be applied. 504 505Example 506~~~~~~~ 507 508Below commands applies a scheme saying "If a memory region of size in [4KiB, 5098KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 510interval in [10, 20], page out the region. For the paging out, use only up to 51110ms per second, and also don't page out more than 1GiB per second. Under the 512limitation, page out memory regions having longer age first. Also, check the 513free memory rate of the system every 5 seconds, start the monitoring and paging 514out when the free memory rate becomes lower than 50%, but stop it if the free 515memory rate becomes larger than 60%, or lower than 30%". :: 516 517 # cd <sysfs>/kernel/mm/damon/admin 518 # # populate directories 519 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 520 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 521 # cd kdamonds/0/contexts/0/schemes/0 522 # # set the basic access pattern and the action 523 # echo 4096 > access_pattern/sz/min 524 # echo 8192 > access_pattern/sz/max 525 # echo 0 > access_pattern/nr_accesses/min 526 # echo 5 > access_pattern/nr_accesses/max 527 # echo 10 > access_pattern/age/min 528 # echo 20 > access_pattern/age/max 529 # echo pageout > action 530 # # set quotas 531 # echo 10 > quotas/ms 532 # echo $((1024*1024*1024)) > quotas/bytes 533 # echo 1000 > quotas/reset_interval_ms 534 # # set watermark 535 # echo free_mem_rate > watermarks/metric 536 # echo 5000000 > watermarks/interval_us 537 # echo 600 > watermarks/high 538 # echo 500 > watermarks/mid 539 # echo 300 > watermarks/low 540 541Please note that it's highly recommended to use user space tools like `damo 542<https://github.com/awslabs/damo>`_ rather than manually reading and writing 543the files as above. Above is only for an example. 544 545.. _tracepoint: 546 547Tracepoints for Monitoring Results 548================================== 549 550Users can get the monitoring results via the :ref:`tried_regions 551<sysfs_schemes_tried_regions>`. The interface is useful for getting a 552snapshot, but it could be inefficient for fully recording all the monitoring 553results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 554and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 555provides the whole monitoring results, while ``damon:damos_before_apply`` 556provides the monitoring results for regions that each DAMON-based Operation 557Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 558``damon:damos_before_apply`` is more useful for recording internal behavior of 559DAMOS, or DAMOS target access 560:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 561monitoring results recording. 562 563While the monitoring is turned on, you could record the tracepoint events and 564show results using tracepoint supporting tools like ``perf``. For example:: 565 566 # echo on > kdamonds/0/state 567 # perf record -e damon:damon_aggregated & 568 # sleep 5 569 # kill 9 $(pidof perf) 570 # echo off > kdamonds/0/state 571 # perf script 572 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 573 [...] 574 575Each line of the perf script output represents each monitoring region. The 576first five fields are as usual other tracepoint outputs. The sixth field 577(``target_id=X``) shows the ide of the monitoring target of the region. The 578seventh field (``nr_regions=X``) shows the total number of monitoring regions 579for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 580(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 581``nr_accesses`` of the region (refer to 582:ref:`design <damon_design_region_based_sampling>` for more details of the 583counter). Finally the tenth field (``X``) shows the ``age`` of the region 584(refer to :ref:`design <damon_design_age_tracking>` for more details of the 585counter). 586 587If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would 588be somewhat like below:: 589 590 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 591 [...] 592 593Each line of the output represents each monitoring region that each DAMON-based 594Operation Scheme was about to be applied at the traced time. The first five 595fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 596of the scheme in the list of the contexts of the context's kdamond, the index 597of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 598addition to the output of ``damon_aggregated`` tracepoint. 599 600 601.. _debugfs_interface: 602 603debugfs Interface (DEPRECATED!) 604=============================== 605 606.. note:: 607 608 THIS IS DEPRECATED! 609 610 DAMON debugfs interface is deprecated, so users should move to the 611 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 612 move, please report your usecase to damon@lists.linux.dev and 613 linux-mm@kvack.org. 614 615DAMON exports nine files, ``DEPRECATED``, ``attrs``, ``target_ids``, 616``init_regions``, ``schemes``, ``monitor_on_DEPRECATED``, ``kdamond_pid``, 617``mk_contexts`` and ``rm_contexts`` under its debugfs directory, 618``<debugfs>/damon/``. 619 620 621``DEPRECATED`` is a read-only file for the DAMON debugfs interface deprecation 622notice. Reading it returns the deprecation notice, as below:: 623 624 # cat DEPRECATED 625 DAMON debugfs interface is deprecated, so users should move to DAMON_SYSFS. If you cannot, please report your usecase to damon@lists.linux.dev and linux-mm@kvack.org. 626 627 628Attributes 629---------- 630 631Users can get and set the ``sampling interval``, ``aggregation interval``, 632``update interval``, and min/max number of monitoring target regions by 633reading from and writing to the ``attrs`` file. To know about the monitoring 634attributes in detail, please refer to the :doc:`/mm/damon/design`. For 635example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 6361000, and then check it again:: 637 638 # cd <debugfs>/damon 639 # echo 5000 100000 1000000 10 1000 > attrs 640 # cat attrs 641 5000 100000 1000000 10 1000 642 643 644Target IDs 645---------- 646 647Some types of address spaces supports multiple monitoring target. For example, 648the virtual memory address spaces monitoring can have multiple processes as the 649monitoring targets. Users can set the targets by writing relevant id values of 650the targets to, and get the ids of the current targets by reading from the 651``target_ids`` file. In case of the virtual address spaces monitoring, the 652values should be pids of the monitoring target processes. For example, below 653commands set processes having pids 42 and 4242 as the monitoring targets and 654check it again:: 655 656 # cd <debugfs>/damon 657 # echo 42 4242 > target_ids 658 # cat target_ids 659 42 4242 660 661Users can also monitor the physical memory address space of the system by 662writing a special keyword, "``paddr\n``" to the file. Because physical address 663space monitoring doesn't support multiple targets, reading the file will show a 664fake value, ``42``, as below:: 665 666 # cd <debugfs>/damon 667 # echo paddr > target_ids 668 # cat target_ids 669 42 670 671Note that setting the target ids doesn't start the monitoring. 672 673 674Initial Monitoring Target Regions 675--------------------------------- 676 677In case of the virtual address space monitoring, DAMON automatically sets and 678updates the monitoring target regions so that entire memory mappings of target 679processes can be covered. However, users can want to limit the monitoring 680region to specific address ranges, such as the heap, the stack, or specific 681file-mapped area. Or, some users can know the initial access pattern of their 682workloads and therefore want to set optimal initial regions for the 'adaptive 683regions adjustment'. 684 685In contrast, DAMON do not automatically sets and updates the monitoring target 686regions in case of physical memory monitoring. Therefore, users should set the 687monitoring target regions by themselves. 688 689In such cases, users can explicitly set the initial monitoring target regions 690as they want, by writing proper values to the ``init_regions`` file. The input 691should be a sequence of three integers separated by white spaces that represent 692one region in below form.:: 693 694 <target idx> <start address> <end address> 695 696The ``target idx`` should be the index of the target in ``target_ids`` file, 697starting from ``0``, and the regions should be passed in address order. For 698example, below commands will set a couple of address ranges, ``1-100`` and 699``100-200`` as the initial monitoring target region of pid 42, which is the 700first one (index ``0``) in ``target_ids``, and another couple of address 701ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one 702(index ``1``) in ``target_ids``.:: 703 704 # cd <debugfs>/damon 705 # cat target_ids 706 42 4242 707 # echo "0 1 100 \ 708 0 100 200 \ 709 1 20 40 \ 710 1 50 100" > init_regions 711 712Note that this sets the initial monitoring target regions only. In case of 713virtual memory monitoring, DAMON will automatically updates the boundary of the 714regions after one ``update interval``. Therefore, users should set the 715``update interval`` large enough in this case, if they don't want the 716update. 717 718 719Schemes 720------- 721 722Users can get and set the DAMON-based operation :ref:`schemes 723<damon_design_damos>` by reading from and writing to ``schemes`` debugfs file. 724Reading the file also shows the statistics of each scheme. To the file, each 725of the schemes should be represented in each line in below form:: 726 727 <target access pattern> <action> <quota> <watermarks> 728 729You can disable schemes by simply writing an empty string to the file. 730 731Target Access Pattern 732~~~~~~~~~~~~~~~~~~~~~ 733 734The target access :ref:`pattern <damon_design_damos_access_pattern>` of the 735scheme. The ``<target access pattern>`` is constructed with three ranges in 736below form:: 737 738 min-size max-size min-acc max-acc min-age max-age 739 740Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 741number of monitored accesses per aggregate interval for access frequency 742(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 743regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 744closed interval. 745 746Action 747~~~~~~ 748 749The ``<action>`` is a predefined integer for memory management :ref:`actions 750<damon_design_damos_action>`. The mapping between the ``<action>`` values and 751the memory management actions is as below. For the detailed meaning of the 752action and DAMON operations set supporting each action, please refer to the 753list on :ref:`design doc <damon_design_damos_action>`. 754 755 - 0: ``willneed`` 756 - 1: ``cold`` 757 - 2: ``pageout`` 758 - 3: ``hugepage`` 759 - 4: ``nohugepage`` 760 - 5: ``stat`` 761 762Quota 763~~~~~ 764 765Users can set the :ref:`quotas <damon_design_damos_quotas>` of the given scheme 766via the ``<quota>`` in below form:: 767 768 <ms> <sz> <reset interval> <priority weights> 769 770This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 771the action to memory regions of the ``target access pattern`` within the 772``<reset interval>`` milliseconds, and to apply the action to only up to 773``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 774``<ms>`` and ``<sz>`` zero disables the quota limits. 775 776For the :ref:`prioritization <damon_design_damos_quotas_prioritization>`, users 777can set the weights for the three properties in ``<priority weights>`` in below 778form:: 779 780 <size weight> <access frequency weight> <age weight> 781 782Watermarks 783~~~~~~~~~~ 784 785Users can specify :ref:`watermarks <damon_design_damos_watermarks>` of the 786given scheme via ``<watermarks>`` in below form:: 787 788 <metric> <check interval> <high mark> <middle mark> <low mark> 789 790``<metric>`` is a predefined integer for the metric to be checked. The 791supported numbers and their meanings are as below. 792 793 - 0: Ignore the watermarks 794 - 1: System's free memory rate (per thousand) 795 796The value of the metric is checked every ``<check interval>`` microseconds. 797 798If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 799scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 800is activated. 801 802.. _damos_stats: 803 804Statistics 805~~~~~~~~~~ 806 807It also counts the total number and bytes of regions that each scheme is tried 808to be applied, the two numbers for the regions that each scheme is successfully 809applied, and the total number of the quota limit exceeds. This statistics can 810be used for online analysis or tuning of the schemes. 811 812The statistics can be shown by reading the ``schemes`` file. Reading the file 813will show each scheme you entered in each line, and the five numbers for the 814statistics will be added at the end of each line. 815 816Example 817~~~~~~~ 818 819Below commands applies a scheme saying "If a memory region of size in [4KiB, 8208KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 821interval in [10, 20], page out the region. For the paging out, use only up to 82210ms per second, and also don't page out more than 1GiB per second. Under the 823limitation, page out memory regions having longer age first. Also, check the 824free memory rate of the system every 5 seconds, start the monitoring and paging 825out when the free memory rate becomes lower than 50%, but stop it if the free 826memory rate becomes larger than 60%, or lower than 30%".:: 827 828 # cd <debugfs>/damon 829 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 830 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 831 # scheme+=" 0 0 100" # prioritization weights 832 # scheme+=" 1 5000000 600 500 300" # watermarks 833 # echo "$scheme" > schemes 834 835 836Turning On/Off 837-------------- 838 839Setting the files as described above doesn't incur effect unless you explicitly 840start the monitoring. You can start, stop, and check the current status of the 841monitoring by writing to and reading from the ``monitor_on_DEPRECATED`` file. 842Writing ``on`` to the file starts the monitoring of the targets with the 843attributes. Writing ``off`` to the file stops those. DAMON also stops if 844every target process is terminated. Below example commands turn on, off, and 845check the status of DAMON:: 846 847 # cd <debugfs>/damon 848 # echo on > monitor_on_DEPRECATED 849 # echo off > monitor_on_DEPRECATED 850 # cat monitor_on_DEPRECATED 851 off 852 853Please note that you cannot write to the above-mentioned debugfs files while 854the monitoring is turned on. If you write to the files while DAMON is running, 855an error code such as ``-EBUSY`` will be returned. 856 857 858Monitoring Thread PID 859--------------------- 860 861DAMON does requested monitoring with a kernel thread called ``kdamond``. You 862can get the pid of the thread by reading the ``kdamond_pid`` file. When the 863monitoring is turned off, reading the file returns ``none``. :: 864 865 # cd <debugfs>/damon 866 # cat monitor_on_DEPRECATED 867 off 868 # cat kdamond_pid 869 none 870 # echo on > monitor_on_DEPRECATED 871 # cat kdamond_pid 872 18594 873 874 875Using Multiple Monitoring Threads 876--------------------------------- 877 878One ``kdamond`` thread is created for each monitoring context. You can create 879and remove monitoring contexts for multiple ``kdamond`` required use case using 880the ``mk_contexts`` and ``rm_contexts`` files. 881 882Writing the name of the new context to the ``mk_contexts`` file creates a 883directory of the name on the DAMON debugfs directory. The directory will have 884DAMON debugfs files for the context. :: 885 886 # cd <debugfs>/damon 887 # ls foo 888 # ls: cannot access 'foo': No such file or directory 889 # echo foo > mk_contexts 890 # ls foo 891 # attrs init_regions kdamond_pid schemes target_ids 892 893If the context is not needed anymore, you can remove it and the corresponding 894directory by putting the name of the context to the ``rm_contexts`` file. :: 895 896 # echo foo > rm_contexts 897 # ls foo 898 # ls: cannot access 'foo': No such file or directory 899 900Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on_DEPRECATED`` files 901are in the root directory only. 902