1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *Special-purpose DAMON modules.* 10 :ref:`This <damon_modules_special_purpose>` is for people who are building, 11 distributing, and/or administrating the kernel with special-purpose DAMON 12 usages. Using this, users can use DAMON's major features for the given 13 purposes in build, boot, or runtime in simple ways. 14- *DAMON user space tool.* 15 `This <https://github.com/damonitor/damo>`_ is for privileged people such as 16 system administrators who want a just-working human-friendly interface. 17 Using this, users can use the DAMON’s major features in a human-friendly way. 18 It may not be highly tuned for special cases, though. For more detail, 19 please refer to its `usage document 20 <https://github.com/damonitor/damo/blob/next/USAGE.md>`_. 21- *sysfs interface.* 22 :ref:`This <sysfs_interface>` is for privileged user space programmers who 23 want more optimized use of DAMON. Using this, users can use DAMON’s major 24 features by reading from and writing to special sysfs files. Therefore, 25 you can write and use your personalized DAMON sysfs wrapper programs that 26 reads/writes the sysfs files instead of you. The `DAMON user space tool 27 <https://github.com/damonitor/damo>`_ is one example of such programs. 28- *Kernel Space Programming Interface.* 29 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </mm/damon/api>`. 34 35.. _sysfs_interface: 36 37sysfs Interface 38=============== 39 40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 41creates multiple directories and files under its sysfs directory, 42``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 43from the files under the directory. 44 45For a short example, users can monitor the virtual address space of a given 46workload as below. :: 47 48 # cd /sys/kernel/mm/damon/admin/ 49 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 50 # echo vaddr > kdamonds/0/contexts/0/operations 51 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 52 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 53 # echo on > kdamonds/0/state 54 55Files Hierarchy 56--------------- 57 58The files hierarchy of DAMON sysfs interface is shown below. In the below 59figure, parents-children relations are represented with indentations, each 60directory is having ``/`` suffix, and files in each directory are separated by 61comma (","). 62 63.. parsed-literal:: 64 65 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 66 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 67 │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms 68 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 69 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit, 70 │ │ │ │ pause 71 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 72 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 73 │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us 74 │ │ │ │ │ │ nr_regions/min,max 75 │ │ │ │ │ │ :ref:`probes <damon_usage_sysfs_probes>`/nr_probes 76 │ │ │ │ │ │ │ 0/weight 77 │ │ │ │ │ │ │ │ filters/nr_filters 78 │ │ │ │ │ │ │ │ │ 0/type,matching,allow,path 79 │ │ │ │ │ │ │ │ │ ... 80 │ │ │ │ │ │ │ ... 81 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 82 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target 83 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 84 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 85 │ │ │ │ │ │ │ │ ... 86 │ │ │ │ │ │ ... 87 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 88 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us 89 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 90 │ │ │ │ │ │ │ │ sz/min,max 91 │ │ │ │ │ │ │ │ nr_accesses/min,max 92 │ │ │ │ │ │ │ │ age/min,max 93 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms, 94 │ │ │ │ │ │ │ effective_bytes,goal_tuner, 95 │ │ │ │ │ │ │ fail_charge_num,fail_charge_denom 96 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 97 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 98 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path 99 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 100 │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters 101 │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,damon_target_idx,min,max 102 │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests 103 │ │ │ │ │ │ │ │ 0/id,weight 104 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots 105 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 106 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed 107 │ │ │ │ │ │ │ │ │ probes 108 │ │ │ │ │ │ │ │ │ │ 0/hits 109 │ │ │ │ │ │ │ │ │ │ ... 110 │ │ │ │ │ │ │ │ ... 111 │ │ │ │ │ │ ... 112 │ │ │ │ ... 113 │ │ ... 114 115.. _sysfs_root: 116 117Root 118---- 119 120The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 121has one directory named ``admin``. The directory contains the files for 122privileged user space programs' control of DAMON. User space tools or daemons 123having the root permission could use this directory. 124 125.. _sysfs_kdamonds: 126 127kdamonds/ 128--------- 129 130Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 131controlling the kdamonds (refer to 132:ref:`design <damon_design_execution_model_and_data_structures>` for more 133details) exists. In the beginning, this directory has only one file, 134``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 135child directories named ``0`` to ``N-1``. Each directory represents each 136kdamond. 137 138.. _sysfs_kdamond: 139 140kdamonds/<N>/ 141------------- 142 143In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``) 144and one directory (``contexts``) exist. 145 146Reading ``state`` returns ``on`` if the kdamond is currently running, or 147``off`` if it is not running. 148 149Users can write below commands for the kdamond to the ``state`` file. 150 151- ``on``: Start running. 152- ``off``: Stop running. 153- ``commit``: Read the user inputs in the sysfs files except ``state`` file 154 again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be 155 ignored if no target region is specified. 156- ``update_tuned_intervals``: Update the contents of ``sample_us`` and 157 ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling 158 interval`` and ``aggregation interval`` for the files. Please refer to 159 :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>` 160 for more details. 161- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 162 :ref:`quota goals <sysfs_schemes_quota_goals>`. 163- ``update_schemes_stats``: Update the contents of stats files for each 164 DAMON-based operation scheme of the kdamond. For details of the stats, 165 please refer to :ref:`stats section <sysfs_schemes_stats>`. 166- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 167 action tried regions directory for each DAMON-based operation scheme of the 168 kdamond. For details of the DAMON-based operation scheme action tried 169 regions directory, please refer to 170 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 171- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 172 files. 173- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 174 action tried regions directory for each DAMON-based operation scheme of the 175 kdamond. 176- ``update_schemes_effective_quotas``: Update the contents of 177 ``effective_bytes`` files for each DAMON-based operation scheme of the 178 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`. 179 180If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 181 182Users can ask the kernel to periodically update files showing auto-tuned 183parameters and DAMOS stats instead of manually writing 184``update_tuned_intervals`` like keywords to ``state`` file. For this, users 185should write the desired update time interval in milliseconds to ``refresh_ms`` 186file. If the interval is zero, the periodic update is disabled. Reading the 187file shows currently set time interval. 188 189``contexts`` directory contains files for controlling the monitoring contexts 190that this kdamond will execute. 191 192.. _sysfs_contexts: 193 194kdamonds/<N>/contexts/ 195---------------------- 196 197In the beginning, this directory has only one file, ``nr_contexts``. Writing a 198number (``N``) to the file creates the number of child directories named as 199``0`` to ``N-1``. Each directory represents each monitoring context (refer to 200:ref:`design <damon_design_execution_model_and_data_structures>` for more 201details). At the moment, only one context per kdamond is supported, so only 202``0`` or ``1`` can be written to the file. 203 204.. _sysfs_context: 205 206contexts/<N>/ 207------------- 208 209In each context directory, four files (``avail_operations``, ``operations``, 210``addr_unit`` and ``pause``) and three directories (``monitoring_attrs``, 211``targets``, and ``schemes``) exist. 212 213DAMON supports multiple types of :ref:`monitoring operations 214<damon_design_configurable_operations_set>`, including those for virtual address 215space and the physical address space. You can get the list of available 216monitoring operations set on the currently running kernel by reading 217``avail_operations`` file. Based on the kernel configuration, the file will 218list different available operation sets. Please refer to the :ref:`design 219<damon_operations_set>` for the list of all available operation sets and their 220brief explanations. 221 222You can set and get what type of monitoring operations DAMON will use for the 223context by writing one of the keywords listed in ``avail_operations`` file and 224reading from the ``operations`` file. 225 226``addr_unit`` file is for setting and getting the :ref:`address unit 227<damon_design_addr_unit>` parameter of the operations set. 228 229``pause`` file is for setting and getting the :ref:`pause request 230<damon_design_execution_model_and_data_structures>` parameter of the context. 231 232.. _sysfs_monitoring_attrs: 233 234contexts/<N>/monitoring_attrs/ 235------------------------------ 236 237Files for specifying attributes of the monitoring including required quality 238and efficiency of the monitoring are in ``monitoring_attrs`` directory. 239Specifically, three directories, ``intervals``, ``nr_regions`` and ``probes`` 240exist in this directory. 241 242Under ``intervals`` directory, three files for DAMON's sampling interval 243(``sample_us``), aggregation interval (``aggr_us``), and update interval 244(``update_us``) exist. You can set and get the values in micro-seconds by 245writing to and reading from the files. 246 247Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 248of DAMON's monitoring regions (``min`` and ``max``, respectively), which 249controls the monitoring overhead, exist. You can set and get the values by 250writing to and reading from the files. 251 252For more details about the intervals and monitoring regions range, please refer 253to the Design document (:doc:`/mm/damon/design`). 254 255.. _damon_usage_sysfs_monitoring_intervals_goal: 256 257contexts/<N>/monitoring_attrs/intervals/intervals_goal/ 258------------------------------------------------------- 259 260Under the ``intervals`` directory, one directory for automated tuning of 261``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists. 262Under the directory, four files for the auto-tuning control, namely 263``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist. 264Please refer to the :ref:`design document of the feature 265<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning 266mechanism. Reading and writing the four files under ``intervals_goal`` 267directory shows and updates the tuning parameters that described in the 268:ref:`design doc <damon_design_monitoring_intervals_autotuning>` with the same 269names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The 270tuning-applied current values of the two intervals can be read from the 271``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to 272the ``state`` file. 273 274.. _damon_usage_sysfs_probes: 275 276contexts/<N>/monitoring_attrs/probes/ 277------------------------------------- 278 279A directory for registering :ref:`data attributes monitoring 280<damon_design_data_attrs_monitoring>` probes. 281 282In the beginning, this directory has only one file, ``nr_probes``. Writing a 283number (``N``) to the file creates the number of child directories named ``0`` 284to ``N-1``. Each directory represents each monitoring probe. 285 286In each probe directory, one directory, ``filters`` exists. The directory 287contains files for installing filters for the probe, that is used to determine 288the data attribute for the probe. 289 290Each probe directory also contains ``weight`` file. Reading from and writing 291to the file gets and sets the :ref:`attributes-only monitoring 292<damon_design_attrs_only_monitoring>` weight for the attribute of the probe. 293 294In the beginning, ``filters`` directory has only one file, ``nr_filters``. 295Writing a number (``N``) to the file creates the number of child directories 296named ``0`` to ``N-1``. Each directory represents each filter and works in a 297way similar to that for :ref:`DAMOS filter <sysfs_filters>`. When the filter 298``type`` is ``memcg``, ``path`` file acts as ``memcg_path`` for :ref:`DAMOS 299filter <sysfs_filters>`. 300 301.. _sysfs_targets: 302 303contexts/<N>/targets/ 304--------------------- 305 306In the beginning, this directory has only one file, ``nr_targets``. Writing a 307number (``N``) to the file creates the number of child directories named ``0`` 308to ``N-1``. Each directory represents each monitoring target. 309 310.. _sysfs_target: 311 312targets/<N>/ 313------------ 314 315In each target directory, two files (``pid_target`` and ``obsolete_target``) 316and one directory (``regions``) exist. 317 318If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 319be a process. You can specify the process to DAMON by writing the pid of the 320process to the ``pid_target`` file. 321 322Users can selectively remove targets in the middle of the targets array by 323writing non-zero value to ``obsolete_target`` file and committing it (writing 324``commit`` to ``state`` file). DAMON will remove the matching targets from its 325internal targets array. Users are responsible to construct target directories 326again, so that those correctly represent the changed internal targets array. 327 328 329.. _sysfs_regions: 330 331targets/<N>/regions 332------------------- 333 334In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are 335required to set the monitoring target address ranges. In case of ``vaddr`` 336operations set, it is not mandatory, but users can optionally set the initial 337monitoring region to specific address ranges. Please refer to the :ref:`design 338<damon_design_vaddr_target_regions_construction>` for more details. 339 340For such cases, users can explicitly set the initial monitoring target regions 341as they want, by writing proper values to the files under this directory. 342 343In the beginning, this directory has only one file, ``nr_regions``. Writing a 344number (``N``) to the file creates the number of child directories named ``0`` 345to ``N-1``. Each directory represents each initial monitoring target region. 346 347If ``nr_regions`` is zero when committing new DAMON parameters online (writing 348``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit 349logic ignores the target regions. In other words, the current monitoring 350results for the target are preserved. 351 352.. _sysfs_region: 353 354regions/<N>/ 355------------ 356 357In each region directory, you will find two files (``start`` and ``end``). You 358can set and get the start and end addresses of the initial monitoring target 359region by writing to and reading from the files, respectively. 360 361Each region should not overlap with others. ``end`` of directory ``N`` should 362be equal or smaller than ``start`` of directory ``N+1``. 363 364.. _sysfs_schemes: 365 366contexts/<N>/schemes/ 367--------------------- 368 369The directory for DAMON-based Operation Schemes (:ref:`DAMOS 370<damon_design_damos>`). Users can get and set the schemes by reading from and 371writing to files under this directory. 372 373In the beginning, this directory has only one file, ``nr_schemes``. Writing a 374number (``N``) to the file creates the number of child directories named ``0`` 375to ``N-1``. Each directory represents each DAMON-based operation scheme. 376 377.. _sysfs_scheme: 378 379schemes/<N>/ 380------------ 381 382In each scheme directory, nine directories (``access_pattern``, ``quotas``, 383``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``, 384``stats``, and ``tried_regions``) and three files (``action``, ``target_nid`` 385and ``apply_interval_us``) exist. 386 387The ``action`` file is for setting and getting the scheme's :ref:`action 388<damon_design_damos_action>`. The keywords that can be written to and read 389from the file and their meaning are same to those of the list on 390:ref:`design doc <damon_design_damos_action>`. 391 392The ``target_nid`` file is for setting the migration target node, which is 393only meaningful when the ``action`` is either ``migrate_hot`` or 394``migrate_cold``. 395 396The ``apply_interval_us`` file is for setting and getting the scheme's 397:ref:`apply_interval <damon_design_damos>` in microseconds. 398 399.. _sysfs_access_pattern: 400 401schemes/<N>/access_pattern/ 402--------------------------- 403 404The directory for the target access :ref:`pattern 405<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 406 407Under the ``access_pattern`` directory, three directories (``sz``, 408``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 409exist. You can set and get the access pattern for the given scheme by writing 410to and reading from the ``min`` and ``max`` files under ``sz``, 411``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 412and the ``max`` form a closed interval. 413 414.. _sysfs_quotas: 415 416schemes/<N>/quotas/ 417------------------- 418 419The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 420DAMON-based operation scheme. 421 422Under ``quotas`` directory, seven files (``ms``, ``bytes``, 423``reset_interval_ms``, ``effective_bytes``, ``goal_tuner``, ``fail_charge_num`` 424and ``fail_charge_denom``) and two directories (``weights`` and ``goals``) 425exist. 426 427You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 428``reset interval`` in milliseconds by writing the values to the three files, 429respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 430for applying the ``action`` to memory regions of the ``access_pattern``, and to 431apply the action to only up to ``bytes`` bytes of memory regions within the 432``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 433quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is 434set. 435 436You can set the goal-based effective quota auto-tuning algorithm to use, by 437writing the algorithm name to ``goal_tuner`` file. Reading the file returns 438the currently selected tuner algorithm. Refer to the design documentation of 439:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for 440the background design of the feature and the name of the selectable algorithms. 441Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals 442setup. 443 444You can set the action-failed memory quota charging ratio by writing the 445numerator and the denominator for the ratio to ``fail_charge_num`` and 446``fail_charge_denom`` files, respectively. Reading those files will return the 447current set values. Refer to :ref:`design 448<damon_design_damos_quotas_failed_memory_charging_ratio>` for more details of 449the ratio feature. 450 451The time quota is internally transformed to a size quota. Between the 452transformed size quota and user-specified size quota, smaller one is applied. 453Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the 454effective size quota is further adjusted. Reading ``effective_bytes`` returns 455the current effective size quota. The file is not updated in real time, so 456users should ask DAMON sysfs interface to update the content of the file for 457the stats by writing a special keyword, ``update_schemes_effective_quotas`` to 458the relevant ``kdamonds/<N>/state`` file. 459 460Under ``weights`` directory, three files (``sz_permil``, 461``nr_accesses_permil``, and ``age_permil``) exist. 462You can set the :ref:`prioritization weights 463<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 464in per-thousand unit by writing the values to the three files under the 465``weights`` directory. 466 467.. _sysfs_schemes_quota_goals: 468 469schemes/<N>/quotas/goals/ 470------------------------- 471 472The directory for the :ref:`automatic quota tuning goals 473<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 474scheme. 475 476In the beginning, this directory has only one file, ``nr_goals``. Writing a 477number (``N``) to the file creates the number of child directories named ``0`` 478to ``N-1``. Each directory represents each goal and current achievement. 479Among the multiple feedback, the best one is used. 480 481Each goal directory contains five files, namely ``target_metric``, 482``target_value``, ``current_value``, ``nid``, and ``path``. Users can set and 483get the five parameters for the quota auto-tuning goals that specified on the 484:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and 485reading from each of the files. Because the kernel does not update 486``current_value``, reading it only makes sense when ``target_metric`` is 487``user_input``. Note that users should further write 488``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 489directory <sysfs_kdamond>` to pass the feedback to DAMON. 490 491.. _sysfs_watermarks: 492 493schemes/<N>/watermarks/ 494----------------------- 495 496The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 497given DAMON-based operation scheme. 498 499Under the watermarks directory, five files (``metric``, ``interval_us``, 500``high``, ``mid``, and ``low``) for setting the metric, the time interval 501between check of the metric, and the three watermarks exist. You can set and 502get the five values by writing to and reading from the files, respectively. 503 504Keywords and meanings of those that can be written to the ``metric`` file are 505as below. 506 507 - none: Ignore the watermarks 508 - free_mem_rate: System's free memory rate (per thousand) 509 510The ``interval_us`` should be written in microseconds unit. 511 512.. _sysfs_filters: 513 514schemes/<N>/{core\_,ops\_,}filters/ 515----------------------------------- 516 517Directories for :ref:`filters <damon_design_damos_filters>` of the given 518DAMON-based operation scheme. 519 520``core_filters`` and ``ops_filters`` directories are for the filters handled by 521the DAMON core layer and operations set layer, respectively. ``filters`` 522directory can be used for installing filters regardless of their handled 523layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be 524installed before those of ``filters``. All three directories have same files. 525 526Use of ``filters`` directory can make filters evaluation orders confusing to 527expect. For this reason, ``filters`` directory is deprecated. It is still 528functioning, but is scheduled for removal in the near future. Users should use 529``core_filters`` and ``ops_filters`` directories instead. 530 531In the beginning, the directory has only one file, ``nr_filters``. Writing a 532number (``N``) to the file creates the number of child directories named ``0`` 533to ``N-1``. Each directory represents each filter. The filters are evaluated 534in the numeric order. 535 536Each filter directory contains nine files, namely ``type``, ``matching``, 537``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max`` 538and ``damon_target_idx``. To ``type`` file, you can write the type of the 539filter. Refer to :ref:`the design doc <damon_design_damos_filters>` for 540available type names, their meaning and on what layer those are handled. 541 542For ``memcg`` type, you can specify the memory cgroup of the interest by 543writing the path of the memory cgroup from the cgroups mount point to 544``memcg_path`` file. For ``addr`` type, you can specify the start and end 545address of the range (open-ended interval) to ``addr_start`` and ``addr_end`` 546files, respectively. For ``hugepage_size`` type, you can specify the minimum 547and maximum size of the range (closed interval) to ``min`` and ``max`` files, 548respectively. For ``target`` type, you can specify the index of the target 549between the list of the DAMON context's monitoring targets list to 550``damon_target_idx`` file. 551 552You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter 553is for memory that matches the ``type``. You can write ``Y`` or ``N`` to 554``allow`` file to specify if applying the action to the memory that satisfies 555the ``type`` and ``matching`` should be allowed or not. 556 557For example, below restricts a DAMOS action to be applied to only non-anonymous 558pages of all memory cgroups except ``/having_care_already``.:: 559 560 # cd ops_filters/0/ 561 # echo 2 > nr_filters 562 # # disallow anonymous pages 563 echo anon > 0/type 564 echo Y > 0/matching 565 echo N > 0/allow 566 # # further filter out all cgroups except one at '/having_care_already' 567 echo memcg > 1/type 568 echo /having_care_already > 1/memcg_path 569 echo Y > 1/matching 570 echo N > 1/allow 571 572Refer to the :ref:`DAMOS filters design documentation 573<damon_design_damos_filters>` for more details including how multiple filters 574of different ``allow`` works, when each of the filters are supported, and 575differences on stats. 576 577.. _damon_sysfs_dests: 578 579schemes/<N>/dests/ 580------------------ 581 582Directory for specifying the destinations of given DAMON-based operation 583scheme's action. This directory is ignored if the action of the given scheme 584is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}`` 585actions are supporting multiple destinations. 586 587In the beginning, the directory has only one file, ``nr_dests``. Writing a 588number (``N``) to the file creates the number of child directories named ``0`` 589to ``N-1``. Each directory represents each action destination. 590 591Each destination directory contains two files, namely ``id`` and ``weight``. 592Users can write and read the identifier of the destination to ``id`` file. 593For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node 594id should be written to ``id`` file. Users can write and read the weight of 595the destination among the given destinations to the ``weight`` file. The 596weight can be an arbitrary integer. When DAMOS apply the action to each entity 597of the memory region, it will select the destination of the action based on the 598relative weights of the destinations. 599 600.. _sysfs_schemes_stats: 601 602schemes/<N>/stats/ 603------------------ 604 605DAMON counts statistics for each scheme. This statistics can be used for 606online analysis or tuning of the schemes. Refer to :ref:`design doc 607<damon_design_damos_stat>` for more details about the stats. 608 609The statistics can be retrieved by reading the files under ``stats`` directory 610(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, 611``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and 612``max_nr_snapshots``), respectively. 613 614The files are not updated in real time by default. Users should ask DAMON 615sysfs interface to periodically update those using ``refresh_ms``, or do a one 616time update by writing a special keyword, ``update_schemes_stats`` to the 617relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory 618<sysfs_kdamond>` for more details. 619 620.. _sysfs_schemes_tried_regions: 621 622schemes/<N>/tried_regions/ 623-------------------------- 624 625This directory initially has one file, ``total_bytes``. 626 627When a special keyword, ``update_schemes_tried_regions``, is written to the 628relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 629that reading it returns the total size of the scheme tried regions, and creates 630directories named integer starting from ``0`` under this directory. Each 631directory contains files exposing detailed information about each of the memory 632region that the corresponding scheme's ``action`` has tried to be applied under 633this directory, during next :ref:`apply interval <damon_design_damos>` of the 634corresponding scheme. The information includes address range, ``nr_accesses``, 635and ``age`` of the region. 636 637Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 638file will only update the ``total_bytes`` file, and will not create the 639subdirectories. 640 641The directories will be removed when another special keyword, 642``clear_schemes_tried_regions``, is written to the relevant 643``kdamonds/<N>/state`` file. 644 645The expected usage of this directory is investigations of schemes' behaviors, 646and query-like efficient data access monitoring results retrievals. For the 647latter use case, in particular, users can set the ``action`` as ``stat`` and 648set the ``access pattern`` as their interested pattern that they want to query. 649 650.. _sysfs_schemes_tried_region: 651 652tried_regions/<N>/ 653------------------ 654 655In each region directory, you will find five files (``start``, ``end``, 656``nr_accesses``, ``age`` and ``sz_filter_passed``). Reading the files will 657show the properties of the region that corresponding DAMON-based operation 658scheme ``action`` has tried to be applied. 659 660tried_regions/<N>/probes/ 661------------------------- 662 663In each region directory, one directory (``probes``) also exists. In the 664directory, subdirectories named ``0`` to ``N-1`` exists. ``N`` is the number 665of installed probes. In each number-named directory, a file (``hits``) exist. 666Reading the file shows the number of data attributes monitoring probe-hit 667positive samples of the region. 668 669Example 670~~~~~~~ 671 672Below commands applies a scheme saying "If a memory region of size in [4KiB, 6738KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 674interval in [10, 20], page out the region. For the paging out, use only up to 67510ms per second, and also don't page out more than 1GiB per second. Under the 676limitation, page out memory regions having longer age first. Also, check the 677free memory rate of the system every 5 seconds, start the monitoring and paging 678out when the free memory rate becomes lower than 50%, but stop it if the free 679memory rate becomes larger than 60%, or lower than 30%". :: 680 681 # cd <sysfs>/kernel/mm/damon/admin 682 # # populate directories 683 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 684 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 685 # cd kdamonds/0/contexts/0/schemes/0 686 # # set the basic access pattern and the action 687 # echo 4096 > access_pattern/sz/min 688 # echo 8192 > access_pattern/sz/max 689 # echo 0 > access_pattern/nr_accesses/min 690 # echo 5 > access_pattern/nr_accesses/max 691 # echo 10 > access_pattern/age/min 692 # echo 20 > access_pattern/age/max 693 # echo pageout > action 694 # # set quotas 695 # echo 10 > quotas/ms 696 # echo $((1024*1024*1024)) > quotas/bytes 697 # echo 1000 > quotas/reset_interval_ms 698 # # set watermark 699 # echo free_mem_rate > watermarks/metric 700 # echo 5000000 > watermarks/interval_us 701 # echo 600 > watermarks/high 702 # echo 500 > watermarks/mid 703 # echo 300 > watermarks/low 704 705Please note that it's highly recommended to use user space tools like `damo 706<https://github.com/damonitor/damo>`_ rather than manually reading and writing 707the files as above. Above is only for an example. 708 709.. _tracepoint: 710 711Tracepoints for Monitoring Results 712================================== 713 714Users can get the monitoring results via the :ref:`tried_regions 715<sysfs_schemes_tried_regions>`. The interface is useful for getting a 716snapshot, but it could be inefficient for fully recording all the monitoring 717results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 718and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 719provides the whole monitoring results, while ``damon:damos_before_apply`` 720provides the monitoring results for regions that each DAMON-based Operation 721Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 722``damon:damos_before_apply`` is more useful for recording internal behavior of 723DAMOS, or DAMOS target access 724:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 725monitoring results recording. 726 727While the monitoring is turned on, you could record the tracepoint events and 728show results using tracepoint supporting tools like ``perf``. For example:: 729 730 # echo on > kdamonds/0/state 731 # perf record -e damon:damon_aggregated & 732 # sleep 5 733 # kill 9 $(pidof perf) 734 # echo off > kdamonds/0/state 735 # perf script 736 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 737 [...] 738 739Each line of the perf script output represents each monitoring region. The 740first five fields are as usual other tracepoint outputs. The sixth field 741(``target_id=X``) shows the id of the monitoring target of the region. The 742seventh field (``nr_regions=X``) shows the total number of monitoring regions 743for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 744(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 745``nr_accesses`` of the region (refer to 746:ref:`design <damon_design_region_based_sampling>` for more details of the 747counter). Finally the tenth field (``X``) shows the ``age`` of the region 748(refer to :ref:`design <damon_design_age_tracking>` for more details of the 749counter). 750 751If the event was ``damon:damos_before_apply``, the ``perf script`` output would 752be somewhat like below:: 753 754 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 755 [...] 756 757Each line of the output represents each monitoring region that each DAMON-based 758Operation Scheme was about to be applied at the traced time. The first five 759fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 760of the scheme in the list of the contexts of the context's kdamond, the index 761of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 762addition to the output of ``damon_aggregated`` tracepoint. 763