1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *Special-purpose DAMON modules.* 10 :ref:`This <damon_modules_special_purpose>` is for people who are building, 11 distributing, and/or administrating the kernel with special-purpose DAMON 12 usages. Using this, users can use DAMON's major features for the given 13 purposes in build, boot, or runtime in simple ways. 14- *DAMON user space tool.* 15 `This <https://github.com/damonitor/damo>`_ is for privileged people such as 16 system administrators who want a just-working human-friendly interface. 17 Using this, users can use the DAMON’s major features in a human-friendly way. 18 It may not be highly tuned for special cases, though. For more detail, 19 please refer to its `usage document 20 <https://github.com/damonitor/damo/blob/next/USAGE.md>`_. 21- *sysfs interface.* 22 :ref:`This <sysfs_interface>` is for privileged user space programmers who 23 want more optimized use of DAMON. Using this, users can use DAMON’s major 24 features by reading from and writing to special sysfs files. Therefore, 25 you can write and use your personalized DAMON sysfs wrapper programs that 26 reads/writes the sysfs files instead of you. The `DAMON user space tool 27 <https://github.com/damonitor/damo>`_ is one example of such programs. 28- *Kernel Space Programming Interface.* 29 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </mm/damon/api>`. 34 35.. _sysfs_interface: 36 37sysfs Interface 38=============== 39 40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 41creates multiple directories and files under its sysfs directory, 42``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 43from the files under the directory. 44 45For a short example, users can monitor the virtual address space of a given 46workload as below. :: 47 48 # cd /sys/kernel/mm/damon/admin/ 49 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 50 # echo vaddr > kdamonds/0/contexts/0/operations 51 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 52 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 53 # echo on > kdamonds/0/state 54 55Files Hierarchy 56--------------- 57 58The files hierarchy of DAMON sysfs interface is shown below. In the below 59figure, parents-children relations are represented with indentations, each 60directory is having ``/`` suffix, and files in each directory are separated by 61comma (","). 62 63.. parsed-literal:: 64 65 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 66 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 67 │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms 68 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 69 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit, 70 │ │ │ │ pause 71 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 72 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 73 │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us 74 │ │ │ │ │ │ nr_regions/min,max 75 │ │ │ │ │ │ :ref:`probes <damon_usage_sysfs_probes>`/nr_probes 76 │ │ │ │ │ │ │ 0/weight 77 │ │ │ │ │ │ │ │ preps/nr_preps 78 │ │ │ │ │ │ │ │ │ 0/prep_action 79 │ │ │ │ │ │ │ │ │ ... 80 │ │ │ │ │ │ │ │ filters/nr_filters 81 │ │ │ │ │ │ │ │ │ 0/type,matching,allow,path,min,max 82 │ │ │ │ │ │ │ │ │ ... 83 │ │ │ │ │ │ │ ... 84 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 85 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target 86 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 87 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 88 │ │ │ │ │ │ │ │ ... 89 │ │ │ │ │ │ ... 90 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 91 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us 92 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 93 │ │ │ │ │ │ │ │ sz/min,max 94 │ │ │ │ │ │ │ │ nr_accesses/min,max 95 │ │ │ │ │ │ │ │ age/min,max 96 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms, 97 │ │ │ │ │ │ │ effective_bytes,goal_tuner, 98 │ │ │ │ │ │ │ fail_charge_num,fail_charge_denom 99 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 100 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 101 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path 102 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 103 │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters 104 │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,damon_target_idx,min,max 105 │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests 106 │ │ │ │ │ │ │ │ 0/id,weight 107 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots 108 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 109 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed 110 │ │ │ │ │ │ │ │ │ probes 111 │ │ │ │ │ │ │ │ │ │ 0/hits 112 │ │ │ │ │ │ │ │ │ │ ... 113 │ │ │ │ │ │ │ │ ... 114 │ │ │ │ │ │ ... 115 │ │ │ │ ... 116 │ │ ... 117 118.. _sysfs_root: 119 120Root 121---- 122 123The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 124has one directory named ``admin``. The directory contains the files for 125privileged user space programs' control of DAMON. User space tools or daemons 126having the root permission could use this directory. 127 128.. _sysfs_kdamonds: 129 130kdamonds/ 131--------- 132 133Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 134controlling the kdamonds (refer to 135:ref:`design <damon_design_execution_model_and_data_structures>` for more 136details) exists. In the beginning, this directory has only one file, 137``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 138child directories named ``0`` to ``N-1``. Each directory represents each 139kdamond. 140 141.. _sysfs_kdamond: 142 143kdamonds/<N>/ 144------------- 145 146In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``) 147and one directory (``contexts``) exist. 148 149Reading ``state`` returns ``on`` if the kdamond is currently running, or 150``off`` if it is not running. 151 152Users can write below commands for the kdamond to the ``state`` file. 153 154- ``on``: Start running. 155- ``off``: Stop running. 156- ``commit``: Read the user inputs in the sysfs files except ``state`` file 157 again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be 158 ignored if no target region is specified. 159- ``update_tuned_intervals``: Update the contents of ``sample_us`` and 160 ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling 161 interval`` and ``aggregation interval`` for the files. Please refer to 162 :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>` 163 for more details. 164- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 165 :ref:`quota goals <sysfs_schemes_quota_goals>`. 166- ``update_schemes_stats``: Update the contents of stats files for each 167 DAMON-based operation scheme of the kdamond. For details of the stats, 168 please refer to :ref:`stats section <sysfs_schemes_stats>`. 169- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 170 action tried regions directory for each DAMON-based operation scheme of the 171 kdamond. For details of the DAMON-based operation scheme action tried 172 regions directory, please refer to 173 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 174- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 175 files. 176- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 177 action tried regions directory for each DAMON-based operation scheme of the 178 kdamond. 179- ``update_schemes_effective_quotas``: Update the contents of 180 ``effective_bytes`` files for each DAMON-based operation scheme of the 181 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`. 182 183If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 184 185Users can ask the kernel to periodically update files showing auto-tuned 186parameters and DAMOS stats instead of manually writing 187``update_tuned_intervals`` like keywords to ``state`` file. For this, users 188should write the desired update time interval in milliseconds to ``refresh_ms`` 189file. If the interval is zero, the periodic update is disabled. Reading the 190file shows currently set time interval. 191 192``contexts`` directory contains files for controlling the monitoring contexts 193that this kdamond will execute. 194 195.. _sysfs_contexts: 196 197kdamonds/<N>/contexts/ 198---------------------- 199 200In the beginning, this directory has only one file, ``nr_contexts``. Writing a 201number (``N``) to the file creates the number of child directories named as 202``0`` to ``N-1``. Each directory represents each monitoring context (refer to 203:ref:`design <damon_design_execution_model_and_data_structures>` for more 204details). At the moment, only one context per kdamond is supported, so only 205``0`` or ``1`` can be written to the file. 206 207.. _sysfs_context: 208 209contexts/<N>/ 210------------- 211 212In each context directory, four files (``avail_operations``, ``operations``, 213``addr_unit`` and ``pause``) and three directories (``monitoring_attrs``, 214``targets``, and ``schemes``) exist. 215 216DAMON supports multiple types of :ref:`monitoring operations 217<damon_design_configurable_operations_set>`, including those for virtual address 218space and the physical address space. You can get the list of available 219monitoring operations set on the currently running kernel by reading 220``avail_operations`` file. Based on the kernel configuration, the file will 221list different available operation sets. Please refer to the :ref:`design 222<damon_operations_set>` for the list of all available operation sets and their 223brief explanations. 224 225You can set and get what type of monitoring operations DAMON will use for the 226context by writing one of the keywords listed in ``avail_operations`` file and 227reading from the ``operations`` file. 228 229``addr_unit`` file is for setting and getting the :ref:`address unit 230<damon_design_addr_unit>` parameter of the operations set. 231 232``pause`` file is for setting and getting the :ref:`pause request 233<damon_design_execution_model_and_data_structures>` parameter of the context. 234 235.. _sysfs_monitoring_attrs: 236 237contexts/<N>/monitoring_attrs/ 238------------------------------ 239 240Files for specifying attributes of the monitoring including required quality 241and efficiency of the monitoring are in ``monitoring_attrs`` directory. 242Specifically, three directories, ``intervals``, ``nr_regions`` and ``probes`` 243exist in this directory. 244 245Under ``intervals`` directory, three files for DAMON's sampling interval 246(``sample_us``), aggregation interval (``aggr_us``), and update interval 247(``update_us``) exist. You can set and get the values in micro-seconds by 248writing to and reading from the files. 249 250Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 251of DAMON's monitoring regions (``min`` and ``max``, respectively), which 252controls the monitoring overhead, exist. You can set and get the values by 253writing to and reading from the files. 254 255For more details about the intervals and monitoring regions range, please refer 256to the Design document (:doc:`/mm/damon/design`). 257 258.. _damon_usage_sysfs_monitoring_intervals_goal: 259 260contexts/<N>/monitoring_attrs/intervals/intervals_goal/ 261------------------------------------------------------- 262 263Under the ``intervals`` directory, one directory for automated tuning of 264``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists. 265Under the directory, four files for the auto-tuning control, namely 266``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist. 267Please refer to the :ref:`design document of the feature 268<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning 269mechanism. Reading and writing the four files under ``intervals_goal`` 270directory shows and updates the tuning parameters that described in the 271:ref:`design doc <damon_design_monitoring_intervals_autotuning>` with the same 272names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The 273tuning-applied current values of the two intervals can be read from the 274``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to 275the ``state`` file. 276 277.. _damon_usage_sysfs_probes: 278 279contexts/<N>/monitoring_attrs/probes/ 280------------------------------------- 281 282A directory for registering :ref:`data attributes monitoring 283<damon_design_data_attrs_monitoring>` probes. 284 285In the beginning, this directory has only one file, ``nr_probes``. Writing a 286number (``N``) to the file creates the number of child directories named ``0`` 287to ``N-1``. Each directory represents each monitoring probe. 288 289In each probe directory, two directories, ``preps`` and ``filters`` exist. The 290directories contain files for installing probing preparation actions and 291filters for the probe, that are used to determine the data attribute for the 292probe. 293 294In the beginning, ``preps`` directory has only one file, ``nr_preps``. 295Writing a number (``N``) to the file creates the number of child directories 296named ``0`` to ``N-1``. Each directory represents each preparation action. 297Each directory has one file, ``prep_action``. The preparation action can be 298selected by writing the name of the action to the ``prep_action`` file. Refer 299to the :ref:`design doc <damon_design_data_attrs_monitoring>` for the list of 300supported actions. 301 302Each probe directory also contains ``weight`` file. Reading from and writing 303to the file gets and sets the :ref:`attributes-only monitoring 304<damon_design_attrs_only_monitoring>` weight for the attribute of the probe. 305 306In the beginning, ``filters`` directory has only one file, ``nr_filters``. 307Writing a number (``N``) to the file creates the number of child directories 308named ``0`` to ``N-1``. Each directory represents each filter and works in a 309way similar to that for :ref:`DAMOS filter <sysfs_filters>`. When the filter 310``type`` is ``memcg``, ``path`` file acts as ``memcg_path`` for :ref:`DAMOS 311filter <sysfs_filters>`. When the filter ``type`` is ``hugepage_size``, 312``min`` and ``max`` files act as files of the same names for :ref:`DAMOS 313filter <sysfs_filters>`. 314 315.. _sysfs_targets: 316 317contexts/<N>/targets/ 318--------------------- 319 320In the beginning, this directory has only one file, ``nr_targets``. Writing a 321number (``N``) to the file creates the number of child directories named ``0`` 322to ``N-1``. Each directory represents each monitoring target. 323 324.. _sysfs_target: 325 326targets/<N>/ 327------------ 328 329In each target directory, two files (``pid_target`` and ``obsolete_target``) 330and one directory (``regions``) exist. 331 332If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 333be a process. You can specify the process to DAMON by writing the pid of the 334process to the ``pid_target`` file. 335 336Users can selectively remove targets in the middle of the targets array by 337writing non-zero value to ``obsolete_target`` file and committing it (writing 338``commit`` to ``state`` file). DAMON will remove the matching targets from its 339internal targets array. Users are responsible to construct target directories 340again, so that those correctly represent the changed internal targets array. 341 342 343.. _sysfs_regions: 344 345targets/<N>/regions 346------------------- 347 348In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are 349required to set the monitoring target address ranges. In case of ``vaddr`` 350operations set, it is not mandatory, but users can optionally set the initial 351monitoring region to specific address ranges. Please refer to the :ref:`design 352<damon_design_vaddr_target_regions_construction>` for more details. 353 354For such cases, users can explicitly set the initial monitoring target regions 355as they want, by writing proper values to the files under this directory. 356 357In the beginning, this directory has only one file, ``nr_regions``. Writing a 358number (``N``) to the file creates the number of child directories named ``0`` 359to ``N-1``. Each directory represents each initial monitoring target region. 360 361If ``nr_regions`` is zero when committing new DAMON parameters online (writing 362``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit 363logic ignores the target regions. In other words, the current monitoring 364results for the target are preserved. 365 366.. _sysfs_region: 367 368regions/<N>/ 369------------ 370 371In each region directory, you will find two files (``start`` and ``end``). You 372can set and get the start and end addresses of the initial monitoring target 373region by writing to and reading from the files, respectively. 374 375Each region should not overlap with others. ``end`` of directory ``N`` should 376be equal or smaller than ``start`` of directory ``N+1``. 377 378.. _sysfs_schemes: 379 380contexts/<N>/schemes/ 381--------------------- 382 383The directory for DAMON-based Operation Schemes (:ref:`DAMOS 384<damon_design_damos>`). Users can get and set the schemes by reading from and 385writing to files under this directory. 386 387In the beginning, this directory has only one file, ``nr_schemes``. Writing a 388number (``N``) to the file creates the number of child directories named ``0`` 389to ``N-1``. Each directory represents each DAMON-based operation scheme. 390 391.. _sysfs_scheme: 392 393schemes/<N>/ 394------------ 395 396In each scheme directory, nine directories (``access_pattern``, ``quotas``, 397``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``, 398``stats``, and ``tried_regions``) and three files (``action``, ``target_nid`` 399and ``apply_interval_us``) exist. 400 401The ``action`` file is for setting and getting the scheme's :ref:`action 402<damon_design_damos_action>`. The keywords that can be written to and read 403from the file and their meaning are same to those of the list on 404:ref:`design doc <damon_design_damos_action>`. 405 406The ``target_nid`` file is for setting the migration target node, which is 407only meaningful when the ``action`` is either ``migrate_hot`` or 408``migrate_cold``. 409 410The ``apply_interval_us`` file is for setting and getting the scheme's 411:ref:`apply_interval <damon_design_damos>` in microseconds. 412 413.. _sysfs_access_pattern: 414 415schemes/<N>/access_pattern/ 416--------------------------- 417 418The directory for the target access :ref:`pattern 419<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 420 421Under the ``access_pattern`` directory, three directories (``sz``, 422``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 423exist. You can set and get the access pattern for the given scheme by writing 424to and reading from the ``min`` and ``max`` files under ``sz``, 425``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 426and the ``max`` form a closed interval. 427 428.. _sysfs_quotas: 429 430schemes/<N>/quotas/ 431------------------- 432 433The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 434DAMON-based operation scheme. 435 436Under ``quotas`` directory, seven files (``ms``, ``bytes``, 437``reset_interval_ms``, ``effective_bytes``, ``goal_tuner``, ``fail_charge_num`` 438and ``fail_charge_denom``) and two directories (``weights`` and ``goals``) 439exist. 440 441You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 442``reset interval`` in milliseconds by writing the values to the three files, 443respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 444for applying the ``action`` to memory regions of the ``access_pattern``, and to 445apply the action to only up to ``bytes`` bytes of memory regions within the 446``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 447quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is 448set. 449 450You can set the goal-based effective quota auto-tuning algorithm to use, by 451writing the algorithm name to ``goal_tuner`` file. Reading the file returns 452the currently selected tuner algorithm. Refer to the design documentation of 453:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for 454the background design of the feature and the name of the selectable algorithms. 455Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals 456setup. 457 458You can set the action-failed memory quota charging ratio by writing the 459numerator and the denominator for the ratio to ``fail_charge_num`` and 460``fail_charge_denom`` files, respectively. Reading those files will return the 461current set values. Refer to :ref:`design 462<damon_design_damos_quotas_failed_memory_charging_ratio>` for more details of 463the ratio feature. 464 465The time quota is internally transformed to a size quota. Between the 466transformed size quota and user-specified size quota, smaller one is applied. 467Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the 468effective size quota is further adjusted. Reading ``effective_bytes`` returns 469the current effective size quota. The file is not updated in real time, so 470users should ask DAMON sysfs interface to update the content of the file for 471the stats by writing a special keyword, ``update_schemes_effective_quotas`` to 472the relevant ``kdamonds/<N>/state`` file. 473 474Under ``weights`` directory, three files (``sz_permil``, 475``nr_accesses_permil``, and ``age_permil``) exist. 476You can set the :ref:`prioritization weights 477<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 478in per-thousand unit by writing the values to the three files under the 479``weights`` directory. 480 481.. _sysfs_schemes_quota_goals: 482 483schemes/<N>/quotas/goals/ 484------------------------- 485 486The directory for the :ref:`automatic quota tuning goals 487<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 488scheme. 489 490In the beginning, this directory has only one file, ``nr_goals``. Writing a 491number (``N``) to the file creates the number of child directories named ``0`` 492to ``N-1``. Each directory represents each goal and current achievement. 493Among the multiple feedback, the best one is used. 494 495Each goal directory contains five files, namely ``target_metric``, 496``target_value``, ``current_value``, ``nid``, and ``path``. Users can set and 497get the five parameters for the quota auto-tuning goals that specified on the 498:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and 499reading from each of the files. Because the kernel does not update 500``current_value``, reading it only makes sense when ``target_metric`` is 501``user_input``. Note that users should further write 502``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 503directory <sysfs_kdamond>` to pass the feedback to DAMON. 504 505.. _sysfs_watermarks: 506 507schemes/<N>/watermarks/ 508----------------------- 509 510The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 511given DAMON-based operation scheme. 512 513Under the watermarks directory, five files (``metric``, ``interval_us``, 514``high``, ``mid``, and ``low``) for setting the metric, the time interval 515between check of the metric, and the three watermarks exist. You can set and 516get the five values by writing to and reading from the files, respectively. 517 518Keywords and meanings of those that can be written to the ``metric`` file are 519as below. 520 521 - none: Ignore the watermarks 522 - free_mem_rate: System's free memory rate (per thousand) 523 524The ``interval_us`` should be written in microseconds unit. 525 526.. _sysfs_filters: 527 528schemes/<N>/{core\_,ops\_,}filters/ 529----------------------------------- 530 531Directories for :ref:`filters <damon_design_damos_filters>` of the given 532DAMON-based operation scheme. 533 534``core_filters`` and ``ops_filters`` directories are for the filters handled by 535the DAMON core layer and operations set layer, respectively. ``filters`` 536directory can be used for installing filters regardless of their handled 537layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be 538installed before those of ``filters``. All three directories have same files. 539 540Use of ``filters`` directory can make filters evaluation orders confusing to 541expect. For this reason, ``filters`` directory is deprecated. It is still 542functioning, but is scheduled for removal in the near future. Users should use 543``core_filters`` and ``ops_filters`` directories instead. 544 545In the beginning, the directory has only one file, ``nr_filters``. Writing a 546number (``N``) to the file creates the number of child directories named ``0`` 547to ``N-1``. Each directory represents each filter. The filters are evaluated 548in the numeric order. 549 550Each filter directory contains nine files, namely ``type``, ``matching``, 551``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max`` 552and ``damon_target_idx``. To ``type`` file, you can write the type of the 553filter. Refer to :ref:`the design doc <damon_design_damos_filters>` for 554available type names, their meaning and on what layer those are handled. 555 556For ``probe_hits_wsum`` type, you can specify the minimum and maximum probe 557hits weighted sum value for the filter to ``min`` and ``max`` files, 558respectively. 559 560For ``memcg`` type, you can specify the memory cgroup of the interest by 561writing the path of the memory cgroup from the cgroups mount point to 562``memcg_path`` file. For ``addr`` type, you can specify the start and end 563address of the range (open-ended interval) to ``addr_start`` and ``addr_end`` 564files, respectively. For ``hugepage_size`` type, you can specify the minimum 565and maximum size of the range (closed interval) to ``min`` and ``max`` files, 566respectively. For ``target`` type, you can specify the index of the target 567between the list of the DAMON context's monitoring targets list to 568``damon_target_idx`` file. 569 570You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter 571is for memory that matches the ``type``. You can write ``Y`` or ``N`` to 572``allow`` file to specify if applying the action to the memory that satisfies 573the ``type`` and ``matching`` should be allowed or not. 574 575For example, below restricts a DAMOS action to be applied to only non-anonymous 576pages of all memory cgroups except ``/having_care_already``.:: 577 578 # cd ops_filters/0/ 579 # echo 2 > nr_filters 580 # # disallow anonymous pages 581 echo anon > 0/type 582 echo Y > 0/matching 583 echo N > 0/allow 584 # # further filter out all cgroups except one at '/having_care_already' 585 echo memcg > 1/type 586 echo /having_care_already > 1/memcg_path 587 echo Y > 1/matching 588 echo N > 1/allow 589 590Refer to the :ref:`DAMOS filters design documentation 591<damon_design_damos_filters>` for more details including how multiple filters 592of different ``allow`` works, when each of the filters are supported, and 593differences on stats. 594 595.. _damon_sysfs_dests: 596 597schemes/<N>/dests/ 598------------------ 599 600Directory for specifying the destinations of given DAMON-based operation 601scheme's action. This directory is ignored if the action of the given scheme 602is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}`` 603actions are supporting multiple destinations. 604 605In the beginning, the directory has only one file, ``nr_dests``. Writing a 606number (``N``) to the file creates the number of child directories named ``0`` 607to ``N-1``. Each directory represents each action destination. 608 609Each destination directory contains two files, namely ``id`` and ``weight``. 610Users can write and read the identifier of the destination to ``id`` file. 611For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node 612id should be written to ``id`` file. Users can write and read the weight of 613the destination among the given destinations to the ``weight`` file. The 614weight can be an arbitrary integer. When DAMOS apply the action to each entity 615of the memory region, it will select the destination of the action based on the 616relative weights of the destinations. 617 618.. _sysfs_schemes_stats: 619 620schemes/<N>/stats/ 621------------------ 622 623DAMON counts statistics for each scheme. This statistics can be used for 624online analysis or tuning of the schemes. Refer to :ref:`design doc 625<damon_design_damos_stat>` for more details about the stats. 626 627The statistics can be retrieved by reading the files under ``stats`` directory 628(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, 629``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and 630``max_nr_snapshots``), respectively. 631 632The files are not updated in real time by default. Users should ask DAMON 633sysfs interface to periodically update those using ``refresh_ms``, or do a one 634time update by writing a special keyword, ``update_schemes_stats`` to the 635relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory 636<sysfs_kdamond>` for more details. 637 638.. _sysfs_schemes_tried_regions: 639 640schemes/<N>/tried_regions/ 641-------------------------- 642 643This directory initially has one file, ``total_bytes``. 644 645When a special keyword, ``update_schemes_tried_regions``, is written to the 646relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 647that reading it returns the total size of the scheme tried regions, and creates 648directories named integer starting from ``0`` under this directory. Each 649directory contains files exposing detailed information about each of the memory 650region that the corresponding scheme's ``action`` has tried to be applied under 651this directory, during next :ref:`apply interval <damon_design_damos>` of the 652corresponding scheme. The information includes address range, ``nr_accesses``, 653and ``age`` of the region. 654 655Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 656file will only update the ``total_bytes`` file, and will not create the 657subdirectories. 658 659The directories will be removed when another special keyword, 660``clear_schemes_tried_regions``, is written to the relevant 661``kdamonds/<N>/state`` file. 662 663The expected usage of this directory is investigations of schemes' behaviors, 664and query-like efficient data access monitoring results retrievals. For the 665latter use case, in particular, users can set the ``action`` as ``stat`` and 666set the ``access pattern`` as their interested pattern that they want to query. 667 668.. _sysfs_schemes_tried_region: 669 670tried_regions/<N>/ 671------------------ 672 673In each region directory, you will find five files (``start``, ``end``, 674``nr_accesses``, ``age`` and ``sz_filter_passed``). Reading the files will 675show the properties of the region that corresponding DAMON-based operation 676scheme ``action`` has tried to be applied. 677 678tried_regions/<N>/probes/ 679------------------------- 680 681In each region directory, one directory (``probes``) also exists. In the 682directory, subdirectories named ``0`` to ``N-1`` exists. ``N`` is the number 683of installed probes. In each number-named directory, a file (``hits``) exist. 684Reading the file shows the number of data attributes monitoring probe-hit 685positive samples of the region. 686 687Example 688~~~~~~~ 689 690Below commands applies a scheme saying "If a memory region of size in [4KiB, 6918KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 692interval in [10, 20], page out the region. For the paging out, use only up to 69310ms per second, and also don't page out more than 1GiB per second. Under the 694limitation, page out memory regions having longer age first. Also, check the 695free memory rate of the system every 5 seconds, start the monitoring and paging 696out when the free memory rate becomes lower than 50%, but stop it if the free 697memory rate becomes larger than 60%, or lower than 30%". :: 698 699 # cd <sysfs>/kernel/mm/damon/admin 700 # # populate directories 701 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 702 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 703 # cd kdamonds/0/contexts/0/schemes/0 704 # # set the basic access pattern and the action 705 # echo 4096 > access_pattern/sz/min 706 # echo 8192 > access_pattern/sz/max 707 # echo 0 > access_pattern/nr_accesses/min 708 # echo 5 > access_pattern/nr_accesses/max 709 # echo 10 > access_pattern/age/min 710 # echo 20 > access_pattern/age/max 711 # echo pageout > action 712 # # set quotas 713 # echo 10 > quotas/ms 714 # echo $((1024*1024*1024)) > quotas/bytes 715 # echo 1000 > quotas/reset_interval_ms 716 # # set watermark 717 # echo free_mem_rate > watermarks/metric 718 # echo 5000000 > watermarks/interval_us 719 # echo 600 > watermarks/high 720 # echo 500 > watermarks/mid 721 # echo 300 > watermarks/low 722 723Please note that it's highly recommended to use user space tools like `damo 724<https://github.com/damonitor/damo>`_ rather than manually reading and writing 725the files as above. Above is only for an example. 726 727.. _tracepoint: 728 729Tracepoints for Monitoring Results 730================================== 731 732Users can get the monitoring results via the :ref:`tried_regions 733<sysfs_schemes_tried_regions>`. The interface is useful for getting a 734snapshot, but it could be inefficient for fully recording all the monitoring 735results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 736and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 737provides the whole monitoring results, while ``damon:damos_before_apply`` 738provides the monitoring results for regions that each DAMON-based Operation 739Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 740``damon:damos_before_apply`` is more useful for recording internal behavior of 741DAMOS, or DAMOS target access 742:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 743monitoring results recording. 744 745While the monitoring is turned on, you could record the tracepoint events and 746show results using tracepoint supporting tools like ``perf``. For example:: 747 748 # echo on > kdamonds/0/state 749 # perf record -e damon:damon_aggregated & 750 # sleep 5 751 # kill 9 $(pidof perf) 752 # echo off > kdamonds/0/state 753 # perf script 754 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 755 [...] 756 757Each line of the perf script output represents each monitoring region. The 758first five fields are as usual other tracepoint outputs. The sixth field 759(``target_id=X``) shows the id of the monitoring target of the region. The 760seventh field (``nr_regions=X``) shows the total number of monitoring regions 761for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 762(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 763``nr_accesses`` of the region (refer to 764:ref:`design <damon_design_region_based_sampling>` for more details of the 765counter). Finally the tenth field (``X``) shows the ``age`` of the region 766(refer to :ref:`design <damon_design_age_tracking>` for more details of the 767counter). 768 769If the event was ``damon:damos_before_apply``, the ``perf script`` output would 770be somewhat like below:: 771 772 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 773 [...] 774 775Each line of the output represents each monitoring region that each DAMON-based 776Operation Scheme was about to be applied at the traced time. The first five 777fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 778of the scheme in the list of the contexts of the context's kdamond, the index 779of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 780addition to the output of ``damon_aggregated`` tracepoint. 781