1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *Special-purpose DAMON modules.* 10 :ref:`This <damon_modules_special_purpose>` is for people who are building, 11 distributing, and/or administrating the kernel with special-purpose DAMON 12 usages. Using this, users can use DAMON's major features for the given 13 purposes in build, boot, or runtime in simple ways. 14- *DAMON user space tool.* 15 `This <https://github.com/damonitor/damo>`_ is for privileged people such as 16 system administrators who want a just-working human-friendly interface. 17 Using this, users can use the DAMON’s major features in a human-friendly way. 18 It may not be highly tuned for special cases, though. For more detail, 19 please refer to its `usage document 20 <https://github.com/damonitor/damo/blob/next/USAGE.md>`_. 21- *sysfs interface.* 22 :ref:`This <sysfs_interface>` is for privileged user space programmers who 23 want more optimized use of DAMON. Using this, users can use DAMON’s major 24 features by reading from and writing to special sysfs files. Therefore, 25 you can write and use your personalized DAMON sysfs wrapper programs that 26 reads/writes the sysfs files instead of you. The `DAMON user space tool 27 <https://github.com/damonitor/damo>`_ is one example of such programs. 28- *Kernel Space Programming Interface.* 29 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </mm/damon/api>`. 34 35.. _sysfs_interface: 36 37sysfs Interface 38=============== 39 40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 41creates multiple directories and files under its sysfs directory, 42``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 43from the files under the directory. 44 45For a short example, users can monitor the virtual address space of a given 46workload as below. :: 47 48 # cd /sys/kernel/mm/damon/admin/ 49 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 50 # echo vaddr > kdamonds/0/contexts/0/operations 51 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 52 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 53 # echo on > kdamonds/0/state 54 55Files Hierarchy 56--------------- 57 58The files hierarchy of DAMON sysfs interface is shown below. In the below 59figure, parents-children relations are represented with indentations, each 60directory is having ``/`` suffix, and files in each directory are separated by 61comma (","). 62 63.. parsed-literal:: 64 65 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 66 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 67 │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms 68 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 69 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit 70 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 71 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 72 │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us 73 │ │ │ │ │ │ nr_regions/min,max 74 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 75 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target 76 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 77 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 78 │ │ │ │ │ │ │ │ ... 79 │ │ │ │ │ │ ... 80 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 81 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us 82 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 83 │ │ │ │ │ │ │ │ sz/min,max 84 │ │ │ │ │ │ │ │ nr_accesses/min,max 85 │ │ │ │ │ │ │ │ age/min,max 86 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,effective_bytes,goal_tuner 87 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 88 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 89 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path 90 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 91 │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters 92 │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max 93 │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests 94 │ │ │ │ │ │ │ │ 0/id,weight 95 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots 96 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 97 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed 98 │ │ │ │ │ │ │ │ ... 99 │ │ │ │ │ │ ... 100 │ │ │ │ ... 101 │ │ ... 102 103.. _sysfs_root: 104 105Root 106---- 107 108The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 109has one directory named ``admin``. The directory contains the files for 110privileged user space programs' control of DAMON. User space tools or daemons 111having the root permission could use this directory. 112 113.. _sysfs_kdamonds: 114 115kdamonds/ 116--------- 117 118Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 119controlling the kdamonds (refer to 120:ref:`design <damon_design_execution_model_and_data_structures>` for more 121details) exists. In the beginning, this directory has only one file, 122``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 123child directories named ``0`` to ``N-1``. Each directory represents each 124kdamond. 125 126.. _sysfs_kdamond: 127 128kdamonds/<N>/ 129------------- 130 131In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``) 132and one directory (``contexts``) exist. 133 134Reading ``state`` returns ``on`` if the kdamond is currently running, or 135``off`` if it is not running. 136 137Users can write below commands for the kdamond to the ``state`` file. 138 139- ``on``: Start running. 140- ``off``: Stop running. 141- ``commit``: Read the user inputs in the sysfs files except ``state`` file 142 again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be 143 ignored if no target region is specified. 144- ``update_tuned_intervals``: Update the contents of ``sample_us`` and 145 ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling 146 interval`` and ``aggregation interval`` for the files. Please refer to 147 :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>` 148 for more details. 149- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 150 :ref:`quota goals <sysfs_schemes_quota_goals>`. 151- ``update_schemes_stats``: Update the contents of stats files for each 152 DAMON-based operation scheme of the kdamond. For details of the stats, 153 please refer to :ref:`stats section <sysfs_schemes_stats>`. 154- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 155 action tried regions directory for each DAMON-based operation scheme of the 156 kdamond. For details of the DAMON-based operation scheme action tried 157 regions directory, please refer to 158 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 159- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 160 files. 161- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 162 action tried regions directory for each DAMON-based operation scheme of the 163 kdamond. 164- ``update_schemes_effective_quotas``: Update the contents of 165 ``effective_bytes`` files for each DAMON-based operation scheme of the 166 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`. 167 168If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 169 170Users can ask the kernel to periodically update files showing auto-tuned 171parameters and DAMOS stats instead of manually writing 172``update_tuned_intervals`` like keywords to ``state`` file. For this, users 173should write the desired update time interval in milliseconds to ``refresh_ms`` 174file. If the interval is zero, the periodic update is disabled. Reading the 175file shows currently set time interval. 176 177``contexts`` directory contains files for controlling the monitoring contexts 178that this kdamond will execute. 179 180.. _sysfs_contexts: 181 182kdamonds/<N>/contexts/ 183---------------------- 184 185In the beginning, this directory has only one file, ``nr_contexts``. Writing a 186number (``N``) to the file creates the number of child directories named as 187``0`` to ``N-1``. Each directory represents each monitoring context (refer to 188:ref:`design <damon_design_execution_model_and_data_structures>` for more 189details). At the moment, only one context per kdamond is supported, so only 190``0`` or ``1`` can be written to the file. 191 192.. _sysfs_context: 193 194contexts/<N>/ 195------------- 196 197In each context directory, three files (``avail_operations``, ``operations`` 198and ``addr_unit``) and three directories (``monitoring_attrs``, ``targets``, 199and ``schemes``) exist. 200 201DAMON supports multiple types of :ref:`monitoring operations 202<damon_design_configurable_operations_set>`, including those for virtual address 203space and the physical address space. You can get the list of available 204monitoring operations set on the currently running kernel by reading 205``avail_operations`` file. Based on the kernel configuration, the file will 206list different available operation sets. Please refer to the :ref:`design 207<damon_operations_set>` for the list of all available operation sets and their 208brief explanations. 209 210You can set and get what type of monitoring operations DAMON will use for the 211context by writing one of the keywords listed in ``avail_operations`` file and 212reading from the ``operations`` file. 213 214``addr_unit`` file is for setting and getting the :ref:`address unit 215<damon_design_addr_unit>` parameter of the operations set. 216 217.. _sysfs_monitoring_attrs: 218 219contexts/<N>/monitoring_attrs/ 220------------------------------ 221 222Files for specifying attributes of the monitoring including required quality 223and efficiency of the monitoring are in ``monitoring_attrs`` directory. 224Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 225directory. 226 227Under ``intervals`` directory, three files for DAMON's sampling interval 228(``sample_us``), aggregation interval (``aggr_us``), and update interval 229(``update_us``) exist. You can set and get the values in micro-seconds by 230writing to and reading from the files. 231 232Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 233of DAMON's monitoring regions (``min`` and ``max``, respectively), which 234controls the monitoring overhead, exist. You can set and get the values by 235writing to and rading from the files. 236 237For more details about the intervals and monitoring regions range, please refer 238to the Design document (:doc:`/mm/damon/design`). 239 240.. _damon_usage_sysfs_monitoring_intervals_goal: 241 242contexts/<N>/monitoring_attrs/intervals/intervals_goal/ 243------------------------------------------------------- 244 245Under the ``intervals`` directory, one directory for automated tuning of 246``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists. 247Under the directory, four files for the auto-tuning control, namely 248``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist. 249Please refer to the :ref:`design document of the feature 250<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning 251mechanism. Reading and writing the four files under ``intervals_goal`` 252directory shows and updates the tuning parameters that described in the 253:ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same 254names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The 255tuning-applied current values of the two intervals can be read from the 256``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to 257the ``state`` file. 258 259.. _sysfs_targets: 260 261contexts/<N>/targets/ 262--------------------- 263 264In the beginning, this directory has only one file, ``nr_targets``. Writing a 265number (``N``) to the file creates the number of child directories named ``0`` 266to ``N-1``. Each directory represents each monitoring target. 267 268.. _sysfs_target: 269 270targets/<N>/ 271------------ 272 273In each target directory, two files (``pid_target`` and ``obsolete_target``) 274and one directory (``regions``) exist. 275 276If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 277be a process. You can specify the process to DAMON by writing the pid of the 278process to the ``pid_target`` file. 279 280Users can selectively remove targets in the middle of the targets array by 281writing non-zero value to ``obsolete_target`` file and committing it (writing 282``commit`` to ``state`` file). DAMON will remove the matching targets from its 283internal targets array. Users are responsible to construct target directories 284again, so that those correctly represent the changed internal targets array. 285 286 287.. _sysfs_regions: 288 289targets/<N>/regions 290------------------- 291 292In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are 293required to set the monitoring target address ranges. In case of ``vaddr`` 294operations set, it is not mandatory, but users can optionally set the initial 295monitoring region to specific address ranges. Please refer to the :ref:`design 296<damon_design_vaddr_target_regions_construction>` for more details. 297 298For such cases, users can explicitly set the initial monitoring target regions 299as they want, by writing proper values to the files under this directory. 300 301In the beginning, this directory has only one file, ``nr_regions``. Writing a 302number (``N``) to the file creates the number of child directories named ``0`` 303to ``N-1``. Each directory represents each initial monitoring target region. 304 305If ``nr_regions`` is zero when committing new DAMON parameters online (writing 306``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit 307logic ignores the target regions. In other words, the current monitoring 308results for the target are preserved. 309 310.. _sysfs_region: 311 312regions/<N>/ 313------------ 314 315In each region directory, you will find two files (``start`` and ``end``). You 316can set and get the start and end addresses of the initial monitoring target 317region by writing to and reading from the files, respectively. 318 319Each region should not overlap with others. ``end`` of directory ``N`` should 320be equal or smaller than ``start`` of directory ``N+1``. 321 322.. _sysfs_schemes: 323 324contexts/<N>/schemes/ 325--------------------- 326 327The directory for DAMON-based Operation Schemes (:ref:`DAMOS 328<damon_design_damos>`). Users can get and set the schemes by reading from and 329writing to files under this directory. 330 331In the beginning, this directory has only one file, ``nr_schemes``. Writing a 332number (``N``) to the file creates the number of child directories named ``0`` 333to ``N-1``. Each directory represents each DAMON-based operation scheme. 334 335.. _sysfs_scheme: 336 337schemes/<N>/ 338------------ 339 340In each scheme directory, eight directories (``access_pattern``, ``quotas``, 341``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``, 342``stats``, and ``tried_regions``) and three files (``action``, ``target_nid`` 343and ``apply_interval``) exist. 344 345The ``action`` file is for setting and getting the scheme's :ref:`action 346<damon_design_damos_action>`. The keywords that can be written to and read 347from the file and their meaning are same to those of the list on 348:ref:`design doc <damon_design_damos_action>`. 349 350The ``target_nid`` file is for setting the migration target node, which is 351only meaningful when the ``action`` is either ``migrate_hot`` or 352``migrate_cold``. 353 354The ``apply_interval_us`` file is for setting and getting the scheme's 355:ref:`apply_interval <damon_design_damos>` in microseconds. 356 357.. _sysfs_access_pattern: 358 359schemes/<N>/access_pattern/ 360--------------------------- 361 362The directory for the target access :ref:`pattern 363<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 364 365Under the ``access_pattern`` directory, three directories (``sz``, 366``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 367exist. You can set and get the access pattern for the given scheme by writing 368to and reading from the ``min`` and ``max`` files under ``sz``, 369``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 370and the ``max`` form a closed interval. 371 372.. _sysfs_quotas: 373 374schemes/<N>/quotas/ 375------------------- 376 377The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 378DAMON-based operation scheme. 379 380Under ``quotas`` directory, five files (``ms``, ``bytes``, 381``reset_interval_ms``, ``effective_bytes`` and ``goal_tuner``) and two 382directories (``weights`` and ``goals``) exist. 383 384You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 385``reset interval`` in milliseconds by writing the values to the three files, 386respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 387for applying the ``action`` to memory regions of the ``access_pattern``, and to 388apply the action to only up to ``bytes`` bytes of memory regions within the 389``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 390quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is 391set. 392 393You can set the goal-based effective quota auto-tuning algorithm to use, by 394writing the algorithm name to ``goal_tuner`` file. Reading the file returns 395the currently selected tuner algorithm. Refer to the design documentation of 396:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for 397the background design of the feature and the name of the selectable algorithms. 398Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals 399setup. 400 401The time quota is internally transformed to a size quota. Between the 402transformed size quota and user-specified size quota, smaller one is applied. 403Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the 404effective size quota is further adjusted. Reading ``effective_bytes`` returns 405the current effective size quota. The file is not updated in real time, so 406users should ask DAMON sysfs interface to update the content of the file for 407the stats by writing a special keyword, ``update_schemes_effective_quotas`` to 408the relevant ``kdamonds/<N>/state`` file. 409 410Under ``weights`` directory, three files (``sz_permil``, 411``nr_accesses_permil``, and ``age_permil``) exist. 412You can set the :ref:`prioritization weights 413<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 414in per-thousand unit by writing the values to the three files under the 415``weights`` directory. 416 417.. _sysfs_schemes_quota_goals: 418 419schemes/<N>/quotas/goals/ 420------------------------- 421 422The directory for the :ref:`automatic quota tuning goals 423<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 424scheme. 425 426In the beginning, this directory has only one file, ``nr_goals``. Writing a 427number (``N``) to the file creates the number of child directories named ``0`` 428to ``N-1``. Each directory represents each goal and current achievement. 429Among the multiple feedback, the best one is used. 430 431Each goal directory contains five files, namely ``target_metric``, 432``target_value``, ``current_value`` ``nid`` and ``path``. Users can set and 433get the five parameters for the quota auto-tuning goals that specified on the 434:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and 435reading from each of the files. Note that users should further write 436``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 437directory <sysfs_kdamond>` to pass the feedback to DAMON. 438 439.. _sysfs_watermarks: 440 441schemes/<N>/watermarks/ 442----------------------- 443 444The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 445given DAMON-based operation scheme. 446 447Under the watermarks directory, five files (``metric``, ``interval_us``, 448``high``, ``mid``, and ``low``) for setting the metric, the time interval 449between check of the metric, and the three watermarks exist. You can set and 450get the five values by writing to the files, respectively. 451 452Keywords and meanings of those that can be written to the ``metric`` file are 453as below. 454 455 - none: Ignore the watermarks 456 - free_mem_rate: System's free memory rate (per thousand) 457 458The ``interval`` should written in microseconds unit. 459 460.. _sysfs_filters: 461 462schemes/<N>/{core\_,ops\_,}filters/ 463----------------------------------- 464 465Directories for :ref:`filters <damon_design_damos_filters>` of the given 466DAMON-based operation scheme. 467 468``core_filters`` and ``ops_filters`` directories are for the filters handled by 469the DAMON core layer and operations set layer, respectively. ``filters`` 470directory can be used for installing filters regardless of their handled 471layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be 472installed before those of ``filters``. All three directories have same files. 473 474Use of ``filters`` directory can make expecting evaluation orders of given 475filters with the files under directory bit confusing. Users are hence 476recommended to use ``core_filters`` and ``ops_filters`` directories. The 477``filters`` directory could be deprecated in future. 478 479In the beginning, the directory has only one file, ``nr_filters``. Writing a 480number (``N``) to the file creates the number of child directories named ``0`` 481to ``N-1``. Each directory represents each filter. The filters are evaluated 482in the numeric order. 483 484Each filter directory contains nine files, namely ``type``, ``matching``, 485``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max`` 486and ``target_idx``. To ``type`` file, you can write the type of the filter. 487Refer to :ref:`the design doc <damon_design_damos_filters>` for available type 488names, their meaning and on what layer those are handled. 489 490For ``memcg`` type, you can specify the memory cgroup of the interest by 491writing the path of the memory cgroup from the cgroups mount point to 492``memcg_path`` file. For ``addr`` type, you can specify the start and end 493address of the range (open-ended interval) to ``addr_start`` and ``addr_end`` 494files, respectively. For ``hugepage_size`` type, you can specify the minimum 495and maximum size of the range (closed interval) to ``min`` and ``max`` files, 496respectively. For ``target`` type, you can specify the index of the target 497between the list of the DAMON context's monitoring targets list to 498``target_idx`` file. 499 500You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter 501is for memory that matches the ``type``. You can write ``Y`` or ``N`` to 502``allow`` file to specify if applying the action to the memory that satisfies 503the ``type`` and ``matching`` should be allowed or not. 504 505For example, below restricts a DAMOS action to be applied to only non-anonymous 506pages of all memory cgroups except ``/having_care_already``.:: 507 508 # cd ops_filters/0/ 509 # echo 2 > nr_filters 510 # # disallow anonymous pages 511 echo anon > 0/type 512 echo Y > 0/matching 513 echo N > 0/allow 514 # # further filter out all cgroups except one at '/having_care_already' 515 echo memcg > 1/type 516 echo /having_care_already > 1/memcg_path 517 echo Y > 1/matching 518 echo N > 1/allow 519 520Refer to the :ref:`DAMOS filters design documentation 521<damon_design_damos_filters>` for more details including how multiple filters 522of different ``allow`` works, when each of the filters are supported, and 523differences on stats. 524 525.. _damon_sysfs_dests: 526 527schemes/<N>/dests/ 528------------------ 529 530Directory for specifying the destinations of given DAMON-based operation 531scheme's action. This directory is ignored if the action of the given scheme 532is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}`` 533actions are supporting multiple destinations. 534 535In the beginning, the directory has only one file, ``nr_dests``. Writing a 536number (``N``) to the file creates the number of child directories named ``0`` 537to ``N-1``. Each directory represents each action destination. 538 539Each destination directory contains two files, namely ``id`` and ``weight``. 540Users can write and read the identifier of the destination to ``id`` file. 541For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node 542id should be written to ``id`` file. Users can write and read the weight of 543the destination among the given destinations to the ``weight`` file. The 544weight can be an arbitrary integer. When DAMOS apply the action to each entity 545of the memory region, it will select the destination of the action based on the 546relative weights of the destinations. 547 548.. _sysfs_schemes_stats: 549 550schemes/<N>/stats/ 551------------------ 552 553DAMON counts statistics for each scheme. This statistics can be used for 554online analysis or tuning of the schemes. Refer to :ref:`design doc 555<damon_design_damos_stat>` for more details about the stats. 556 557The statistics can be retrieved by reading the files under ``stats`` directory 558(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, 559``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and 560``max_nr_snapshots``), respectively. 561 562The files are not updated in real time by default. Users should ask DAMON 563sysfs interface to periodically update those using ``refresh_ms``, or do a one 564time update by writing a special keyword, ``update_schemes_stats`` to the 565relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory 566<sysfs_kdamond>` for more details. 567 568.. _sysfs_schemes_tried_regions: 569 570schemes/<N>/tried_regions/ 571-------------------------- 572 573This directory initially has one file, ``total_bytes``. 574 575When a special keyword, ``update_schemes_tried_regions``, is written to the 576relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 577that reading it returns the total size of the scheme tried regions, and creates 578directories named integer starting from ``0`` under this directory. Each 579directory contains files exposing detailed information about each of the memory 580region that the corresponding scheme's ``action`` has tried to be applied under 581this directory, during next :ref:`apply interval <damon_design_damos>` of the 582corresponding scheme. The information includes address range, ``nr_accesses``, 583and ``age`` of the region. 584 585Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 586file will only update the ``total_bytes`` file, and will not create the 587subdirectories. 588 589The directories will be removed when another special keyword, 590``clear_schemes_tried_regions``, is written to the relevant 591``kdamonds/<N>/state`` file. 592 593The expected usage of this directory is investigations of schemes' behaviors, 594and query-like efficient data access monitoring results retrievals. For the 595latter use case, in particular, users can set the ``action`` as ``stat`` and 596set the ``access pattern`` as their interested pattern that they want to query. 597 598.. _sysfs_schemes_tried_region: 599 600tried_regions/<N>/ 601------------------ 602 603In each region directory, you will find five files (``start``, ``end``, 604``nr_accesses``, ``age``, and ``sz_filter_passed``). Reading the files will 605show the properties of the region that corresponding DAMON-based operation 606scheme ``action`` has tried to be applied. 607 608Example 609~~~~~~~ 610 611Below commands applies a scheme saying "If a memory region of size in [4KiB, 6128KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 613interval in [10, 20], page out the region. For the paging out, use only up to 61410ms per second, and also don't page out more than 1GiB per second. Under the 615limitation, page out memory regions having longer age first. Also, check the 616free memory rate of the system every 5 seconds, start the monitoring and paging 617out when the free memory rate becomes lower than 50%, but stop it if the free 618memory rate becomes larger than 60%, or lower than 30%". :: 619 620 # cd <sysfs>/kernel/mm/damon/admin 621 # # populate directories 622 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 623 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 624 # cd kdamonds/0/contexts/0/schemes/0 625 # # set the basic access pattern and the action 626 # echo 4096 > access_pattern/sz/min 627 # echo 8192 > access_pattern/sz/max 628 # echo 0 > access_pattern/nr_accesses/min 629 # echo 5 > access_pattern/nr_accesses/max 630 # echo 10 > access_pattern/age/min 631 # echo 20 > access_pattern/age/max 632 # echo pageout > action 633 # # set quotas 634 # echo 10 > quotas/ms 635 # echo $((1024*1024*1024)) > quotas/bytes 636 # echo 1000 > quotas/reset_interval_ms 637 # # set watermark 638 # echo free_mem_rate > watermarks/metric 639 # echo 5000000 > watermarks/interval_us 640 # echo 600 > watermarks/high 641 # echo 500 > watermarks/mid 642 # echo 300 > watermarks/low 643 644Please note that it's highly recommended to use user space tools like `damo 645<https://github.com/damonitor/damo>`_ rather than manually reading and writing 646the files as above. Above is only for an example. 647 648.. _tracepoint: 649 650Tracepoints for Monitoring Results 651================================== 652 653Users can get the monitoring results via the :ref:`tried_regions 654<sysfs_schemes_tried_regions>`. The interface is useful for getting a 655snapshot, but it could be inefficient for fully recording all the monitoring 656results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 657and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 658provides the whole monitoring results, while ``damon:damos_before_apply`` 659provides the monitoring results for regions that each DAMON-based Operation 660Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 661``damon:damos_before_apply`` is more useful for recording internal behavior of 662DAMOS, or DAMOS target access 663:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 664monitoring results recording. 665 666While the monitoring is turned on, you could record the tracepoint events and 667show results using tracepoint supporting tools like ``perf``. For example:: 668 669 # echo on > kdamonds/0/state 670 # perf record -e damon:damon_aggregated & 671 # sleep 5 672 # kill 9 $(pidof perf) 673 # echo off > kdamonds/0/state 674 # perf script 675 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 676 [...] 677 678Each line of the perf script output represents each monitoring region. The 679first five fields are as usual other tracepoint outputs. The sixth field 680(``target_id=X``) shows the ide of the monitoring target of the region. The 681seventh field (``nr_regions=X``) shows the total number of monitoring regions 682for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 683(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 684``nr_accesses`` of the region (refer to 685:ref:`design <damon_design_region_based_sampling>` for more details of the 686counter). Finally the tenth field (``X``) shows the ``age`` of the region 687(refer to :ref:`design <damon_design_age_tracking>` for more details of the 688counter). 689 690If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would 691be somewhat like below:: 692 693 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 694 [...] 695 696Each line of the output represents each monitoring region that each DAMON-based 697Operation Scheme was about to be applied at the traced time. The first five 698fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 699of the scheme in the list of the contexts of the context's kdamond, the index 700of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 701addition to the output of ``damon_aggregated`` tracepoint. 702