1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *Special-purpose DAMON modules.* 10 :ref:`This <damon_modules_special_purpose>` is for people who are building, 11 distributing, and/or administrating the kernel with special-purpose DAMON 12 usages. Using this, users can use DAMON's major features for the given 13 purposes in build, boot, or runtime in simple ways. 14- *DAMON user space tool.* 15 `This <https://github.com/damonitor/damo>`_ is for privileged people such as 16 system administrators who want a just-working human-friendly interface. 17 Using this, users can use the DAMON’s major features in a human-friendly way. 18 It may not be highly tuned for special cases, though. For more detail, 19 please refer to its `usage document 20 <https://github.com/damonitor/damo/blob/next/USAGE.md>`_. 21- *sysfs interface.* 22 :ref:`This <sysfs_interface>` is for privileged user space programmers who 23 want more optimized use of DAMON. Using this, users can use DAMON’s major 24 features by reading from and writing to special sysfs files. Therefore, 25 you can write and use your personalized DAMON sysfs wrapper programs that 26 reads/writes the sysfs files instead of you. The `DAMON user space tool 27 <https://github.com/damonitor/damo>`_ is one example of such programs. 28- *Kernel Space Programming Interface.* 29 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </mm/damon/api>`. 34 35.. _sysfs_interface: 36 37sysfs Interface 38=============== 39 40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 41creates multiple directories and files under its sysfs directory, 42``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 43from the files under the directory. 44 45For a short example, users can monitor the virtual address space of a given 46workload as below. :: 47 48 # cd /sys/kernel/mm/damon/admin/ 49 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 50 # echo vaddr > kdamonds/0/contexts/0/operations 51 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 52 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 53 # echo on > kdamonds/0/state 54 55Files Hierarchy 56--------------- 57 58The files hierarchy of DAMON sysfs interface is shown below. In the below 59figure, parents-children relations are represented with indentations, each 60directory is having ``/`` suffix, and files in each directory are separated by 61comma (","). 62 63.. parsed-literal:: 64 65 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 66 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 67 │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms 68 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 69 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit 70 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 71 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 72 │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us 73 │ │ │ │ │ │ nr_regions/min,max 74 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 75 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target 76 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 77 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 78 │ │ │ │ │ │ │ │ ... 79 │ │ │ │ │ │ ... 80 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 81 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us 82 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 83 │ │ │ │ │ │ │ │ sz/min,max 84 │ │ │ │ │ │ │ │ nr_accesses/min,max 85 │ │ │ │ │ │ │ │ age/min,max 86 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,effective_bytes 87 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 88 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 89 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path 90 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 91 │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters 92 │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max 93 │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests 94 │ │ │ │ │ │ │ │ 0/id,weight 95 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots 96 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 97 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed 98 │ │ │ │ │ │ │ │ ... 99 │ │ │ │ │ │ ... 100 │ │ │ │ ... 101 │ │ ... 102 103.. _sysfs_root: 104 105Root 106---- 107 108The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 109has one directory named ``admin``. The directory contains the files for 110privileged user space programs' control of DAMON. User space tools or daemons 111having the root permission could use this directory. 112 113.. _sysfs_kdamonds: 114 115kdamonds/ 116--------- 117 118Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 119controlling the kdamonds (refer to 120:ref:`design <damon_design_execution_model_and_data_structures>` for more 121details) exists. In the beginning, this directory has only one file, 122``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 123child directories named ``0`` to ``N-1``. Each directory represents each 124kdamond. 125 126.. _sysfs_kdamond: 127 128kdamonds/<N>/ 129------------- 130 131In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``) 132and one directory (``contexts``) exist. 133 134Reading ``state`` returns ``on`` if the kdamond is currently running, or 135``off`` if it is not running. 136 137Users can write below commands for the kdamond to the ``state`` file. 138 139- ``on``: Start running. 140- ``off``: Stop running. 141- ``commit``: Read the user inputs in the sysfs files except ``state`` file 142 again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be 143 ignored if no target region is specified. 144- ``update_tuned_intervals``: Update the contents of ``sample_us`` and 145 ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling 146 interval`` and ``aggregation interval`` for the files. Please refer to 147 :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>` 148 for more details. 149- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 150 :ref:`quota goals <sysfs_schemes_quota_goals>`. 151- ``update_schemes_stats``: Update the contents of stats files for each 152 DAMON-based operation scheme of the kdamond. For details of the stats, 153 please refer to :ref:`stats section <sysfs_schemes_stats>`. 154- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 155 action tried regions directory for each DAMON-based operation scheme of the 156 kdamond. For details of the DAMON-based operation scheme action tried 157 regions directory, please refer to 158 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 159- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 160 files. 161- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 162 action tried regions directory for each DAMON-based operation scheme of the 163 kdamond. 164- ``update_schemes_effective_quotas``: Update the contents of 165 ``effective_bytes`` files for each DAMON-based operation scheme of the 166 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`. 167 168If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 169 170Users can ask the kernel to periodically update files showing auto-tuned 171parameters and DAMOS stats instead of manually writing 172``update_tuned_intervals`` like keywords to ``state`` file. For this, users 173should write the desired update time interval in milliseconds to ``refresh_ms`` 174file. If the interval is zero, the periodic update is disabled. Reading the 175file shows currently set time interval. 176 177``contexts`` directory contains files for controlling the monitoring contexts 178that this kdamond will execute. 179 180.. _sysfs_contexts: 181 182kdamonds/<N>/contexts/ 183---------------------- 184 185In the beginning, this directory has only one file, ``nr_contexts``. Writing a 186number (``N``) to the file creates the number of child directories named as 187``0`` to ``N-1``. Each directory represents each monitoring context (refer to 188:ref:`design <damon_design_execution_model_and_data_structures>` for more 189details). At the moment, only one context per kdamond is supported, so only 190``0`` or ``1`` can be written to the file. 191 192.. _sysfs_context: 193 194contexts/<N>/ 195------------- 196 197In each context directory, three files (``avail_operations``, ``operations`` 198and ``addr_unit``) and three directories (``monitoring_attrs``, ``targets``, 199and ``schemes``) exist. 200 201DAMON supports multiple types of :ref:`monitoring operations 202<damon_design_configurable_operations_set>`, including those for virtual address 203space and the physical address space. You can get the list of available 204monitoring operations set on the currently running kernel by reading 205``avail_operations`` file. Based on the kernel configuration, the file will 206list different available operation sets. Please refer to the :ref:`design 207<damon_operations_set>` for the list of all available operation sets and their 208brief explanations. 209 210You can set and get what type of monitoring operations DAMON will use for the 211context by writing one of the keywords listed in ``avail_operations`` file and 212reading from the ``operations`` file. 213 214``addr_unit`` file is for setting and getting the :ref:`address unit 215<damon_design_addr_unit>` parameter of the operations set. 216 217.. _sysfs_monitoring_attrs: 218 219contexts/<N>/monitoring_attrs/ 220------------------------------ 221 222Files for specifying attributes of the monitoring including required quality 223and efficiency of the monitoring are in ``monitoring_attrs`` directory. 224Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 225directory. 226 227Under ``intervals`` directory, three files for DAMON's sampling interval 228(``sample_us``), aggregation interval (``aggr_us``), and update interval 229(``update_us``) exist. You can set and get the values in micro-seconds by 230writing to and reading from the files. 231 232Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 233of DAMON's monitoring regions (``min`` and ``max``, respectively), which 234controls the monitoring overhead, exist. You can set and get the values by 235writing to and rading from the files. 236 237For more details about the intervals and monitoring regions range, please refer 238to the Design document (:doc:`/mm/damon/design`). 239 240.. _damon_usage_sysfs_monitoring_intervals_goal: 241 242contexts/<N>/monitoring_attrs/intervals/intervals_goal/ 243------------------------------------------------------- 244 245Under the ``intervals`` directory, one directory for automated tuning of 246``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists. 247Under the directory, four files for the auto-tuning control, namely 248``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist. 249Please refer to the :ref:`design document of the feature 250<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning 251mechanism. Reading and writing the four files under ``intervals_goal`` 252directory shows and updates the tuning parameters that described in the 253:ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same 254names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The 255tuning-applied current values of the two intervals can be read from the 256``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to 257the ``state`` file. 258 259.. _sysfs_targets: 260 261contexts/<N>/targets/ 262--------------------- 263 264In the beginning, this directory has only one file, ``nr_targets``. Writing a 265number (``N``) to the file creates the number of child directories named ``0`` 266to ``N-1``. Each directory represents each monitoring target. 267 268.. _sysfs_target: 269 270targets/<N>/ 271------------ 272 273In each target directory, two files (``pid_target`` and ``obsolete_target``) 274and one directory (``regions``) exist. 275 276If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 277be a process. You can specify the process to DAMON by writing the pid of the 278process to the ``pid_target`` file. 279 280Users can selectively remove targets in the middle of the targets array by 281writing non-zero value to ``obsolete_target`` file and committing it (writing 282``commit`` to ``state`` file). DAMON will remove the matching targets from its 283internal targets array. Users are responsible to construct target directories 284again, so that those correctly represent the changed internal targets array. 285 286 287.. _sysfs_regions: 288 289targets/<N>/regions 290------------------- 291 292In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are 293required to set the monitoring target address ranges. In case of ``vaddr`` 294operations set, it is not mandatory, but users can optionally set the initial 295monitoring region to specific address ranges. Please refer to the :ref:`design 296<damon_design_vaddr_target_regions_construction>` for more details. 297 298For such cases, users can explicitly set the initial monitoring target regions 299as they want, by writing proper values to the files under this directory. 300 301In the beginning, this directory has only one file, ``nr_regions``. Writing a 302number (``N``) to the file creates the number of child directories named ``0`` 303to ``N-1``. Each directory represents each initial monitoring target region. 304 305If ``nr_regions`` is zero when committing new DAMON parameters online (writing 306``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit 307logic ignores the target regions. In other words, the current monitoring 308results for the target are preserved. 309 310.. _sysfs_region: 311 312regions/<N>/ 313------------ 314 315In each region directory, you will find two files (``start`` and ``end``). You 316can set and get the start and end addresses of the initial monitoring target 317region by writing to and reading from the files, respectively. 318 319Each region should not overlap with others. ``end`` of directory ``N`` should 320be equal or smaller than ``start`` of directory ``N+1``. 321 322.. _sysfs_schemes: 323 324contexts/<N>/schemes/ 325--------------------- 326 327The directory for DAMON-based Operation Schemes (:ref:`DAMOS 328<damon_design_damos>`). Users can get and set the schemes by reading from and 329writing to files under this directory. 330 331In the beginning, this directory has only one file, ``nr_schemes``. Writing a 332number (``N``) to the file creates the number of child directories named ``0`` 333to ``N-1``. Each directory represents each DAMON-based operation scheme. 334 335.. _sysfs_scheme: 336 337schemes/<N>/ 338------------ 339 340In each scheme directory, eight directories (``access_pattern``, ``quotas``, 341``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``, 342``stats``, and ``tried_regions``) and three files (``action``, ``target_nid`` 343and ``apply_interval``) exist. 344 345The ``action`` file is for setting and getting the scheme's :ref:`action 346<damon_design_damos_action>`. The keywords that can be written to and read 347from the file and their meaning are same to those of the list on 348:ref:`design doc <damon_design_damos_action>`. 349 350The ``target_nid`` file is for setting the migration target node, which is 351only meaningful when the ``action`` is either ``migrate_hot`` or 352``migrate_cold``. 353 354The ``apply_interval_us`` file is for setting and getting the scheme's 355:ref:`apply_interval <damon_design_damos>` in microseconds. 356 357.. _sysfs_access_pattern: 358 359schemes/<N>/access_pattern/ 360--------------------------- 361 362The directory for the target access :ref:`pattern 363<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 364 365Under the ``access_pattern`` directory, three directories (``sz``, 366``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 367exist. You can set and get the access pattern for the given scheme by writing 368to and reading from the ``min`` and ``max`` files under ``sz``, 369``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 370and the ``max`` form a closed interval. 371 372.. _sysfs_quotas: 373 374schemes/<N>/quotas/ 375------------------- 376 377The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 378DAMON-based operation scheme. 379 380Under ``quotas`` directory, four files (``ms``, ``bytes``, 381``reset_interval_ms``, ``effective_bytes``) and two directories (``weights`` and 382``goals``) exist. 383 384You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 385``reset interval`` in milliseconds by writing the values to the three files, 386respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 387for applying the ``action`` to memory regions of the ``access_pattern``, and to 388apply the action to only up to ``bytes`` bytes of memory regions within the 389``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 390quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is 391set. 392 393The time quota is internally transformed to a size quota. Between the 394transformed size quota and user-specified size quota, smaller one is applied. 395Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the 396effective size quota is further adjusted. Reading ``effective_bytes`` returns 397the current effective size quota. The file is not updated in real time, so 398users should ask DAMON sysfs interface to update the content of the file for 399the stats by writing a special keyword, ``update_schemes_effective_quotas`` to 400the relevant ``kdamonds/<N>/state`` file. 401 402Under ``weights`` directory, three files (``sz_permil``, 403``nr_accesses_permil``, and ``age_permil``) exist. 404You can set the :ref:`prioritization weights 405<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 406in per-thousand unit by writing the values to the three files under the 407``weights`` directory. 408 409.. _sysfs_schemes_quota_goals: 410 411schemes/<N>/quotas/goals/ 412------------------------- 413 414The directory for the :ref:`automatic quota tuning goals 415<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 416scheme. 417 418In the beginning, this directory has only one file, ``nr_goals``. Writing a 419number (``N``) to the file creates the number of child directories named ``0`` 420to ``N-1``. Each directory represents each goal and current achievement. 421Among the multiple feedback, the best one is used. 422 423Each goal directory contains five files, namely ``target_metric``, 424``target_value``, ``current_value`` ``nid`` and ``path``. Users can set and 425get the five parameters for the quota auto-tuning goals that specified on the 426:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and 427reading from each of the files. Note that users should further write 428``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 429directory <sysfs_kdamond>` to pass the feedback to DAMON. 430 431.. _sysfs_watermarks: 432 433schemes/<N>/watermarks/ 434----------------------- 435 436The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 437given DAMON-based operation scheme. 438 439Under the watermarks directory, five files (``metric``, ``interval_us``, 440``high``, ``mid``, and ``low``) for setting the metric, the time interval 441between check of the metric, and the three watermarks exist. You can set and 442get the five values by writing to the files, respectively. 443 444Keywords and meanings of those that can be written to the ``metric`` file are 445as below. 446 447 - none: Ignore the watermarks 448 - free_mem_rate: System's free memory rate (per thousand) 449 450The ``interval`` should written in microseconds unit. 451 452.. _sysfs_filters: 453 454schemes/<N>/{core\_,ops\_,}filters/ 455----------------------------------- 456 457Directories for :ref:`filters <damon_design_damos_filters>` of the given 458DAMON-based operation scheme. 459 460``core_filters`` and ``ops_filters`` directories are for the filters handled by 461the DAMON core layer and operations set layer, respectively. ``filters`` 462directory can be used for installing filters regardless of their handled 463layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be 464installed before those of ``filters``. All three directories have same files. 465 466Use of ``filters`` directory can make expecting evaluation orders of given 467filters with the files under directory bit confusing. Users are hence 468recommended to use ``core_filters`` and ``ops_filters`` directories. The 469``filters`` directory could be deprecated in future. 470 471In the beginning, the directory has only one file, ``nr_filters``. Writing a 472number (``N``) to the file creates the number of child directories named ``0`` 473to ``N-1``. Each directory represents each filter. The filters are evaluated 474in the numeric order. 475 476Each filter directory contains nine files, namely ``type``, ``matching``, 477``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max`` 478and ``target_idx``. To ``type`` file, you can write the type of the filter. 479Refer to :ref:`the design doc <damon_design_damos_filters>` for available type 480names, their meaning and on what layer those are handled. 481 482For ``memcg`` type, you can specify the memory cgroup of the interest by 483writing the path of the memory cgroup from the cgroups mount point to 484``memcg_path`` file. For ``addr`` type, you can specify the start and end 485address of the range (open-ended interval) to ``addr_start`` and ``addr_end`` 486files, respectively. For ``hugepage_size`` type, you can specify the minimum 487and maximum size of the range (closed interval) to ``min`` and ``max`` files, 488respectively. For ``target`` type, you can specify the index of the target 489between the list of the DAMON context's monitoring targets list to 490``target_idx`` file. 491 492You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter 493is for memory that matches the ``type``. You can write ``Y`` or ``N`` to 494``allow`` file to specify if applying the action to the memory that satisfies 495the ``type`` and ``matching`` should be allowed or not. 496 497For example, below restricts a DAMOS action to be applied to only non-anonymous 498pages of all memory cgroups except ``/having_care_already``.:: 499 500 # cd ops_filters/0/ 501 # echo 2 > nr_filters 502 # # disallow anonymous pages 503 echo anon > 0/type 504 echo Y > 0/matching 505 echo N > 0/allow 506 # # further filter out all cgroups except one at '/having_care_already' 507 echo memcg > 1/type 508 echo /having_care_already > 1/memcg_path 509 echo Y > 1/matching 510 echo N > 1/allow 511 512Refer to the :ref:`DAMOS filters design documentation 513<damon_design_damos_filters>` for more details including how multiple filters 514of different ``allow`` works, when each of the filters are supported, and 515differences on stats. 516 517.. _damon_sysfs_dests: 518 519schemes/<N>/dests/ 520------------------ 521 522Directory for specifying the destinations of given DAMON-based operation 523scheme's action. This directory is ignored if the action of the given scheme 524is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}`` 525actions are supporting multiple destinations. 526 527In the beginning, the directory has only one file, ``nr_dests``. Writing a 528number (``N``) to the file creates the number of child directories named ``0`` 529to ``N-1``. Each directory represents each action destination. 530 531Each destination directory contains two files, namely ``id`` and ``weight``. 532Users can write and read the identifier of the destination to ``id`` file. 533For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node 534id should be written to ``id`` file. Users can write and read the weight of 535the destination among the given destinations to the ``weight`` file. The 536weight can be an arbitrary integer. When DAMOS apply the action to each entity 537of the memory region, it will select the destination of the action based on the 538relative weights of the destinations. 539 540.. _sysfs_schemes_stats: 541 542schemes/<N>/stats/ 543------------------ 544 545DAMON counts statistics for each scheme. This statistics can be used for 546online analysis or tuning of the schemes. Refer to :ref:`design doc 547<damon_design_damos_stat>` for more details about the stats. 548 549The statistics can be retrieved by reading the files under ``stats`` directory 550(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, 551``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and 552``max_nr_snapshots``), respectively. 553 554The files are not updated in real time by default. Users should ask DAMON 555sysfs interface to periodically update those using ``refresh_ms``, or do a one 556time update by writing a special keyword, ``update_schemes_stats`` to the 557relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory 558<sysfs_kdamond>` for more details. 559 560.. _sysfs_schemes_tried_regions: 561 562schemes/<N>/tried_regions/ 563-------------------------- 564 565This directory initially has one file, ``total_bytes``. 566 567When a special keyword, ``update_schemes_tried_regions``, is written to the 568relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 569that reading it returns the total size of the scheme tried regions, and creates 570directories named integer starting from ``0`` under this directory. Each 571directory contains files exposing detailed information about each of the memory 572region that the corresponding scheme's ``action`` has tried to be applied under 573this directory, during next :ref:`apply interval <damon_design_damos>` of the 574corresponding scheme. The information includes address range, ``nr_accesses``, 575and ``age`` of the region. 576 577Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 578file will only update the ``total_bytes`` file, and will not create the 579subdirectories. 580 581The directories will be removed when another special keyword, 582``clear_schemes_tried_regions``, is written to the relevant 583``kdamonds/<N>/state`` file. 584 585The expected usage of this directory is investigations of schemes' behaviors, 586and query-like efficient data access monitoring results retrievals. For the 587latter use case, in particular, users can set the ``action`` as ``stat`` and 588set the ``access pattern`` as their interested pattern that they want to query. 589 590.. _sysfs_schemes_tried_region: 591 592tried_regions/<N>/ 593------------------ 594 595In each region directory, you will find five files (``start``, ``end``, 596``nr_accesses``, ``age``, and ``sz_filter_passed``). Reading the files will 597show the properties of the region that corresponding DAMON-based operation 598scheme ``action`` has tried to be applied. 599 600Example 601~~~~~~~ 602 603Below commands applies a scheme saying "If a memory region of size in [4KiB, 6048KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 605interval in [10, 20], page out the region. For the paging out, use only up to 60610ms per second, and also don't page out more than 1GiB per second. Under the 607limitation, page out memory regions having longer age first. Also, check the 608free memory rate of the system every 5 seconds, start the monitoring and paging 609out when the free memory rate becomes lower than 50%, but stop it if the free 610memory rate becomes larger than 60%, or lower than 30%". :: 611 612 # cd <sysfs>/kernel/mm/damon/admin 613 # # populate directories 614 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 615 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 616 # cd kdamonds/0/contexts/0/schemes/0 617 # # set the basic access pattern and the action 618 # echo 4096 > access_pattern/sz/min 619 # echo 8192 > access_pattern/sz/max 620 # echo 0 > access_pattern/nr_accesses/min 621 # echo 5 > access_pattern/nr_accesses/max 622 # echo 10 > access_pattern/age/min 623 # echo 20 > access_pattern/age/max 624 # echo pageout > action 625 # # set quotas 626 # echo 10 > quotas/ms 627 # echo $((1024*1024*1024)) > quotas/bytes 628 # echo 1000 > quotas/reset_interval_ms 629 # # set watermark 630 # echo free_mem_rate > watermarks/metric 631 # echo 5000000 > watermarks/interval_us 632 # echo 600 > watermarks/high 633 # echo 500 > watermarks/mid 634 # echo 300 > watermarks/low 635 636Please note that it's highly recommended to use user space tools like `damo 637<https://github.com/damonitor/damo>`_ rather than manually reading and writing 638the files as above. Above is only for an example. 639 640.. _tracepoint: 641 642Tracepoints for Monitoring Results 643================================== 644 645Users can get the monitoring results via the :ref:`tried_regions 646<sysfs_schemes_tried_regions>`. The interface is useful for getting a 647snapshot, but it could be inefficient for fully recording all the monitoring 648results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 649and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 650provides the whole monitoring results, while ``damon:damos_before_apply`` 651provides the monitoring results for regions that each DAMON-based Operation 652Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 653``damon:damos_before_apply`` is more useful for recording internal behavior of 654DAMOS, or DAMOS target access 655:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 656monitoring results recording. 657 658While the monitoring is turned on, you could record the tracepoint events and 659show results using tracepoint supporting tools like ``perf``. For example:: 660 661 # echo on > kdamonds/0/state 662 # perf record -e damon:damon_aggregated & 663 # sleep 5 664 # kill 9 $(pidof perf) 665 # echo off > kdamonds/0/state 666 # perf script 667 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 668 [...] 669 670Each line of the perf script output represents each monitoring region. The 671first five fields are as usual other tracepoint outputs. The sixth field 672(``target_id=X``) shows the ide of the monitoring target of the region. The 673seventh field (``nr_regions=X``) shows the total number of monitoring regions 674for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 675(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 676``nr_accesses`` of the region (refer to 677:ref:`design <damon_design_region_based_sampling>` for more details of the 678counter). Finally the tenth field (``X``) shows the ``age`` of the region 679(refer to :ref:`design <damon_design_age_tracking>` for more details of the 680counter). 681 682If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would 683be somewhat like below:: 684 685 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 686 [...] 687 688Each line of the output represents each monitoring region that each DAMON-based 689Operation Scheme was about to be applied at the traced time. The first five 690fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 691of the scheme in the list of the contexts of the context's kdamond, the index 692of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 693addition to the output of ``damon_aggregated`` tracepoint. 694