1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *Special-purpose DAMON modules.* 10 :ref:`This <damon_modules_special_purpose>` is for people who are building, 11 distributing, and/or administrating the kernel with special-purpose DAMON 12 usages. Using this, users can use DAMON's major features for the given 13 purposes in build, boot, or runtime in simple ways. 14- *DAMON user space tool.* 15 `This <https://github.com/damonitor/damo>`_ is for privileged people such as 16 system administrators who want a just-working human-friendly interface. 17 Using this, users can use the DAMON’s major features in a human-friendly way. 18 It may not be highly tuned for special cases, though. For more detail, 19 please refer to its `usage document 20 <https://github.com/damonitor/damo/blob/next/USAGE.md>`_. 21- *sysfs interface.* 22 :ref:`This <sysfs_interface>` is for privileged user space programmers who 23 want more optimized use of DAMON. Using this, users can use DAMON’s major 24 features by reading from and writing to special sysfs files. Therefore, 25 you can write and use your personalized DAMON sysfs wrapper programs that 26 reads/writes the sysfs files instead of you. The `DAMON user space tool 27 <https://github.com/damonitor/damo>`_ is one example of such programs. 28- *Kernel Space Programming Interface.* 29 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </mm/damon/api>`. 34 35.. _sysfs_interface: 36 37sysfs Interface 38=============== 39 40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 41creates multiple directories and files under its sysfs directory, 42``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 43from the files under the directory. 44 45For a short example, users can monitor the virtual address space of a given 46workload as below. :: 47 48 # cd /sys/kernel/mm/damon/admin/ 49 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 50 # echo vaddr > kdamonds/0/contexts/0/operations 51 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 52 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 53 # echo on > kdamonds/0/state 54 55Files Hierarchy 56--------------- 57 58The files hierarchy of DAMON sysfs interface is shown below. In the below 59figure, parents-children relations are represented with indentations, each 60directory is having ``/`` suffix, and files in each directory are separated by 61comma (","). 62 63.. parsed-literal:: 64 65 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 66 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 67 │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms 68 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 69 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit, 70 │ │ │ │ pause 71 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 72 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 73 │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us 74 │ │ │ │ │ │ nr_regions/min,max 75 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 76 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target 77 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 78 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 79 │ │ │ │ │ │ │ │ ... 80 │ │ │ │ │ │ ... 81 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 82 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us 83 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 84 │ │ │ │ │ │ │ │ sz/min,max 85 │ │ │ │ │ │ │ │ nr_accesses/min,max 86 │ │ │ │ │ │ │ │ age/min,max 87 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms, 88 │ │ │ │ │ │ │ effective_bytes,goal_tuner, 89 │ │ │ │ │ │ │ fail_charge_num,fail_charge_denom 90 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 91 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 92 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path 93 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 94 │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters 95 │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max 96 │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests 97 │ │ │ │ │ │ │ │ 0/id,weight 98 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots 99 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 100 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed 101 │ │ │ │ │ │ │ │ ... 102 │ │ │ │ │ │ ... 103 │ │ │ │ ... 104 │ │ ... 105 106.. _sysfs_root: 107 108Root 109---- 110 111The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 112has one directory named ``admin``. The directory contains the files for 113privileged user space programs' control of DAMON. User space tools or daemons 114having the root permission could use this directory. 115 116.. _sysfs_kdamonds: 117 118kdamonds/ 119--------- 120 121Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 122controlling the kdamonds (refer to 123:ref:`design <damon_design_execution_model_and_data_structures>` for more 124details) exists. In the beginning, this directory has only one file, 125``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 126child directories named ``0`` to ``N-1``. Each directory represents each 127kdamond. 128 129.. _sysfs_kdamond: 130 131kdamonds/<N>/ 132------------- 133 134In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``) 135and one directory (``contexts``) exist. 136 137Reading ``state`` returns ``on`` if the kdamond is currently running, or 138``off`` if it is not running. 139 140Users can write below commands for the kdamond to the ``state`` file. 141 142- ``on``: Start running. 143- ``off``: Stop running. 144- ``commit``: Read the user inputs in the sysfs files except ``state`` file 145 again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be 146 ignored if no target region is specified. 147- ``update_tuned_intervals``: Update the contents of ``sample_us`` and 148 ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling 149 interval`` and ``aggregation interval`` for the files. Please refer to 150 :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>` 151 for more details. 152- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 153 :ref:`quota goals <sysfs_schemes_quota_goals>`. 154- ``update_schemes_stats``: Update the contents of stats files for each 155 DAMON-based operation scheme of the kdamond. For details of the stats, 156 please refer to :ref:`stats section <sysfs_schemes_stats>`. 157- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 158 action tried regions directory for each DAMON-based operation scheme of the 159 kdamond. For details of the DAMON-based operation scheme action tried 160 regions directory, please refer to 161 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 162- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 163 files. 164- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 165 action tried regions directory for each DAMON-based operation scheme of the 166 kdamond. 167- ``update_schemes_effective_quotas``: Update the contents of 168 ``effective_bytes`` files for each DAMON-based operation scheme of the 169 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`. 170 171If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 172 173Users can ask the kernel to periodically update files showing auto-tuned 174parameters and DAMOS stats instead of manually writing 175``update_tuned_intervals`` like keywords to ``state`` file. For this, users 176should write the desired update time interval in milliseconds to ``refresh_ms`` 177file. If the interval is zero, the periodic update is disabled. Reading the 178file shows currently set time interval. 179 180``contexts`` directory contains files for controlling the monitoring contexts 181that this kdamond will execute. 182 183.. _sysfs_contexts: 184 185kdamonds/<N>/contexts/ 186---------------------- 187 188In the beginning, this directory has only one file, ``nr_contexts``. Writing a 189number (``N``) to the file creates the number of child directories named as 190``0`` to ``N-1``. Each directory represents each monitoring context (refer to 191:ref:`design <damon_design_execution_model_and_data_structures>` for more 192details). At the moment, only one context per kdamond is supported, so only 193``0`` or ``1`` can be written to the file. 194 195.. _sysfs_context: 196 197contexts/<N>/ 198------------- 199 200In each context directory, four files (``avail_operations``, ``operations``, 201``addr_unit`` and ``pause``) and three directories (``monitoring_attrs``, 202``targets``, and ``schemes``) exist. 203 204DAMON supports multiple types of :ref:`monitoring operations 205<damon_design_configurable_operations_set>`, including those for virtual address 206space and the physical address space. You can get the list of available 207monitoring operations set on the currently running kernel by reading 208``avail_operations`` file. Based on the kernel configuration, the file will 209list different available operation sets. Please refer to the :ref:`design 210<damon_operations_set>` for the list of all available operation sets and their 211brief explanations. 212 213You can set and get what type of monitoring operations DAMON will use for the 214context by writing one of the keywords listed in ``avail_operations`` file and 215reading from the ``operations`` file. 216 217``addr_unit`` file is for setting and getting the :ref:`address unit 218<damon_design_addr_unit>` parameter of the operations set. 219 220``pause`` file is for setting and getting the :ref:`pause request 221<damon_design_execution_model_and_data_structures>` parameter of the context. 222 223.. _sysfs_monitoring_attrs: 224 225contexts/<N>/monitoring_attrs/ 226------------------------------ 227 228Files for specifying attributes of the monitoring including required quality 229and efficiency of the monitoring are in ``monitoring_attrs`` directory. 230Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 231directory. 232 233Under ``intervals`` directory, three files for DAMON's sampling interval 234(``sample_us``), aggregation interval (``aggr_us``), and update interval 235(``update_us``) exist. You can set and get the values in micro-seconds by 236writing to and reading from the files. 237 238Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 239of DAMON's monitoring regions (``min`` and ``max``, respectively), which 240controls the monitoring overhead, exist. You can set and get the values by 241writing to and rading from the files. 242 243For more details about the intervals and monitoring regions range, please refer 244to the Design document (:doc:`/mm/damon/design`). 245 246.. _damon_usage_sysfs_monitoring_intervals_goal: 247 248contexts/<N>/monitoring_attrs/intervals/intervals_goal/ 249------------------------------------------------------- 250 251Under the ``intervals`` directory, one directory for automated tuning of 252``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists. 253Under the directory, four files for the auto-tuning control, namely 254``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist. 255Please refer to the :ref:`design document of the feature 256<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning 257mechanism. Reading and writing the four files under ``intervals_goal`` 258directory shows and updates the tuning parameters that described in the 259:ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same 260names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The 261tuning-applied current values of the two intervals can be read from the 262``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to 263the ``state`` file. 264 265.. _sysfs_targets: 266 267contexts/<N>/targets/ 268--------------------- 269 270In the beginning, this directory has only one file, ``nr_targets``. Writing a 271number (``N``) to the file creates the number of child directories named ``0`` 272to ``N-1``. Each directory represents each monitoring target. 273 274.. _sysfs_target: 275 276targets/<N>/ 277------------ 278 279In each target directory, two files (``pid_target`` and ``obsolete_target``) 280and one directory (``regions``) exist. 281 282If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 283be a process. You can specify the process to DAMON by writing the pid of the 284process to the ``pid_target`` file. 285 286Users can selectively remove targets in the middle of the targets array by 287writing non-zero value to ``obsolete_target`` file and committing it (writing 288``commit`` to ``state`` file). DAMON will remove the matching targets from its 289internal targets array. Users are responsible to construct target directories 290again, so that those correctly represent the changed internal targets array. 291 292 293.. _sysfs_regions: 294 295targets/<N>/regions 296------------------- 297 298In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are 299required to set the monitoring target address ranges. In case of ``vaddr`` 300operations set, it is not mandatory, but users can optionally set the initial 301monitoring region to specific address ranges. Please refer to the :ref:`design 302<damon_design_vaddr_target_regions_construction>` for more details. 303 304For such cases, users can explicitly set the initial monitoring target regions 305as they want, by writing proper values to the files under this directory. 306 307In the beginning, this directory has only one file, ``nr_regions``. Writing a 308number (``N``) to the file creates the number of child directories named ``0`` 309to ``N-1``. Each directory represents each initial monitoring target region. 310 311If ``nr_regions`` is zero when committing new DAMON parameters online (writing 312``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit 313logic ignores the target regions. In other words, the current monitoring 314results for the target are preserved. 315 316.. _sysfs_region: 317 318regions/<N>/ 319------------ 320 321In each region directory, you will find two files (``start`` and ``end``). You 322can set and get the start and end addresses of the initial monitoring target 323region by writing to and reading from the files, respectively. 324 325Each region should not overlap with others. ``end`` of directory ``N`` should 326be equal or smaller than ``start`` of directory ``N+1``. 327 328.. _sysfs_schemes: 329 330contexts/<N>/schemes/ 331--------------------- 332 333The directory for DAMON-based Operation Schemes (:ref:`DAMOS 334<damon_design_damos>`). Users can get and set the schemes by reading from and 335writing to files under this directory. 336 337In the beginning, this directory has only one file, ``nr_schemes``. Writing a 338number (``N``) to the file creates the number of child directories named ``0`` 339to ``N-1``. Each directory represents each DAMON-based operation scheme. 340 341.. _sysfs_scheme: 342 343schemes/<N>/ 344------------ 345 346In each scheme directory, eight directories (``access_pattern``, ``quotas``, 347``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``, 348``stats``, and ``tried_regions``) and three files (``action``, ``target_nid`` 349and ``apply_interval``) exist. 350 351The ``action`` file is for setting and getting the scheme's :ref:`action 352<damon_design_damos_action>`. The keywords that can be written to and read 353from the file and their meaning are same to those of the list on 354:ref:`design doc <damon_design_damos_action>`. 355 356The ``target_nid`` file is for setting the migration target node, which is 357only meaningful when the ``action`` is either ``migrate_hot`` or 358``migrate_cold``. 359 360The ``apply_interval_us`` file is for setting and getting the scheme's 361:ref:`apply_interval <damon_design_damos>` in microseconds. 362 363.. _sysfs_access_pattern: 364 365schemes/<N>/access_pattern/ 366--------------------------- 367 368The directory for the target access :ref:`pattern 369<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 370 371Under the ``access_pattern`` directory, three directories (``sz``, 372``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 373exist. You can set and get the access pattern for the given scheme by writing 374to and reading from the ``min`` and ``max`` files under ``sz``, 375``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 376and the ``max`` form a closed interval. 377 378.. _sysfs_quotas: 379 380schemes/<N>/quotas/ 381------------------- 382 383The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 384DAMON-based operation scheme. 385 386Under ``quotas`` directory, seven files (``ms``, ``bytes``, 387``reset_interval_ms``, ``effective_bytes``, ``goal_tuner``, ``fail_charge_num`` 388and ``fail_charge_denom``) and two directories (``weights`` and ``goals``) 389exist. 390 391You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 392``reset interval`` in milliseconds by writing the values to the three files, 393respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 394for applying the ``action`` to memory regions of the ``access_pattern``, and to 395apply the action to only up to ``bytes`` bytes of memory regions within the 396``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 397quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is 398set. 399 400You can set the goal-based effective quota auto-tuning algorithm to use, by 401writing the algorithm name to ``goal_tuner`` file. Reading the file returns 402the currently selected tuner algorithm. Refer to the design documentation of 403:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for 404the background design of the feature and the name of the selectable algorithms. 405Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals 406setup. 407 408You can set the action-failed memory quota charging ratio by writing the 409numerator and the denominator for the ratio to ``fail_charge_num`` and 410``fail_charge_denom`` files, respectively. Reading those files will return the 411current set values. Refer to :ref:`design 412<damon_design_damos_quotas_failed_memory_charging_ratio>` for more details of 413the ratio feature. 414 415The time quota is internally transformed to a size quota. Between the 416transformed size quota and user-specified size quota, smaller one is applied. 417Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the 418effective size quota is further adjusted. Reading ``effective_bytes`` returns 419the current effective size quota. The file is not updated in real time, so 420users should ask DAMON sysfs interface to update the content of the file for 421the stats by writing a special keyword, ``update_schemes_effective_quotas`` to 422the relevant ``kdamonds/<N>/state`` file. 423 424Under ``weights`` directory, three files (``sz_permil``, 425``nr_accesses_permil``, and ``age_permil``) exist. 426You can set the :ref:`prioritization weights 427<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 428in per-thousand unit by writing the values to the three files under the 429``weights`` directory. 430 431.. _sysfs_schemes_quota_goals: 432 433schemes/<N>/quotas/goals/ 434------------------------- 435 436The directory for the :ref:`automatic quota tuning goals 437<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 438scheme. 439 440In the beginning, this directory has only one file, ``nr_goals``. Writing a 441number (``N``) to the file creates the number of child directories named ``0`` 442to ``N-1``. Each directory represents each goal and current achievement. 443Among the multiple feedback, the best one is used. 444 445Each goal directory contains five files, namely ``target_metric``, 446``target_value``, ``current_value`` ``nid`` and ``path``. Users can set and 447get the five parameters for the quota auto-tuning goals that specified on the 448:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and 449reading from each of the files. Note that users should further write 450``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 451directory <sysfs_kdamond>` to pass the feedback to DAMON. 452 453.. _sysfs_watermarks: 454 455schemes/<N>/watermarks/ 456----------------------- 457 458The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 459given DAMON-based operation scheme. 460 461Under the watermarks directory, five files (``metric``, ``interval_us``, 462``high``, ``mid``, and ``low``) for setting the metric, the time interval 463between check of the metric, and the three watermarks exist. You can set and 464get the five values by writing to the files, respectively. 465 466Keywords and meanings of those that can be written to the ``metric`` file are 467as below. 468 469 - none: Ignore the watermarks 470 - free_mem_rate: System's free memory rate (per thousand) 471 472The ``interval`` should written in microseconds unit. 473 474.. _sysfs_filters: 475 476schemes/<N>/{core\_,ops\_,}filters/ 477----------------------------------- 478 479Directories for :ref:`filters <damon_design_damos_filters>` of the given 480DAMON-based operation scheme. 481 482``core_filters`` and ``ops_filters`` directories are for the filters handled by 483the DAMON core layer and operations set layer, respectively. ``filters`` 484directory can be used for installing filters regardless of their handled 485layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be 486installed before those of ``filters``. All three directories have same files. 487 488Use of ``filters`` directory can make filters evaluation orders confusing to 489expect. For this reason, ``filters`` directory is deprecated. It is still 490functioning, but is scheduled for removal in the near future. Users should use 491``core_filters`` and ``ops_filters`` directories instead. 492 493In the beginning, the directory has only one file, ``nr_filters``. Writing a 494number (``N``) to the file creates the number of child directories named ``0`` 495to ``N-1``. Each directory represents each filter. The filters are evaluated 496in the numeric order. 497 498Each filter directory contains nine files, namely ``type``, ``matching``, 499``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max`` 500and ``target_idx``. To ``type`` file, you can write the type of the filter. 501Refer to :ref:`the design doc <damon_design_damos_filters>` for available type 502names, their meaning and on what layer those are handled. 503 504For ``memcg`` type, you can specify the memory cgroup of the interest by 505writing the path of the memory cgroup from the cgroups mount point to 506``memcg_path`` file. For ``addr`` type, you can specify the start and end 507address of the range (open-ended interval) to ``addr_start`` and ``addr_end`` 508files, respectively. For ``hugepage_size`` type, you can specify the minimum 509and maximum size of the range (closed interval) to ``min`` and ``max`` files, 510respectively. For ``target`` type, you can specify the index of the target 511between the list of the DAMON context's monitoring targets list to 512``target_idx`` file. 513 514You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter 515is for memory that matches the ``type``. You can write ``Y`` or ``N`` to 516``allow`` file to specify if applying the action to the memory that satisfies 517the ``type`` and ``matching`` should be allowed or not. 518 519For example, below restricts a DAMOS action to be applied to only non-anonymous 520pages of all memory cgroups except ``/having_care_already``.:: 521 522 # cd ops_filters/0/ 523 # echo 2 > nr_filters 524 # # disallow anonymous pages 525 echo anon > 0/type 526 echo Y > 0/matching 527 echo N > 0/allow 528 # # further filter out all cgroups except one at '/having_care_already' 529 echo memcg > 1/type 530 echo /having_care_already > 1/memcg_path 531 echo Y > 1/matching 532 echo N > 1/allow 533 534Refer to the :ref:`DAMOS filters design documentation 535<damon_design_damos_filters>` for more details including how multiple filters 536of different ``allow`` works, when each of the filters are supported, and 537differences on stats. 538 539.. _damon_sysfs_dests: 540 541schemes/<N>/dests/ 542------------------ 543 544Directory for specifying the destinations of given DAMON-based operation 545scheme's action. This directory is ignored if the action of the given scheme 546is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}`` 547actions are supporting multiple destinations. 548 549In the beginning, the directory has only one file, ``nr_dests``. Writing a 550number (``N``) to the file creates the number of child directories named ``0`` 551to ``N-1``. Each directory represents each action destination. 552 553Each destination directory contains two files, namely ``id`` and ``weight``. 554Users can write and read the identifier of the destination to ``id`` file. 555For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node 556id should be written to ``id`` file. Users can write and read the weight of 557the destination among the given destinations to the ``weight`` file. The 558weight can be an arbitrary integer. When DAMOS apply the action to each entity 559of the memory region, it will select the destination of the action based on the 560relative weights of the destinations. 561 562.. _sysfs_schemes_stats: 563 564schemes/<N>/stats/ 565------------------ 566 567DAMON counts statistics for each scheme. This statistics can be used for 568online analysis or tuning of the schemes. Refer to :ref:`design doc 569<damon_design_damos_stat>` for more details about the stats. 570 571The statistics can be retrieved by reading the files under ``stats`` directory 572(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, 573``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and 574``max_nr_snapshots``), respectively. 575 576The files are not updated in real time by default. Users should ask DAMON 577sysfs interface to periodically update those using ``refresh_ms``, or do a one 578time update by writing a special keyword, ``update_schemes_stats`` to the 579relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory 580<sysfs_kdamond>` for more details. 581 582.. _sysfs_schemes_tried_regions: 583 584schemes/<N>/tried_regions/ 585-------------------------- 586 587This directory initially has one file, ``total_bytes``. 588 589When a special keyword, ``update_schemes_tried_regions``, is written to the 590relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 591that reading it returns the total size of the scheme tried regions, and creates 592directories named integer starting from ``0`` under this directory. Each 593directory contains files exposing detailed information about each of the memory 594region that the corresponding scheme's ``action`` has tried to be applied under 595this directory, during next :ref:`apply interval <damon_design_damos>` of the 596corresponding scheme. The information includes address range, ``nr_accesses``, 597and ``age`` of the region. 598 599Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 600file will only update the ``total_bytes`` file, and will not create the 601subdirectories. 602 603The directories will be removed when another special keyword, 604``clear_schemes_tried_regions``, is written to the relevant 605``kdamonds/<N>/state`` file. 606 607The expected usage of this directory is investigations of schemes' behaviors, 608and query-like efficient data access monitoring results retrievals. For the 609latter use case, in particular, users can set the ``action`` as ``stat`` and 610set the ``access pattern`` as their interested pattern that they want to query. 611 612.. _sysfs_schemes_tried_region: 613 614tried_regions/<N>/ 615------------------ 616 617In each region directory, you will find five files (``start``, ``end``, 618``nr_accesses``, ``age``, and ``sz_filter_passed``). Reading the files will 619show the properties of the region that corresponding DAMON-based operation 620scheme ``action`` has tried to be applied. 621 622Example 623~~~~~~~ 624 625Below commands applies a scheme saying "If a memory region of size in [4KiB, 6268KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 627interval in [10, 20], page out the region. For the paging out, use only up to 62810ms per second, and also don't page out more than 1GiB per second. Under the 629limitation, page out memory regions having longer age first. Also, check the 630free memory rate of the system every 5 seconds, start the monitoring and paging 631out when the free memory rate becomes lower than 50%, but stop it if the free 632memory rate becomes larger than 60%, or lower than 30%". :: 633 634 # cd <sysfs>/kernel/mm/damon/admin 635 # # populate directories 636 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 637 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 638 # cd kdamonds/0/contexts/0/schemes/0 639 # # set the basic access pattern and the action 640 # echo 4096 > access_pattern/sz/min 641 # echo 8192 > access_pattern/sz/max 642 # echo 0 > access_pattern/nr_accesses/min 643 # echo 5 > access_pattern/nr_accesses/max 644 # echo 10 > access_pattern/age/min 645 # echo 20 > access_pattern/age/max 646 # echo pageout > action 647 # # set quotas 648 # echo 10 > quotas/ms 649 # echo $((1024*1024*1024)) > quotas/bytes 650 # echo 1000 > quotas/reset_interval_ms 651 # # set watermark 652 # echo free_mem_rate > watermarks/metric 653 # echo 5000000 > watermarks/interval_us 654 # echo 600 > watermarks/high 655 # echo 500 > watermarks/mid 656 # echo 300 > watermarks/low 657 658Please note that it's highly recommended to use user space tools like `damo 659<https://github.com/damonitor/damo>`_ rather than manually reading and writing 660the files as above. Above is only for an example. 661 662.. _tracepoint: 663 664Tracepoints for Monitoring Results 665================================== 666 667Users can get the monitoring results via the :ref:`tried_regions 668<sysfs_schemes_tried_regions>`. The interface is useful for getting a 669snapshot, but it could be inefficient for fully recording all the monitoring 670results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 671and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 672provides the whole monitoring results, while ``damon:damos_before_apply`` 673provides the monitoring results for regions that each DAMON-based Operation 674Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 675``damon:damos_before_apply`` is more useful for recording internal behavior of 676DAMOS, or DAMOS target access 677:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 678monitoring results recording. 679 680While the monitoring is turned on, you could record the tracepoint events and 681show results using tracepoint supporting tools like ``perf``. For example:: 682 683 # echo on > kdamonds/0/state 684 # perf record -e damon:damon_aggregated & 685 # sleep 5 686 # kill 9 $(pidof perf) 687 # echo off > kdamonds/0/state 688 # perf script 689 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 690 [...] 691 692Each line of the perf script output represents each monitoring region. The 693first five fields are as usual other tracepoint outputs. The sixth field 694(``target_id=X``) shows the ide of the monitoring target of the region. The 695seventh field (``nr_regions=X``) shows the total number of monitoring regions 696for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 697(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 698``nr_accesses`` of the region (refer to 699:ref:`design <damon_design_region_based_sampling>` for more details of the 700counter). Finally the tenth field (``X``) shows the ``age`` of the region 701(refer to :ref:`design <damon_design_age_tracking>` for more details of the 702counter). 703 704If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would 705be somewhat like below:: 706 707 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 708 [...] 709 710Each line of the output represents each monitoring region that each DAMON-based 711Operation Scheme was about to be applied at the traced time. The first five 712fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 713of the scheme in the list of the contexts of the context's kdamond, the index 714of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 715addition to the output of ``damon_aggregated`` tracepoint. 716