1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *Special-purpose DAMON modules.* 10 :ref:`This <damon_modules_special_purpose>` is for people who are building, 11 distributing, and/or administrating the kernel with special-purpose DAMON 12 usages. Using this, users can use DAMON's major features for the given 13 purposes in build, boot, or runtime in simple ways. 14- *DAMON user space tool.* 15 `This <https://github.com/damonitor/damo>`_ is for privileged people such as 16 system administrators who want a just-working human-friendly interface. 17 Using this, users can use the DAMON’s major features in a human-friendly way. 18 It may not be highly tuned for special cases, though. For more detail, 19 please refer to its `usage document 20 <https://github.com/damonitor/damo/blob/next/USAGE.md>`_. 21- *sysfs interface.* 22 :ref:`This <sysfs_interface>` is for privileged user space programmers who 23 want more optimized use of DAMON. Using this, users can use DAMON’s major 24 features by reading from and writing to special sysfs files. Therefore, 25 you can write and use your personalized DAMON sysfs wrapper programs that 26 reads/writes the sysfs files instead of you. The `DAMON user space tool 27 <https://github.com/damonitor/damo>`_ is one example of such programs. 28- *Kernel Space Programming Interface.* 29 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </mm/damon/api>`. 34 35.. _sysfs_interface: 36 37sysfs Interface 38=============== 39 40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 41creates multiple directories and files under its sysfs directory, 42``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 43from the files under the directory. 44 45For a short example, users can monitor the virtual address space of a given 46workload as below. :: 47 48 # cd /sys/kernel/mm/damon/admin/ 49 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 50 # echo vaddr > kdamonds/0/contexts/0/operations 51 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 52 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 53 # echo on > kdamonds/0/state 54 55Files Hierarchy 56--------------- 57 58The files hierarchy of DAMON sysfs interface is shown below. In the below 59figure, parents-children relations are represented with indentations, each 60directory is having ``/`` suffix, and files in each directory are separated by 61comma (","). 62 63.. parsed-literal:: 64 65 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin 66 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds 67 │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms 68 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts 69 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit 70 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/ 71 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 72 │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us 73 │ │ │ │ │ │ nr_regions/min,max 74 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets 75 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target 76 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions 77 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end 78 │ │ │ │ │ │ │ │ ... 79 │ │ │ │ │ │ ... 80 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes 81 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us 82 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/ 83 │ │ │ │ │ │ │ │ sz/min,max 84 │ │ │ │ │ │ │ │ nr_accesses/min,max 85 │ │ │ │ │ │ │ │ age/min,max 86 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms, 87 │ │ │ │ │ │ │ effective_bytes,goal_tuner, 88 │ │ │ │ │ │ │ fail_charge_num,fail_charge_denom 89 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 90 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals 91 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path 92 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low 93 │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters 94 │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max 95 │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests 96 │ │ │ │ │ │ │ │ 0/id,weight 97 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots 98 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes 99 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed 100 │ │ │ │ │ │ │ │ ... 101 │ │ │ │ │ │ ... 102 │ │ │ │ ... 103 │ │ ... 104 105.. _sysfs_root: 106 107Root 108---- 109 110The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 111has one directory named ``admin``. The directory contains the files for 112privileged user space programs' control of DAMON. User space tools or daemons 113having the root permission could use this directory. 114 115.. _sysfs_kdamonds: 116 117kdamonds/ 118--------- 119 120Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 121controlling the kdamonds (refer to 122:ref:`design <damon_design_execution_model_and_data_structures>` for more 123details) exists. In the beginning, this directory has only one file, 124``nr_kdamonds``. Writing a number (``N``) to the file creates the number of 125child directories named ``0`` to ``N-1``. Each directory represents each 126kdamond. 127 128.. _sysfs_kdamond: 129 130kdamonds/<N>/ 131------------- 132 133In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``) 134and one directory (``contexts``) exist. 135 136Reading ``state`` returns ``on`` if the kdamond is currently running, or 137``off`` if it is not running. 138 139Users can write below commands for the kdamond to the ``state`` file. 140 141- ``on``: Start running. 142- ``off``: Stop running. 143- ``commit``: Read the user inputs in the sysfs files except ``state`` file 144 again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be 145 ignored if no target region is specified. 146- ``update_tuned_intervals``: Update the contents of ``sample_us`` and 147 ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling 148 interval`` and ``aggregation interval`` for the files. Please refer to 149 :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>` 150 for more details. 151- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes' 152 :ref:`quota goals <sysfs_schemes_quota_goals>`. 153- ``update_schemes_stats``: Update the contents of stats files for each 154 DAMON-based operation scheme of the kdamond. For details of the stats, 155 please refer to :ref:`stats section <sysfs_schemes_stats>`. 156- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme 157 action tried regions directory for each DAMON-based operation scheme of the 158 kdamond. For details of the DAMON-based operation scheme action tried 159 regions directory, please refer to 160 :ref:`tried_regions section <sysfs_schemes_tried_regions>`. 161- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes`` 162 files. 163- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme 164 action tried regions directory for each DAMON-based operation scheme of the 165 kdamond. 166- ``update_schemes_effective_quotas``: Update the contents of 167 ``effective_bytes`` files for each DAMON-based operation scheme of the 168 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`. 169 170If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 171 172Users can ask the kernel to periodically update files showing auto-tuned 173parameters and DAMOS stats instead of manually writing 174``update_tuned_intervals`` like keywords to ``state`` file. For this, users 175should write the desired update time interval in milliseconds to ``refresh_ms`` 176file. If the interval is zero, the periodic update is disabled. Reading the 177file shows currently set time interval. 178 179``contexts`` directory contains files for controlling the monitoring contexts 180that this kdamond will execute. 181 182.. _sysfs_contexts: 183 184kdamonds/<N>/contexts/ 185---------------------- 186 187In the beginning, this directory has only one file, ``nr_contexts``. Writing a 188number (``N``) to the file creates the number of child directories named as 189``0`` to ``N-1``. Each directory represents each monitoring context (refer to 190:ref:`design <damon_design_execution_model_and_data_structures>` for more 191details). At the moment, only one context per kdamond is supported, so only 192``0`` or ``1`` can be written to the file. 193 194.. _sysfs_context: 195 196contexts/<N>/ 197------------- 198 199In each context directory, three files (``avail_operations``, ``operations`` 200and ``addr_unit``) and three directories (``monitoring_attrs``, ``targets``, 201and ``schemes``) exist. 202 203DAMON supports multiple types of :ref:`monitoring operations 204<damon_design_configurable_operations_set>`, including those for virtual address 205space and the physical address space. You can get the list of available 206monitoring operations set on the currently running kernel by reading 207``avail_operations`` file. Based on the kernel configuration, the file will 208list different available operation sets. Please refer to the :ref:`design 209<damon_operations_set>` for the list of all available operation sets and their 210brief explanations. 211 212You can set and get what type of monitoring operations DAMON will use for the 213context by writing one of the keywords listed in ``avail_operations`` file and 214reading from the ``operations`` file. 215 216``addr_unit`` file is for setting and getting the :ref:`address unit 217<damon_design_addr_unit>` parameter of the operations set. 218 219.. _sysfs_monitoring_attrs: 220 221contexts/<N>/monitoring_attrs/ 222------------------------------ 223 224Files for specifying attributes of the monitoring including required quality 225and efficiency of the monitoring are in ``monitoring_attrs`` directory. 226Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 227directory. 228 229Under ``intervals`` directory, three files for DAMON's sampling interval 230(``sample_us``), aggregation interval (``aggr_us``), and update interval 231(``update_us``) exist. You can set and get the values in micro-seconds by 232writing to and reading from the files. 233 234Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 235of DAMON's monitoring regions (``min`` and ``max``, respectively), which 236controls the monitoring overhead, exist. You can set and get the values by 237writing to and rading from the files. 238 239For more details about the intervals and monitoring regions range, please refer 240to the Design document (:doc:`/mm/damon/design`). 241 242.. _damon_usage_sysfs_monitoring_intervals_goal: 243 244contexts/<N>/monitoring_attrs/intervals/intervals_goal/ 245------------------------------------------------------- 246 247Under the ``intervals`` directory, one directory for automated tuning of 248``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists. 249Under the directory, four files for the auto-tuning control, namely 250``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist. 251Please refer to the :ref:`design document of the feature 252<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning 253mechanism. Reading and writing the four files under ``intervals_goal`` 254directory shows and updates the tuning parameters that described in the 255:ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same 256names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The 257tuning-applied current values of the two intervals can be read from the 258``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to 259the ``state`` file. 260 261.. _sysfs_targets: 262 263contexts/<N>/targets/ 264--------------------- 265 266In the beginning, this directory has only one file, ``nr_targets``. Writing a 267number (``N``) to the file creates the number of child directories named ``0`` 268to ``N-1``. Each directory represents each monitoring target. 269 270.. _sysfs_target: 271 272targets/<N>/ 273------------ 274 275In each target directory, two files (``pid_target`` and ``obsolete_target``) 276and one directory (``regions``) exist. 277 278If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 279be a process. You can specify the process to DAMON by writing the pid of the 280process to the ``pid_target`` file. 281 282Users can selectively remove targets in the middle of the targets array by 283writing non-zero value to ``obsolete_target`` file and committing it (writing 284``commit`` to ``state`` file). DAMON will remove the matching targets from its 285internal targets array. Users are responsible to construct target directories 286again, so that those correctly represent the changed internal targets array. 287 288 289.. _sysfs_regions: 290 291targets/<N>/regions 292------------------- 293 294In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are 295required to set the monitoring target address ranges. In case of ``vaddr`` 296operations set, it is not mandatory, but users can optionally set the initial 297monitoring region to specific address ranges. Please refer to the :ref:`design 298<damon_design_vaddr_target_regions_construction>` for more details. 299 300For such cases, users can explicitly set the initial monitoring target regions 301as they want, by writing proper values to the files under this directory. 302 303In the beginning, this directory has only one file, ``nr_regions``. Writing a 304number (``N``) to the file creates the number of child directories named ``0`` 305to ``N-1``. Each directory represents each initial monitoring target region. 306 307If ``nr_regions`` is zero when committing new DAMON parameters online (writing 308``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit 309logic ignores the target regions. In other words, the current monitoring 310results for the target are preserved. 311 312.. _sysfs_region: 313 314regions/<N>/ 315------------ 316 317In each region directory, you will find two files (``start`` and ``end``). You 318can set and get the start and end addresses of the initial monitoring target 319region by writing to and reading from the files, respectively. 320 321Each region should not overlap with others. ``end`` of directory ``N`` should 322be equal or smaller than ``start`` of directory ``N+1``. 323 324.. _sysfs_schemes: 325 326contexts/<N>/schemes/ 327--------------------- 328 329The directory for DAMON-based Operation Schemes (:ref:`DAMOS 330<damon_design_damos>`). Users can get and set the schemes by reading from and 331writing to files under this directory. 332 333In the beginning, this directory has only one file, ``nr_schemes``. Writing a 334number (``N``) to the file creates the number of child directories named ``0`` 335to ``N-1``. Each directory represents each DAMON-based operation scheme. 336 337.. _sysfs_scheme: 338 339schemes/<N>/ 340------------ 341 342In each scheme directory, eight directories (``access_pattern``, ``quotas``, 343``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``, 344``stats``, and ``tried_regions``) and three files (``action``, ``target_nid`` 345and ``apply_interval``) exist. 346 347The ``action`` file is for setting and getting the scheme's :ref:`action 348<damon_design_damos_action>`. The keywords that can be written to and read 349from the file and their meaning are same to those of the list on 350:ref:`design doc <damon_design_damos_action>`. 351 352The ``target_nid`` file is for setting the migration target node, which is 353only meaningful when the ``action`` is either ``migrate_hot`` or 354``migrate_cold``. 355 356The ``apply_interval_us`` file is for setting and getting the scheme's 357:ref:`apply_interval <damon_design_damos>` in microseconds. 358 359.. _sysfs_access_pattern: 360 361schemes/<N>/access_pattern/ 362--------------------------- 363 364The directory for the target access :ref:`pattern 365<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 366 367Under the ``access_pattern`` directory, three directories (``sz``, 368``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 369exist. You can set and get the access pattern for the given scheme by writing 370to and reading from the ``min`` and ``max`` files under ``sz``, 371``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 372and the ``max`` form a closed interval. 373 374.. _sysfs_quotas: 375 376schemes/<N>/quotas/ 377------------------- 378 379The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 380DAMON-based operation scheme. 381 382Under ``quotas`` directory, seven files (``ms``, ``bytes``, 383``reset_interval_ms``, ``effective_bytes``, ``goal_tuner``, ``fail_charge_num`` 384and ``fail_charge_denom``) and two directories (``weights`` and ``goals``) 385exist. 386 387You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 388``reset interval`` in milliseconds by writing the values to the three files, 389respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 390for applying the ``action`` to memory regions of the ``access_pattern``, and to 391apply the action to only up to ``bytes`` bytes of memory regions within the 392``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 393quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is 394set. 395 396You can set the goal-based effective quota auto-tuning algorithm to use, by 397writing the algorithm name to ``goal_tuner`` file. Reading the file returns 398the currently selected tuner algorithm. Refer to the design documentation of 399:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for 400the background design of the feature and the name of the selectable algorithms. 401Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals 402setup. 403 404You can set the action-failed memory quota charging ratio by writing the 405numerator and the denominator for the ratio to ``fail_charge_num`` and 406``fail_charge_denom`` files, respectively. Reading those files will return the 407current set values. Refer to :ref:`design 408<damon_design_damos_quotas_failed_memory_charging_ratio>` for more details of 409the ratio feature. 410 411The time quota is internally transformed to a size quota. Between the 412transformed size quota and user-specified size quota, smaller one is applied. 413Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the 414effective size quota is further adjusted. Reading ``effective_bytes`` returns 415the current effective size quota. The file is not updated in real time, so 416users should ask DAMON sysfs interface to update the content of the file for 417the stats by writing a special keyword, ``update_schemes_effective_quotas`` to 418the relevant ``kdamonds/<N>/state`` file. 419 420Under ``weights`` directory, three files (``sz_permil``, 421``nr_accesses_permil``, and ``age_permil``) exist. 422You can set the :ref:`prioritization weights 423<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 424in per-thousand unit by writing the values to the three files under the 425``weights`` directory. 426 427.. _sysfs_schemes_quota_goals: 428 429schemes/<N>/quotas/goals/ 430------------------------- 431 432The directory for the :ref:`automatic quota tuning goals 433<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation 434scheme. 435 436In the beginning, this directory has only one file, ``nr_goals``. Writing a 437number (``N``) to the file creates the number of child directories named ``0`` 438to ``N-1``. Each directory represents each goal and current achievement. 439Among the multiple feedback, the best one is used. 440 441Each goal directory contains five files, namely ``target_metric``, 442``target_value``, ``current_value`` ``nid`` and ``path``. Users can set and 443get the five parameters for the quota auto-tuning goals that specified on the 444:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and 445reading from each of the files. Note that users should further write 446``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond 447directory <sysfs_kdamond>` to pass the feedback to DAMON. 448 449.. _sysfs_watermarks: 450 451schemes/<N>/watermarks/ 452----------------------- 453 454The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 455given DAMON-based operation scheme. 456 457Under the watermarks directory, five files (``metric``, ``interval_us``, 458``high``, ``mid``, and ``low``) for setting the metric, the time interval 459between check of the metric, and the three watermarks exist. You can set and 460get the five values by writing to the files, respectively. 461 462Keywords and meanings of those that can be written to the ``metric`` file are 463as below. 464 465 - none: Ignore the watermarks 466 - free_mem_rate: System's free memory rate (per thousand) 467 468The ``interval`` should written in microseconds unit. 469 470.. _sysfs_filters: 471 472schemes/<N>/{core\_,ops\_,}filters/ 473----------------------------------- 474 475Directories for :ref:`filters <damon_design_damos_filters>` of the given 476DAMON-based operation scheme. 477 478``core_filters`` and ``ops_filters`` directories are for the filters handled by 479the DAMON core layer and operations set layer, respectively. ``filters`` 480directory can be used for installing filters regardless of their handled 481layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be 482installed before those of ``filters``. All three directories have same files. 483 484Use of ``filters`` directory can make expecting evaluation orders of given 485filters with the files under directory bit confusing. Users are hence 486recommended to use ``core_filters`` and ``ops_filters`` directories. The 487``filters`` directory could be deprecated in future. 488 489In the beginning, the directory has only one file, ``nr_filters``. Writing a 490number (``N``) to the file creates the number of child directories named ``0`` 491to ``N-1``. Each directory represents each filter. The filters are evaluated 492in the numeric order. 493 494Each filter directory contains nine files, namely ``type``, ``matching``, 495``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max`` 496and ``target_idx``. To ``type`` file, you can write the type of the filter. 497Refer to :ref:`the design doc <damon_design_damos_filters>` for available type 498names, their meaning and on what layer those are handled. 499 500For ``memcg`` type, you can specify the memory cgroup of the interest by 501writing the path of the memory cgroup from the cgroups mount point to 502``memcg_path`` file. For ``addr`` type, you can specify the start and end 503address of the range (open-ended interval) to ``addr_start`` and ``addr_end`` 504files, respectively. For ``hugepage_size`` type, you can specify the minimum 505and maximum size of the range (closed interval) to ``min`` and ``max`` files, 506respectively. For ``target`` type, you can specify the index of the target 507between the list of the DAMON context's monitoring targets list to 508``target_idx`` file. 509 510You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter 511is for memory that matches the ``type``. You can write ``Y`` or ``N`` to 512``allow`` file to specify if applying the action to the memory that satisfies 513the ``type`` and ``matching`` should be allowed or not. 514 515For example, below restricts a DAMOS action to be applied to only non-anonymous 516pages of all memory cgroups except ``/having_care_already``.:: 517 518 # cd ops_filters/0/ 519 # echo 2 > nr_filters 520 # # disallow anonymous pages 521 echo anon > 0/type 522 echo Y > 0/matching 523 echo N > 0/allow 524 # # further filter out all cgroups except one at '/having_care_already' 525 echo memcg > 1/type 526 echo /having_care_already > 1/memcg_path 527 echo Y > 1/matching 528 echo N > 1/allow 529 530Refer to the :ref:`DAMOS filters design documentation 531<damon_design_damos_filters>` for more details including how multiple filters 532of different ``allow`` works, when each of the filters are supported, and 533differences on stats. 534 535.. _damon_sysfs_dests: 536 537schemes/<N>/dests/ 538------------------ 539 540Directory for specifying the destinations of given DAMON-based operation 541scheme's action. This directory is ignored if the action of the given scheme 542is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}`` 543actions are supporting multiple destinations. 544 545In the beginning, the directory has only one file, ``nr_dests``. Writing a 546number (``N``) to the file creates the number of child directories named ``0`` 547to ``N-1``. Each directory represents each action destination. 548 549Each destination directory contains two files, namely ``id`` and ``weight``. 550Users can write and read the identifier of the destination to ``id`` file. 551For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node 552id should be written to ``id`` file. Users can write and read the weight of 553the destination among the given destinations to the ``weight`` file. The 554weight can be an arbitrary integer. When DAMOS apply the action to each entity 555of the memory region, it will select the destination of the action based on the 556relative weights of the destinations. 557 558.. _sysfs_schemes_stats: 559 560schemes/<N>/stats/ 561------------------ 562 563DAMON counts statistics for each scheme. This statistics can be used for 564online analysis or tuning of the schemes. Refer to :ref:`design doc 565<damon_design_damos_stat>` for more details about the stats. 566 567The statistics can be retrieved by reading the files under ``stats`` directory 568(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, 569``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and 570``max_nr_snapshots``), respectively. 571 572The files are not updated in real time by default. Users should ask DAMON 573sysfs interface to periodically update those using ``refresh_ms``, or do a one 574time update by writing a special keyword, ``update_schemes_stats`` to the 575relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory 576<sysfs_kdamond>` for more details. 577 578.. _sysfs_schemes_tried_regions: 579 580schemes/<N>/tried_regions/ 581-------------------------- 582 583This directory initially has one file, ``total_bytes``. 584 585When a special keyword, ``update_schemes_tried_regions``, is written to the 586relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 587that reading it returns the total size of the scheme tried regions, and creates 588directories named integer starting from ``0`` under this directory. Each 589directory contains files exposing detailed information about each of the memory 590region that the corresponding scheme's ``action`` has tried to be applied under 591this directory, during next :ref:`apply interval <damon_design_damos>` of the 592corresponding scheme. The information includes address range, ``nr_accesses``, 593and ``age`` of the region. 594 595Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 596file will only update the ``total_bytes`` file, and will not create the 597subdirectories. 598 599The directories will be removed when another special keyword, 600``clear_schemes_tried_regions``, is written to the relevant 601``kdamonds/<N>/state`` file. 602 603The expected usage of this directory is investigations of schemes' behaviors, 604and query-like efficient data access monitoring results retrievals. For the 605latter use case, in particular, users can set the ``action`` as ``stat`` and 606set the ``access pattern`` as their interested pattern that they want to query. 607 608.. _sysfs_schemes_tried_region: 609 610tried_regions/<N>/ 611------------------ 612 613In each region directory, you will find five files (``start``, ``end``, 614``nr_accesses``, ``age``, and ``sz_filter_passed``). Reading the files will 615show the properties of the region that corresponding DAMON-based operation 616scheme ``action`` has tried to be applied. 617 618Example 619~~~~~~~ 620 621Below commands applies a scheme saying "If a memory region of size in [4KiB, 6228KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 623interval in [10, 20], page out the region. For the paging out, use only up to 62410ms per second, and also don't page out more than 1GiB per second. Under the 625limitation, page out memory regions having longer age first. Also, check the 626free memory rate of the system every 5 seconds, start the monitoring and paging 627out when the free memory rate becomes lower than 50%, but stop it if the free 628memory rate becomes larger than 60%, or lower than 30%". :: 629 630 # cd <sysfs>/kernel/mm/damon/admin 631 # # populate directories 632 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 633 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 634 # cd kdamonds/0/contexts/0/schemes/0 635 # # set the basic access pattern and the action 636 # echo 4096 > access_pattern/sz/min 637 # echo 8192 > access_pattern/sz/max 638 # echo 0 > access_pattern/nr_accesses/min 639 # echo 5 > access_pattern/nr_accesses/max 640 # echo 10 > access_pattern/age/min 641 # echo 20 > access_pattern/age/max 642 # echo pageout > action 643 # # set quotas 644 # echo 10 > quotas/ms 645 # echo $((1024*1024*1024)) > quotas/bytes 646 # echo 1000 > quotas/reset_interval_ms 647 # # set watermark 648 # echo free_mem_rate > watermarks/metric 649 # echo 5000000 > watermarks/interval_us 650 # echo 600 > watermarks/high 651 # echo 500 > watermarks/mid 652 # echo 300 > watermarks/low 653 654Please note that it's highly recommended to use user space tools like `damo 655<https://github.com/damonitor/damo>`_ rather than manually reading and writing 656the files as above. Above is only for an example. 657 658.. _tracepoint: 659 660Tracepoints for Monitoring Results 661================================== 662 663Users can get the monitoring results via the :ref:`tried_regions 664<sysfs_schemes_tried_regions>`. The interface is useful for getting a 665snapshot, but it could be inefficient for fully recording all the monitoring 666results. For the purpose, two trace points, namely ``damon:damon_aggregated`` 667and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated`` 668provides the whole monitoring results, while ``damon:damos_before_apply`` 669provides the monitoring results for regions that each DAMON-based Operation 670Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence, 671``damon:damos_before_apply`` is more useful for recording internal behavior of 672DAMOS, or DAMOS target access 673:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient 674monitoring results recording. 675 676While the monitoring is turned on, you could record the tracepoint events and 677show results using tracepoint supporting tools like ``perf``. For example:: 678 679 # echo on > kdamonds/0/state 680 # perf record -e damon:damon_aggregated & 681 # sleep 5 682 # kill 9 $(pidof perf) 683 # echo off > kdamonds/0/state 684 # perf script 685 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864 686 [...] 687 688Each line of the perf script output represents each monitoring region. The 689first five fields are as usual other tracepoint outputs. The sixth field 690(``target_id=X``) shows the ide of the monitoring target of the region. The 691seventh field (``nr_regions=X``) shows the total number of monitoring regions 692for the target. The eighth field (``X-Y:``) shows the start (``X``) and end 693(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the 694``nr_accesses`` of the region (refer to 695:ref:`design <damon_design_region_based_sampling>` for more details of the 696counter). Finally the tenth field (``X``) shows the ``age`` of the region 697(refer to :ref:`design <damon_design_age_tracking>` for more details of the 698counter). 699 700If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would 701be somewhat like below:: 702 703 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136 704 [...] 705 706Each line of the output represents each monitoring region that each DAMON-based 707Operation Scheme was about to be applied at the traced time. The first five 708fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``) 709of the scheme in the list of the contexts of the context's kdamond, the index 710of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in 711addition to the output of ``damon_aggregated`` tracepoint. 712