xref: /linux/Documentation/admin-guide/mm/damon/usage.rst (revision aec2f682d47c54ef434b2d440992626d80b1ebdc)
1.. SPDX-License-Identifier: GPL-2.0
2
3===============
4Detailed Usages
5===============
6
7DAMON provides below interfaces for different users.
8
9- *Special-purpose DAMON modules.*
10  :ref:`This <damon_modules_special_purpose>` is for people who are building,
11  distributing, and/or administrating the kernel with special-purpose DAMON
12  usages.  Using this, users can use DAMON's major features for the given
13  purposes in build, boot, or runtime in simple ways.
14- *DAMON user space tool.*
15  `This <https://github.com/damonitor/damo>`_ is for privileged people such as
16  system administrators who want a just-working human-friendly interface.
17  Using this, users can use the DAMON’s major features in a human-friendly way.
18  It may not be highly tuned for special cases, though.  For more detail,
19  please refer to its `usage document
20  <https://github.com/damonitor/damo/blob/next/USAGE.md>`_.
21- *sysfs interface.*
22  :ref:`This <sysfs_interface>` is for privileged user space programmers who
23  want more optimized use of DAMON.  Using this, users can use DAMON’s major
24  features by reading from and writing to special sysfs files.  Therefore,
25  you can write and use your personalized DAMON sysfs wrapper programs that
26  reads/writes the sysfs files instead of you.  The `DAMON user space tool
27  <https://github.com/damonitor/damo>`_ is one example of such programs.
28- *Kernel Space Programming Interface.*
29  :doc:`This </mm/damon/api>` is for kernel space programmers.  Using this,
30  users can utilize every feature of DAMON most flexibly and efficiently by
31  writing kernel space DAMON application programs for you.  You can even extend
32  DAMON for various address spaces.  For detail, please refer to the interface
33  :doc:`document </mm/damon/api>`.
34
35.. _sysfs_interface:
36
37sysfs Interface
38===============
39
40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined.  It
41creates multiple directories and files under its sysfs directory,
42``<sysfs>/kernel/mm/damon/``.  You can control DAMON by writing to and reading
43from the files under the directory.
44
45For a short example, users can monitor the virtual address space of a given
46workload as below. ::
47
48    # cd /sys/kernel/mm/damon/admin/
49    # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts
50    # echo vaddr > kdamonds/0/contexts/0/operations
51    # echo 1 > kdamonds/0/contexts/0/targets/nr_targets
52    # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target
53    # echo on > kdamonds/0/state
54
55Files Hierarchy
56---------------
57
58The files hierarchy of DAMON sysfs interface is shown below.  In the below
59figure, parents-children relations are represented with indentations, each
60directory is having ``/`` suffix, and files in each directory are separated by
61comma (",").
62
63.. parsed-literal::
64
65    :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin
66    │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds
67    │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms
68    │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts
69    │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit
70    │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/
71    │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
72    │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us
73    │ │ │ │ │ │ nr_regions/min,max
74    │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets
75    │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target
76    │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions
77    │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end
78    │ │ │ │ │ │ │ │ ...
79    │ │ │ │ │ │ ...
80    │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes
81    │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us
82    │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/
83    │ │ │ │ │ │ │ │ sz/min,max
84    │ │ │ │ │ │ │ │ nr_accesses/min,max
85    │ │ │ │ │ │ │ │ age/min,max
86    │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,effective_bytes,goal_tuner
87    │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
88    │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals
89    │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path
90    │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
91    │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters
92    │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max
93    │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests
94    │ │ │ │ │ │ │ │ 0/id,weight
95    │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots
96    │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
97    │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
98    │ │ │ │ │ │ │ │ ...
99    │ │ │ │ │ │ ...
100    │ │ │ │ ...
101    │ │ ...
102
103.. _sysfs_root:
104
105Root
106----
107
108The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
109has one directory named ``admin``.  The directory contains the files for
110privileged user space programs' control of DAMON.  User space tools or daemons
111having the root permission could use this directory.
112
113.. _sysfs_kdamonds:
114
115kdamonds/
116---------
117
118Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
119controlling the kdamonds (refer to
120:ref:`design <damon_design_execution_model_and_data_structures>` for more
121details) exists.  In the beginning, this directory has only one file,
122``nr_kdamonds``.  Writing a number (``N``) to the file creates the number of
123child directories named ``0`` to ``N-1``.  Each directory represents each
124kdamond.
125
126.. _sysfs_kdamond:
127
128kdamonds/<N>/
129-------------
130
131In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``)
132and one directory (``contexts``) exist.
133
134Reading ``state`` returns ``on`` if the kdamond is currently running, or
135``off`` if it is not running.
136
137Users can write below commands for the kdamond to the ``state`` file.
138
139- ``on``: Start running.
140- ``off``: Stop running.
141- ``commit``: Read the user inputs in the sysfs files except ``state`` file
142  again.  Monitoring :ref:`target region <sysfs_regions>` inputs are also be
143  ignored if no target region is specified.
144- ``update_tuned_intervals``: Update the contents of ``sample_us`` and
145  ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling
146  interval`` and ``aggregation interval`` for the files.  Please refer to
147  :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>`
148  for more details.
149- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes'
150  :ref:`quota goals <sysfs_schemes_quota_goals>`.
151- ``update_schemes_stats``: Update the contents of stats files for each
152  DAMON-based operation scheme of the kdamond.  For details of the stats,
153  please refer to :ref:`stats section <sysfs_schemes_stats>`.
154- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme
155  action tried regions directory for each DAMON-based operation scheme of the
156  kdamond.  For details of the DAMON-based operation scheme action tried
157  regions directory, please refer to
158  :ref:`tried_regions section <sysfs_schemes_tried_regions>`.
159- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes``
160  files.
161- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme
162  action tried regions directory for each DAMON-based operation scheme of the
163  kdamond.
164- ``update_schemes_effective_quotas``: Update the contents of
165  ``effective_bytes`` files for each DAMON-based operation scheme of the
166  kdamond.  For more details, refer to :ref:`quotas directory <sysfs_quotas>`.
167
168If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
169
170Users can ask the kernel to periodically update files showing auto-tuned
171parameters and DAMOS stats instead of manually writing
172``update_tuned_intervals`` like keywords to ``state`` file.  For this, users
173should write the desired update time interval in milliseconds to ``refresh_ms``
174file.  If the interval is zero, the periodic update is disabled.  Reading the
175file shows currently set time interval.
176
177``contexts`` directory contains files for controlling the monitoring contexts
178that this kdamond will execute.
179
180.. _sysfs_contexts:
181
182kdamonds/<N>/contexts/
183----------------------
184
185In the beginning, this directory has only one file, ``nr_contexts``.  Writing a
186number (``N``) to the file creates the number of child directories named as
187``0`` to ``N-1``.  Each directory represents each monitoring context (refer to
188:ref:`design <damon_design_execution_model_and_data_structures>` for more
189details).  At the moment, only one context per kdamond is supported, so only
190``0`` or ``1`` can be written to the file.
191
192.. _sysfs_context:
193
194contexts/<N>/
195-------------
196
197In each context directory, three files (``avail_operations``, ``operations``
198and ``addr_unit``) and three directories (``monitoring_attrs``, ``targets``,
199and ``schemes``) exist.
200
201DAMON supports multiple types of :ref:`monitoring operations
202<damon_design_configurable_operations_set>`, including those for virtual address
203space and the physical address space.  You can get the list of available
204monitoring operations set on the currently running kernel by reading
205``avail_operations`` file.  Based on the kernel configuration, the file will
206list different available operation sets.  Please refer to the :ref:`design
207<damon_operations_set>` for the list of all available operation sets and their
208brief explanations.
209
210You can set and get what type of monitoring operations DAMON will use for the
211context by writing one of the keywords listed in ``avail_operations`` file and
212reading from the ``operations`` file.
213
214``addr_unit`` file is for setting and getting the :ref:`address unit
215<damon_design_addr_unit>` parameter of the operations set.
216
217.. _sysfs_monitoring_attrs:
218
219contexts/<N>/monitoring_attrs/
220------------------------------
221
222Files for specifying attributes of the monitoring including required quality
223and efficiency of the monitoring are in ``monitoring_attrs`` directory.
224Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this
225directory.
226
227Under ``intervals`` directory, three files for DAMON's sampling interval
228(``sample_us``), aggregation interval (``aggr_us``), and update interval
229(``update_us``) exist.  You can set and get the values in micro-seconds by
230writing to and reading from the files.
231
232Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
233of DAMON's monitoring regions (``min`` and ``max``, respectively), which
234controls the monitoring overhead, exist.  You can set and get the values by
235writing to and rading from the files.
236
237For more details about the intervals and monitoring regions range, please refer
238to the Design document (:doc:`/mm/damon/design`).
239
240.. _damon_usage_sysfs_monitoring_intervals_goal:
241
242contexts/<N>/monitoring_attrs/intervals/intervals_goal/
243-------------------------------------------------------
244
245Under the ``intervals`` directory, one directory for automated tuning of
246``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists.
247Under the directory, four files for the auto-tuning control, namely
248``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist.
249Please refer to  the :ref:`design document of the feature
250<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning
251mechanism.  Reading and writing the four files under ``intervals_goal``
252directory shows and updates the tuning parameters that described in the
253:ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same
254names.  The tuning starts with the user-set ``sample_us`` and ``aggr_us``.  The
255tuning-applied current values of the two intervals can be read from the
256``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to
257the ``state`` file.
258
259.. _sysfs_targets:
260
261contexts/<N>/targets/
262---------------------
263
264In the beginning, this directory has only one file, ``nr_targets``.  Writing a
265number (``N``) to the file creates the number of child directories named ``0``
266to ``N-1``.  Each directory represents each monitoring target.
267
268.. _sysfs_target:
269
270targets/<N>/
271------------
272
273In each target directory, two files (``pid_target`` and ``obsolete_target``)
274and one directory (``regions``) exist.
275
276If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
277be a process.  You can specify the process to DAMON by writing the pid of the
278process to the ``pid_target`` file.
279
280Users can selectively remove targets in the middle of the targets array by
281writing non-zero value to ``obsolete_target`` file and committing it (writing
282``commit`` to ``state`` file).  DAMON will remove the matching targets from its
283internal targets array.  Users are responsible to construct target directories
284again, so that those correctly represent the changed internal targets array.
285
286
287.. _sysfs_regions:
288
289targets/<N>/regions
290-------------------
291
292In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are
293required to set the monitoring target address ranges.  In case of ``vaddr``
294operations set, it is not mandatory, but users can optionally set the initial
295monitoring region to specific address ranges.  Please refer to the :ref:`design
296<damon_design_vaddr_target_regions_construction>` for more details.
297
298For such cases, users can explicitly set the initial monitoring target regions
299as they want, by writing proper values to the files under this directory.
300
301In the beginning, this directory has only one file, ``nr_regions``.  Writing a
302number (``N``) to the file creates the number of child directories named ``0``
303to ``N-1``.  Each directory represents each initial monitoring target region.
304
305If ``nr_regions`` is zero when committing new DAMON parameters online (writing
306``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit
307logic ignores the target regions.  In other words, the current monitoring
308results for the target are preserved.
309
310.. _sysfs_region:
311
312regions/<N>/
313------------
314
315In each region directory, you will find two files (``start`` and ``end``).  You
316can set and get the start and end addresses of the initial monitoring target
317region by writing to and reading from the files, respectively.
318
319Each region should not overlap with others.  ``end`` of directory ``N`` should
320be equal or smaller than ``start`` of directory ``N+1``.
321
322.. _sysfs_schemes:
323
324contexts/<N>/schemes/
325---------------------
326
327The directory for DAMON-based Operation Schemes (:ref:`DAMOS
328<damon_design_damos>`).  Users can get and set the schemes by reading from and
329writing to files under this directory.
330
331In the beginning, this directory has only one file, ``nr_schemes``.  Writing a
332number (``N``) to the file creates the number of child directories named ``0``
333to ``N-1``.  Each directory represents each DAMON-based operation scheme.
334
335.. _sysfs_scheme:
336
337schemes/<N>/
338------------
339
340In each scheme directory, eight directories (``access_pattern``, ``quotas``,
341``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``,
342``stats``, and ``tried_regions``) and three files (``action``, ``target_nid``
343and ``apply_interval``) exist.
344
345The ``action`` file is for setting and getting the scheme's :ref:`action
346<damon_design_damos_action>`.  The keywords that can be written to and read
347from the file and their meaning are same to those of the list on
348:ref:`design doc <damon_design_damos_action>`.
349
350The ``target_nid`` file is for setting the migration target node, which is
351only meaningful when the ``action`` is either ``migrate_hot`` or
352``migrate_cold``.
353
354The ``apply_interval_us`` file is for setting and getting the scheme's
355:ref:`apply_interval <damon_design_damos>` in microseconds.
356
357.. _sysfs_access_pattern:
358
359schemes/<N>/access_pattern/
360---------------------------
361
362The directory for the target access :ref:`pattern
363<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme.
364
365Under the ``access_pattern`` directory, three directories (``sz``,
366``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
367exist.  You can set and get the access pattern for the given scheme by writing
368to and reading from the ``min`` and ``max`` files under ``sz``,
369``nr_accesses``, and ``age`` directories, respectively.  Note that the ``min``
370and the ``max`` form a closed interval.
371
372.. _sysfs_quotas:
373
374schemes/<N>/quotas/
375-------------------
376
377The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given
378DAMON-based operation scheme.
379
380Under ``quotas`` directory, five files (``ms``, ``bytes``,
381``reset_interval_ms``, ``effective_bytes`` and ``goal_tuner``) and two
382directories (``weights`` and ``goals``) exist.
383
384You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
385``reset interval`` in milliseconds by writing the values to the three files,
386respectively.  Then, DAMON tries to use only up to ``time quota`` milliseconds
387for applying the ``action`` to memory regions of the ``access_pattern``, and to
388apply the action to only up to ``bytes`` bytes of memory regions within the
389``reset_interval_ms``.  Setting both ``ms`` and ``bytes`` zero disables the
390quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is
391set.
392
393You can set the goal-based effective quota auto-tuning algorithm to use, by
394writing the algorithm name to ``goal_tuner`` file.  Reading the file returns
395the currently selected tuner algorithm.  Refer to the design documentation of
396:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for
397the background design of the feature and the name of the selectable algorithms.
398Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals
399setup.
400
401The time quota is internally transformed to a size quota.  Between the
402transformed size quota and user-specified size quota, smaller one is applied.
403Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the
404effective size quota is further adjusted.  Reading ``effective_bytes`` returns
405the current effective size quota.  The file is not updated in real time, so
406users should ask DAMON sysfs interface to update the content of the file for
407the stats by writing a special keyword, ``update_schemes_effective_quotas`` to
408the relevant ``kdamonds/<N>/state`` file.
409
410Under ``weights`` directory, three files (``sz_permil``,
411``nr_accesses_permil``, and ``age_permil``) exist.
412You can set the :ref:`prioritization weights
413<damon_design_damos_quotas_prioritization>` for size, access frequency, and age
414in per-thousand unit by writing the values to the three files under the
415``weights`` directory.
416
417.. _sysfs_schemes_quota_goals:
418
419schemes/<N>/quotas/goals/
420-------------------------
421
422The directory for the :ref:`automatic quota tuning goals
423<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation
424scheme.
425
426In the beginning, this directory has only one file, ``nr_goals``.  Writing a
427number (``N``) to the file creates the number of child directories named ``0``
428to ``N-1``.  Each directory represents each goal and current achievement.
429Among the multiple feedback, the best one is used.
430
431Each goal directory contains five files, namely ``target_metric``,
432``target_value``, ``current_value`` ``nid`` and ``path``.  Users can set and
433get the five parameters for the quota auto-tuning goals that specified on the
434:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and
435reading from each of the files.  Note that users should further write
436``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond
437directory <sysfs_kdamond>` to pass the feedback to DAMON.
438
439.. _sysfs_watermarks:
440
441schemes/<N>/watermarks/
442-----------------------
443
444The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the
445given DAMON-based operation scheme.
446
447Under the watermarks directory, five files (``metric``, ``interval_us``,
448``high``, ``mid``, and ``low``) for setting the metric, the time interval
449between check of the metric, and the three watermarks exist.  You can set and
450get the five values by writing to the files, respectively.
451
452Keywords and meanings of those that can be written to the ``metric`` file are
453as below.
454
455 - none: Ignore the watermarks
456 - free_mem_rate: System's free memory rate (per thousand)
457
458The ``interval`` should written in microseconds unit.
459
460.. _sysfs_filters:
461
462schemes/<N>/{core\_,ops\_,}filters/
463-----------------------------------
464
465Directories for :ref:`filters <damon_design_damos_filters>` of the given
466DAMON-based operation scheme.
467
468``core_filters`` and ``ops_filters`` directories are for the filters handled by
469the DAMON core layer and operations set layer, respectively.  ``filters``
470directory can be used for installing filters regardless of their handled
471layers.  Filters that requested by ``core_filters`` and ``ops_filters`` will be
472installed before those of ``filters``.  All three directories have same files.
473
474Use of ``filters`` directory can make expecting evaluation orders of given
475filters with the files under directory bit confusing.  Users are hence
476recommended to use ``core_filters`` and ``ops_filters`` directories.  The
477``filters`` directory could be deprecated in future.
478
479In the beginning, the directory has only one file, ``nr_filters``.  Writing a
480number (``N``) to the file creates the number of child directories named ``0``
481to ``N-1``.  Each directory represents each filter.  The filters are evaluated
482in the numeric order.
483
484Each filter directory contains nine files, namely ``type``, ``matching``,
485``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max``
486and ``target_idx``.  To ``type`` file, you can write the type of the filter.
487Refer to :ref:`the design doc <damon_design_damos_filters>` for available type
488names, their meaning and on what layer those are handled.
489
490For ``memcg`` type, you can specify the memory cgroup of the interest by
491writing the path of the memory cgroup from the cgroups mount point to
492``memcg_path`` file.  For ``addr`` type, you can specify the start and end
493address of the range (open-ended interval) to ``addr_start`` and ``addr_end``
494files, respectively.  For ``hugepage_size`` type, you can specify the minimum
495and maximum size of the range (closed interval) to ``min`` and ``max`` files,
496respectively.  For ``target`` type, you can specify the index of the target
497between the list of the DAMON context's monitoring targets list to
498``target_idx`` file.
499
500You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter
501is for memory that matches the ``type``.  You can write ``Y`` or ``N`` to
502``allow`` file to specify if applying the action to the memory that satisfies
503the ``type`` and ``matching`` should be allowed or not.
504
505For example, below restricts a DAMOS action to be applied to only non-anonymous
506pages of all memory cgroups except ``/having_care_already``.::
507
508    # cd ops_filters/0/
509    # echo 2 > nr_filters
510    # # disallow anonymous pages
511    echo anon > 0/type
512    echo Y > 0/matching
513    echo N > 0/allow
514    # # further filter out all cgroups except one at '/having_care_already'
515    echo memcg > 1/type
516    echo /having_care_already > 1/memcg_path
517    echo Y > 1/matching
518    echo N > 1/allow
519
520Refer to the :ref:`DAMOS filters design documentation
521<damon_design_damos_filters>` for more details including how multiple filters
522of different ``allow`` works, when each of the filters are supported, and
523differences on stats.
524
525.. _damon_sysfs_dests:
526
527schemes/<N>/dests/
528------------------
529
530Directory for specifying the destinations of given DAMON-based operation
531scheme's action.  This directory is ignored if the action of the given scheme
532is not supporting multiple destinations.  Only ``DAMOS_MIGRATE_{HOT,COLD}``
533actions are supporting multiple destinations.
534
535In the beginning, the directory has only one file, ``nr_dests``.  Writing a
536number (``N``) to the file creates the number of child directories named ``0``
537to ``N-1``.  Each directory represents each action destination.
538
539Each destination directory contains two files, namely ``id`` and ``weight``.
540Users can write and read the identifier of the destination to ``id`` file.
541For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node
542id should be written to ``id`` file.  Users can write and read the weight of
543the destination among the given destinations to the ``weight`` file.  The
544weight can be an arbitrary integer.  When DAMOS apply the action to each entity
545of the memory region, it will select the destination of the action based on the
546relative weights of the destinations.
547
548.. _sysfs_schemes_stats:
549
550schemes/<N>/stats/
551------------------
552
553DAMON counts statistics for each scheme.  This statistics can be used for
554online analysis or tuning of the schemes.  Refer to :ref:`design doc
555<damon_design_damos_stat>` for more details about the stats.
556
557The statistics can be retrieved by reading the files under ``stats`` directory
558(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``,
559``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and
560``max_nr_snapshots``), respectively.
561
562The files are not updated in real time by default.  Users should ask DAMON
563sysfs interface to periodically update those using ``refresh_ms``, or do a one
564time update by writing a special keyword, ``update_schemes_stats`` to the
565relevant ``kdamonds/<N>/state`` file.  Refer to :ref:`kdamond directory
566<sysfs_kdamond>` for more details.
567
568.. _sysfs_schemes_tried_regions:
569
570schemes/<N>/tried_regions/
571--------------------------
572
573This directory initially has one file, ``total_bytes``.
574
575When a special keyword, ``update_schemes_tried_regions``, is written to the
576relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so
577that reading it returns the total size of the scheme tried regions, and creates
578directories named integer starting from ``0`` under this directory.  Each
579directory contains files exposing detailed information about each of the memory
580region that the corresponding scheme's ``action`` has tried to be applied under
581this directory, during next :ref:`apply interval <damon_design_damos>` of the
582corresponding scheme.  The information includes address range, ``nr_accesses``,
583and ``age`` of the region.
584
585Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state``
586file will only update the ``total_bytes`` file, and will not create the
587subdirectories.
588
589The directories will be removed when another special keyword,
590``clear_schemes_tried_regions``, is written to the relevant
591``kdamonds/<N>/state`` file.
592
593The expected usage of this directory is investigations of schemes' behaviors,
594and query-like efficient data access monitoring results retrievals.  For the
595latter use case, in particular, users can set the ``action`` as ``stat`` and
596set the ``access pattern`` as their interested pattern that they want to query.
597
598.. _sysfs_schemes_tried_region:
599
600tried_regions/<N>/
601------------------
602
603In each region directory, you will find five files (``start``, ``end``,
604``nr_accesses``, ``age``, and ``sz_filter_passed``).  Reading the files will
605show the properties of the region that corresponding DAMON-based operation
606scheme ``action`` has tried to be applied.
607
608Example
609~~~~~~~
610
611Below commands applies a scheme saying "If a memory region of size in [4KiB,
6128KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
613interval in [10, 20], page out the region.  For the paging out, use only up to
61410ms per second, and also don't page out more than 1GiB per second.  Under the
615limitation, page out memory regions having longer age first.  Also, check the
616free memory rate of the system every 5 seconds, start the monitoring and paging
617out when the free memory rate becomes lower than 50%, but stop it if the free
618memory rate becomes larger than 60%, or lower than 30%". ::
619
620    # cd <sysfs>/kernel/mm/damon/admin
621    # # populate directories
622    # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts;
623    # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes
624    # cd kdamonds/0/contexts/0/schemes/0
625    # # set the basic access pattern and the action
626    # echo 4096 > access_pattern/sz/min
627    # echo 8192 > access_pattern/sz/max
628    # echo 0 > access_pattern/nr_accesses/min
629    # echo 5 > access_pattern/nr_accesses/max
630    # echo 10 > access_pattern/age/min
631    # echo 20 > access_pattern/age/max
632    # echo pageout > action
633    # # set quotas
634    # echo 10 > quotas/ms
635    # echo $((1024*1024*1024)) > quotas/bytes
636    # echo 1000 > quotas/reset_interval_ms
637    # # set watermark
638    # echo free_mem_rate > watermarks/metric
639    # echo 5000000 > watermarks/interval_us
640    # echo 600 > watermarks/high
641    # echo 500 > watermarks/mid
642    # echo 300 > watermarks/low
643
644Please note that it's highly recommended to use user space tools like `damo
645<https://github.com/damonitor/damo>`_ rather than manually reading and writing
646the files as above.  Above is only for an example.
647
648.. _tracepoint:
649
650Tracepoints for Monitoring Results
651==================================
652
653Users can get the monitoring results via the :ref:`tried_regions
654<sysfs_schemes_tried_regions>`.  The interface is useful for getting a
655snapshot, but it could be inefficient for fully recording all the monitoring
656results.  For the purpose, two trace points, namely ``damon:damon_aggregated``
657and ``damon:damos_before_apply``, are provided.  ``damon:damon_aggregated``
658provides the whole monitoring results, while ``damon:damos_before_apply``
659provides the monitoring results for regions that each DAMON-based Operation
660Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied.  Hence,
661``damon:damos_before_apply`` is more useful for recording internal behavior of
662DAMOS, or DAMOS target access
663:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient
664monitoring results recording.
665
666While the monitoring is turned on, you could record the tracepoint events and
667show results using tracepoint supporting tools like ``perf``.  For example::
668
669    # echo on > kdamonds/0/state
670    # perf record -e damon:damon_aggregated &
671    # sleep 5
672    # kill 9 $(pidof perf)
673    # echo off > kdamonds/0/state
674    # perf script
675    kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864
676    [...]
677
678Each line of the perf script output represents each monitoring region.  The
679first five fields are as usual other tracepoint outputs.  The sixth field
680(``target_id=X``) shows the ide of the monitoring target of the region.  The
681seventh field (``nr_regions=X``) shows the total number of monitoring regions
682for the target.  The eighth field (``X-Y:``) shows the start (``X``) and end
683(``Y``) addresses of the region in bytes.  The ninth field (``X``) shows the
684``nr_accesses`` of the region (refer to
685:ref:`design <damon_design_region_based_sampling>` for more details of the
686counter).  Finally the tenth field (``X``) shows the ``age`` of the region
687(refer to :ref:`design <damon_design_age_tracking>` for more details of the
688counter).
689
690If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would
691be somewhat like below::
692
693    kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136
694    [...]
695
696Each line of the output represents each monitoring region that each DAMON-based
697Operation Scheme was about to be applied at the traced time.  The first five
698fields are as usual.  It shows the index of the DAMON context (``ctx_idx=X``)
699of the scheme in the list of the contexts of the context's kdamond, the index
700of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in
701addition to the output of ``damon_aggregated`` tracepoint.
702