xref: /linux/Documentation/admin-guide/mm/damon/usage.rst (revision 6aacab308a5dfd222b2d23662bbae60c11007cfb)
1.. SPDX-License-Identifier: GPL-2.0
2
3===============
4Detailed Usages
5===============
6
7DAMON provides below interfaces for different users.
8
9- *Special-purpose DAMON modules.*
10  :ref:`This <damon_modules_special_purpose>` is for people who are building,
11  distributing, and/or administrating the kernel with special-purpose DAMON
12  usages.  Using this, users can use DAMON's major features for the given
13  purposes in build, boot, or runtime in simple ways.
14- *DAMON user space tool.*
15  `This <https://github.com/damonitor/damo>`_ is for privileged people such as
16  system administrators who want a just-working human-friendly interface.
17  Using this, users can use the DAMON’s major features in a human-friendly way.
18  It may not be highly tuned for special cases, though.  For more detail,
19  please refer to its `usage document
20  <https://github.com/damonitor/damo/blob/next/USAGE.md>`_.
21- *sysfs interface.*
22  :ref:`This <sysfs_interface>` is for privileged user space programmers who
23  want more optimized use of DAMON.  Using this, users can use DAMON’s major
24  features by reading from and writing to special sysfs files.  Therefore,
25  you can write and use your personalized DAMON sysfs wrapper programs that
26  reads/writes the sysfs files instead of you.  The `DAMON user space tool
27  <https://github.com/damonitor/damo>`_ is one example of such programs.
28- *Kernel Space Programming Interface.*
29  :doc:`This </mm/damon/api>` is for kernel space programmers.  Using this,
30  users can utilize every feature of DAMON most flexibly and efficiently by
31  writing kernel space DAMON application programs for you.  You can even extend
32  DAMON for various address spaces.  For detail, please refer to the interface
33  :doc:`document </mm/damon/api>`.
34
35.. _sysfs_interface:
36
37sysfs Interface
38===============
39
40DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined.  It
41creates multiple directories and files under its sysfs directory,
42``<sysfs>/kernel/mm/damon/``.  You can control DAMON by writing to and reading
43from the files under the directory.
44
45For a short example, users can monitor the virtual address space of a given
46workload as below. ::
47
48    # cd /sys/kernel/mm/damon/admin/
49    # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts
50    # echo vaddr > kdamonds/0/contexts/0/operations
51    # echo 1 > kdamonds/0/contexts/0/targets/nr_targets
52    # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target
53    # echo on > kdamonds/0/state
54
55Files Hierarchy
56---------------
57
58The files hierarchy of DAMON sysfs interface is shown below.  In the below
59figure, parents-children relations are represented with indentations, each
60directory is having ``/`` suffix, and files in each directory are separated by
61comma (",").
62
63.. parsed-literal::
64
65    :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin
66    │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds
67    │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms
68    │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts
69    │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit
70    │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/
71    │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
72    │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us
73    │ │ │ │ │ │ nr_regions/min,max
74    │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets
75    │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target
76    │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions
77    │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end
78    │ │ │ │ │ │ │ │ ...
79    │ │ │ │ │ │ ...
80    │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes
81    │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us
82    │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/
83    │ │ │ │ │ │ │ │ sz/min,max
84    │ │ │ │ │ │ │ │ nr_accesses/min,max
85    │ │ │ │ │ │ │ │ age/min,max
86    │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,effective_bytes
87    │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
88    │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals
89    │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path
90    │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
91    │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters
92    │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max
93    │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests
94    │ │ │ │ │ │ │ │ 0/id,weight
95    │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots
96    │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
97    │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
98    │ │ │ │ │ │ │ │ ...
99    │ │ │ │ │ │ ...
100    │ │ │ │ ...
101    │ │ ...
102
103.. _sysfs_root:
104
105Root
106----
107
108The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
109has one directory named ``admin``.  The directory contains the files for
110privileged user space programs' control of DAMON.  User space tools or daemons
111having the root permission could use this directory.
112
113.. _sysfs_kdamonds:
114
115kdamonds/
116---------
117
118Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
119controlling the kdamonds (refer to
120:ref:`design <damon_design_execution_model_and_data_structures>` for more
121details) exists.  In the beginning, this directory has only one file,
122``nr_kdamonds``.  Writing a number (``N``) to the file creates the number of
123child directories named ``0`` to ``N-1``.  Each directory represents each
124kdamond.
125
126.. _sysfs_kdamond:
127
128kdamonds/<N>/
129-------------
130
131In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``)
132and one directory (``contexts``) exist.
133
134Reading ``state`` returns ``on`` if the kdamond is currently running, or
135``off`` if it is not running.
136
137Users can write below commands for the kdamond to the ``state`` file.
138
139- ``on``: Start running.
140- ``off``: Stop running.
141- ``commit``: Read the user inputs in the sysfs files except ``state`` file
142  again.  Monitoring :ref:`target region <sysfs_regions>` inputs are also be
143  ignored if no target region is specified.
144- ``update_tuned_intervals``: Update the contents of ``sample_us`` and
145  ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling
146  interval`` and ``aggregation interval`` for the files.  Please refer to
147  :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>`
148  for more details.
149- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes'
150  :ref:`quota goals <sysfs_schemes_quota_goals>`.
151- ``update_schemes_stats``: Update the contents of stats files for each
152  DAMON-based operation scheme of the kdamond.  For details of the stats,
153  please refer to :ref:`stats section <sysfs_schemes_stats>`.
154- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme
155  action tried regions directory for each DAMON-based operation scheme of the
156  kdamond.  For details of the DAMON-based operation scheme action tried
157  regions directory, please refer to
158  :ref:`tried_regions section <sysfs_schemes_tried_regions>`.
159- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes``
160  files.
161- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme
162  action tried regions directory for each DAMON-based operation scheme of the
163  kdamond.
164- ``update_schemes_effective_quotas``: Update the contents of
165  ``effective_bytes`` files for each DAMON-based operation scheme of the
166  kdamond.  For more details, refer to :ref:`quotas directory <sysfs_quotas>`.
167
168If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
169
170Users can ask the kernel to periodically update files showing auto-tuned
171parameters and DAMOS stats instead of manually writing
172``update_tuned_intervals`` like keywords to ``state`` file.  For this, users
173should write the desired update time interval in milliseconds to ``refresh_ms``
174file.  If the interval is zero, the periodic update is disabled.  Reading the
175file shows currently set time interval.
176
177``contexts`` directory contains files for controlling the monitoring contexts
178that this kdamond will execute.
179
180.. _sysfs_contexts:
181
182kdamonds/<N>/contexts/
183----------------------
184
185In the beginning, this directory has only one file, ``nr_contexts``.  Writing a
186number (``N``) to the file creates the number of child directories named as
187``0`` to ``N-1``.  Each directory represents each monitoring context (refer to
188:ref:`design <damon_design_execution_model_and_data_structures>` for more
189details).  At the moment, only one context per kdamond is supported, so only
190``0`` or ``1`` can be written to the file.
191
192.. _sysfs_context:
193
194contexts/<N>/
195-------------
196
197In each context directory, three files (``avail_operations``, ``operations``
198and ``addr_unit``) and three directories (``monitoring_attrs``, ``targets``,
199and ``schemes``) exist.
200
201DAMON supports multiple types of :ref:`monitoring operations
202<damon_design_configurable_operations_set>`, including those for virtual address
203space and the physical address space.  You can get the list of available
204monitoring operations set on the currently running kernel by reading
205``avail_operations`` file.  Based on the kernel configuration, the file will
206list different available operation sets.  Please refer to the :ref:`design
207<damon_operations_set>` for the list of all available operation sets and their
208brief explanations.
209
210You can set and get what type of monitoring operations DAMON will use for the
211context by writing one of the keywords listed in ``avail_operations`` file and
212reading from the ``operations`` file.
213
214``addr_unit`` file is for setting and getting the :ref:`address unit
215<damon_design_addr_unit>` parameter of the operations set.
216
217.. _sysfs_monitoring_attrs:
218
219contexts/<N>/monitoring_attrs/
220------------------------------
221
222Files for specifying attributes of the monitoring including required quality
223and efficiency of the monitoring are in ``monitoring_attrs`` directory.
224Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this
225directory.
226
227Under ``intervals`` directory, three files for DAMON's sampling interval
228(``sample_us``), aggregation interval (``aggr_us``), and update interval
229(``update_us``) exist.  You can set and get the values in micro-seconds by
230writing to and reading from the files.
231
232Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
233of DAMON's monitoring regions (``min`` and ``max``, respectively), which
234controls the monitoring overhead, exist.  You can set and get the values by
235writing to and rading from the files.
236
237For more details about the intervals and monitoring regions range, please refer
238to the Design document (:doc:`/mm/damon/design`).
239
240.. _damon_usage_sysfs_monitoring_intervals_goal:
241
242contexts/<N>/monitoring_attrs/intervals/intervals_goal/
243-------------------------------------------------------
244
245Under the ``intervals`` directory, one directory for automated tuning of
246``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists.
247Under the directory, four files for the auto-tuning control, namely
248``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist.
249Please refer to  the :ref:`design document of the feature
250<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning
251mechanism.  Reading and writing the four files under ``intervals_goal``
252directory shows and updates the tuning parameters that described in the
253:ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same
254names.  The tuning starts with the user-set ``sample_us`` and ``aggr_us``.  The
255tuning-applied current values of the two intervals can be read from the
256``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to
257the ``state`` file.
258
259.. _sysfs_targets:
260
261contexts/<N>/targets/
262---------------------
263
264In the beginning, this directory has only one file, ``nr_targets``.  Writing a
265number (``N``) to the file creates the number of child directories named ``0``
266to ``N-1``.  Each directory represents each monitoring target.
267
268.. _sysfs_target:
269
270targets/<N>/
271------------
272
273In each target directory, two files (``pid_target`` and ``obsolete_target``)
274and one directory (``regions``) exist.
275
276If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
277be a process.  You can specify the process to DAMON by writing the pid of the
278process to the ``pid_target`` file.
279
280Users can selectively remove targets in the middle of the targets array by
281writing non-zero value to ``obsolete_target`` file and committing it (writing
282``commit`` to ``state`` file).  DAMON will remove the matching targets from its
283internal targets array.  Users are responsible to construct target directories
284again, so that those correctly represent the changed internal targets array.
285
286
287.. _sysfs_regions:
288
289targets/<N>/regions
290-------------------
291
292In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are
293required to set the monitoring target address ranges.  In case of ``vaddr``
294operations set, it is not mandatory, but users can optionally set the initial
295monitoring region to specific address ranges.  Please refer to the :ref:`design
296<damon_design_vaddr_target_regions_construction>` for more details.
297
298For such cases, users can explicitly set the initial monitoring target regions
299as they want, by writing proper values to the files under this directory.
300
301In the beginning, this directory has only one file, ``nr_regions``.  Writing a
302number (``N``) to the file creates the number of child directories named ``0``
303to ``N-1``.  Each directory represents each initial monitoring target region.
304
305If ``nr_regions`` is zero when committing new DAMON parameters online (writing
306``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit
307logic ignores the target regions.  In other words, the current monitoring
308results for the target are preserved.
309
310.. _sysfs_region:
311
312regions/<N>/
313------------
314
315In each region directory, you will find two files (``start`` and ``end``).  You
316can set and get the start and end addresses of the initial monitoring target
317region by writing to and reading from the files, respectively.
318
319Each region should not overlap with others.  ``end`` of directory ``N`` should
320be equal or smaller than ``start`` of directory ``N+1``.
321
322.. _sysfs_schemes:
323
324contexts/<N>/schemes/
325---------------------
326
327The directory for DAMON-based Operation Schemes (:ref:`DAMOS
328<damon_design_damos>`).  Users can get and set the schemes by reading from and
329writing to files under this directory.
330
331In the beginning, this directory has only one file, ``nr_schemes``.  Writing a
332number (``N``) to the file creates the number of child directories named ``0``
333to ``N-1``.  Each directory represents each DAMON-based operation scheme.
334
335.. _sysfs_scheme:
336
337schemes/<N>/
338------------
339
340In each scheme directory, eight directories (``access_pattern``, ``quotas``,
341``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``,
342``stats``, and ``tried_regions``) and three files (``action``, ``target_nid``
343and ``apply_interval``) exist.
344
345The ``action`` file is for setting and getting the scheme's :ref:`action
346<damon_design_damos_action>`.  The keywords that can be written to and read
347from the file and their meaning are same to those of the list on
348:ref:`design doc <damon_design_damos_action>`.
349
350The ``target_nid`` file is for setting the migration target node, which is
351only meaningful when the ``action`` is either ``migrate_hot`` or
352``migrate_cold``.
353
354The ``apply_interval_us`` file is for setting and getting the scheme's
355:ref:`apply_interval <damon_design_damos>` in microseconds.
356
357.. _sysfs_access_pattern:
358
359schemes/<N>/access_pattern/
360---------------------------
361
362The directory for the target access :ref:`pattern
363<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme.
364
365Under the ``access_pattern`` directory, three directories (``sz``,
366``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
367exist.  You can set and get the access pattern for the given scheme by writing
368to and reading from the ``min`` and ``max`` files under ``sz``,
369``nr_accesses``, and ``age`` directories, respectively.  Note that the ``min``
370and the ``max`` form a closed interval.
371
372.. _sysfs_quotas:
373
374schemes/<N>/quotas/
375-------------------
376
377The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given
378DAMON-based operation scheme.
379
380Under ``quotas`` directory, four files (``ms``, ``bytes``,
381``reset_interval_ms``, ``effective_bytes``) and two directories (``weights`` and
382``goals``) exist.
383
384You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
385``reset interval`` in milliseconds by writing the values to the three files,
386respectively.  Then, DAMON tries to use only up to ``time quota`` milliseconds
387for applying the ``action`` to memory regions of the ``access_pattern``, and to
388apply the action to only up to ``bytes`` bytes of memory regions within the
389``reset_interval_ms``.  Setting both ``ms`` and ``bytes`` zero disables the
390quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is
391set.
392
393The time quota is internally transformed to a size quota.  Between the
394transformed size quota and user-specified size quota, smaller one is applied.
395Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the
396effective size quota is further adjusted.  Reading ``effective_bytes`` returns
397the current effective size quota.  The file is not updated in real time, so
398users should ask DAMON sysfs interface to update the content of the file for
399the stats by writing a special keyword, ``update_schemes_effective_quotas`` to
400the relevant ``kdamonds/<N>/state`` file.
401
402Under ``weights`` directory, three files (``sz_permil``,
403``nr_accesses_permil``, and ``age_permil``) exist.
404You can set the :ref:`prioritization weights
405<damon_design_damos_quotas_prioritization>` for size, access frequency, and age
406in per-thousand unit by writing the values to the three files under the
407``weights`` directory.
408
409.. _sysfs_schemes_quota_goals:
410
411schemes/<N>/quotas/goals/
412-------------------------
413
414The directory for the :ref:`automatic quota tuning goals
415<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation
416scheme.
417
418In the beginning, this directory has only one file, ``nr_goals``.  Writing a
419number (``N``) to the file creates the number of child directories named ``0``
420to ``N-1``.  Each directory represents each goal and current achievement.
421Among the multiple feedback, the best one is used.
422
423Each goal directory contains five files, namely ``target_metric``,
424``target_value``, ``current_value`` ``nid`` and ``path``.  Users can set and
425get the five parameters for the quota auto-tuning goals that specified on the
426:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and
427reading from each of the files.  Note that users should further write
428``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond
429directory <sysfs_kdamond>` to pass the feedback to DAMON.
430
431.. _sysfs_watermarks:
432
433schemes/<N>/watermarks/
434-----------------------
435
436The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the
437given DAMON-based operation scheme.
438
439Under the watermarks directory, five files (``metric``, ``interval_us``,
440``high``, ``mid``, and ``low``) for setting the metric, the time interval
441between check of the metric, and the three watermarks exist.  You can set and
442get the five values by writing to the files, respectively.
443
444Keywords and meanings of those that can be written to the ``metric`` file are
445as below.
446
447 - none: Ignore the watermarks
448 - free_mem_rate: System's free memory rate (per thousand)
449
450The ``interval`` should written in microseconds unit.
451
452.. _sysfs_filters:
453
454schemes/<N>/{core\_,ops\_,}filters/
455-----------------------------------
456
457Directories for :ref:`filters <damon_design_damos_filters>` of the given
458DAMON-based operation scheme.
459
460``core_filters`` and ``ops_filters`` directories are for the filters handled by
461the DAMON core layer and operations set layer, respectively.  ``filters``
462directory can be used for installing filters regardless of their handled
463layers.  Filters that requested by ``core_filters`` and ``ops_filters`` will be
464installed before those of ``filters``.  All three directories have same files.
465
466Use of ``filters`` directory can make expecting evaluation orders of given
467filters with the files under directory bit confusing.  Users are hence
468recommended to use ``core_filters`` and ``ops_filters`` directories.  The
469``filters`` directory could be deprecated in future.
470
471In the beginning, the directory has only one file, ``nr_filters``.  Writing a
472number (``N``) to the file creates the number of child directories named ``0``
473to ``N-1``.  Each directory represents each filter.  The filters are evaluated
474in the numeric order.
475
476Each filter directory contains nine files, namely ``type``, ``matching``,
477``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max``
478and ``target_idx``.  To ``type`` file, you can write the type of the filter.
479Refer to :ref:`the design doc <damon_design_damos_filters>` for available type
480names, their meaning and on what layer those are handled.
481
482For ``memcg`` type, you can specify the memory cgroup of the interest by
483writing the path of the memory cgroup from the cgroups mount point to
484``memcg_path`` file.  For ``addr`` type, you can specify the start and end
485address of the range (open-ended interval) to ``addr_start`` and ``addr_end``
486files, respectively.  For ``hugepage_size`` type, you can specify the minimum
487and maximum size of the range (closed interval) to ``min`` and ``max`` files,
488respectively.  For ``target`` type, you can specify the index of the target
489between the list of the DAMON context's monitoring targets list to
490``target_idx`` file.
491
492You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter
493is for memory that matches the ``type``.  You can write ``Y`` or ``N`` to
494``allow`` file to specify if applying the action to the memory that satisfies
495the ``type`` and ``matching`` should be allowed or not.
496
497For example, below restricts a DAMOS action to be applied to only non-anonymous
498pages of all memory cgroups except ``/having_care_already``.::
499
500    # cd ops_filters/0/
501    # echo 2 > nr_filters
502    # # disallow anonymous pages
503    echo anon > 0/type
504    echo Y > 0/matching
505    echo N > 0/allow
506    # # further filter out all cgroups except one at '/having_care_already'
507    echo memcg > 1/type
508    echo /having_care_already > 1/memcg_path
509    echo Y > 1/matching
510    echo N > 1/allow
511
512Refer to the :ref:`DAMOS filters design documentation
513<damon_design_damos_filters>` for more details including how multiple filters
514of different ``allow`` works, when each of the filters are supported, and
515differences on stats.
516
517.. _damon_sysfs_dests:
518
519schemes/<N>/dests/
520------------------
521
522Directory for specifying the destinations of given DAMON-based operation
523scheme's action.  This directory is ignored if the action of the given scheme
524is not supporting multiple destinations.  Only ``DAMOS_MIGRATE_{HOT,COLD}``
525actions are supporting multiple destinations.
526
527In the beginning, the directory has only one file, ``nr_dests``.  Writing a
528number (``N``) to the file creates the number of child directories named ``0``
529to ``N-1``.  Each directory represents each action destination.
530
531Each destination directory contains two files, namely ``id`` and ``weight``.
532Users can write and read the identifier of the destination to ``id`` file.
533For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node
534id should be written to ``id`` file.  Users can write and read the weight of
535the destination among the given destinations to the ``weight`` file.  The
536weight can be an arbitrary integer.  When DAMOS apply the action to each entity
537of the memory region, it will select the destination of the action based on the
538relative weights of the destinations.
539
540.. _sysfs_schemes_stats:
541
542schemes/<N>/stats/
543------------------
544
545DAMON counts statistics for each scheme.  This statistics can be used for
546online analysis or tuning of the schemes.  Refer to :ref:`design doc
547<damon_design_damos_stat>` for more details about the stats.
548
549The statistics can be retrieved by reading the files under ``stats`` directory
550(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``,
551``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and
552``max_nr_snapshots``), respectively.
553
554The files are not updated in real time by default.  Users should ask DAMON
555sysfs interface to periodically update those using ``refresh_ms``, or do a one
556time update by writing a special keyword, ``update_schemes_stats`` to the
557relevant ``kdamonds/<N>/state`` file.  Refer to :ref:`kdamond directory
558<sysfs_kdamond>` for more details.
559
560.. _sysfs_schemes_tried_regions:
561
562schemes/<N>/tried_regions/
563--------------------------
564
565This directory initially has one file, ``total_bytes``.
566
567When a special keyword, ``update_schemes_tried_regions``, is written to the
568relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so
569that reading it returns the total size of the scheme tried regions, and creates
570directories named integer starting from ``0`` under this directory.  Each
571directory contains files exposing detailed information about each of the memory
572region that the corresponding scheme's ``action`` has tried to be applied under
573this directory, during next :ref:`apply interval <damon_design_damos>` of the
574corresponding scheme.  The information includes address range, ``nr_accesses``,
575and ``age`` of the region.
576
577Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state``
578file will only update the ``total_bytes`` file, and will not create the
579subdirectories.
580
581The directories will be removed when another special keyword,
582``clear_schemes_tried_regions``, is written to the relevant
583``kdamonds/<N>/state`` file.
584
585The expected usage of this directory is investigations of schemes' behaviors,
586and query-like efficient data access monitoring results retrievals.  For the
587latter use case, in particular, users can set the ``action`` as ``stat`` and
588set the ``access pattern`` as their interested pattern that they want to query.
589
590.. _sysfs_schemes_tried_region:
591
592tried_regions/<N>/
593------------------
594
595In each region directory, you will find five files (``start``, ``end``,
596``nr_accesses``, ``age``, and ``sz_filter_passed``).  Reading the files will
597show the properties of the region that corresponding DAMON-based operation
598scheme ``action`` has tried to be applied.
599
600Example
601~~~~~~~
602
603Below commands applies a scheme saying "If a memory region of size in [4KiB,
6048KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
605interval in [10, 20], page out the region.  For the paging out, use only up to
60610ms per second, and also don't page out more than 1GiB per second.  Under the
607limitation, page out memory regions having longer age first.  Also, check the
608free memory rate of the system every 5 seconds, start the monitoring and paging
609out when the free memory rate becomes lower than 50%, but stop it if the free
610memory rate becomes larger than 60%, or lower than 30%". ::
611
612    # cd <sysfs>/kernel/mm/damon/admin
613    # # populate directories
614    # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts;
615    # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes
616    # cd kdamonds/0/contexts/0/schemes/0
617    # # set the basic access pattern and the action
618    # echo 4096 > access_pattern/sz/min
619    # echo 8192 > access_pattern/sz/max
620    # echo 0 > access_pattern/nr_accesses/min
621    # echo 5 > access_pattern/nr_accesses/max
622    # echo 10 > access_pattern/age/min
623    # echo 20 > access_pattern/age/max
624    # echo pageout > action
625    # # set quotas
626    # echo 10 > quotas/ms
627    # echo $((1024*1024*1024)) > quotas/bytes
628    # echo 1000 > quotas/reset_interval_ms
629    # # set watermark
630    # echo free_mem_rate > watermarks/metric
631    # echo 5000000 > watermarks/interval_us
632    # echo 600 > watermarks/high
633    # echo 500 > watermarks/mid
634    # echo 300 > watermarks/low
635
636Please note that it's highly recommended to use user space tools like `damo
637<https://github.com/damonitor/damo>`_ rather than manually reading and writing
638the files as above.  Above is only for an example.
639
640.. _tracepoint:
641
642Tracepoints for Monitoring Results
643==================================
644
645Users can get the monitoring results via the :ref:`tried_regions
646<sysfs_schemes_tried_regions>`.  The interface is useful for getting a
647snapshot, but it could be inefficient for fully recording all the monitoring
648results.  For the purpose, two trace points, namely ``damon:damon_aggregated``
649and ``damon:damos_before_apply``, are provided.  ``damon:damon_aggregated``
650provides the whole monitoring results, while ``damon:damos_before_apply``
651provides the monitoring results for regions that each DAMON-based Operation
652Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied.  Hence,
653``damon:damos_before_apply`` is more useful for recording internal behavior of
654DAMOS, or DAMOS target access
655:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient
656monitoring results recording.
657
658While the monitoring is turned on, you could record the tracepoint events and
659show results using tracepoint supporting tools like ``perf``.  For example::
660
661    # echo on > kdamonds/0/state
662    # perf record -e damon:damon_aggregated &
663    # sleep 5
664    # kill 9 $(pidof perf)
665    # echo off > kdamonds/0/state
666    # perf script
667    kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864
668    [...]
669
670Each line of the perf script output represents each monitoring region.  The
671first five fields are as usual other tracepoint outputs.  The sixth field
672(``target_id=X``) shows the ide of the monitoring target of the region.  The
673seventh field (``nr_regions=X``) shows the total number of monitoring regions
674for the target.  The eighth field (``X-Y:``) shows the start (``X``) and end
675(``Y``) addresses of the region in bytes.  The ninth field (``X``) shows the
676``nr_accesses`` of the region (refer to
677:ref:`design <damon_design_region_based_sampling>` for more details of the
678counter).  Finally the tenth field (``X``) shows the ``age`` of the region
679(refer to :ref:`design <damon_design_age_tracking>` for more details of the
680counter).
681
682If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would
683be somewhat like below::
684
685    kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136
686    [...]
687
688Each line of the output represents each monitoring region that each DAMON-based
689Operation Scheme was about to be applied at the traced time.  The first five
690fields are as usual.  It shows the index of the DAMON context (``ctx_idx=X``)
691of the scheme in the list of the contexts of the context's kdamond, the index
692of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in
693addition to the output of ``damon_aggregated`` tracepoint.
694