xref: /linux/Documentation/filesystems/caching/fscache.rst (revision 170aafe35cb98e0f3fbacb446ea86389fbce22ea)
1.. SPDX-License-Identifier: GPL-2.0
2
3==========================
4General Filesystem Caching
5==========================
6
7Overview
8========
9
10This facility is a general purpose cache for network filesystems, though it
11could be used for caching other things such as ISO9660 filesystems too.
12
13FS-Cache mediates between cache backends (such as CacheFiles) and network
14filesystems::
15
16	+---------+
17	|         |                                    +--------------+
18	|   NFS   |--+                                 |              |
19	|         |  |                             +-->|   CacheFS    |
20	+---------+  |               +----------+  |   |  /dev/hda5   |
21	             |               |          |  |   +--------------+
22	+---------+  +-------------->|          |  |
23	|         |      +-------+   |          |--+
24	|   AFS   |----->|       |   | FS-Cache |
25	|         |      | netfs |-->|          |--+
26	+---------+  +-->|  lib  |   |          |  |
27	             |   |       |   |          |  |   +--------------+
28	+---------+  |   +-------+   +----------+  |   |              |
29	|         |  |                             +-->|  CacheFiles  |
30	|   9P    |--+                                 |  /var/cache  |
31	|         |                                    +--------------+
32	+---------+
33
34Or to look at it another way, FS-Cache is a module that provides a caching
35facility to a network filesystem such that the cache is transparent to the
36user::
37
38	+---------+
39	|         |
40	| Server  |
41	|         |
42	+---------+
43	     |                  NETWORK
44	~~~~~|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
45	     |
46	     |           +----------+
47	     V           |          |
48	+---------+      |          |
49	|         |      |          |
50	|   NFS   |----->| FS-Cache |
51	|         |      |          |--+
52	+---------+      |          |  |   +--------------+   +--------------+
53	     |           |          |  |   |              |   |              |
54	     V           +----------+  +-->|  CacheFiles  |-->|  Ext3        |
55	+---------+                        |  /var/cache  |   |  /dev/sda6   |
56	|         |                        +--------------+   +--------------+
57	|   VFS   |                                ^                     ^
58	|         |                                |                     |
59	+---------+                                +--------------+      |
60	     |                  KERNEL SPACE                      |      |
61	~~~~~|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|~~~~~~|~~~~
62	     |                  USER SPACE                        |      |
63	     V                                                    |      |
64	+---------+                                           +--------------+
65	|         |                                           |              |
66	| Process |                                           | cachefilesd  |
67	|         |                                           |              |
68	+---------+                                           +--------------+
69
70
71FS-Cache does not follow the idea of completely loading every netfs file
72opened in its entirety into a cache before permitting it to be accessed and
73then serving the pages out of that cache rather than the netfs inode because:
74
75 (1) It must be practical to operate without a cache.
76
77 (2) The size of any accessible file must not be limited to the size of the
78     cache.
79
80 (3) The combined size of all opened files (this includes mapped libraries)
81     must not be limited to the size of the cache.
82
83 (4) The user should not be forced to download an entire file just to do a
84     one-off access of a small portion of it (such as might be done with the
85     "file" program).
86
87It instead serves the cache out in chunks as and when requested by the netfs
88using it.
89
90
91FS-Cache provides the following facilities:
92
93   * More than one cache can be used at once.  Caches can be selected
94     explicitly by use of tags.
95
96   * Caches can be added / removed at any time, even whilst being accessed.
97
98   * The netfs is provided with an interface that allows either party to
99     withdraw caching facilities from a file (required for (2)).
100
101   * The interface to the netfs returns as few errors as possible, preferring
102     rather to let the netfs remain oblivious.
103
104   * There are three types of cookie: cache, volume and data file cookies.
105     Cache cookies represent the cache as a whole and are not normally visible
106     to the netfs; the netfs gets a volume cookie to represent a collection of
107     files (typically something that a netfs would get for a superblock); and
108     data file cookies are used to cache data (something that would be got for
109     an inode).
110
111   * Volumes are matched using a key.  This is a printable string that is used
112     to encode all the information that might be needed to distinguish one
113     superblock, say, from another.  This would be a compound of things like
114     cell name or server address, volume name or share path.  It must be a
115     valid pathname.
116
117   * Cookies are matched using a key.  This is a binary blob and is used to
118     represent the object within a volume (so the volume key need not form
119     part of the blob).  This might include things like an inode number and
120     uniquifier or a file handle.
121
122   * Cookie resources are set up and pinned by marking the cookie in-use.
123     This prevents the backing resources from being culled.  Timed garbage
124     collection is employed to eliminate cookies that haven't been used for a
125     short while, thereby reducing resource overload.  This is intended to be
126     used when a file is opened or closed.
127
128     A cookie can be marked in-use multiple times simultaneously; each mark
129     must be unused.
130
131   * Begin/end access functions are provided to delay cache withdrawal for the
132     duration of an operation and prevent structs from being freed whilst
133     we're looking at them.
134
135   * Data I/O is done by asynchronous DIO to/from a buffer described by the
136     netfs using an iov_iter.
137
138   * An invalidation facility is available to discard data from the cache and
139     to deal with I/O that's in progress that is accessing old data.
140
141   * Cookies can be "retired" upon release, thereby causing the object to be
142     removed from the cache.
143
144
145The netfs API to FS-Cache can be found in:
146
147	Documentation/filesystems/caching/netfs-api.rst
148
149The cache backend API to FS-Cache can be found in:
150
151	Documentation/filesystems/caching/backend-api.rst
152
153
154Statistical Information
155=======================
156
157If FS-Cache is compiled with the following options enabled::
158
159	CONFIG_FSCACHE_STATS=y
160
161then it will gather certain statistics and display them through:
162
163	/proc/fs/fscache/stats
164
165This shows counts of a number of events that can happen in FS-Cache:
166
167+--------------+-------+-------------------------------------------------------+
168|CLASS         |EVENT  |MEANING                                                |
169+==============+=======+=======================================================+
170|Cookies       |n=N    |Number of data storage cookies allocated               |
171+              +-------+-------------------------------------------------------+
172|              |v=N    |Number of volume index cookies allocated               |
173+              +-------+-------------------------------------------------------+
174|              |vcol=N |Number of volume index key collisions                  |
175+              +-------+-------------------------------------------------------+
176|              |voom=N |Number of OOM events when allocating volume cookies    |
177+--------------+-------+-------------------------------------------------------+
178|Acquire       |n=N    |Number of acquire cookie requests seen                 |
179+              +-------+-------------------------------------------------------+
180|              |ok=N   |Number of acq reqs succeeded                           |
181+              +-------+-------------------------------------------------------+
182|              |oom=N  |Number of acq reqs failed on ENOMEM                    |
183+--------------+-------+-------------------------------------------------------+
184|LRU           |n=N    |Number of cookies currently on the LRU                 |
185+              +-------+-------------------------------------------------------+
186|              |exp=N  |Number of cookies expired off of the LRU               |
187+              +-------+-------------------------------------------------------+
188|              |rmv=N  |Number of cookies removed from the LRU                 |
189+              +-------+-------------------------------------------------------+
190|              |drp=N  |Number of LRU'd cookies relinquished/withdrawn         |
191+              +-------+-------------------------------------------------------+
192|              |at=N   |Time till next LRU cull (jiffies)                      |
193+--------------+-------+-------------------------------------------------------+
194|Invals        |n=N    |Number of invalidations                                |
195+--------------+-------+-------------------------------------------------------+
196|Updates       |n=N    |Number of update cookie requests seen                  |
197+              +-------+-------------------------------------------------------+
198|              |rsz=N  |Number of resize requests                              |
199+              +-------+-------------------------------------------------------+
200|              |rsn=N  |Number of skipped resize requests                      |
201+--------------+-------+-------------------------------------------------------+
202|Relinqs       |n=N    |Number of relinquish cookie requests seen              |
203+              +-------+-------------------------------------------------------+
204|              |rtr=N  |Number of rlq reqs with retire=true                    |
205+              +-------+-------------------------------------------------------+
206|              |drop=N |Number of cookies no longer blocking re-acquisition    |
207+--------------+-------+-------------------------------------------------------+
208|NoSpace       |nwr=N  |Number of write requests refused due to lack of space  |
209+              +-------+-------------------------------------------------------+
210|              |ncr=N  |Number of create requests refused due to lack of space |
211+              +-------+-------------------------------------------------------+
212|              |cull=N |Number of objects culled to make space                 |
213+--------------+-------+-------------------------------------------------------+
214|IO            |rd=N   |Number of read operations in the cache                 |
215+              +-------+-------------------------------------------------------+
216|              |wr=N   |Number of write operations in the cache                |
217+--------------+-------+-------------------------------------------------------+
218
219Netfslib will also add some stats counters of its own.
220
221
222Cache List
223==========
224
225FS-Cache provides a list of cache cookies:
226
227	/proc/fs/fscache/cookies
228
229This will look something like::
230
231	# cat /proc/fs/fscache/caches
232	CACHE    REF   VOLS  OBJS  ACCES S NAME
233	======== ===== ===== ===== ===== = ===============
234	00000001     2     1  2123     1 A default
235
236where the columns are:
237
238	=======	===============================================================
239	COLUMN	DESCRIPTION
240	=======	===============================================================
241	CACHE	Cache cookie debug ID (also appears in traces)
242	REF	Number of references on the cache cookie
243	VOLS	Number of volumes cookies in this cache
244	OBJS	Number of cache objects in use
245	ACCES	Number of accesses pinning the cache
246	S	State
247	NAME	Name of the cache.
248	=======	===============================================================
249
250The state can be (-) Inactive, (P)reparing, (A)ctive, (E)rror or (W)ithdrawing.
251
252
253Volume List
254===========
255
256FS-Cache provides a list of volume cookies:
257
258	/proc/fs/fscache/volumes
259
260This will look something like::
261
262	VOLUME   REF   nCOOK ACC FL CACHE           KEY
263	======== ===== ===== === == =============== ================
264	00000001    55    54   1 00 default         afs,example.com,100058
265
266where the columns are:
267
268	=======	===============================================================
269	COLUMN	DESCRIPTION
270	=======	===============================================================
271	VOLUME	The volume cookie debug ID (also appears in traces)
272	REF	Number of references on the volume cookie
273	nCOOK	Number of cookies in the volume
274	ACC	Number of accesses pinning the cache
275	FL	Flags on the volume cookie
276	CACHE	Name of the cache or "-"
277	KEY	The indexing key for the volume
278	=======	===============================================================
279
280
281Cookie List
282===========
283
284FS-Cache provides a list of cookies:
285
286	/proc/fs/fscache/cookies
287
288This will look something like::
289
290	# head /proc/fs/fscache/cookies
291	COOKIE   VOLUME   REF ACT ACC S FL DEF
292	======== ======== === === === = == ================
293	00000435 00000001   1   0  -1 - 08 0000000201d080070000000000000000, 0000000000000000
294	00000436 00000001   1   0  -1 - 00 0000005601d080080000000000000000, 0000000000000051
295	00000437 00000001   1   0  -1 - 08 00023b3001d0823f0000000000000000, 0000000000000000
296	00000438 00000001   1   0  -1 - 08 0000005801d0807b0000000000000000, 0000000000000000
297	00000439 00000001   1   0  -1 - 08 00023b3201d080a10000000000000000, 0000000000000000
298	0000043a 00000001   1   0  -1 - 08 00023b3401d080a30000000000000000, 0000000000000000
299	0000043b 00000001   1   0  -1 - 08 00023b3601d080b30000000000000000, 0000000000000000
300	0000043c 00000001   1   0  -1 - 08 00023b3801d080b40000000000000000, 0000000000000000
301
302where the columns are:
303
304	=======	===============================================================
305	COLUMN	DESCRIPTION
306	=======	===============================================================
307	COOKIE	The cookie debug ID (also appears in traces)
308	VOLUME	The parent volume cookie debug ID
309	REF	Number of references on the volume cookie
310	ACT	Number of times the cookie is marked for in use
311	ACC	Number of access pins in the cookie
312	S	State of the cookie
313	FL	Flags on the cookie
314	DEF	Key, auxiliary data
315	=======	===============================================================
316
317
318Debugging
319=========
320
321If CONFIG_NETFS_DEBUG is enabled, the FS-Cache facility and NETFS support can
322have runtime debugging enabled by adjusting the value in::
323
324	/sys/module/netfs/parameters/debug
325
326This is a bitmask of debugging streams to enable:
327
328	=======	=======	===============================	=======================
329	BIT	VALUE	STREAM				POINT
330	=======	=======	===============================	=======================
331	0	1	Cache management		Function entry trace
332	1	2					Function exit trace
333	2	4					General
334	3	8	Cookie management		Function entry trace
335	4	16					Function exit trace
336	5	32					General
337	6-8						(Not used)
338	9	512	I/O operation management	Function entry trace
339	10	1024					Function exit trace
340	11	2048					General
341	=======	=======	===============================	=======================
342
343The appropriate set of values should be OR'd together and the result written to
344the control file.  For example::
345
346	echo $((1|8|512)) >/sys/module/netfs/parameters/debug
347
348will turn on all function entry debugging.
349