xref: /linux/Documentation/filesystems/f2fs.rst (revision 89272ca1102e000f7dbca724b7b106e688199a5d)
1*89272ca1SMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0
2*89272ca1SMauro Carvalho Chehab
3*89272ca1SMauro Carvalho Chehab==========================================
4*89272ca1SMauro Carvalho ChehabWHAT IS Flash-Friendly File System (F2FS)?
5*89272ca1SMauro Carvalho Chehab==========================================
6*89272ca1SMauro Carvalho Chehab
7*89272ca1SMauro Carvalho ChehabNAND flash memory-based storage devices, such as SSD, eMMC, and SD cards, have
8*89272ca1SMauro Carvalho Chehabbeen equipped on a variety systems ranging from mobile to server systems. Since
9*89272ca1SMauro Carvalho Chehabthey are known to have different characteristics from the conventional rotating
10*89272ca1SMauro Carvalho Chehabdisks, a file system, an upper layer to the storage device, should adapt to the
11*89272ca1SMauro Carvalho Chehabchanges from the sketch in the design level.
12*89272ca1SMauro Carvalho Chehab
13*89272ca1SMauro Carvalho ChehabF2FS is a file system exploiting NAND flash memory-based storage devices, which
14*89272ca1SMauro Carvalho Chehabis based on Log-structured File System (LFS). The design has been focused on
15*89272ca1SMauro Carvalho Chehabaddressing the fundamental issues in LFS, which are snowball effect of wandering
16*89272ca1SMauro Carvalho Chehabtree and high cleaning overhead.
17*89272ca1SMauro Carvalho Chehab
18*89272ca1SMauro Carvalho ChehabSince a NAND flash memory-based storage device shows different characteristic
19*89272ca1SMauro Carvalho Chehabaccording to its internal geometry or flash memory management scheme, namely FTL,
20*89272ca1SMauro Carvalho ChehabF2FS and its tools support various parameters not only for configuring on-disk
21*89272ca1SMauro Carvalho Chehablayout, but also for selecting allocation and cleaning algorithms.
22*89272ca1SMauro Carvalho Chehab
23*89272ca1SMauro Carvalho ChehabThe following git tree provides the file system formatting tool (mkfs.f2fs),
24*89272ca1SMauro Carvalho Chehaba consistency checking tool (fsck.f2fs), and a debugging tool (dump.f2fs).
25*89272ca1SMauro Carvalho Chehab
26*89272ca1SMauro Carvalho Chehab- git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git
27*89272ca1SMauro Carvalho Chehab
28*89272ca1SMauro Carvalho ChehabFor reporting bugs and sending patches, please use the following mailing list:
29*89272ca1SMauro Carvalho Chehab
30*89272ca1SMauro Carvalho Chehab- linux-f2fs-devel@lists.sourceforge.net
31*89272ca1SMauro Carvalho Chehab
32*89272ca1SMauro Carvalho ChehabBackground and Design issues
33*89272ca1SMauro Carvalho Chehab============================
34*89272ca1SMauro Carvalho Chehab
35*89272ca1SMauro Carvalho ChehabLog-structured File System (LFS)
36*89272ca1SMauro Carvalho Chehab--------------------------------
37*89272ca1SMauro Carvalho Chehab"A log-structured file system writes all modifications to disk sequentially in
38*89272ca1SMauro Carvalho Chehaba log-like structure, thereby speeding up  both file writing and crash recovery.
39*89272ca1SMauro Carvalho ChehabThe log is the only structure on disk; it contains indexing information so that
40*89272ca1SMauro Carvalho Chehabfiles can be read back from the log efficiently. In order to maintain large free
41*89272ca1SMauro Carvalho Chehabareas on disk for fast writing, we divide  the log into segments and use a
42*89272ca1SMauro Carvalho Chehabsegment cleaner to compress the live information from heavily fragmented
43*89272ca1SMauro Carvalho Chehabsegments." from Rosenblum, M. and Ousterhout, J. K., 1992, "The design and
44*89272ca1SMauro Carvalho Chehabimplementation of a log-structured file system", ACM Trans. Computer Systems
45*89272ca1SMauro Carvalho Chehab10, 1, 26–52.
46*89272ca1SMauro Carvalho Chehab
47*89272ca1SMauro Carvalho ChehabWandering Tree Problem
48*89272ca1SMauro Carvalho Chehab----------------------
49*89272ca1SMauro Carvalho ChehabIn LFS, when a file data is updated and written to the end of log, its direct
50*89272ca1SMauro Carvalho Chehabpointer block is updated due to the changed location. Then the indirect pointer
51*89272ca1SMauro Carvalho Chehabblock is also updated due to the direct pointer block update. In this manner,
52*89272ca1SMauro Carvalho Chehabthe upper index structures such as inode, inode map, and checkpoint block are
53*89272ca1SMauro Carvalho Chehabalso updated recursively. This problem is called as wandering tree problem [1],
54*89272ca1SMauro Carvalho Chehaband in order to enhance the performance, it should eliminate or relax the update
55*89272ca1SMauro Carvalho Chehabpropagation as much as possible.
56*89272ca1SMauro Carvalho Chehab
57*89272ca1SMauro Carvalho Chehab[1] Bityutskiy, A. 2005. JFFS3 design issues. http://www.linux-mtd.infradead.org/
58*89272ca1SMauro Carvalho Chehab
59*89272ca1SMauro Carvalho ChehabCleaning Overhead
60*89272ca1SMauro Carvalho Chehab-----------------
61*89272ca1SMauro Carvalho ChehabSince LFS is based on out-of-place writes, it produces so many obsolete blocks
62*89272ca1SMauro Carvalho Chehabscattered across the whole storage. In order to serve new empty log space, it
63*89272ca1SMauro Carvalho Chehabneeds to reclaim these obsolete blocks seamlessly to users. This job is called
64*89272ca1SMauro Carvalho Chehabas a cleaning process.
65*89272ca1SMauro Carvalho Chehab
66*89272ca1SMauro Carvalho ChehabThe process consists of three operations as follows.
67*89272ca1SMauro Carvalho Chehab
68*89272ca1SMauro Carvalho Chehab1. A victim segment is selected through referencing segment usage table.
69*89272ca1SMauro Carvalho Chehab2. It loads parent index structures of all the data in the victim identified by
70*89272ca1SMauro Carvalho Chehab   segment summary blocks.
71*89272ca1SMauro Carvalho Chehab3. It checks the cross-reference between the data and its parent index structure.
72*89272ca1SMauro Carvalho Chehab4. It moves valid data selectively.
73*89272ca1SMauro Carvalho Chehab
74*89272ca1SMauro Carvalho ChehabThis cleaning job may cause unexpected long delays, so the most important goal
75*89272ca1SMauro Carvalho Chehabis to hide the latencies to users. And also definitely, it should reduce the
76*89272ca1SMauro Carvalho Chehabamount of valid data to be moved, and move them quickly as well.
77*89272ca1SMauro Carvalho Chehab
78*89272ca1SMauro Carvalho ChehabKey Features
79*89272ca1SMauro Carvalho Chehab============
80*89272ca1SMauro Carvalho Chehab
81*89272ca1SMauro Carvalho ChehabFlash Awareness
82*89272ca1SMauro Carvalho Chehab---------------
83*89272ca1SMauro Carvalho Chehab- Enlarge the random write area for better performance, but provide the high
84*89272ca1SMauro Carvalho Chehab  spatial locality
85*89272ca1SMauro Carvalho Chehab- Align FS data structures to the operational units in FTL as best efforts
86*89272ca1SMauro Carvalho Chehab
87*89272ca1SMauro Carvalho ChehabWandering Tree Problem
88*89272ca1SMauro Carvalho Chehab----------------------
89*89272ca1SMauro Carvalho Chehab- Use a term, “node”, that represents inodes as well as various pointer blocks
90*89272ca1SMauro Carvalho Chehab- Introduce Node Address Table (NAT) containing the locations of all the “node”
91*89272ca1SMauro Carvalho Chehab  blocks; this will cut off the update propagation.
92*89272ca1SMauro Carvalho Chehab
93*89272ca1SMauro Carvalho ChehabCleaning Overhead
94*89272ca1SMauro Carvalho Chehab-----------------
95*89272ca1SMauro Carvalho Chehab- Support a background cleaning process
96*89272ca1SMauro Carvalho Chehab- Support greedy and cost-benefit algorithms for victim selection policies
97*89272ca1SMauro Carvalho Chehab- Support multi-head logs for static/dynamic hot and cold data separation
98*89272ca1SMauro Carvalho Chehab- Introduce adaptive logging for efficient block allocation
99*89272ca1SMauro Carvalho Chehab
100*89272ca1SMauro Carvalho ChehabMount Options
101*89272ca1SMauro Carvalho Chehab=============
102*89272ca1SMauro Carvalho Chehab
103*89272ca1SMauro Carvalho Chehab
104*89272ca1SMauro Carvalho Chehab====================== ============================================================
105*89272ca1SMauro Carvalho Chehabbackground_gc=%s       Turn on/off cleaning operations, namely garbage
106*89272ca1SMauro Carvalho Chehab                       collection, triggered in background when I/O subsystem is
107*89272ca1SMauro Carvalho Chehab                       idle. If background_gc=on, it will turn on the garbage
108*89272ca1SMauro Carvalho Chehab                       collection and if background_gc=off, garbage collection
109*89272ca1SMauro Carvalho Chehab                       will be turned off. If background_gc=sync, it will turn
110*89272ca1SMauro Carvalho Chehab                       on synchronous garbage collection running in background.
111*89272ca1SMauro Carvalho Chehab                       Default value for this option is on. So garbage
112*89272ca1SMauro Carvalho Chehab                       collection is on by default.
113*89272ca1SMauro Carvalho Chehabdisable_roll_forward   Disable the roll-forward recovery routine
114*89272ca1SMauro Carvalho Chehabnorecovery             Disable the roll-forward recovery routine, mounted read-
115*89272ca1SMauro Carvalho Chehab                       only (i.e., -o ro,disable_roll_forward)
116*89272ca1SMauro Carvalho Chehabdiscard/nodiscard      Enable/disable real-time discard in f2fs, if discard is
117*89272ca1SMauro Carvalho Chehab                       enabled, f2fs will issue discard/TRIM commands when a
118*89272ca1SMauro Carvalho Chehab		       segment is cleaned.
119*89272ca1SMauro Carvalho Chehabno_heap                Disable heap-style segment allocation which finds free
120*89272ca1SMauro Carvalho Chehab                       segments for data from the beginning of main area, while
121*89272ca1SMauro Carvalho Chehab		       for node from the end of main area.
122*89272ca1SMauro Carvalho Chehabnouser_xattr           Disable Extended User Attributes. Note: xattr is enabled
123*89272ca1SMauro Carvalho Chehab                       by default if CONFIG_F2FS_FS_XATTR is selected.
124*89272ca1SMauro Carvalho Chehabnoacl                  Disable POSIX Access Control List. Note: acl is enabled
125*89272ca1SMauro Carvalho Chehab                       by default if CONFIG_F2FS_FS_POSIX_ACL is selected.
126*89272ca1SMauro Carvalho Chehabactive_logs=%u         Support configuring the number of active logs. In the
127*89272ca1SMauro Carvalho Chehab                       current design, f2fs supports only 2, 4, and 6 logs.
128*89272ca1SMauro Carvalho Chehab                       Default number is 6.
129*89272ca1SMauro Carvalho Chehabdisable_ext_identify   Disable the extension list configured by mkfs, so f2fs
130*89272ca1SMauro Carvalho Chehab                       does not aware of cold files such as media files.
131*89272ca1SMauro Carvalho Chehabinline_xattr           Enable the inline xattrs feature.
132*89272ca1SMauro Carvalho Chehabnoinline_xattr         Disable the inline xattrs feature.
133*89272ca1SMauro Carvalho Chehabinline_xattr_size=%u   Support configuring inline xattr size, it depends on
134*89272ca1SMauro Carvalho Chehab		       flexible inline xattr feature.
135*89272ca1SMauro Carvalho Chehabinline_data            Enable the inline data feature: New created small(<~3.4k)
136*89272ca1SMauro Carvalho Chehab                       files can be written into inode block.
137*89272ca1SMauro Carvalho Chehabinline_dentry          Enable the inline dir feature: data in new created
138*89272ca1SMauro Carvalho Chehab                       directory entries can be written into inode block. The
139*89272ca1SMauro Carvalho Chehab                       space of inode block which is used to store inline
140*89272ca1SMauro Carvalho Chehab                       dentries is limited to ~3.4k.
141*89272ca1SMauro Carvalho Chehabnoinline_dentry        Disable the inline dentry feature.
142*89272ca1SMauro Carvalho Chehabflush_merge	       Merge concurrent cache_flush commands as much as possible
143*89272ca1SMauro Carvalho Chehab                       to eliminate redundant command issues. If the underlying
144*89272ca1SMauro Carvalho Chehab		       device handles the cache_flush command relatively slowly,
145*89272ca1SMauro Carvalho Chehab		       recommend to enable this option.
146*89272ca1SMauro Carvalho Chehabnobarrier              This option can be used if underlying storage guarantees
147*89272ca1SMauro Carvalho Chehab                       its cached data should be written to the novolatile area.
148*89272ca1SMauro Carvalho Chehab		       If this option is set, no cache_flush commands are issued
149*89272ca1SMauro Carvalho Chehab		       but f2fs still guarantees the write ordering of all the
150*89272ca1SMauro Carvalho Chehab		       data writes.
151*89272ca1SMauro Carvalho Chehabfastboot               This option is used when a system wants to reduce mount
152*89272ca1SMauro Carvalho Chehab                       time as much as possible, even though normal performance
153*89272ca1SMauro Carvalho Chehab		       can be sacrificed.
154*89272ca1SMauro Carvalho Chehabextent_cache           Enable an extent cache based on rb-tree, it can cache
155*89272ca1SMauro Carvalho Chehab                       as many as extent which map between contiguous logical
156*89272ca1SMauro Carvalho Chehab                       address and physical address per inode, resulting in
157*89272ca1SMauro Carvalho Chehab                       increasing the cache hit ratio. Set by default.
158*89272ca1SMauro Carvalho Chehabnoextent_cache         Disable an extent cache based on rb-tree explicitly, see
159*89272ca1SMauro Carvalho Chehab                       the above extent_cache mount option.
160*89272ca1SMauro Carvalho Chehabnoinline_data          Disable the inline data feature, inline data feature is
161*89272ca1SMauro Carvalho Chehab                       enabled by default.
162*89272ca1SMauro Carvalho Chehabdata_flush             Enable data flushing before checkpoint in order to
163*89272ca1SMauro Carvalho Chehab                       persist data of regular and symlink.
164*89272ca1SMauro Carvalho Chehabreserve_root=%d        Support configuring reserved space which is used for
165*89272ca1SMauro Carvalho Chehab                       allocation from a privileged user with specified uid or
166*89272ca1SMauro Carvalho Chehab                       gid, unit: 4KB, the default limit is 0.2% of user blocks.
167*89272ca1SMauro Carvalho Chehabresuid=%d              The user ID which may use the reserved blocks.
168*89272ca1SMauro Carvalho Chehabresgid=%d              The group ID which may use the reserved blocks.
169*89272ca1SMauro Carvalho Chehabfault_injection=%d     Enable fault injection in all supported types with
170*89272ca1SMauro Carvalho Chehab                       specified injection rate.
171*89272ca1SMauro Carvalho Chehabfault_type=%d          Support configuring fault injection type, should be
172*89272ca1SMauro Carvalho Chehab                       enabled with fault_injection option, fault type value
173*89272ca1SMauro Carvalho Chehab                       is shown below, it supports single or combined type.
174*89272ca1SMauro Carvalho Chehab
175*89272ca1SMauro Carvalho Chehab                       ===================	===========
176*89272ca1SMauro Carvalho Chehab                       Type_Name		Type_Value
177*89272ca1SMauro Carvalho Chehab                       ===================	===========
178*89272ca1SMauro Carvalho Chehab                       FAULT_KMALLOC		0x000000001
179*89272ca1SMauro Carvalho Chehab                       FAULT_KVMALLOC		0x000000002
180*89272ca1SMauro Carvalho Chehab                       FAULT_PAGE_ALLOC		0x000000004
181*89272ca1SMauro Carvalho Chehab                       FAULT_PAGE_GET		0x000000008
182*89272ca1SMauro Carvalho Chehab                       FAULT_ALLOC_BIO		0x000000010
183*89272ca1SMauro Carvalho Chehab                       FAULT_ALLOC_NID		0x000000020
184*89272ca1SMauro Carvalho Chehab                       FAULT_ORPHAN		0x000000040
185*89272ca1SMauro Carvalho Chehab                       FAULT_BLOCK		0x000000080
186*89272ca1SMauro Carvalho Chehab                       FAULT_DIR_DEPTH		0x000000100
187*89272ca1SMauro Carvalho Chehab                       FAULT_EVICT_INODE	0x000000200
188*89272ca1SMauro Carvalho Chehab                       FAULT_TRUNCATE		0x000000400
189*89272ca1SMauro Carvalho Chehab                       FAULT_READ_IO		0x000000800
190*89272ca1SMauro Carvalho Chehab                       FAULT_CHECKPOINT		0x000001000
191*89272ca1SMauro Carvalho Chehab                       FAULT_DISCARD		0x000002000
192*89272ca1SMauro Carvalho Chehab                       FAULT_WRITE_IO		0x000004000
193*89272ca1SMauro Carvalho Chehab                       ===================	===========
194*89272ca1SMauro Carvalho Chehabmode=%s                Control block allocation mode which supports "adaptive"
195*89272ca1SMauro Carvalho Chehab                       and "lfs". In "lfs" mode, there should be no random
196*89272ca1SMauro Carvalho Chehab                       writes towards main area.
197*89272ca1SMauro Carvalho Chehabio_bits=%u             Set the bit size of write IO requests. It should be set
198*89272ca1SMauro Carvalho Chehab                       with "mode=lfs".
199*89272ca1SMauro Carvalho Chehabusrquota               Enable plain user disk quota accounting.
200*89272ca1SMauro Carvalho Chehabgrpquota               Enable plain group disk quota accounting.
201*89272ca1SMauro Carvalho Chehabprjquota               Enable plain project quota accounting.
202*89272ca1SMauro Carvalho Chehabusrjquota=<file>       Appoint specified file and type during mount, so that quota
203*89272ca1SMauro Carvalho Chehabgrpjquota=<file>       information can be properly updated during recovery flow,
204*89272ca1SMauro Carvalho Chehabprjjquota=<file>       <quota file>: must be in root directory;
205*89272ca1SMauro Carvalho Chehabjqfmt=<quota type>     <quota type>: [vfsold,vfsv0,vfsv1].
206*89272ca1SMauro Carvalho Chehaboffusrjquota           Turn off user journelled quota.
207*89272ca1SMauro Carvalho Chehaboffgrpjquota           Turn off group journelled quota.
208*89272ca1SMauro Carvalho Chehaboffprjjquota           Turn off project journelled quota.
209*89272ca1SMauro Carvalho Chehabquota                  Enable plain user disk quota accounting.
210*89272ca1SMauro Carvalho Chehabnoquota                Disable all plain disk quota option.
211*89272ca1SMauro Carvalho Chehabwhint_mode=%s          Control which write hints are passed down to block
212*89272ca1SMauro Carvalho Chehab                       layer. This supports "off", "user-based", and
213*89272ca1SMauro Carvalho Chehab                       "fs-based".  In "off" mode (default), f2fs does not pass
214*89272ca1SMauro Carvalho Chehab                       down hints. In "user-based" mode, f2fs tries to pass
215*89272ca1SMauro Carvalho Chehab                       down hints given by users. And in "fs-based" mode, f2fs
216*89272ca1SMauro Carvalho Chehab                       passes down hints with its policy.
217*89272ca1SMauro Carvalho Chehaballoc_mode=%s          Adjust block allocation policy, which supports "reuse"
218*89272ca1SMauro Carvalho Chehab                       and "default".
219*89272ca1SMauro Carvalho Chehabfsync_mode=%s          Control the policy of fsync. Currently supports "posix",
220*89272ca1SMauro Carvalho Chehab                       "strict", and "nobarrier". In "posix" mode, which is
221*89272ca1SMauro Carvalho Chehab                       default, fsync will follow POSIX semantics and does a
222*89272ca1SMauro Carvalho Chehab                       light operation to improve the filesystem performance.
223*89272ca1SMauro Carvalho Chehab                       In "strict" mode, fsync will be heavy and behaves in line
224*89272ca1SMauro Carvalho Chehab                       with xfs, ext4 and btrfs, where xfstest generic/342 will
225*89272ca1SMauro Carvalho Chehab                       pass, but the performance will regress. "nobarrier" is
226*89272ca1SMauro Carvalho Chehab                       based on "posix", but doesn't issue flush command for
227*89272ca1SMauro Carvalho Chehab                       non-atomic files likewise "nobarrier" mount option.
228*89272ca1SMauro Carvalho Chehabtest_dummy_encryption  Enable dummy encryption, which provides a fake fscrypt
229*89272ca1SMauro Carvalho Chehab                       context. The fake fscrypt context is used by xfstests.
230*89272ca1SMauro Carvalho Chehabcheckpoint=%s[:%u[%]]  Set to "disable" to turn off checkpointing. Set to "enable"
231*89272ca1SMauro Carvalho Chehab                       to reenable checkpointing. Is enabled by default. While
232*89272ca1SMauro Carvalho Chehab                       disabled, any unmounting or unexpected shutdowns will cause
233*89272ca1SMauro Carvalho Chehab                       the filesystem contents to appear as they did when the
234*89272ca1SMauro Carvalho Chehab                       filesystem was mounted with that option.
235*89272ca1SMauro Carvalho Chehab                       While mounting with checkpoint=disabled, the filesystem must
236*89272ca1SMauro Carvalho Chehab                       run garbage collection to ensure that all available space can
237*89272ca1SMauro Carvalho Chehab                       be used. If this takes too much time, the mount may return
238*89272ca1SMauro Carvalho Chehab                       EAGAIN. You may optionally add a value to indicate how much
239*89272ca1SMauro Carvalho Chehab                       of the disk you would be willing to temporarily give up to
240*89272ca1SMauro Carvalho Chehab                       avoid additional garbage collection. This can be given as a
241*89272ca1SMauro Carvalho Chehab                       number of blocks, or as a percent. For instance, mounting
242*89272ca1SMauro Carvalho Chehab                       with checkpoint=disable:100% would always succeed, but it may
243*89272ca1SMauro Carvalho Chehab                       hide up to all remaining free space. The actual space that
244*89272ca1SMauro Carvalho Chehab                       would be unusable can be viewed at /sys/fs/f2fs/<disk>/unusable
245*89272ca1SMauro Carvalho Chehab                       This space is reclaimed once checkpoint=enable.
246*89272ca1SMauro Carvalho Chehabcompress_algorithm=%s  Control compress algorithm, currently f2fs supports "lzo"
247*89272ca1SMauro Carvalho Chehab                       and "lz4" algorithm.
248*89272ca1SMauro Carvalho Chehabcompress_log_size=%u   Support configuring compress cluster size, the size will
249*89272ca1SMauro Carvalho Chehab                       be 4KB * (1 << %u), 16KB is minimum size, also it's
250*89272ca1SMauro Carvalho Chehab                       default size.
251*89272ca1SMauro Carvalho Chehabcompress_extension=%s  Support adding specified extension, so that f2fs can enable
252*89272ca1SMauro Carvalho Chehab                       compression on those corresponding files, e.g. if all files
253*89272ca1SMauro Carvalho Chehab                       with '.ext' has high compression rate, we can set the '.ext'
254*89272ca1SMauro Carvalho Chehab                       on compression extension list and enable compression on
255*89272ca1SMauro Carvalho Chehab                       these file by default rather than to enable it via ioctl.
256*89272ca1SMauro Carvalho Chehab                       For other files, we can still enable compression via ioctl.
257*89272ca1SMauro Carvalho Chehab====================== ============================================================
258*89272ca1SMauro Carvalho Chehab
259*89272ca1SMauro Carvalho ChehabDebugfs Entries
260*89272ca1SMauro Carvalho Chehab===============
261*89272ca1SMauro Carvalho Chehab
262*89272ca1SMauro Carvalho Chehab/sys/kernel/debug/f2fs/ contains information about all the partitions mounted as
263*89272ca1SMauro Carvalho Chehabf2fs. Each file shows the whole f2fs information.
264*89272ca1SMauro Carvalho Chehab
265*89272ca1SMauro Carvalho Chehab/sys/kernel/debug/f2fs/status includes:
266*89272ca1SMauro Carvalho Chehab
267*89272ca1SMauro Carvalho Chehab - major file system information managed by f2fs currently
268*89272ca1SMauro Carvalho Chehab - average SIT information about whole segments
269*89272ca1SMauro Carvalho Chehab - current memory footprint consumed by f2fs.
270*89272ca1SMauro Carvalho Chehab
271*89272ca1SMauro Carvalho ChehabSysfs Entries
272*89272ca1SMauro Carvalho Chehab=============
273*89272ca1SMauro Carvalho Chehab
274*89272ca1SMauro Carvalho ChehabInformation about mounted f2fs file systems can be found in
275*89272ca1SMauro Carvalho Chehab/sys/fs/f2fs.  Each mounted filesystem will have a directory in
276*89272ca1SMauro Carvalho Chehab/sys/fs/f2fs based on its device name (i.e., /sys/fs/f2fs/sda).
277*89272ca1SMauro Carvalho ChehabThe files in each per-device directory are shown in table below.
278*89272ca1SMauro Carvalho Chehab
279*89272ca1SMauro Carvalho ChehabFiles in /sys/fs/f2fs/<devname>
280*89272ca1SMauro Carvalho Chehab(see also Documentation/ABI/testing/sysfs-fs-f2fs)
281*89272ca1SMauro Carvalho Chehab
282*89272ca1SMauro Carvalho ChehabUsage
283*89272ca1SMauro Carvalho Chehab=====
284*89272ca1SMauro Carvalho Chehab
285*89272ca1SMauro Carvalho Chehab1. Download userland tools and compile them.
286*89272ca1SMauro Carvalho Chehab
287*89272ca1SMauro Carvalho Chehab2. Skip, if f2fs was compiled statically inside kernel.
288*89272ca1SMauro Carvalho Chehab   Otherwise, insert the f2fs.ko module::
289*89272ca1SMauro Carvalho Chehab
290*89272ca1SMauro Carvalho Chehab	# insmod f2fs.ko
291*89272ca1SMauro Carvalho Chehab
292*89272ca1SMauro Carvalho Chehab3. Create a directory trying to mount::
293*89272ca1SMauro Carvalho Chehab
294*89272ca1SMauro Carvalho Chehab	# mkdir /mnt/f2fs
295*89272ca1SMauro Carvalho Chehab
296*89272ca1SMauro Carvalho Chehab4. Format the block device, and then mount as f2fs::
297*89272ca1SMauro Carvalho Chehab
298*89272ca1SMauro Carvalho Chehab	# mkfs.f2fs -l label /dev/block_device
299*89272ca1SMauro Carvalho Chehab	# mount -t f2fs /dev/block_device /mnt/f2fs
300*89272ca1SMauro Carvalho Chehab
301*89272ca1SMauro Carvalho Chehabmkfs.f2fs
302*89272ca1SMauro Carvalho Chehab---------
303*89272ca1SMauro Carvalho ChehabThe mkfs.f2fs is for the use of formatting a partition as the f2fs filesystem,
304*89272ca1SMauro Carvalho Chehabwhich builds a basic on-disk layout.
305*89272ca1SMauro Carvalho Chehab
306*89272ca1SMauro Carvalho ChehabThe options consist of:
307*89272ca1SMauro Carvalho Chehab
308*89272ca1SMauro Carvalho Chehab===============    ===========================================================
309*89272ca1SMauro Carvalho Chehab``-l [label]``     Give a volume label, up to 512 unicode name.
310*89272ca1SMauro Carvalho Chehab``-a [0 or 1]``    Split start location of each area for heap-based allocation.
311*89272ca1SMauro Carvalho Chehab
312*89272ca1SMauro Carvalho Chehab                   1 is set by default, which performs this.
313*89272ca1SMauro Carvalho Chehab``-o [int]``       Set overprovision ratio in percent over volume size.
314*89272ca1SMauro Carvalho Chehab
315*89272ca1SMauro Carvalho Chehab                   5 is set by default.
316*89272ca1SMauro Carvalho Chehab``-s [int]``       Set the number of segments per section.
317*89272ca1SMauro Carvalho Chehab
318*89272ca1SMauro Carvalho Chehab                   1 is set by default.
319*89272ca1SMauro Carvalho Chehab``-z [int]``       Set the number of sections per zone.
320*89272ca1SMauro Carvalho Chehab
321*89272ca1SMauro Carvalho Chehab                   1 is set by default.
322*89272ca1SMauro Carvalho Chehab``-e [str]``       Set basic extension list. e.g. "mp3,gif,mov"
323*89272ca1SMauro Carvalho Chehab``-t [0 or 1]``    Disable discard command or not.
324*89272ca1SMauro Carvalho Chehab
325*89272ca1SMauro Carvalho Chehab                   1 is set by default, which conducts discard.
326*89272ca1SMauro Carvalho Chehab===============    ===========================================================
327*89272ca1SMauro Carvalho Chehab
328*89272ca1SMauro Carvalho Chehabfsck.f2fs
329*89272ca1SMauro Carvalho Chehab---------
330*89272ca1SMauro Carvalho ChehabThe fsck.f2fs is a tool to check the consistency of an f2fs-formatted
331*89272ca1SMauro Carvalho Chehabpartition, which examines whether the filesystem metadata and user-made data
332*89272ca1SMauro Carvalho Chehabare cross-referenced correctly or not.
333*89272ca1SMauro Carvalho ChehabNote that, initial version of the tool does not fix any inconsistency.
334*89272ca1SMauro Carvalho Chehab
335*89272ca1SMauro Carvalho ChehabThe options consist of::
336*89272ca1SMauro Carvalho Chehab
337*89272ca1SMauro Carvalho Chehab  -d debug level [default:0]
338*89272ca1SMauro Carvalho Chehab
339*89272ca1SMauro Carvalho Chehabdump.f2fs
340*89272ca1SMauro Carvalho Chehab---------
341*89272ca1SMauro Carvalho ChehabThe dump.f2fs shows the information of specific inode and dumps SSA and SIT to
342*89272ca1SMauro Carvalho Chehabfile. Each file is dump_ssa and dump_sit.
343*89272ca1SMauro Carvalho Chehab
344*89272ca1SMauro Carvalho ChehabThe dump.f2fs is used to debug on-disk data structures of the f2fs filesystem.
345*89272ca1SMauro Carvalho ChehabIt shows on-disk inode information recognized by a given inode number, and is
346*89272ca1SMauro Carvalho Chehabable to dump all the SSA and SIT entries into predefined files, ./dump_ssa and
347*89272ca1SMauro Carvalho Chehab./dump_sit respectively.
348*89272ca1SMauro Carvalho Chehab
349*89272ca1SMauro Carvalho ChehabThe options consist of::
350*89272ca1SMauro Carvalho Chehab
351*89272ca1SMauro Carvalho Chehab  -d debug level [default:0]
352*89272ca1SMauro Carvalho Chehab  -i inode no (hex)
353*89272ca1SMauro Carvalho Chehab  -s [SIT dump segno from #1~#2 (decimal), for all 0~-1]
354*89272ca1SMauro Carvalho Chehab  -a [SSA dump segno from #1~#2 (decimal), for all 0~-1]
355*89272ca1SMauro Carvalho Chehab
356*89272ca1SMauro Carvalho ChehabExamples::
357*89272ca1SMauro Carvalho Chehab
358*89272ca1SMauro Carvalho Chehab    # dump.f2fs -i [ino] /dev/sdx
359*89272ca1SMauro Carvalho Chehab    # dump.f2fs -s 0~-1 /dev/sdx (SIT dump)
360*89272ca1SMauro Carvalho Chehab    # dump.f2fs -a 0~-1 /dev/sdx (SSA dump)
361*89272ca1SMauro Carvalho Chehab
362*89272ca1SMauro Carvalho ChehabDesign
363*89272ca1SMauro Carvalho Chehab======
364*89272ca1SMauro Carvalho Chehab
365*89272ca1SMauro Carvalho ChehabOn-disk Layout
366*89272ca1SMauro Carvalho Chehab--------------
367*89272ca1SMauro Carvalho Chehab
368*89272ca1SMauro Carvalho ChehabF2FS divides the whole volume into a number of segments, each of which is fixed
369*89272ca1SMauro Carvalho Chehabto 2MB in size. A section is composed of consecutive segments, and a zone
370*89272ca1SMauro Carvalho Chehabconsists of a set of sections. By default, section and zone sizes are set to one
371*89272ca1SMauro Carvalho Chehabsegment size identically, but users can easily modify the sizes by mkfs.
372*89272ca1SMauro Carvalho Chehab
373*89272ca1SMauro Carvalho ChehabF2FS splits the entire volume into six areas, and all the areas except superblock
374*89272ca1SMauro Carvalho Chehabconsists of multiple segments as described below::
375*89272ca1SMauro Carvalho Chehab
376*89272ca1SMauro Carvalho Chehab                                            align with the zone size <-|
377*89272ca1SMauro Carvalho Chehab                 |-> align with the segment size
378*89272ca1SMauro Carvalho Chehab     _________________________________________________________________________
379*89272ca1SMauro Carvalho Chehab    |            |            |   Segment   |    Node     |   Segment  |      |
380*89272ca1SMauro Carvalho Chehab    | Superblock | Checkpoint |    Info.    |   Address   |   Summary  | Main |
381*89272ca1SMauro Carvalho Chehab    |    (SB)    |   (CP)     | Table (SIT) | Table (NAT) | Area (SSA) |      |
382*89272ca1SMauro Carvalho Chehab    |____________|_____2______|______N______|______N______|______N_____|__N___|
383*89272ca1SMauro Carvalho Chehab                                                                       .      .
384*89272ca1SMauro Carvalho Chehab                                                             .                .
385*89272ca1SMauro Carvalho Chehab                                                 .                            .
386*89272ca1SMauro Carvalho Chehab                                    ._________________________________________.
387*89272ca1SMauro Carvalho Chehab                                    |_Segment_|_..._|_Segment_|_..._|_Segment_|
388*89272ca1SMauro Carvalho Chehab                                    .           .
389*89272ca1SMauro Carvalho Chehab                                    ._________._________
390*89272ca1SMauro Carvalho Chehab                                    |_section_|__...__|_
391*89272ca1SMauro Carvalho Chehab                                    .            .
392*89272ca1SMauro Carvalho Chehab		                    .________.
393*89272ca1SMauro Carvalho Chehab	                            |__zone__|
394*89272ca1SMauro Carvalho Chehab
395*89272ca1SMauro Carvalho Chehab- Superblock (SB)
396*89272ca1SMauro Carvalho Chehab   It is located at the beginning of the partition, and there exist two copies
397*89272ca1SMauro Carvalho Chehab   to avoid file system crash. It contains basic partition information and some
398*89272ca1SMauro Carvalho Chehab   default parameters of f2fs.
399*89272ca1SMauro Carvalho Chehab
400*89272ca1SMauro Carvalho Chehab- Checkpoint (CP)
401*89272ca1SMauro Carvalho Chehab   It contains file system information, bitmaps for valid NAT/SIT sets, orphan
402*89272ca1SMauro Carvalho Chehab   inode lists, and summary entries of current active segments.
403*89272ca1SMauro Carvalho Chehab
404*89272ca1SMauro Carvalho Chehab- Segment Information Table (SIT)
405*89272ca1SMauro Carvalho Chehab   It contains segment information such as valid block count and bitmap for the
406*89272ca1SMauro Carvalho Chehab   validity of all the blocks.
407*89272ca1SMauro Carvalho Chehab
408*89272ca1SMauro Carvalho Chehab- Node Address Table (NAT)
409*89272ca1SMauro Carvalho Chehab   It is composed of a block address table for all the node blocks stored in
410*89272ca1SMauro Carvalho Chehab   Main area.
411*89272ca1SMauro Carvalho Chehab
412*89272ca1SMauro Carvalho Chehab- Segment Summary Area (SSA)
413*89272ca1SMauro Carvalho Chehab   It contains summary entries which contains the owner information of all the
414*89272ca1SMauro Carvalho Chehab   data and node blocks stored in Main area.
415*89272ca1SMauro Carvalho Chehab
416*89272ca1SMauro Carvalho Chehab- Main Area
417*89272ca1SMauro Carvalho Chehab   It contains file and directory data including their indices.
418*89272ca1SMauro Carvalho Chehab
419*89272ca1SMauro Carvalho ChehabIn order to avoid misalignment between file system and flash-based storage, F2FS
420*89272ca1SMauro Carvalho Chehabaligns the start block address of CP with the segment size. Also, it aligns the
421*89272ca1SMauro Carvalho Chehabstart block address of Main area with the zone size by reserving some segments
422*89272ca1SMauro Carvalho Chehabin SSA area.
423*89272ca1SMauro Carvalho Chehab
424*89272ca1SMauro Carvalho ChehabReference the following survey for additional technical details.
425*89272ca1SMauro Carvalho Chehabhttps://wiki.linaro.org/WorkingGroups/Kernel/Projects/FlashCardSurvey
426*89272ca1SMauro Carvalho Chehab
427*89272ca1SMauro Carvalho ChehabFile System Metadata Structure
428*89272ca1SMauro Carvalho Chehab------------------------------
429*89272ca1SMauro Carvalho Chehab
430*89272ca1SMauro Carvalho ChehabF2FS adopts the checkpointing scheme to maintain file system consistency. At
431*89272ca1SMauro Carvalho Chehabmount time, F2FS first tries to find the last valid checkpoint data by scanning
432*89272ca1SMauro Carvalho ChehabCP area. In order to reduce the scanning time, F2FS uses only two copies of CP.
433*89272ca1SMauro Carvalho ChehabOne of them always indicates the last valid data, which is called as shadow copy
434*89272ca1SMauro Carvalho Chehabmechanism. In addition to CP, NAT and SIT also adopt the shadow copy mechanism.
435*89272ca1SMauro Carvalho Chehab
436*89272ca1SMauro Carvalho ChehabFor file system consistency, each CP points to which NAT and SIT copies are
437*89272ca1SMauro Carvalho Chehabvalid, as shown as below::
438*89272ca1SMauro Carvalho Chehab
439*89272ca1SMauro Carvalho Chehab  +--------+----------+---------+
440*89272ca1SMauro Carvalho Chehab  |   CP   |    SIT   |   NAT   |
441*89272ca1SMauro Carvalho Chehab  +--------+----------+---------+
442*89272ca1SMauro Carvalho Chehab  .         .          .          .
443*89272ca1SMauro Carvalho Chehab  .            .              .              .
444*89272ca1SMauro Carvalho Chehab  .               .                 .                 .
445*89272ca1SMauro Carvalho Chehab  +-------+-------+--------+--------+--------+--------+
446*89272ca1SMauro Carvalho Chehab  | CP #0 | CP #1 | SIT #0 | SIT #1 | NAT #0 | NAT #1 |
447*89272ca1SMauro Carvalho Chehab  +-------+-------+--------+--------+--------+--------+
448*89272ca1SMauro Carvalho Chehab     |             ^                          ^
449*89272ca1SMauro Carvalho Chehab     |             |                          |
450*89272ca1SMauro Carvalho Chehab     `----------------------------------------'
451*89272ca1SMauro Carvalho Chehab
452*89272ca1SMauro Carvalho ChehabIndex Structure
453*89272ca1SMauro Carvalho Chehab---------------
454*89272ca1SMauro Carvalho Chehab
455*89272ca1SMauro Carvalho ChehabThe key data structure to manage the data locations is a "node". Similar to
456*89272ca1SMauro Carvalho Chehabtraditional file structures, F2FS has three types of node: inode, direct node,
457*89272ca1SMauro Carvalho Chehabindirect node. F2FS assigns 4KB to an inode block which contains 923 data block
458*89272ca1SMauro Carvalho Chehabindices, two direct node pointers, two indirect node pointers, and one double
459*89272ca1SMauro Carvalho Chehabindirect node pointer as described below. One direct node block contains 1018
460*89272ca1SMauro Carvalho Chehabdata blocks, and one indirect node block contains also 1018 node blocks. Thus,
461*89272ca1SMauro Carvalho Chehabone inode block (i.e., a file) covers::
462*89272ca1SMauro Carvalho Chehab
463*89272ca1SMauro Carvalho Chehab  4KB * (923 + 2 * 1018 + 2 * 1018 * 1018 + 1018 * 1018 * 1018) := 3.94TB.
464*89272ca1SMauro Carvalho Chehab
465*89272ca1SMauro Carvalho Chehab   Inode block (4KB)
466*89272ca1SMauro Carvalho Chehab     |- data (923)
467*89272ca1SMauro Carvalho Chehab     |- direct node (2)
468*89272ca1SMauro Carvalho Chehab     |          `- data (1018)
469*89272ca1SMauro Carvalho Chehab     |- indirect node (2)
470*89272ca1SMauro Carvalho Chehab     |            `- direct node (1018)
471*89272ca1SMauro Carvalho Chehab     |                       `- data (1018)
472*89272ca1SMauro Carvalho Chehab     `- double indirect node (1)
473*89272ca1SMauro Carvalho Chehab                         `- indirect node (1018)
474*89272ca1SMauro Carvalho Chehab			              `- direct node (1018)
475*89272ca1SMauro Carvalho Chehab	                                         `- data (1018)
476*89272ca1SMauro Carvalho Chehab
477*89272ca1SMauro Carvalho ChehabNote that, all the node blocks are mapped by NAT which means the location of
478*89272ca1SMauro Carvalho Chehabeach node is translated by the NAT table. In the consideration of the wandering
479*89272ca1SMauro Carvalho Chehabtree problem, F2FS is able to cut off the propagation of node updates caused by
480*89272ca1SMauro Carvalho Chehableaf data writes.
481*89272ca1SMauro Carvalho Chehab
482*89272ca1SMauro Carvalho ChehabDirectory Structure
483*89272ca1SMauro Carvalho Chehab-------------------
484*89272ca1SMauro Carvalho Chehab
485*89272ca1SMauro Carvalho ChehabA directory entry occupies 11 bytes, which consists of the following attributes.
486*89272ca1SMauro Carvalho Chehab
487*89272ca1SMauro Carvalho Chehab- hash		hash value of the file name
488*89272ca1SMauro Carvalho Chehab- ino		inode number
489*89272ca1SMauro Carvalho Chehab- len		the length of file name
490*89272ca1SMauro Carvalho Chehab- type		file type such as directory, symlink, etc
491*89272ca1SMauro Carvalho Chehab
492*89272ca1SMauro Carvalho ChehabA dentry block consists of 214 dentry slots and file names. Therein a bitmap is
493*89272ca1SMauro Carvalho Chehabused to represent whether each dentry is valid or not. A dentry block occupies
494*89272ca1SMauro Carvalho Chehab4KB with the following composition.
495*89272ca1SMauro Carvalho Chehab
496*89272ca1SMauro Carvalho Chehab::
497*89272ca1SMauro Carvalho Chehab
498*89272ca1SMauro Carvalho Chehab  Dentry Block(4 K) = bitmap (27 bytes) + reserved (3 bytes) +
499*89272ca1SMauro Carvalho Chehab	              dentries(11 * 214 bytes) + file name (8 * 214 bytes)
500*89272ca1SMauro Carvalho Chehab
501*89272ca1SMauro Carvalho Chehab                         [Bucket]
502*89272ca1SMauro Carvalho Chehab             +--------------------------------+
503*89272ca1SMauro Carvalho Chehab             |dentry block 1 | dentry block 2 |
504*89272ca1SMauro Carvalho Chehab             +--------------------------------+
505*89272ca1SMauro Carvalho Chehab             .               .
506*89272ca1SMauro Carvalho Chehab       .                             .
507*89272ca1SMauro Carvalho Chehab  .       [Dentry Block Structure: 4KB]       .
508*89272ca1SMauro Carvalho Chehab  +--------+----------+----------+------------+
509*89272ca1SMauro Carvalho Chehab  | bitmap | reserved | dentries | file names |
510*89272ca1SMauro Carvalho Chehab  +--------+----------+----------+------------+
511*89272ca1SMauro Carvalho Chehab  [Dentry Block: 4KB] .   .
512*89272ca1SMauro Carvalho Chehab		 .               .
513*89272ca1SMauro Carvalho Chehab            .                          .
514*89272ca1SMauro Carvalho Chehab            +------+------+-----+------+
515*89272ca1SMauro Carvalho Chehab            | hash | ino  | len | type |
516*89272ca1SMauro Carvalho Chehab            +------+------+-----+------+
517*89272ca1SMauro Carvalho Chehab            [Dentry Structure: 11 bytes]
518*89272ca1SMauro Carvalho Chehab
519*89272ca1SMauro Carvalho ChehabF2FS implements multi-level hash tables for directory structure. Each level has
520*89272ca1SMauro Carvalho Chehaba hash table with dedicated number of hash buckets as shown below. Note that
521*89272ca1SMauro Carvalho Chehab"A(2B)" means a bucket includes 2 data blocks.
522*89272ca1SMauro Carvalho Chehab
523*89272ca1SMauro Carvalho Chehab::
524*89272ca1SMauro Carvalho Chehab
525*89272ca1SMauro Carvalho Chehab    ----------------------
526*89272ca1SMauro Carvalho Chehab    A : bucket
527*89272ca1SMauro Carvalho Chehab    B : block
528*89272ca1SMauro Carvalho Chehab    N : MAX_DIR_HASH_DEPTH
529*89272ca1SMauro Carvalho Chehab    ----------------------
530*89272ca1SMauro Carvalho Chehab
531*89272ca1SMauro Carvalho Chehab    level #0   | A(2B)
532*89272ca1SMauro Carvalho Chehab	    |
533*89272ca1SMauro Carvalho Chehab    level #1   | A(2B) - A(2B)
534*89272ca1SMauro Carvalho Chehab	    |
535*89272ca1SMauro Carvalho Chehab    level #2   | A(2B) - A(2B) - A(2B) - A(2B)
536*89272ca1SMauro Carvalho Chehab	.     |   .       .       .       .
537*89272ca1SMauro Carvalho Chehab    level #N/2 | A(2B) - A(2B) - A(2B) - A(2B) - A(2B) - ... - A(2B)
538*89272ca1SMauro Carvalho Chehab	.     |   .       .       .       .
539*89272ca1SMauro Carvalho Chehab    level #N   | A(4B) - A(4B) - A(4B) - A(4B) - A(4B) - ... - A(4B)
540*89272ca1SMauro Carvalho Chehab
541*89272ca1SMauro Carvalho ChehabThe number of blocks and buckets are determined by::
542*89272ca1SMauro Carvalho Chehab
543*89272ca1SMauro Carvalho Chehab                            ,- 2, if n < MAX_DIR_HASH_DEPTH / 2,
544*89272ca1SMauro Carvalho Chehab  # of blocks in level #n = |
545*89272ca1SMauro Carvalho Chehab                            `- 4, Otherwise
546*89272ca1SMauro Carvalho Chehab
547*89272ca1SMauro Carvalho Chehab                             ,- 2^(n + dir_level),
548*89272ca1SMauro Carvalho Chehab			     |        if n + dir_level < MAX_DIR_HASH_DEPTH / 2,
549*89272ca1SMauro Carvalho Chehab  # of buckets in level #n = |
550*89272ca1SMauro Carvalho Chehab                             `- 2^((MAX_DIR_HASH_DEPTH / 2) - 1),
551*89272ca1SMauro Carvalho Chehab			              Otherwise
552*89272ca1SMauro Carvalho Chehab
553*89272ca1SMauro Carvalho ChehabWhen F2FS finds a file name in a directory, at first a hash value of the file
554*89272ca1SMauro Carvalho Chehabname is calculated. Then, F2FS scans the hash table in level #0 to find the
555*89272ca1SMauro Carvalho Chehabdentry consisting of the file name and its inode number. If not found, F2FS
556*89272ca1SMauro Carvalho Chehabscans the next hash table in level #1. In this way, F2FS scans hash tables in
557*89272ca1SMauro Carvalho Chehabeach levels incrementally from 1 to N. In each levels F2FS needs to scan only
558*89272ca1SMauro Carvalho Chehabone bucket determined by the following equation, which shows O(log(# of files))
559*89272ca1SMauro Carvalho Chehabcomplexity::
560*89272ca1SMauro Carvalho Chehab
561*89272ca1SMauro Carvalho Chehab  bucket number to scan in level #n = (hash value) % (# of buckets in level #n)
562*89272ca1SMauro Carvalho Chehab
563*89272ca1SMauro Carvalho ChehabIn the case of file creation, F2FS finds empty consecutive slots that cover the
564*89272ca1SMauro Carvalho Chehabfile name. F2FS searches the empty slots in the hash tables of whole levels from
565*89272ca1SMauro Carvalho Chehab1 to N in the same way as the lookup operation.
566*89272ca1SMauro Carvalho Chehab
567*89272ca1SMauro Carvalho ChehabThe following figure shows an example of two cases holding children::
568*89272ca1SMauro Carvalho Chehab
569*89272ca1SMauro Carvalho Chehab       --------------> Dir <--------------
570*89272ca1SMauro Carvalho Chehab       |                                 |
571*89272ca1SMauro Carvalho Chehab    child                             child
572*89272ca1SMauro Carvalho Chehab
573*89272ca1SMauro Carvalho Chehab    child - child                     [hole] - child
574*89272ca1SMauro Carvalho Chehab
575*89272ca1SMauro Carvalho Chehab    child - child - child             [hole] - [hole] - child
576*89272ca1SMauro Carvalho Chehab
577*89272ca1SMauro Carvalho Chehab   Case 1:                           Case 2:
578*89272ca1SMauro Carvalho Chehab   Number of children = 6,           Number of children = 3,
579*89272ca1SMauro Carvalho Chehab   File size = 7                     File size = 7
580*89272ca1SMauro Carvalho Chehab
581*89272ca1SMauro Carvalho ChehabDefault Block Allocation
582*89272ca1SMauro Carvalho Chehab------------------------
583*89272ca1SMauro Carvalho Chehab
584*89272ca1SMauro Carvalho ChehabAt runtime, F2FS manages six active logs inside "Main" area: Hot/Warm/Cold node
585*89272ca1SMauro Carvalho Chehaband Hot/Warm/Cold data.
586*89272ca1SMauro Carvalho Chehab
587*89272ca1SMauro Carvalho Chehab- Hot node	contains direct node blocks of directories.
588*89272ca1SMauro Carvalho Chehab- Warm node	contains direct node blocks except hot node blocks.
589*89272ca1SMauro Carvalho Chehab- Cold node	contains indirect node blocks
590*89272ca1SMauro Carvalho Chehab- Hot data	contains dentry blocks
591*89272ca1SMauro Carvalho Chehab- Warm data	contains data blocks except hot and cold data blocks
592*89272ca1SMauro Carvalho Chehab- Cold data	contains multimedia data or migrated data blocks
593*89272ca1SMauro Carvalho Chehab
594*89272ca1SMauro Carvalho ChehabLFS has two schemes for free space management: threaded log and copy-and-compac-
595*89272ca1SMauro Carvalho Chehabtion. The copy-and-compaction scheme which is known as cleaning, is well-suited
596*89272ca1SMauro Carvalho Chehabfor devices showing very good sequential write performance, since free segments
597*89272ca1SMauro Carvalho Chehabare served all the time for writing new data. However, it suffers from cleaning
598*89272ca1SMauro Carvalho Chehaboverhead under high utilization. Contrarily, the threaded log scheme suffers
599*89272ca1SMauro Carvalho Chehabfrom random writes, but no cleaning process is needed. F2FS adopts a hybrid
600*89272ca1SMauro Carvalho Chehabscheme where the copy-and-compaction scheme is adopted by default, but the
601*89272ca1SMauro Carvalho Chehabpolicy is dynamically changed to the threaded log scheme according to the file
602*89272ca1SMauro Carvalho Chehabsystem status.
603*89272ca1SMauro Carvalho Chehab
604*89272ca1SMauro Carvalho ChehabIn order to align F2FS with underlying flash-based storage, F2FS allocates a
605*89272ca1SMauro Carvalho Chehabsegment in a unit of section. F2FS expects that the section size would be the
606*89272ca1SMauro Carvalho Chehabsame as the unit size of garbage collection in FTL. Furthermore, with respect
607*89272ca1SMauro Carvalho Chehabto the mapping granularity in FTL, F2FS allocates each section of the active
608*89272ca1SMauro Carvalho Chehablogs from different zones as much as possible, since FTL can write the data in
609*89272ca1SMauro Carvalho Chehabthe active logs into one allocation unit according to its mapping granularity.
610*89272ca1SMauro Carvalho Chehab
611*89272ca1SMauro Carvalho ChehabCleaning process
612*89272ca1SMauro Carvalho Chehab----------------
613*89272ca1SMauro Carvalho Chehab
614*89272ca1SMauro Carvalho ChehabF2FS does cleaning both on demand and in the background. On-demand cleaning is
615*89272ca1SMauro Carvalho Chehabtriggered when there are not enough free segments to serve VFS calls. Background
616*89272ca1SMauro Carvalho Chehabcleaner is operated by a kernel thread, and triggers the cleaning job when the
617*89272ca1SMauro Carvalho Chehabsystem is idle.
618*89272ca1SMauro Carvalho Chehab
619*89272ca1SMauro Carvalho ChehabF2FS supports two victim selection policies: greedy and cost-benefit algorithms.
620*89272ca1SMauro Carvalho ChehabIn the greedy algorithm, F2FS selects a victim segment having the smallest number
621*89272ca1SMauro Carvalho Chehabof valid blocks. In the cost-benefit algorithm, F2FS selects a victim segment
622*89272ca1SMauro Carvalho Chehabaccording to the segment age and the number of valid blocks in order to address
623*89272ca1SMauro Carvalho Chehablog block thrashing problem in the greedy algorithm. F2FS adopts the greedy
624*89272ca1SMauro Carvalho Chehabalgorithm for on-demand cleaner, while background cleaner adopts cost-benefit
625*89272ca1SMauro Carvalho Chehabalgorithm.
626*89272ca1SMauro Carvalho Chehab
627*89272ca1SMauro Carvalho ChehabIn order to identify whether the data in the victim segment are valid or not,
628*89272ca1SMauro Carvalho ChehabF2FS manages a bitmap. Each bit represents the validity of a block, and the
629*89272ca1SMauro Carvalho Chehabbitmap is composed of a bit stream covering whole blocks in main area.
630*89272ca1SMauro Carvalho Chehab
631*89272ca1SMauro Carvalho ChehabWrite-hint Policy
632*89272ca1SMauro Carvalho Chehab-----------------
633*89272ca1SMauro Carvalho Chehab
634*89272ca1SMauro Carvalho Chehab1) whint_mode=off. F2FS only passes down WRITE_LIFE_NOT_SET.
635*89272ca1SMauro Carvalho Chehab
636*89272ca1SMauro Carvalho Chehab2) whint_mode=user-based. F2FS tries to pass down hints given by
637*89272ca1SMauro Carvalho Chehabusers.
638*89272ca1SMauro Carvalho Chehab
639*89272ca1SMauro Carvalho Chehab===================== ======================== ===================
640*89272ca1SMauro Carvalho ChehabUser                  F2FS                     Block
641*89272ca1SMauro Carvalho Chehab===================== ======================== ===================
642*89272ca1SMauro Carvalho Chehab                      META                     WRITE_LIFE_NOT_SET
643*89272ca1SMauro Carvalho Chehab                      HOT_NODE                 "
644*89272ca1SMauro Carvalho Chehab                      WARM_NODE                "
645*89272ca1SMauro Carvalho Chehab                      COLD_NODE                "
646*89272ca1SMauro Carvalho Chehabioctl(COLD)           COLD_DATA                WRITE_LIFE_EXTREME
647*89272ca1SMauro Carvalho Chehabextension list        "                        "
648*89272ca1SMauro Carvalho Chehab
649*89272ca1SMauro Carvalho Chehab-- buffered io
650*89272ca1SMauro Carvalho ChehabWRITE_LIFE_EXTREME    COLD_DATA                WRITE_LIFE_EXTREME
651*89272ca1SMauro Carvalho ChehabWRITE_LIFE_SHORT      HOT_DATA                 WRITE_LIFE_SHORT
652*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NOT_SET    WARM_DATA                WRITE_LIFE_NOT_SET
653*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NONE       "                        "
654*89272ca1SMauro Carvalho ChehabWRITE_LIFE_MEDIUM     "                        "
655*89272ca1SMauro Carvalho ChehabWRITE_LIFE_LONG       "                        "
656*89272ca1SMauro Carvalho Chehab
657*89272ca1SMauro Carvalho Chehab-- direct io
658*89272ca1SMauro Carvalho ChehabWRITE_LIFE_EXTREME    COLD_DATA                WRITE_LIFE_EXTREME
659*89272ca1SMauro Carvalho ChehabWRITE_LIFE_SHORT      HOT_DATA                 WRITE_LIFE_SHORT
660*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NOT_SET    WARM_DATA                WRITE_LIFE_NOT_SET
661*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NONE       "                        WRITE_LIFE_NONE
662*89272ca1SMauro Carvalho ChehabWRITE_LIFE_MEDIUM     "                        WRITE_LIFE_MEDIUM
663*89272ca1SMauro Carvalho ChehabWRITE_LIFE_LONG       "                        WRITE_LIFE_LONG
664*89272ca1SMauro Carvalho Chehab===================== ======================== ===================
665*89272ca1SMauro Carvalho Chehab
666*89272ca1SMauro Carvalho Chehab3) whint_mode=fs-based. F2FS passes down hints with its policy.
667*89272ca1SMauro Carvalho Chehab
668*89272ca1SMauro Carvalho Chehab===================== ======================== ===================
669*89272ca1SMauro Carvalho ChehabUser                  F2FS                     Block
670*89272ca1SMauro Carvalho Chehab===================== ======================== ===================
671*89272ca1SMauro Carvalho Chehab                      META                     WRITE_LIFE_MEDIUM;
672*89272ca1SMauro Carvalho Chehab                      HOT_NODE                 WRITE_LIFE_NOT_SET
673*89272ca1SMauro Carvalho Chehab                      WARM_NODE                "
674*89272ca1SMauro Carvalho Chehab                      COLD_NODE                WRITE_LIFE_NONE
675*89272ca1SMauro Carvalho Chehabioctl(COLD)           COLD_DATA                WRITE_LIFE_EXTREME
676*89272ca1SMauro Carvalho Chehabextension list        "                        "
677*89272ca1SMauro Carvalho Chehab
678*89272ca1SMauro Carvalho Chehab-- buffered io
679*89272ca1SMauro Carvalho ChehabWRITE_LIFE_EXTREME    COLD_DATA                WRITE_LIFE_EXTREME
680*89272ca1SMauro Carvalho ChehabWRITE_LIFE_SHORT      HOT_DATA                 WRITE_LIFE_SHORT
681*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NOT_SET    WARM_DATA                WRITE_LIFE_LONG
682*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NONE       "                        "
683*89272ca1SMauro Carvalho ChehabWRITE_LIFE_MEDIUM     "                        "
684*89272ca1SMauro Carvalho ChehabWRITE_LIFE_LONG       "                        "
685*89272ca1SMauro Carvalho Chehab
686*89272ca1SMauro Carvalho Chehab-- direct io
687*89272ca1SMauro Carvalho ChehabWRITE_LIFE_EXTREME    COLD_DATA                WRITE_LIFE_EXTREME
688*89272ca1SMauro Carvalho ChehabWRITE_LIFE_SHORT      HOT_DATA                 WRITE_LIFE_SHORT
689*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NOT_SET    WARM_DATA                WRITE_LIFE_NOT_SET
690*89272ca1SMauro Carvalho ChehabWRITE_LIFE_NONE       "                        WRITE_LIFE_NONE
691*89272ca1SMauro Carvalho ChehabWRITE_LIFE_MEDIUM     "                        WRITE_LIFE_MEDIUM
692*89272ca1SMauro Carvalho ChehabWRITE_LIFE_LONG       "                        WRITE_LIFE_LONG
693*89272ca1SMauro Carvalho Chehab===================== ======================== ===================
694*89272ca1SMauro Carvalho Chehab
695*89272ca1SMauro Carvalho ChehabFallocate(2) Policy
696*89272ca1SMauro Carvalho Chehab-------------------
697*89272ca1SMauro Carvalho Chehab
698*89272ca1SMauro Carvalho ChehabThe default policy follows the below posix rule.
699*89272ca1SMauro Carvalho Chehab
700*89272ca1SMauro Carvalho ChehabAllocating disk space
701*89272ca1SMauro Carvalho Chehab    The default operation (i.e., mode is zero) of fallocate() allocates
702*89272ca1SMauro Carvalho Chehab    the disk space within the range specified by offset and len.  The
703*89272ca1SMauro Carvalho Chehab    file size (as reported by stat(2)) will be changed if offset+len is
704*89272ca1SMauro Carvalho Chehab    greater than the file size.  Any subregion within the range specified
705*89272ca1SMauro Carvalho Chehab    by offset and len that did not contain data before the call will be
706*89272ca1SMauro Carvalho Chehab    initialized to zero.  This default behavior closely resembles the
707*89272ca1SMauro Carvalho Chehab    behavior of the posix_fallocate(3) library function, and is intended
708*89272ca1SMauro Carvalho Chehab    as a method of optimally implementing that function.
709*89272ca1SMauro Carvalho Chehab
710*89272ca1SMauro Carvalho ChehabHowever, once F2FS receives ioctl(fd, F2FS_IOC_SET_PIN_FILE) in prior to
711*89272ca1SMauro Carvalho Chehabfallocate(fd, DEFAULT_MODE), it allocates on-disk blocks addressess having
712*89272ca1SMauro Carvalho Chehabzero or random data, which is useful to the below scenario where:
713*89272ca1SMauro Carvalho Chehab
714*89272ca1SMauro Carvalho Chehab 1. create(fd)
715*89272ca1SMauro Carvalho Chehab 2. ioctl(fd, F2FS_IOC_SET_PIN_FILE)
716*89272ca1SMauro Carvalho Chehab 3. fallocate(fd, 0, 0, size)
717*89272ca1SMauro Carvalho Chehab 4. address = fibmap(fd, offset)
718*89272ca1SMauro Carvalho Chehab 5. open(blkdev)
719*89272ca1SMauro Carvalho Chehab 6. write(blkdev, address)
720*89272ca1SMauro Carvalho Chehab
721*89272ca1SMauro Carvalho ChehabCompression implementation
722*89272ca1SMauro Carvalho Chehab--------------------------
723*89272ca1SMauro Carvalho Chehab
724*89272ca1SMauro Carvalho Chehab- New term named cluster is defined as basic unit of compression, file can
725*89272ca1SMauro Carvalho Chehab  be divided into multiple clusters logically. One cluster includes 4 << n
726*89272ca1SMauro Carvalho Chehab  (n >= 0) logical pages, compression size is also cluster size, each of
727*89272ca1SMauro Carvalho Chehab  cluster can be compressed or not.
728*89272ca1SMauro Carvalho Chehab
729*89272ca1SMauro Carvalho Chehab- In cluster metadata layout, one special block address is used to indicate
730*89272ca1SMauro Carvalho Chehab  cluster is compressed one or normal one, for compressed cluster, following
731*89272ca1SMauro Carvalho Chehab  metadata maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs
732*89272ca1SMauro Carvalho Chehab  stores data including compress header and compressed data.
733*89272ca1SMauro Carvalho Chehab
734*89272ca1SMauro Carvalho Chehab- In order to eliminate write amplification during overwrite, F2FS only
735*89272ca1SMauro Carvalho Chehab  support compression on write-once file, data can be compressed only when
736*89272ca1SMauro Carvalho Chehab  all logical blocks in file are valid and cluster compress ratio is lower
737*89272ca1SMauro Carvalho Chehab  than specified threshold.
738*89272ca1SMauro Carvalho Chehab
739*89272ca1SMauro Carvalho Chehab- To enable compression on regular inode, there are three ways:
740*89272ca1SMauro Carvalho Chehab
741*89272ca1SMauro Carvalho Chehab  * chattr +c file
742*89272ca1SMauro Carvalho Chehab  * chattr +c dir; touch dir/file
743*89272ca1SMauro Carvalho Chehab  * mount w/ -o compress_extension=ext; touch file.ext
744*89272ca1SMauro Carvalho Chehab
745*89272ca1SMauro Carvalho ChehabCompress metadata layout::
746*89272ca1SMauro Carvalho Chehab
747*89272ca1SMauro Carvalho Chehab				[Dnode Structure]
748*89272ca1SMauro Carvalho Chehab		+-----------------------------------------------+
749*89272ca1SMauro Carvalho Chehab		| cluster 1 | cluster 2 | ......... | cluster N |
750*89272ca1SMauro Carvalho Chehab		+-----------------------------------------------+
751*89272ca1SMauro Carvalho Chehab		.           .                       .           .
752*89272ca1SMauro Carvalho Chehab	.                       .                .                      .
753*89272ca1SMauro Carvalho Chehab    .         Compressed Cluster       .        .        Normal Cluster            .
754*89272ca1SMauro Carvalho Chehab    +----------+---------+---------+---------+  +---------+---------+---------+---------+
755*89272ca1SMauro Carvalho Chehab    |compr flag| block 1 | block 2 | block 3 |  | block 1 | block 2 | block 3 | block 4 |
756*89272ca1SMauro Carvalho Chehab    +----------+---------+---------+---------+  +---------+---------+---------+---------+
757*89272ca1SMauro Carvalho Chehab	    .                             .
758*89272ca1SMauro Carvalho Chehab	    .                                           .
759*89272ca1SMauro Carvalho Chehab	.                                                           .
760*89272ca1SMauro Carvalho Chehab	+-------------+-------------+----------+----------------------------+
761*89272ca1SMauro Carvalho Chehab	| data length | data chksum | reserved |      compressed data       |
762*89272ca1SMauro Carvalho Chehab	+-------------+-------------+----------+----------------------------+
763