1:orphan: 2 3Making Filesystems Exportable 4============================= 5 6Overview 7-------- 8 9All filesystem operations require a dentry (or two) as a starting 10point. Local applications have a reference-counted hold on suitable 11dentries via open file descriptors or cwd/root. However remote 12applications that access a filesystem via a remote filesystem protocol 13such as NFS may not be able to hold such a reference, and so need a 14different way to refer to a particular dentry. As the alternative 15form of reference needs to be stable across renames, truncates, and 16server-reboot (among other things, though these tend to be the most 17problematic), there is no simple answer like 'filename'. 18 19The mechanism discussed here allows each filesystem implementation to 20specify how to generate an opaque (outside of the filesystem) byte 21string for any dentry, and how to find an appropriate dentry for any 22given opaque byte string. 23This byte string will be called a "filehandle fragment" as it 24corresponds to part of an NFS filehandle. 25 26A filesystem which supports the mapping between filehandle fragments 27and dentries will be termed "exportable". 28 29 30 31Dcache Issues 32------------- 33 34The dcache normally contains a proper prefix of any given filesystem 35tree. This means that if any filesystem object is in the dcache, then 36all of the ancestors of that filesystem object are also in the dcache. 37As normal access is by filename this prefix is created naturally and 38maintained easily (by each object maintaining a reference count on 39its parent). 40 41However when objects are included into the dcache by interpreting a 42filehandle fragment, there is no automatic creation of a path prefix 43for the object. This leads to two related but distinct features of 44the dcache that are not needed for normal filesystem access. 45 461. The dcache must sometimes contain objects that are not part of the 47 proper prefix. i.e that are not connected to the root. 482. The dcache must be prepared for a newly found (via ->lookup) directory 49 to already have a (non-connected) dentry, and must be able to move 50 that dentry into place (based on the parent and name in the 51 ->lookup). This is particularly needed for directories as 52 it is a dcache invariant that directories only have one dentry. 53 54To implement these features, the dcache has: 55 56a. A dentry flag DCACHE_DISCONNECTED which is set on 57 any dentry that might not be part of the proper prefix. 58 This is set when anonymous dentries are created, and cleared when a 59 dentry is noticed to be a child of a dentry which is in the proper 60 prefix. If the refcount on a dentry with this flag set 61 becomes zero, the dentry is immediately discarded, rather than being 62 kept in the dcache. If a dentry that is not already in the dcache 63 is repeatedly accessed by filehandle (as NFSD might do), an new dentry 64 will be a allocated for each access, and discarded at the end of 65 the access. 66 67 Note that such a dentry can acquire children, name, ancestors, etc. 68 without losing DCACHE_DISCONNECTED - that flag is only cleared when 69 subtree is successfully reconnected to root. Until then dentries 70 in such subtree are retained only as long as there are references; 71 refcount reaching zero means immediate eviction, same as for unhashed 72 dentries. That guarantees that we won't need to hunt them down upon 73 umount. 74 75b. A primitive for creation of secondary roots - d_obtain_root(inode). 76 Those do _not_ bear DCACHE_DISCONNECTED. They are placed on the 77 per-superblock list (->s_roots), so they can be located at umount 78 time for eviction purposes. 79 80c. Helper routines to allocate anonymous dentries, and to help attach 81 loose directory dentries at lookup time. They are: 82 83 d_obtain_alias(inode) will return a dentry for the given inode. 84 If the inode already has a dentry, one of those is returned. 85 86 If it doesn't, a new anonymous (IS_ROOT and 87 DCACHE_DISCONNECTED) dentry is allocated and attached. 88 89 In the case of a directory, care is taken that only one dentry 90 can ever be attached. 91 92 d_splice_alias(inode, dentry) will introduce a new dentry into the tree; 93 either the passed-in dentry or a preexisting alias for the given inode 94 (such as an anonymous one created by d_obtain_alias), if appropriate. 95 It returns NULL when the passed-in dentry is used, following the calling 96 convention of ->lookup. 97 98Filesystem Issues 99----------------- 100 101For a filesystem to be exportable it must: 102 103 1. provide the filehandle fragment routines described below. 104 2. make sure that d_splice_alias is used rather than d_add 105 when ->lookup finds an inode for a given parent and name. 106 107 If inode is NULL, d_splice_alias(inode, dentry) is equivalent to:: 108 109 d_add(dentry, inode), NULL 110 111 Similarly, d_splice_alias(ERR_PTR(err), dentry) = ERR_PTR(err) 112 113 Typically the ->lookup routine will simply end with a:: 114 115 return d_splice_alias(inode, dentry); 116 } 117 118 119 120A file system implementation declares that instances of the filesystem 121are exportable by setting the s_export_op field in the struct 122super_block. This field must point to a "struct export_operations" 123struct which has the following members: 124 125 encode_fh (mandatory) 126 Takes a dentry and creates a filehandle fragment which may later be used 127 to find or create a dentry for the same object. 128 129 fh_to_dentry (mandatory) 130 Given a filehandle fragment, this should find the implied object and 131 create a dentry for it (possibly with d_obtain_alias). 132 133 fh_to_parent (optional but strongly recommended) 134 Given a filehandle fragment, this should find the parent of the 135 implied object and create a dentry for it (possibly with 136 d_obtain_alias). May fail if the filehandle fragment is too small. 137 138 get_parent (optional but strongly recommended) 139 When given a dentry for a directory, this should return a dentry for 140 the parent. Quite possibly the parent dentry will have been allocated 141 by d_alloc_anon. The default get_parent function just returns an error 142 so any filehandle lookup that requires finding a parent will fail. 143 ->lookup("..") is *not* used as a default as it can leave ".." entries 144 in the dcache which are too messy to work with. 145 146 get_name (optional) 147 When given a parent dentry and a child dentry, this should find a name 148 in the directory identified by the parent dentry, which leads to the 149 object identified by the child dentry. If no get_name function is 150 supplied, a default implementation is provided which uses vfs_readdir 151 to find potential names, and matches inode numbers to find the correct 152 match. 153 154 flags 155 Some filesystems may need to be handled differently than others. The 156 export_operations struct also includes a flags field that allows the 157 filesystem to communicate such information to nfsd. See the Export 158 Operations Flags section below for more explanation. 159 160A filehandle fragment consists of an array of 1 or more 4byte words, 161together with a one byte "type". 162The decode_fh routine should not depend on the stated size that is 163passed to it. This size may be larger than the original filehandle 164generated by encode_fh, in which case it will have been padded with 165nuls. Rather, the encode_fh routine should choose a "type" which 166indicates the decode_fh how much of the filehandle is valid, and how 167it should be interpreted. 168 169Export Operations Flags 170----------------------- 171In addition to the operation vector pointers, struct export_operations also 172contains a "flags" field that allows the filesystem to communicate to nfsd 173that it may want to do things differently when dealing with it. The 174following flags are defined: 175 176 EXPORT_OP_NOWCC - disable NFSv3 WCC attributes on this filesystem 177 RFC 1813 recommends that servers always send weak cache consistency 178 (WCC) data to the client after each operation. The server should 179 atomically collect attributes about the inode, do an operation on it, 180 and then collect the attributes afterward. This allows the client to 181 skip issuing GETATTRs in some situations but means that the server 182 is calling vfs_getattr for almost all RPCs. On some filesystems 183 (particularly those that are clustered or networked) this is expensive 184 and atomicity is difficult to guarantee. This flag indicates to nfsd 185 that it should skip providing WCC attributes to the client in NFSv3 186 replies when doing operations on this filesystem. Consider enabling 187 this on filesystems that have an expensive ->getattr inode operation, 188 or when atomicity between pre and post operation attribute collection 189 is impossible to guarantee. 190 191 EXPORT_OP_NOSUBTREECHK - disallow subtree checking on this fs 192 Many NFS operations deal with filehandles, which the server must then 193 vet to ensure that they live inside of an exported tree. When the 194 export consists of an entire filesystem, this is trivial. nfsd can just 195 ensure that the filehandle live on the filesystem. When only part of a 196 filesystem is exported however, then nfsd must walk the ancestors of the 197 inode to ensure that it's within an exported subtree. This is an 198 expensive operation and not all filesystems can support it properly. 199 This flag exempts the filesystem from subtree checking and causes 200 exportfs to get back an error if it tries to enable subtree checking 201 on it. 202 203 EXPORT_OP_CLOSE_BEFORE_UNLINK - always close cached files before unlinking 204 On some exportable filesystems (such as NFS) unlinking a file that 205 is still open can cause a fair bit of extra work. For instance, 206 the NFS client will do a "sillyrename" to ensure that the file 207 sticks around while it's still open. When reexporting, that open 208 file is held by nfsd so we usually end up doing a sillyrename, and 209 then immediately deleting the sillyrenamed file just afterward when 210 the link count actually goes to zero. Sometimes this delete can race 211 with other operations (for instance an rmdir of the parent directory). 212 This flag causes nfsd to close any open files for this inode _before_ 213 calling into the vfs to do an unlink or a rename that would replace 214 an existing file. 215 216 EXPORT_OP_REMOTE_FS - Backing storage for this filesystem is remote 217 PF_LOCAL_THROTTLE exists for loopback NFSD, where a thread needs to 218 write to one bdi (the final bdi) in order to free up writes queued 219 to another bdi (the client bdi). Such threads get a private balance 220 of dirty pages so that dirty pages for the client bdi do not imact 221 the daemon writing to the final bdi. For filesystems whose durable 222 storage is not local (such as exported NFS filesystems), this 223 constraint has negative consequences. EXPORT_OP_REMOTE_FS enables 224 an export to disable writeback throttling. 225 226 EXPORT_OP_NOATOMIC_ATTR - Filesystem does not update attributes atomically 227 EXPORT_OP_NOATOMIC_ATTR indicates that the exported filesystem 228 cannot provide the semantics required by the "atomic" boolean in 229 NFSv4's change_info4. This boolean indicates to a client whether the 230 returned before and after change attributes were obtained atomically 231 with the respect to the requested metadata operation (UNLINK, 232 OPEN/CREATE, MKDIR, etc). 233 234 EXPORT_OP_FLUSH_ON_CLOSE - Filesystem flushes file data on close(2) 235 On most filesystems, inodes can remain under writeback after the 236 file is closed. NFSD relies on client activity or local flusher 237 threads to handle writeback. Certain filesystems, such as NFS, flush 238 all of an inode's dirty data on last close. Exports that behave this 239 way should set EXPORT_OP_FLUSH_ON_CLOSE so that NFSD knows to skip 240 waiting for writeback when closing such files. 241 242 EXPORT_OP_ASYNC_LOCK - Indicates a capable filesystem to do async lock 243 requests from lockd. Only set EXPORT_OP_ASYNC_LOCK if the filesystem has 244 it's own ->lock() functionality as core posix_lock_file() implementation 245 has no async lock request handling yet. For more information about how to 246 indicate an async lock request from a ->lock() file_operations struct, see 247 fs/locks.c and comment for the function vfs_lock_file(). 248