1*dc451bbcSJ. Bruce FieldsReexporting NFS filesystems 2*dc451bbcSJ. Bruce Fields=========================== 3*dc451bbcSJ. Bruce Fields 4*dc451bbcSJ. Bruce FieldsOverview 5*dc451bbcSJ. Bruce Fields-------- 6*dc451bbcSJ. Bruce Fields 7*dc451bbcSJ. Bruce FieldsIt is possible to reexport an NFS filesystem over NFS. However, this 8*dc451bbcSJ. Bruce Fieldsfeature comes with a number of limitations. Before trying it, we 9*dc451bbcSJ. Bruce Fieldsrecommend some careful research to determine whether it will work for 10*dc451bbcSJ. Bruce Fieldsyour purposes. 11*dc451bbcSJ. Bruce Fields 12*dc451bbcSJ. Bruce FieldsA discussion of current known limitations follows. 13*dc451bbcSJ. Bruce Fields 14*dc451bbcSJ. Bruce Fields"fsid=" required, crossmnt broken 15*dc451bbcSJ. Bruce Fields--------------------------------- 16*dc451bbcSJ. Bruce Fields 17*dc451bbcSJ. Bruce FieldsWe require the "fsid=" export option on any reexport of an NFS 18*dc451bbcSJ. Bruce Fieldsfilesystem. You can use "uuidgen -r" to generate a unique argument. 19*dc451bbcSJ. Bruce Fields 20*dc451bbcSJ. Bruce FieldsThe "crossmnt" export does not propagate "fsid=", so it will not allow 21*dc451bbcSJ. Bruce Fieldstraversing into further nfs filesystems; if you wish to export nfs 22*dc451bbcSJ. Bruce Fieldsfilesystems mounted under the exported filesystem, you'll need to export 23*dc451bbcSJ. Bruce Fieldsthem explicitly, assigning each its own unique "fsid= option. 24*dc451bbcSJ. Bruce Fields 25*dc451bbcSJ. Bruce FieldsReboot recovery 26*dc451bbcSJ. Bruce Fields--------------- 27*dc451bbcSJ. Bruce Fields 28*dc451bbcSJ. Bruce FieldsThe NFS protocol's normal reboot recovery mechanisms don't work for the 29*dc451bbcSJ. Bruce Fieldscase when the reexport server reboots. Clients will lose any locks 30*dc451bbcSJ. Bruce Fieldsthey held before the reboot, and further IO will result in errors. 31*dc451bbcSJ. Bruce FieldsClosing and reopening files should clear the errors. 32*dc451bbcSJ. Bruce Fields 33*dc451bbcSJ. Bruce FieldsFilehandle limits 34*dc451bbcSJ. Bruce Fields----------------- 35*dc451bbcSJ. Bruce Fields 36*dc451bbcSJ. Bruce FieldsIf the original server uses an X byte filehandle for a given object, the 37*dc451bbcSJ. Bruce Fieldsreexport server's filehandle for the reexported object will be X+22 38*dc451bbcSJ. Bruce Fieldsbytes, rounded up to the nearest multiple of four bytes. 39*dc451bbcSJ. Bruce Fields 40*dc451bbcSJ. Bruce FieldsThe result must fit into the RFC-mandated filehandle size limits: 41*dc451bbcSJ. Bruce Fields 42*dc451bbcSJ. Bruce Fields+-------+-----------+ 43*dc451bbcSJ. Bruce Fields| NFSv2 | 32 bytes | 44*dc451bbcSJ. Bruce Fields+-------+-----------+ 45*dc451bbcSJ. Bruce Fields| NFSv3 | 64 bytes | 46*dc451bbcSJ. Bruce Fields+-------+-----------+ 47*dc451bbcSJ. Bruce Fields| NFSv4 | 128 bytes | 48*dc451bbcSJ. Bruce Fields+-------+-----------+ 49*dc451bbcSJ. Bruce Fields 50*dc451bbcSJ. Bruce FieldsSo, for example, you will only be able to reexport a filesystem over 51*dc451bbcSJ. Bruce FieldsNFSv2 if the original server gives you filehandles that fit in 10 52*dc451bbcSJ. Bruce Fieldsbytes--which is unlikely. 53*dc451bbcSJ. Bruce Fields 54*dc451bbcSJ. Bruce FieldsIn general there's no way to know the maximum filehandle size given out 55*dc451bbcSJ. Bruce Fieldsby an NFS server without asking the server vendor. 56*dc451bbcSJ. Bruce Fields 57*dc451bbcSJ. Bruce FieldsBut the following table gives a few examples. The first column is the 58*dc451bbcSJ. Bruce Fieldstypical length of the filehandle from a Linux server exporting the given 59*dc451bbcSJ. Bruce Fieldsfilesystem, the second is the length after that nfs export is reexported 60*dc451bbcSJ. Bruce Fieldsby another Linux host: 61*dc451bbcSJ. Bruce Fields 62*dc451bbcSJ. Bruce Fields+--------+-------------------+----------------+ 63*dc451bbcSJ. Bruce Fields| | filehandle length | after reexport | 64*dc451bbcSJ. Bruce Fields+========+===================+================+ 65*dc451bbcSJ. Bruce Fields| ext4: | 28 bytes | 52 bytes | 66*dc451bbcSJ. Bruce Fields+--------+-------------------+----------------+ 67*dc451bbcSJ. Bruce Fields| xfs: | 32 bytes | 56 bytes | 68*dc451bbcSJ. Bruce Fields+--------+-------------------+----------------+ 69*dc451bbcSJ. Bruce Fields| btrfs: | 40 bytes | 64 bytes | 70*dc451bbcSJ. Bruce Fields+--------+-------------------+----------------+ 71*dc451bbcSJ. Bruce Fields 72*dc451bbcSJ. Bruce FieldsAll will therefore fit in an NFSv3 or NFSv4 filehandle after reexport, 73*dc451bbcSJ. Bruce Fieldsbut none are reexportable over NFSv2. 74*dc451bbcSJ. Bruce Fields 75*dc451bbcSJ. Bruce FieldsLinux server filehandles are a bit more complicated than this, though; 76*dc451bbcSJ. Bruce Fieldsfor example: 77*dc451bbcSJ. Bruce Fields 78*dc451bbcSJ. Bruce Fields - The (non-default) "subtreecheck" export option generally 79*dc451bbcSJ. Bruce Fields requires another 4 to 8 bytes in the filehandle. 80*dc451bbcSJ. Bruce Fields - If you export a subdirectory of a filesystem (instead of 81*dc451bbcSJ. Bruce Fields exporting the filesystem root), that also usually adds 4 to 8 82*dc451bbcSJ. Bruce Fields bytes. 83*dc451bbcSJ. Bruce Fields - If you export over NFSv2, knfsd usually uses a shorter 84*dc451bbcSJ. Bruce Fields filesystem identifier that saves 8 bytes. 85*dc451bbcSJ. Bruce Fields - The root directory of an export uses a filehandle that is 86*dc451bbcSJ. Bruce Fields shorter. 87*dc451bbcSJ. Bruce Fields 88*dc451bbcSJ. Bruce FieldsAs you can see, the 128-byte NFSv4 filehandle is large enough that 89*dc451bbcSJ. Bruce Fieldsyou're unlikely to have trouble using NFSv4 to reexport any filesystem 90*dc451bbcSJ. Bruce Fieldsexported from a Linux server. In general, if the original server is 91*dc451bbcSJ. Bruce Fieldssomething that also supports NFSv3, you're *probably* OK. Re-exporting 92*dc451bbcSJ. Bruce Fieldsover NFSv3 may be dicier, and reexporting over NFSv2 will probably 93*dc451bbcSJ. Bruce Fieldsnever work. 94*dc451bbcSJ. Bruce Fields 95*dc451bbcSJ. Bruce FieldsFor more details of Linux filehandle structure, the best reference is 96*dc451bbcSJ. Bruce Fieldsthe source code and comments; see in particular: 97*dc451bbcSJ. Bruce Fields 98*dc451bbcSJ. Bruce Fields - include/linux/exportfs.h:enum fid_type 99*dc451bbcSJ. Bruce Fields - include/uapi/linux/nfsd/nfsfh.h:struct nfs_fhbase_new 100*dc451bbcSJ. Bruce Fields - fs/nfsd/nfsfh.c:set_version_and_fsid_type 101*dc451bbcSJ. Bruce Fields - fs/nfs/export.c:nfs_encode_fh 102*dc451bbcSJ. Bruce Fields 103*dc451bbcSJ. Bruce FieldsOpen DENY bits ignored 104*dc451bbcSJ. Bruce Fields---------------------- 105*dc451bbcSJ. Bruce Fields 106*dc451bbcSJ. Bruce FieldsNFS since NFSv4 supports ALLOW and DENY bits taken from Windows, which 107*dc451bbcSJ. Bruce Fieldsallow you, for example, to open a file in a mode which forbids other 108*dc451bbcSJ. Bruce Fieldsread opens or write opens. The Linux client doesn't use them, and the 109*dc451bbcSJ. Bruce Fieldsserver's support has always been incomplete: they are enforced only 110*dc451bbcSJ. Bruce Fieldsagainst other NFS users, not against processes accessing the exported 111*dc451bbcSJ. Bruce Fieldsfilesystem locally. A reexport server will also not pass them along to 112*dc451bbcSJ. Bruce Fieldsthe original server, so they will not be enforced between clients of 113*dc451bbcSJ. Bruce Fieldsdifferent reexport servers. 114