#
788af953 |
| 26-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: automatically update mtime and ctime on write
Writing should implicitly update a file's mtime and ctime. For fuse, the server is supposed to do that. But the client needs to do it too, bec
fusefs: automatically update mtime and ctime on write
Writing should implicitly update a file's mtime and ctime. For fuse, the server is supposed to do that. But the client needs to do it too, because the FUSE_WRITE response does not include time attributes, and it's not desirable to issue a GETATTR after every WRITE. When using the writeback cache, there's another hitch: the kernel should ignore the mtime and ctime fields in any GETATTR response for files with a dirty write cache.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
0d3a88d7 |
| 25-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: writes should update the file size, even when data_cache_mode=0
Writes that extend a file should update the file's size. r344185 restricted that behavior for fusefs to only happen when the
fusefs: writes should update the file size, even when data_cache_mode=0
Writes that extend a file should update the file's size. r344185 restricted that behavior for fusefs to only happen when the data cache was enabled. That probably made sense at the time because the attribute cache wasn't fully baked yet. Now that it is, we should always update the cached file size during write.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
b9e20197 |
| 25-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: rewrite vop_getpages and vop_putpages
Use the standard facilities for getpages and putpages instead of bespoke implementations that don't work well with the writeback cache. This has severa
fusefs: rewrite vop_getpages and vop_putpages
Use the standard facilities for getpages and putpages instead of bespoke implementations that don't work well with the writeback cache. This has several corollaries:
* Change the way we handle short reads _again_. vfs_bio_getpages doesn't provide any way to handle unexpected short reads. Plus, I found some more lock-order problems. So now when the short read is detected we'll just clear the vnode's attribute cache, forcing the file size to be requeried the next time it's needed. VOP_GETPAGES doesn't have any way to indicate a short read to the "caller", so we just bzero the rest of the page whenever a short read happens.
* Change the way we decide when to set the FUSE_WRITE_CACHE bit. We now set it for clustered writes even when the writeback cache is not in use.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
1734e205 |
| 24-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: refine the short read fix from r349332
b_fsprivate1 needs to be initialized even for write operations, probably because a buffer can be used to read, write, and read again with the final rea
fusefs: refine the short read fix from r349332
b_fsprivate1 needs to be initialized even for write operations, probably because a buffer can be used to read, write, and read again with the final read serviced by cache.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
17575bad |
| 24-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: improve the short read fix from r349279
VOP_GETPAGES intentionally tries to read beyond EOF, so fuse_read_biobackend can't rely on bp->b_resid > 0 indicating a short read. And adjusting bp-
fusefs: improve the short read fix from r349279
VOP_GETPAGES intentionally tries to read beyond EOF, so fuse_read_biobackend can't rely on bp->b_resid > 0 indicating a short read. And adjusting bp->b_count after a short read seems to cause some sort of resource leak. Instead, store the shortfall in the bp->b_fsprivate1 field.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
44f654fd |
| 22-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: fix corruption on short reads caused by r349279
Even if a short read is caused by EOF, it's still necessary to bzero the remaining buffer, because that buffer could become valid as a result
fusefs: fix corruption on short reads caused by r349279
Even if a short read is caused by EOF, it's still necessary to bzero the remaining buffer, because that buffer could become valid as a result of a future ftruncate or pwrite operation.
Reported by: fsx Sponsored by: The FreeBSD Foundation
show more ...
|
#
aef22f2d |
| 21-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: correctly handle short reads
A fuse server may return a short read for three reasons:
* The file is opened with FOPEN_DIRECT_IO. In this case, the short read should be returned directly
fusefs: correctly handle short reads
A fuse server may return a short read for three reasons:
* The file is opened with FOPEN_DIRECT_IO. In this case, the short read should be returned directly to userland. We already handled this case correctly.
* The file was truncated server-side, and the read hit EOF. In this case, the kernel should update the file size. Fixed in the case of VOP_READ. Fixing this for VOP_GETPAGES is TODO.
* The file is opened in writeback mode, there are dirty buffers past what the server thinks is the file's EOF, and the read hit what the server thinks is the file's EOF. In this case, the client is trying to read a hole, and should zero-fill it. We already handled this case, and I added a test for it.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
a1c9f4ad |
| 20-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: implement VOP_BMAP
If the fuse daemon supports FUSE_BMAP, then use that for the block mapping. Otherwise, use the same technique used by vop_stdbmap. Report large values for runp and runb i
fusefs: implement VOP_BMAP
If the fuse daemon supports FUSE_BMAP, then use that for the block mapping. Otherwise, use the same technique used by vop_stdbmap. Report large values for runp and runb in order to maximize read clustering and minimize upcalls, even if we don't know the true layout.
The major result of this change is that sequential reads to FUSE files will now usually happen 128KB at a time instead of 64KB.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
84879e46 |
| 18-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: multiple fixes related to the write cache
* Don't always write the last page synchronously. That's not actually required. It was probably just masking another bug that I fixed later, p
fusefs: multiple fixes related to the write cache
* Don't always write the last page synchronously. That's not actually required. It was probably just masking another bug that I fixed later, possibly in r349021.
* Enable the NotifyWriteback tests now that Writeback cache is working.
* Add a test to ensure that the write cache isn't flushed synchronously when in writeback mode.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
402b609c |
| 18-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: use cluster_read for more readahead
fusefs will now use cluster_read. This allows readahead of more than one cache block. However, it won't yet actually cluster the reads because that requ
fusefs: use cluster_read for more readahead
fusefs will now use cluster_read. This allows readahead of more than one cache block. However, it won't yet actually cluster the reads because that requires VOP_BMAP, which fusefs does not yet implement.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
d569012f |
| 17-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: implement non-clustered readahead
fusefs will now read ahead at most one cache block at a time (usually 64 KB). Clustered reads are still TODO. Individual file systems may disable read ahe
fusefs: implement non-clustered readahead
fusefs will now read ahead at most one cache block at a time (usually 64 KB). Clustered reads are still TODO. Individual file systems may disable read ahead by setting fuse_init_out.max_readahead=0 during initialization.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
b5aaf286 |
| 14-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: fix the "write-through" of write-through cacheing
Our fusefs(5) module supports three cache modes: uncached, write-through, and write-back. However, the write-through mode (which is the def
fusefs: fix the "write-through" of write-through cacheing
Our fusefs(5) module supports three cache modes: uncached, write-through, and write-back. However, the write-through mode (which is the default) has never actually worked as its name suggests. Rather, it's always been more like "write-around". It wrote directly, bypassing the cache. The cache would only be populated by a subsequent read of the same data.
This commit fixes that problem. Now the write-through mode works as one would expect: write(2) immediately adds data to the cache and then blocks while the daemon processes the write operation.
A side effect of this change is that non-cache-block-aligned writes will now incur a read-modify-write cycle of the cache block. The old behavior (bypassing write cache entirely) can still be achieved by opening a file with O_DIRECT.
PR: 237588 Sponsored by: The FreeBSD Foundation
show more ...
|
#
8eecd9ce |
| 14-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: enable write clustering
Enable write clustering in fusefs whenever cache mode is set to writeback and the "async" mount option is used. With default values for MAXPHYS, DFLTPHYS, and the fu
fusefs: enable write clustering
Enable write clustering in fusefs whenever cache mode is set to writeback and the "async" mount option is used. With default values for MAXPHYS, DFLTPHYS, and the fuse max_write mount parameter, that means sequential writes will now be written 128KB at a time instead of 64KB.
Also, add a regression test for PR 238565, a panic during unmount that probably affects UFS, ext2, and msdosfs as well as fusefs.
PR: 238565 Sponsored by: The FreeBSD Foundation
show more ...
|
#
dff3a6b4 |
| 13-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: fix a bug with WriteBack cacheing
An errant vfs_bio_clrbuf snuck in in r348931. Surprisingly, it doesn't have any effect most of the time. But under some circumstances it cause the buffer
fusefs: fix a bug with WriteBack cacheing
An errant vfs_bio_clrbuf snuck in in r348931. Surprisingly, it doesn't have any effect most of the time. But under some circumstances it cause the buffer to behave in a write-only fashion.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
a87e0831 |
| 11-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: WIP fixing writeback cacheing
The current "writeback" cache mode, selected by the vfs.fusefs.data_cache_mode sysctl, doesn't do writeback cacheing at all. It merely goes through the motions
fusefs: WIP fixing writeback cacheing
The current "writeback" cache mode, selected by the vfs.fusefs.data_cache_mode sysctl, doesn't do writeback cacheing at all. It merely goes through the motions of using buf(9), but then writes every buffer synchronously. This commit:
* Enables delayed writes when the sysctl is set to writeback cacheing * Fixes a cache-coherency problem when extending a file whose last page has just been written. * Removes the "sync" mount option, which had been set unconditionally. * Adjusts some SDT probes * Adds several new tests that mimic what fsx does but with more control and without a real file system. As I discover failures with fsx, I add regression tests to this file. * Adds a test that ensures we can append to a file without reading any data from it.
This change is still incomplete. Clustered writing is not yet supported, and there are frequent "panic: vm_fault_hold: fault on nofault entry" panics that I need to fix.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
ddc51e45 |
| 06-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: remove some stuff that was copy/pasted from nfsclient
fusefs's I/O methods were originally copy/pasted from nfsclient. This commit removes some irrelevant parts, like stuff involving B_NEED
fusefs: remove some stuff that was copy/pasted from nfsclient
fusefs's I/O methods were originally copy/pasted from nfsclient. This commit removes some irrelevant parts, like stuff involving B_NEEDCOMMIT.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
0269ae4c |
| 06-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
MFHead @348740
Sponsored by: The FreeBSD Foundation
|
#
011bca99 |
| 05-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: simplify fuse_write_biobackend. No functional change.
Sponsored by: The FreeBSD Foundation
|
#
a639731b |
| 04-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: respect RLIMIT_FSIZE
Sponsored by: The FreeBSD Foundation
|
#
d4fd0c81 |
| 28-May-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: set the flags fields of fuse_write_in and fuse_read_in
These fields are supposed to contain the file descriptor flags as supplied to open(2) or set by fcntl(2). The feature is kindof useles
fusefs: set the flags fields of fuse_write_in and fuse_read_in
These fields are supposed to contain the file descriptor flags as supplied to open(2) or set by fcntl(2). The feature is kindof useless on FreeBSD since we don't supply all of these flags to fuse (because of the weak relationship between struct file and struct vnode). But we should at least set the access mode flags (O_RDONLY, etc).
This is the last fusefs change needed to get full protocol 7.9 support. There are still a few options we don't support for good reason (mandatory file locking is dumb, flock support is broken in the protocol until 7.17, etc), but there's nothing else to do at this protocol level.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
bda39894 |
| 27-May-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: set FUSE_WRITE_CACHE when writing from cache
This bit tells the server that we're not sure which uid, gid, and/or pid originated the write. I don't know of a single file system that cares,
fusefs: set FUSE_WRITE_CACHE when writing from cache
This bit tells the server that we're not sure which uid, gid, and/or pid originated the write. I don't know of a single file system that cares, but it's part of the protocol.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
65417f5e |
| 24-May-2019 |
Alan Somers <asomers@FreeBSD.org> |
Remove "struct ucred*" argument from vtruncbuf
vtruncbuf takes a "struct ucred*" argument. AFAICT, it's been unused ever since that function was first added in r34611. Remove it. Also, remove some
Remove "struct ucred*" argument from vtruncbuf
vtruncbuf takes a "struct ucred*" argument. AFAICT, it's been unused ever since that function was first added in r34611. Remove it. Also, remove some "struct ucred" arguments from fuse and nfs functions that were only used by vtruncbuf.
Reviewed by: cem MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D20377
show more ...
|
#
e76986fd |
| 24-May-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: fix exporting fuse filesystems with nfsd
A previous commit made fuse exportable via userland NFS servers. Compatibility with the in-kernel nfsd required two more changes:
* During read and
fusefs: fix exporting fuse filesystems with nfsd
A previous commit made fuse exportable via userland NFS servers. Compatibility with the in-kernel nfsd required two more changes:
* During read and write operations, implicitly do a FUSE_OPEN if there isn't already a valid file handle. That's because nfsd never calls VOP_OPEN. * During VOP_READDIR, if an implicit open was necessary, directory offsets from a previous VOP_READDIR may not be valid, so VOP_READDIR may have to start from the beginning and read until it encounters the requested offset.
I've done only limited testing over NFS, so there are probably still some more bugs. Thanks to rmacklem for all of the readdir changes, which he had made for his pnfs work.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
2013b723 |
| 23-May-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: improve attribute cacheing
Consolidate all calls to fuse_vnode_setsize as a result of a file attribute change to one location in fuse_internal_setattr. There are still a few calls elsewhere
fusefs: improve attribute cacheing
Consolidate all calls to fuse_vnode_setsize as a result of a file attribute change to one location in fuse_internal_setattr. There are still a few calls elsewhere that happen as a result of a write.
Sponsored by: The FreeBSD Foundation
show more ...
|
#
18a2264e |
| 23-May-2019 |
Alan Somers <asomers@FreeBSD.org> |
fusefs: fix "recursing on non recursive lockmgr" panic
When mounted with -o default_permissions and when vfs.fusefs.data_cache_mode=2, fuse_io_strategy would try to clear the suid bit after a succes
fusefs: fix "recursing on non recursive lockmgr" panic
When mounted with -o default_permissions and when vfs.fusefs.data_cache_mode=2, fuse_io_strategy would try to clear the suid bit after a successful write by a non-owner. When combined with a not-yet-committed attribute-caching patch I'm working on, and if the FUSE_SETATTR response indicates an unexpected filesize (legal, if the file system has other clients), this would end up calling vtruncbuf. That would panic, because the buffer lock was already held by bufwrite or bufstrategy or something else upstack from fuse_vnop_strategy.
Sponsored by: The FreeBSD Foundation
show more ...
|