fusefs: fix warnings in the tests reported by GCC

Sponsored by: The FreeBSD Foundation

fusefs: annotate deliberate file descriptor leaks in the tests

closing a file descriptor causes FUSE activity that is superfluous to the
purpose of most tests, but would nonetheless require matching

fusefs: annotate deliberate file descriptor leaks in the tests

closing a file descriptor causes FUSE activity that is superfluous to the
purpose of most tests, but would nonetheless require matching expectations.
Rather than do that, most tests deliberately leak file descriptors instead.
This commit moves the leakage from each test into two trivial functions:
leak and leakdir. Hopefully Coverity will only complain about those
functions and not all of their callers.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: implement protocol 7.23's FUSE_WRITEBACK_CACHE option

As of protocol 7.23, fuse file systems can specify their cache behavior on a
per-mountpoint basis. If they set FUSE_WRITEBACK_CACHE in

fusefs: implement protocol 7.23's FUSE_WRITEBACK_CACHE option

As of protocol 7.23, fuse file systems can specify their cache behavior on a
per-mountpoint basis. If they set FUSE_WRITEBACK_CACHE in
fuse_init_out.flags, then they'll get the writeback cache. If not, then
they'll get the writethrough cache. If they set FOPEN_DIRECT_IO in every
FUSE_OPEN response, then they'll get no cache at all.

The old vfs.fusefs.data_cache_mode sysctl is ignored for servers that use
protocol 7.23 or later. However, it's retained for older servers,
especially for those running in jails that lack access to the new protocol.

This commit also fixes two other minor test bugs:
* WriteCluster:SetUp was using an uninitialized variable.
* Read.direct_io_pread wasn't verifying that the cache was actually
bypassed.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: implement the "time_gran" feature.

If a server supports a timestamp granularity other than 1ns, it can tell the
client this as of protocol 7.23. The client will use that granularity when
up

fusefs: implement the "time_gran" feature.

If a server supports a timestamp granularity other than 1ns, it can tell the
client this as of protocol 7.23. The client will use that granularity when
updating its cached timestamps during write. This way the timestamps won't
appear to change following flush.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: set ctime during FUSE_SETATTR following a write

As of r349396 the kernel will internally update the mtime and ctime of files
on write. It will also flush the mtime should a SETATTR happen b

fusefs: set ctime during FUSE_SETATTR following a write

As of r349396 the kernel will internally update the mtime and ctime of files
on write. It will also flush the mtime should a SETATTR happen before the
data cache gets flushed. Now it will flush the ctime too, if the server is
using protocol 7.23 or higher.

This is the only case in which the kernel will explicitly set a file's
ctime, since neither utimensat(2) nor any other user interfaces allow it.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: automatically update mtime and ctime on write

Writing should implicitly update a file's mtime and ctime. For fuse, the
server is supposed to do that. But the client needs to do it too, bec

fusefs: automatically update mtime and ctime on write

Writing should implicitly update a file's mtime and ctime. For fuse, the
server is supposed to do that. But the client needs to do it too, because
the FUSE_WRITE response does not include time attributes, and it's not
desirable to issue a GETATTR after every WRITE. When using the writeback
cache, there's another hitch: the kernel should ignore the mtime and ctime
fields in any GETATTR response for files with a dirty write cache.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: fix the tests for non-default values of MAXPHYS

Sponsored by: The FreeBSD Foundation

fusefs: fix the tests for nondefault values of vfs.maxbcachebuf

Sponsored by: The FreeBSD Foundation

fusefs: writes should update the file size, even when data_cache_mode=0

Writes that extend a file should update the file's size. r344185 restricted
that behavior for fusefs to only happen when the

fusefs: writes should update the file size, even when data_cache_mode=0

Writes that extend a file should update the file's size. r344185 restricted
that behavior for fusefs to only happen when the data cache was enabled.
That probably made sense at the time because the attribute cache wasn't
fully baked yet. Now that it is, we should always update the cached file
size during write.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: rewrite vop_getpages and vop_putpages

Use the standard facilities for getpages and putpages instead of bespoke
implementations that don't work well with the writeback cache. This has
severa

fusefs: rewrite vop_getpages and vop_putpages

Use the standard facilities for getpages and putpages instead of bespoke
implementations that don't work well with the writeback cache. This has
several corollaries:

* Change the way we handle short reads _again_. vfs_bio_getpages doesn't
provide any way to handle unexpected short reads. Plus, I found some more
lock-order problems. So now when the short read is detected we'll just
clear the vnode's attribute cache, forcing the file size to be requeried
the next time it's needed. VOP_GETPAGES doesn't have any way to indicate
a short read to the "caller", so we just bzero the rest of the page
whenever a short read happens.

* Change the way we decide when to set the FUSE_WRITE_CACHE bit. We now set
it for clustered writes even when the writeback cache is not in use.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: correctly handle short reads

A fuse server may return a short read for three reasons:

* The file is opened with FOPEN_DIRECT_IO. In this case, the short read
should be returned directly

fusefs: correctly handle short reads

A fuse server may return a short read for three reasons:

* The file is opened with FOPEN_DIRECT_IO. In this case, the short read
should be returned directly to userland. We already handled this case
correctly.

* The file was truncated server-side, and the read hit EOF. In this case,
the kernel should update the file size. Fixed in the case of VOP_READ.
Fixing this for VOP_GETPAGES is TODO.

* The file is opened in writeback mode, there are dirty buffers past what
the server thinks is the file's EOF, and the read hit what the server
thinks is the file's EOF. In this case, the client is trying to read a
hole, and should zero-fill it. We already handled this case, and I added
a test for it.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: multiple fixes related to the write cache

* Don't always write the last page synchronously. That's not actually
required. It was probably just masking another bug that I fixed later,
p

fusefs: multiple fixes related to the write cache

* Don't always write the last page synchronously. That's not actually
required. It was probably just masking another bug that I fixed later,
possibly in r349021.

* Enable the NotifyWriteback tests now that Writeback cache is working.

* Add a test to ensure that the write cache isn't flushed synchronously when
in writeback mode.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: skip the Write.mmap test when mmap is not available

fusefs doesn't not allow mmap when data caching is disabled.

Sponsored by: The FreeBSD Foundation

fusefs: fix the "write-through" of write-through cacheing

Our fusefs(5) module supports three cache modes: uncached, write-through,
and write-back. However, the write-through mode (which is the def

fusefs: fix the "write-through" of write-through cacheing

Our fusefs(5) module supports three cache modes: uncached, write-through,
and write-back. However, the write-through mode (which is the default) has
never actually worked as its name suggests. Rather, it's always been more
like "write-around". It wrote directly, bypassing the cache. The cache
would only be populated by a subsequent read of the same data.

This commit fixes that problem. Now the write-through mode works as one
would expect: write(2) immediately adds data to the cache and then blocks
while the daemon processes the write operation.

A side effect of this change is that non-cache-block-aligned writes will now
incur a read-modify-write cycle of the cache block. The old behavior
(bypassing write cache entirely) can still be achieved by opening a file
with O_DIRECT.

PR: 237588
Sponsored by: The FreeBSD Foundation

show more ...

fusefs: enable write clustering

Enable write clustering in fusefs whenever cache mode is set to writeback
and the "async" mount option is used. With default values for MAXPHYS,
DFLTPHYS, and the fu

fusefs: enable write clustering

Enable write clustering in fusefs whenever cache mode is set to writeback
and the "async" mount option is used. With default values for MAXPHYS,
DFLTPHYS, and the fuse max_write mount parameter, that means sequential
writes will now be written 128KB at a time instead of 64KB.

Also, add a regression test for PR 238565, a panic during unmount that
probably affects UFS, ext2, and msdosfs as well as fusefs.

PR: 238565
Sponsored by: The FreeBSD Foundation

show more ...

fusefs: WIP fixing writeback cacheing

The current "writeback" cache mode, selected by the
vfs.fusefs.data_cache_mode sysctl, doesn't do writeback cacheing at all. It
merely goes through the motions

fusefs: WIP fixing writeback cacheing

The current "writeback" cache mode, selected by the
vfs.fusefs.data_cache_mode sysctl, doesn't do writeback cacheing at all. It
merely goes through the motions of using buf(9), but then writes every
buffer synchronously. This commit:

* Enables delayed writes when the sysctl is set to writeback cacheing
* Fixes a cache-coherency problem when extending a file whose last page has
just been written.
* Removes the "sync" mount option, which had been set unconditionally.
* Adjusts some SDT probes
* Adds several new tests that mimic what fsx does but with more control and
without a real file system. As I discover failures with fsx, I add
regression tests to this file.
* Adds a test that ensures we can append to a file without reading any data
from it.

This change is still incomplete. Clustered writing is not yet supported,
and there are frequent "panic: vm_fault_hold: fault on nofault entry" panics
that I need to fix.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: respect RLIMIT_FSIZE

Sponsored by: The FreeBSD Foundation

fusefs: set the flags fields of fuse_write_in and fuse_read_in

These fields are supposed to contain the file descriptor flags as supplied
to open(2) or set by fcntl(2). The feature is kindof useles

fusefs: set the flags fields of fuse_write_in and fuse_read_in

These fields are supposed to contain the file descriptor flags as supplied
to open(2) or set by fcntl(2). The feature is kindof useless on FreeBSD
since we don't supply all of these flags to fuse (because of the weak
relationship between struct file and struct vnode). But we should at least
set the access mode flags (O_RDONLY, etc).

This is the last fusefs change needed to get full protocol 7.9 support.
There are still a few options we don't support for good reason (mandatory
file locking is dumb, flock support is broken in the protocol until 7.17,
etc), but there's nothing else to do at this protocol level.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: set FUSE_WRITE_CACHE when writing from cache

This bit tells the server that we're not sure which uid, gid, and/or pid
originated the write. I don't know of a single file system that cares,

fusefs: set FUSE_WRITE_CACHE when writing from cache

This bit tells the server that we're not sure which uid, gid, and/or pid
originated the write. I don't know of a single file system that cares, but
it's part of the protocol.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: make the tests more cplusplusy

* Prefer std::unique_ptr to raw pointers
* Prefer pass-by-reference to pass-by-pointer
* Prefer static_cast to C-style cast, unless it's too much typing

Repor

fusefs: make the tests more cplusplusy

* Prefer std::unique_ptr to raw pointers
* Prefer pass-by-reference to pass-by-pointer
* Prefer static_cast to C-style cast, unless it's too much typing

Reported by: ngie
Sponsored by: The FreeBSD Foundation

show more ...

fusefs: Upgrade FUSE protocol to version 7.9.

This commit upgrades the FUSE API to protocol 7.9 and adds unit tests for
backwards compatibility with servers built for version 7.8. It doesn't
implem

fusefs: Upgrade FUSE protocol to version 7.9.

This commit upgrades the FUSE API to protocol 7.9 and adds unit tests for
backwards compatibility with servers built for version 7.8. It doesn't
implement any of 7.9's new features yet.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: commit missing file from r347547

Sponsored by: The FreeBSD Foundation

fusefs: enable the Write.mmap test

This test had been disabled because it was designed to check protocol
7.9-specific functionality. Enable it without the 7.9-specific bit.

Sponsored by: The FreeB

fusefs: enable the Write.mmap test

This test had been disabled because it was designed to check protocol
7.9-specific functionality. Enable it without the 7.9-specific bit.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: fix cache invalidation error from r346162

An off-by-one error led to the last page of a write not being removed from
its object, even though that page's buffer was marked as invalid.

PR: 2

fusefs: fix cache invalidation error from r346162

An off-by-one error led to the last page of a write not being removed from
its object, even though that page's buffer was marked as invalid.

PR: 235774
Sponsored by: The FreeBSD Foundation

show more ...

fusefs: evict invalidated cache contents during write-through

fusefs's default cache mode is "writethrough", although it currently works
more like "write-around"; writes bypass the cache completely.

fusefs: evict invalidated cache contents during write-through

fusefs's default cache mode is "writethrough", although it currently works
more like "write-around"; writes bypass the cache completely. Since writes
bypass the cache, they were leaving stale previously-read data in the cache.
This commit invalidates that stale data. It also adds a new global
v_inval_buf_range method, like vtruncbuf but for a range of a file.

PR: 235774
Reported by: cem
Sponsored by: The FreeBSD Foundation

show more ...