Get consistent updates for UFS superblocks.

Superblocks are written by Freebsd clients using the ffs_sbput()
routine (exported from the kernel in sys/ufs/ffs/ffs_subr.c). These
clients include the k

Get consistent updates for UFS superblocks.

Superblocks are written by Freebsd clients using the ffs_sbput()
routine (exported from the kernel in sys/ufs/ffs/ffs_subr.c). These
clients include the kernel, some geom modules, and system utilities
using libufs. Currently backward compatible updates of UFS superblocks
are done only when the kernel writes them. These updates should be
done whenever any call to ffs_sbput() is done. This commit moves
these UFS superblock compatibility updates from the kernel-specific
superblock update to ffs_sbput() so that all clients make these
compatibility updates when they write superblocks.

Reviewed-by: kib
Tested-by: Peter Holm
MFC-after: 1 week
Differential Revision: https://reviews.freebsd.org/D49276
Sponsored-by: Netflix

show more ...

Defer the January 19, 2038 date limit in UFS1 filesystems to February 7, 2106

UFS1 uses a signed 32-bit value for its times. Zero is
January 1, 1970 UTC. Negative values of 32-bit time predate
Janua

Defer the January 19, 2038 date limit in UFS1 filesystems to February 7, 2106

UFS1 uses a signed 32-bit value for its times. Zero is
January 1, 1970 UTC. Negative values of 32-bit time predate
January 1, 1970 back to December 13, 1901. The maximum positive
value for 32-bit time is on January 19, 2038 (my 84th birthday).
On that date, time will go negative and start registering from
December 13, 1901. Note that this issue only affects UFS1 filesystems
since UFS2 has 64-bit times. This fix changes UFS1 times from
signed to unsigned 32-bit values. With this change it will no longer
be possible to represent time from before January 1, 1970, but it
will accurately track time until February 7, 2106. Hopefully there
will not be any FreeBSD systems using UFS1 still in existence by
that time (and by then I will have been dead long enough that no-one
will know at whom to yell :-).

It is possible that some existing UFS1 systems will have set times
predating January 1, 1970. With this commit they will appear as
later than the current time. This commit checks inode times when
they are read into memory and if they are greater than the current
time resets them to the current time. By default this reset happens
silently, but setting the sysctl vfs.ffs.prttimechgs=1 will cause
console messages to be printed whenever a future time is changed.

Reviewed-by: kib
Tested-by: Peter Holm
MFC-after: 1 week
Differential Revision: https://reviews.freebsd.org/D48472

show more ...

Fix backward compatibility with UFS1 filesystems created before June 2002

The code to handle compatibility with old filesystems exists and does
the right things. The loss of compatibility was introd

Fix backward compatibility with UFS1 filesystems created before June 2002

The code to handle compatibility with old filesystems exists and does
the right things. The loss of compatibility was introduced 2.5 years ago
when the superblock integrity checks were added. The problem is that the
compatibility code was being run after the new integrity checks rather
than before them.

The reason that it has taken so long to show up is because it only
affected filesystems created before UFS2 got added in June 2002 and
that had never been mounted read-write (as their superblocks would
be updated by the compatibility code if they could be written).
Hence the problem was not apparent until someone tried to mount a
pre-2002 virgin UFS1 filesystem image.

See the Pahabricator review for more details.

Reviewed-by: kib
Tested-by: Peter Holm
MFC-after: 1 week
Differential-Revision: https://reviews.freebsd.org/D48472

show more ...

Use ffs_sbget() to replace open-coded superblock reload.

No functional change intended.

Tested-by: Peter Holm
MFC-after: 1 week

fs: Add new VFCF_xxx flags for va_filerev

Richard Kojedzinszky <richard@kojedz.in> reported a problem via
email, where the Linux NFSv4.2 client did not detect a change in a
directory on a FreeBSD NF

fs: Add new VFCF_xxx flags for va_filerev

Richard Kojedzinszky <richard@kojedz.in> reported a problem via
email, where the Linux NFSv4.2 client did not detect a change in a
directory on a FreeBSD NFSv4.2 server.

Adding support for the NFSv4.2 change_attr_type attribute seems
to have fixed the problem. This requires that the server file system
indicate if it increments va_filerev by one, since that file attribute
is used for the NFSv4.2 change attribute. Fuse requires an indication
that va_filerev is based on ctime.

This patch adds VFCF_FILEREVINC and VFCF_FILEREVCT to indicate this.

A future patch to the NFS server will use these flags.

Reviewed by: asomers, kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D48436

show more ...

buf: Add a runningbufclaim() helper

No functional change intended.

Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D47696

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Clean up and document UFS/FFS error returns.

The ffs_inotovp() function returns a vnode from a mounted filesystem
for an inode number with specified generation number. We now
consistently return EST

Clean up and document UFS/FFS error returns.

The ffs_inotovp() function returns a vnode from a mounted filesystem
for an inode number with specified generation number. We now
consistently return ESTALE if the inode with given generation number
no longer exists on that filesystem.

The ffs_reload() function reloads all incore data for a filesystem.
It is used after running fsck on a mounted filesystem and finding
things to fix. It now returns the EINTEGRITY error if it is unable
to find a valid superblock.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

UFS: panic rather than forcibly unmount the root fs

If the root fs is forcibly unmounted then basically every process
will die with a SEGV as soon as it tries to run again because libc.so
is gone, w

UFS: panic rather than forcibly unmount the root fs

If the root fs is forcibly unmounted then basically every process
will die with a SEGV as soon as it tries to run again because libc.so
is gone, which leaves the system basically hung. It seems better
to just panic instead, so let's do that.

Requested-by: karels
Reviewed-by: imp, mckusick, karels
Sponsored-by: Netflix
Differential Revision: https://reviews.freebsd.org/D41387

show more ...

UFS/FFS: Migrate to modern uintXX_t from u_intXX_t.

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t. While here also migrate u_char to uint8_

UFS/FFS: Migrate to modern uintXX_t from u_intXX_t.

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t. While here also migrate u_char to uint8_t.
Where other kernel interfaces allow, migrate u_long to uint64_t.

No functional changes intended.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

ufs: stop using LK_SLEEPFAIL in ffs_sync

It provides nothing as either locking succeeds or fails with ENOENT as
is.

Stop cleaning MNT_LOCAL on unmount

There is no point in clearing just this flag. Flags are reset on the
struct mount re-allocation for reuse anyway.

Reviewed by: mckusick
Sponsored by: The FreeBSD

Stop cleaning MNT_LOCAL on unmount

There is no point in clearing just this flag. Flags are reset on the
struct mount re-allocation for reuse anyway.

Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37966

show more ...

vfs: add the concept of vnode state transitions

To quote from a comment above vput_final:
<quote>
* XXX Some filesystems pass in an exclusively locked vnode and strongly depend
* on the lock being h

vfs: add the concept of vnode state transitions

To quote from a comment above vput_final:
<quote>
* XXX Some filesystems pass in an exclusively locked vnode and strongly depend
* on the lock being held all the way until VOP_INACTIVE. This in particular
* happens with UFS which adds half-constructed vnodes to the hash, where they
* can be found by other code.
</quote>

As is there is no mechanism which allows filesystems to denote that a
vnode is fully initialized, consequently problems like the above are
only found the hard way(tm).

Add rudimentary support for state transitions, which in particular allow
to assert the vnode is not legally unlocked until its fate is decided
(either construction finishes or vgone is called to abort it).

The new field lands in a 1-byte hole, thus it does not grow the struct.

Bump __FreeBSD_version to 1400077

Reviewed by: kib (previous version)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D37759

show more ...

mntfs: change mntfs_allocvp API to relock on its own

Reviewed by: kib
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D37759

Provide more precise mount(8) failure message.

Suggested by: Xin LI
Reviewed by: kib
PR: 19683
MFC after: 1 week

When taking a snapshot on a UFS/FFS filesystem, it must be mounted.

The "update" mount option must be specified when the "snapshot"
mount option is used. Return EINVAL if the "snapshot" option is
sp

When taking a snapshot on a UFS/FFS filesystem, it must be mounted.

The "update" mount option must be specified when the "snapshot"
mount option is used. Return EINVAL if the "snapshot" option is
specified without the "update" option also requested.

Reported by: Robert Morris
Reviewed by: kib
PR: 265362
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

show more ...

Explicitly initialize rather than reading newly allocated UFS inodes.

The function ffs_vgetf() is used to find or load UFS inodes into a
vnode. It first looks up the inode and if found in the cache

Explicitly initialize rather than reading newly allocated UFS inodes.

The function ffs_vgetf() is used to find or load UFS inodes into a
vnode. It first looks up the inode and if found in the cache its
vnode is returned. If it is not already in the cache, a new vnode
is allocated and its associated inode read in from the disk. The
read is done even for inodes that are being initially created.
The contents for the inode on the disk are assumed to be empty. If
the on-disk contents had been corrupted either due to a hardware
glitch or an agent deliberately trying to exploit the system, the
UFS code could panic from the unexpected partially-allocated inode.

Rather then having fsck_ffs(8) verify that all unallocated inodes
are properly empty, it is easier and quicker to add a flag to
ffs_vgetf() to indicate that the request is for a newly allocated
inode. When set, the disk read is skipped and the inode is set to
its expected empty (zero'ed out) initial state. As a side benefit,
an unneeded disk I/O is avoided.

Reported by: Peter Holm
Sponsored by: The FreeBSD Foundation

show more ...

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, ha

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.

This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.

The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.

The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.

The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.

Reviewed by: kib
Reviewed by: Warner Losh
Tested by: Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation

show more ...

Add a flags parameter to the ffs_sbget() function that reads UFS superblocks.

Rather than trying to shoehorn flags into the requested superblock
address, create a separate flags parameter to the ffs

Add a flags parameter to the ffs_sbget() function that reads UFS superblocks.

Rather than trying to shoehorn flags into the requested superblock
address, create a separate flags parameter to the ffs_sbget()
function in sys/ufs/ffs/ffs_subr.c. The ffs_sbget() function is
used both in the kernel and in user-level utilities through export
to the sbget() function in the libufs(3) library (see sbget(3)
for details). The kernel uses ffs_sbget() when mounting UFS
filesystems, in the glabel(8) and gjournal(8) GEOM utilities,
and in the standalone library used when booting the system
from a UFS root filesystem.

The ffs_sbget() function reads the superblock located at the byte
offset specified by its sblockloc parameter. The value UFS_STDSB
may be specified for sblockloc to request that the standard
location for the superblock be read.

The two existing options are now flags:

UFS_NOHASHFAIL will note if the check hash is wrong but will still
return the superblock. This is used by the bootstrap code to
give the system a chance to come up so that fsck can be run to
correct the problem.

UFS_NOMSG indicates that superblock inconsistency error messages
should not be printed. It is used by programs like fsck that
want to print their own error message and programs like glabel(8)
that just want to know if a UFS filesystem exists on a partition.

One additional flag is added:

UFS_NOCSUM causes only the superblock itself to be returned, but does
not read in any auxiliary data structures like the cylinder group
summary information. It is used by clients like glabel(8) that
just want to check for possible filesystem types. Using UFS_NOCSUM
skips the superblock checks for csum data which allows superblocks
that have corrupted csum data to be read and used.

The validate_sblock() function checks that the superblock has not
been corrupted in a way that can crash or hang the system. Unless
the UFS_NOMSG flag is specified, it will print out any errors that
it finds. Prior to this commit, validate_sblock() returned as soon
as it found an inconsistency so would print at most one message.
It now does all its checks so when UFS_NOMSG has not been specified
will print out everything that it finds inconsistent.

Sponsored by: The FreeBSD Foundation

show more ...

Rewrite function definitions in the UFS/FFS code base with identifier lists.

The K&R style in UFS and other places in the tree's days are numbered
as this syntax is removed in C2x proposal N2432:

Rewrite function definitions in the UFS/FFS code base with identifier lists.

The K&R style in UFS and other places in the tree's days are numbered
as this syntax is removed in C2x proposal N2432:

https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2432.pdf

Though running to nearly 6000 lines of diffs this update should cause
no functional change to the code.

Requested by: Warner Losh
MFC after: 2 weeks

show more ...

ffs: fix vn_read_from_obj() usage for PAGE_SIZE > block size

vn_read_from_obj() requires that all pages of a vnode (except the last
partial page) be either completely valid or completely invalid,
bu

ffs: fix vn_read_from_obj() usage for PAGE_SIZE > block size

vn_read_from_obj() requires that all pages of a vnode (except the last
partial page) be either completely valid or completely invalid,
but for file systems with block size smaller than PAGE_SIZE,
partially valid pages may exist anywhere in the file.
Do not enable the vn_read_from_obj() path in this case.

Reviewed by: mckusick, kib, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D34836

show more ...

UFS: clear fs_fmod once more, in the buffer data copy.

This is needed for in-kernel copy of the code, where allocation might
happen after fs_fmod is cleared in ffs_sbput() but before the write.

Rep

UFS: clear fs_fmod once more, in the buffer data copy.

This is needed for in-kernel copy of the code, where allocation might
happen after fs_fmod is cleared in ffs_sbput() but before the write.

Reported by: markj
Reviewed by: chs, markj
PR: 263765
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D35149

show more ...

ffs_use_bwrite: make the superblock snapshot more consistent

Copy in-memory struct fs to the superblock buffer under the UFS mutex.

Reviewed by: chs, markj
PR: 263765
Sponsored by: The FreeBSD Foun

ffs_use_bwrite: make the superblock snapshot more consistent

Copy in-memory struct fs to the superblock buffer under the UFS mutex.

Reviewed by: chs, markj
PR: 263765
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D35149

show more ...

ffs: wait for trims earlier during unmount to avoid panic

All softdep processing is supposed to be completed by
softdep_flushfiles() and no more deps are supposed to be created after
that, but if a

ffs: wait for trims earlier during unmount to avoid panic

All softdep processing is supposed to be completed by
softdep_flushfiles() and no more deps are supposed to be created after
that, but if a pending trim completes after softdep_flushfiles() and
before softdep_unmount() then the blkfree that is performed by
ffs_blkfree_trim_task() will create a dep when none should exist, and
if softdep_unmount() is called before that dep is freed then the
kernel will panic. Prevent this by waiting for trims to complete
earlier in the unmount process, in ffs_flushfiles(), so that any deps
will be freed and any modified CG buffers will be flushed by the final
fsync of the devvp in ffs_flushfiles() as intended.

Reviewed by: mckusick, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D34806

show more ...