Purge more stray embedded $FreeBSD$ strings

These do not use __FBSDID but instead use bare char arrays.

Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D41957

Add quitclean command to fsdb(8) to request a filesystem not be marked dirty.

A new command, quitclean, is added to fsdb(8) to request that the
filesystem not be marked as needing a full fsck(8). Th

Add quitclean command to fsdb(8) to request a filesystem not be marked dirty.

A new command, quitclean, is added to fsdb(8) to request that the
filesystem not be marked as needing a full fsck(8). This is useful
when creating deliberately bad filesystem images to be used to check
that fsck is properly able to clean them up.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

Have fsdb(8) only mark a filesystem dirty when it is modified.

Until this update, the fsdb(8) command always marked a filesystem
as needing a full fsck unless it was run with the -n flag which
allow

Have fsdb(8) only mark a filesystem dirty when it is modified.

Until this update, the fsdb(8) command always marked a filesystem
as needing a full fsck unless it was run with the -n flag which
allowed no changes to be made.

This change tracks modifications to the filesystem. Two types of
changes are tracked. The first type of changes are those that are
not critical to the integrity of the filesystem such as changes to
owner, group, time stamps, access mode, and generation number. The
second type of changes are those that do affect the integrity of
the filesystem including zeroing inodes, changing block pointers,
directory entries, link counts, file lengths, file types, and file
flags.

When quitting having made no changes or only changes to data that
is not critical to filesystem integrity, the clean state of the
filesystem is left unchanged. But if filesystem critical data are
changed then fsdb will set the unclean flag which will require a
full fsck to be run before the filesystem can be mounted.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

Add `chdb' command to fsdb(8) to set direct block numbers.

Add the ability to set direct blocks numbers in inodes so that manual
corrections can be made. No checking of the values is attempted so
ac

Add `chdb' command to fsdb(8) to set direct block numbers.

Add the ability to set direct blocks numbers in inodes so that manual
corrections can be made. No checking of the values is attempted so
accidental or deliberate bad values can be set.

Submitted by: Chuck Silvers
MFC after: 1 week

show more ...

Improvement in UFS/FFS directory placement when doing mkdir(2).

The algorithm for laying out new directories was devised in the 1980s
and markedly improved the performance of the filesystem. In thos

Improvement in UFS/FFS directory placement when doing mkdir(2).

The algorithm for laying out new directories was devised in the 1980s
and markedly improved the performance of the filesystem. In those days
large disks had at most 100 cylinder groups and often as few as 10-20.
Modern multi-terrabyte disks have thousands of cylinder groups. The
original algorithm does not handle these large sizes well. This change
attempts to expand the scope of the original algorithm to work well
with these much larger disks while still retaining the properties
of the original algorithm for small disks.

The filesystem implementation is divided into policy routines and
implementation routines. The policy routines can be changed in any
way desired without risk of corrupting the filesystem. The policy
requests are handled by the implementation layer. If the policy
asks for an available resource, it is granted. But if it asks for
an already in-use resource, then the implementation will provide
an available one nearby the request. Thus it is impossible for a
policy to double allocate. This change is limited to the policy
implementation.

This change updates the ffs_dirpref() routine which is responsible
for selecting the cylinder group into which a new directory should
be placed. If we are near the root of the filesystem we aim to
spread them out as much as possible. As we descend deeper from the
root we cluster them closer together around their parent as we
expect them to be more closely interactive. Higher-level directories
like usr/src/sys and usr/src/bin should be separated while the
directories in these areas are more likely to be accessed together
so should be closer. And directories within commands or kernel
subsystems should be closer still.

We pick a range of cylinder groups around the cylinder group of the
directory in which we are being created. The size of the range for
our search is based on our depth from the root of our filesystem.
We then probe that range based on how many directories are already
present. The first new directory is at 1/2 (middle) of the range;
the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8,
5/8, 7/8, 1/16, 3/16, 5/16, etc.

It is desirable to store the depth of a directory in its on-disk
inode so that it is available when we need it. We add a new field
di_dirdepth to track the depth of each directory. Because there are
few spare fields left in the inode, we choose to share an existing
field in the inode rather than having one of our own. Specifically
we create a union with the di_freelink field. The di_freelink field
is used to track inodes that have been unlinked but remain referenced.
It is not needed until a rmdir(2) operation has been done on a
directory. At that point, the directory has no contents and even
if it is kept active as a current directory is no longer able to
have any new directories or files created in it. Thus the use of
di_dirdepth and di_freelink will never coincide.

Reported by: Timo Voelker
Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39246

show more ...

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, ha

Move the ability to search for alternate UFS superblocks from fsck_ffs(8)
into ffs_sbsearch() to allow use by other parts of the system.

Historically only fsck_ffs(8), the UFS filesystem checker, had code
to track down and use alternate UFS superblocks. Since fsdb(8) used
much of the fsck_ffs(8) implementation it had some ability to track
down alternate superblocks.

This change extracts the code to track down alternate superblocks
from fsck_ffs(8) and puts it into a new function ffs_sbsearch() in
sys/ufs/ffs/ffs_subr.c. Like ffs_sbget() and ffs_sbput() also found
in ffs_subr.c, these functions can be used directly by the kernel
subsystems. Additionally they are exported to the UFS library,
libufs(8) so that they can be used by user-level programs. The new
functions added to libufs(8) are sbfind(3) that is an alternative
to sbread(3) and sbsearch(3) that is an alternative to sbget(3).
See their manual pages for further details.

The utilities that have been changed to search for superblocks are
dumpfs(8), fsdb(8), ffsinfo(8), and fsck_ffs(8). Also, the prtblknos(8)
tool found in tools/diag/prtblknos searches for superblocks.

The UFS specific mount code uses the superblock search interface
when mounting the root filesystem and when the administrator doing
a mount(8) command specifies the force flag (-f). The standalone UFS
boot code (found in stand/libsa/ufs.c) uses the superblock search
code in the hope of being able to get the system up and running so
that fsck_ffs(8) can be used to get the filesystem cleaned up.

The following utilities have not been changed to search for
superblocks: clri(8), tunefs(8), snapinfo(8), fstyp(8), quot(8),
dump(8), fsirand(8), growfs(8), quotacheck(8), gjournal(8), and
glabel(8). When these utilities fail, they do report the cause of
the failure. The one exception is the tasting code used to try and
figure what a given disk contains. The tasting code will remain
silent so as not to put out a slew of messages as it trying to taste
every new mass storage device that shows up.

Reviewed by: kib
Reviewed by: Warner Losh
Tested by: Peter Holm
Differential Revision: https://reviews.freebsd.org/D36053
Sponsored by: The FreeBSD Foundation

show more ...

Update fsdb(8) to reflect new structure of fsck_ffs(8).

The cleanup of fsck_ffs(8) in commit c0bfa109b942659f6 broke fsdb(8).
This commit adds the one-line update needed in fsdb(8) to make it
work w

Update fsdb(8) to reflect new structure of fsck_ffs(8).

The cleanup of fsck_ffs(8) in commit c0bfa109b942659f6 broke fsdb(8).
This commit adds the one-line update needed in fsdb(8) to make it
work with the new fsck_ffs(8) structure.

Reported by: Chuck Silvers
Tested by: Chuck Silvers
MFC after: 3 days

show more ...

Revert "fsdb: add missing bufinit() call"

This reverts commit 84768d114951e88288024f09d4beae0956c3cf21.

fsdb: add missing bufinit() call

The bufinit() call in fsck_ffs was moved in commit f190f9193bc10
from a function that is shared with fsdb to one that is private to fsck_ffs,
so add a bufinit() call

fsdb: add missing bufinit() call

The bufinit() call in fsck_ffs was moved in commit f190f9193bc10
from a function that is shared with fsdb to one that is private to fsck_ffs,
so add a bufinit() call in fsdb to compensate for that.

Reviewed by: mckusick
Sponsored by: Netflix

show more ...

Rewrite the disk I/O management system in fsck_ffs(8). Other than
making fsck_ffs(8) run faster, there should be no functional change.

The original fsck_ffs(8) had its own disk I/O management system

Rewrite the disk I/O management system in fsck_ffs(8). Other than
making fsck_ffs(8) run faster, there should be no functional change.

The original fsck_ffs(8) had its own disk I/O management system.
When gjournal(8) was added to FreeBSD 7, code was added to fsck_ffs(8)
to do the necessary gjournal rollback. Rather than use the existing
fsck_ffs(8) disk I/O system, it wrote its own from scratch. Similarly
when journalled soft updates were added in FreeBSD 9, code was added
to fsck_ffs(8) to do the necessary journal rollback. And once again,
rather than using either of the existing fsck_ffs(8) disk I/O
systems, it wrote its own from scratch. Lastly the fsdb(8) utility
uses the fsck_ffs(8) disk I/O management system. In preparation for
making the changes necessary to enable snapshots to be taken when
using journalled soft updates, it was necessary to have a single
disk I/O system used by all the various subsystems in fsck_ffs(8).

This commit merges the functionality required by all the different
subsystems into a single disk I/O system that supports all of their
needs. In so doing it picks up optimizations from each of them
with the results that each of the subsystems does fewer reads and
writes than it did with its own customized I/O system. It also
greatly simplifies making changes to fsck_ffs(8) since everything
goes through a single place. For example the ginode() function
fetches an inode from the disk. When inode check hashes were added,
they previously had to be checked in the code implementing inode
fetch in each of the three different disk I/O systems. Now they
need only be checked in ginode().

Tested by: Peter Holm
Sponsored by: Netflix

show more ...

The fsdb(8) utility uses the fsck_ffs(8) disk I/O interfaces, so
switch from using libufs's bread() to using fsck_ffs's getdatablk()
when importing tools/diag/prtblnos's prtblknos().

Sponsored by: N

The fsdb(8) utility uses the fsck_ffs(8) disk I/O interfaces, so
switch from using libufs's bread() to using fsck_ffs's getdatablk()
when importing tools/diag/prtblnos's prtblknos().

Sponsored by: Netflix

show more ...

fsck_ffs/fsdb: fix -fno-common build

This one is also a small list:

- 3x duplicate definition (ufs2_zino, returntosingle, nflag)
- 5x 'needs extern', 3/5 of which are referenced in fsdb

-fno-commo

fsck_ffs/fsdb: fix -fno-common build

This one is also a small list:

- 3x duplicate definition (ufs2_zino, returntosingle, nflag)
- 5x 'needs extern', 3/5 of which are referenced in fsdb

-fno-common will become the default in GCC10/LLVM11.

MFC after: 1 week

show more ...

MFhead@r344786

Merge ^/head r344549 through r344775.

After a crash, a file that extends into indirect blocks may end up
shorter than its size resulting in a hole as its final block (which
is a violation of the invarients of the UFS filesystem).

Soft u

After a crash, a file that extends into indirect blocks may end up
shorter than its size resulting in a hole as its final block (which
is a violation of the invarients of the UFS filesystem).

Soft updates will always ensure that the file size is correct when
writing inodes to disk for files that contain only direct block
pointers. However soft updates does not roll back sizes for files
with indirect blocks that it has set to unallocated because their
contents have not yet been written to disk. Hence, the file can
appear to have a hole at its end because the block pointer has been
rolled back to zero when its inode was written to disk. Thus,
fsck_ffs calculates the last allocated block in the file. For files
that extend into indirect blocks, fsck_ffs checks for a size past
the last allocated block of the file and if that is found, shortens
the file to reference the last allocated block thus avoiding having
it reference a hole at its end.

Submitted by: Chuck Silvers <chs@netflix.com>
Tested by: Chuck Silvers <chs@netflix.com>
MFC after: 1 week
Sponsored by: Netflix

show more ...

Merge ^/head r339813 through r340125.

In preparation for adding inode check-hashes, change the fsck_ffs
inodirty() function to have a pointer to the inode being dirtied.
No functional change (as for now the parameter is ununsed).

Sponso

In preparation for adding inode check-hashes, change the fsck_ffs
inodirty() function to have a pointer to the inode being dirtied.
No functional change (as for now the parameter is ununsed).

Sponsored by: Netflix

show more ...

Revert r313780 (UFS_ prefix)

Prefix UFS symbols with UFS_ to reduce namespace pollution

Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by: kib
Reviewed by: kib, mckusick
Sponsor

Prefix UFS symbols with UFS_ to reduce namespace pollution

Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by: kib
Reviewed by: kib, mckusick
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9623

show more ...

Fix buildworld after r328075, by also renaming cgget to cglookup in
fsdb.

Reported by: ohartmann@walstatt.org,david@catwhisker.org
Pointy hat to: mckusick

Merge ^/head r327169 through r327340.

The fix in r327273 turns a memory leak into freeing wild pointer.
Fix this by freeing only the initialized pointer.

Plug memory leak by freeing wantedblk{32,64}.

CID: 273655, 273656

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

show more ...

Merge ^/head r313644 through r313895.