makefs: Handle special file types when creating a zpool

Previously, anything other than a regular file, directory or symlink
would cause makefs to exit with an assertion failure. Make it a bit
more

makefs: Handle special file types when creating a zpool

Previously, anything other than a regular file, directory or symlink
would cause makefs to exit with an assertion failure. Make it a bit
more resilient to user error: print a warning and skip the file. Add a
regression test wherein we create an image from a devfs mount.

PR: 283583
MFC after: 2 weeks

show more ...

makefs/zfs: Don't set ZFS_NO_EXECS_DENIED in file flags

This flag was leftover from testing and should have been removed.

PR: 274938
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

makefs/zfs: Ensure that the last block of a file has the right size

When copying a file's contents into the pool, a loop copies blocks of
the maximum size (128KB), allocating space from the vdev for

makefs/zfs: Ensure that the last block of a file has the right size

When copying a file's contents into the pool, a loop copies blocks of
the maximum size (128KB), allocating space from the vdev for each block.
The space allocator rounds up to the nearest block size, but for files
larger than the maximum size, this can result in the last block having a
smaller logical size than the rest of the blocks belonging to that
dnode. This violates some ZFS invariants when appending to that file.

Modify fs_populate_file() to use the same block size for the final
block.

Reported by: cperciva
MFC after: 1 week
Sponsored by: The FreeBSD Foundation

show more ...

makefs/zfs: Remove a nonsensical comment

When populating files, makefs needs to copy their contents into
userspace in order to compute a checksum, so copy_file_range(2) is out
of the question. Thou

makefs/zfs: Remove a nonsensical comment

When populating files, makefs needs to copy their contents into
userspace in order to compute a checksum, so copy_file_range(2) is out
of the question. Though, it could possibly be used when building other
types of filesystems.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

makefs: don't needlessly require directories to exist

If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need

makefs: don't needlessly require directories to exist

If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need to exist. Just let openat fail in that case. It's conceivable
this will make debugging some cases weird, but it's sufficent to handle
the way we add /root/.ssh in CheriBSD VM images.

This is a recommit of 794154149f95d0cbc11aade166f9da919747e397 with
bugfixes.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38029

show more ...

Revert "makefs: don't needlessly require directories to exist"

I pushed prematurely and this version is broken.

This reverts commit 794154149f95d0cbc11aade166f9da919747e397.

makefs: don't needlessly require directories to exist

If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need

makefs: don't needlessly require directories to exist

If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need to exist. Just let openat fail in that case. It's conceivable
this will make debugging some cases weird, but it's sufficent to handle
the way we add /root/.ssh in CheriBSD VM images.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38029

show more ...

makefs: handle mtree link= for ZFS

When a link target is specified use it rather than attempting to read
a potentially non-existant file.

Reviewed by: markj
Differential Revision: https://reviews.f

makefs: handle mtree link= for ZFS

When a link target is specified use it rather than attempting to read
a potentially non-existant file.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38028

show more ...

makefs: handle mtree contents= in zfs

When a source path is provided use it rather than constructing one.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38027

makefs: Fix issues building as a cross-tool on non-FreeBSD

This adds missing includes, uses the standard dirent.h rather than the
BSD-specific sys/dirent.h subset (which works on macOS but not Linux

makefs: Fix issues building as a cross-tool on non-FreeBSD

This adds missing includes, uses the standard dirent.h rather than the
BSD-specific sys/dirent.h subset (which works on macOS but not Linux)
and works around Linux's lack of st_birthtim.

This allows usr.sbin/makefs to be added to LOCAL_XTOOL_DIRS again on
macOS and Linux so that disk images can be cross-built.

Reviewed by: markj
Fixes: 240afd8c1fcc ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36135

show more ...

makefs: Fix 32-bit issues in ZFS time attributes setting

Currently the code copies a struct timespec's raw bits as a pair of
uint64_t. On 64-bit systems this has the same representation, but on
32-b

makefs: Fix 32-bit issues in ZFS time attributes setting

Currently the code copies a struct timespec's raw bits as a pair of
uint64_t. On 64-bit systems this has the same representation, but on
32-bit issues there are two issues:

1. tv_sec is a time_t which is 32-bit on i386 specifically
2. tv_nsec is a long not a 64-bit integer

On i386, this means the assertion should fire as the size doesn't match.
On other 32-bit systems there are 4 bytes of padding after tv_nsec,
which in practice are probably 0, as this data is ultimately coming from
the kernel, so it's deterministic (though the padding bytes are not
required to be preserved by the compiler, so are strictly unspecified).
However, on 32-bit big-endian systems, the padding bytes are in the
wrong half to be harmless, resulting in the nanoseconds being multiplied
by 2^32.

Fix this all by marshalling via a real uint64_t pair like is done by the
real ZFS_TIME_ENCODE.

Reviewed by: markj
Fixes: 240afd8c1fcc ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36131

show more ...

makefs: Handle multiple staging directories when creating ZFS pools

The fsnode tree traversal routines used in ZFS mode assume that all
children of a (directory) fsnode can be accessed using a direc

makefs: Handle multiple staging directories when creating ZFS pools

The fsnode tree traversal routines used in ZFS mode assume that all
children of a (directory) fsnode can be accessed using a directory fd
for the parent and the child name. This is true when populating the
image using an mtree manifest or from a single staging directory, but
doesn't work when multiple staging directories are specified.

Change the traversal routines to use absolute path lookups when an mtree
manifest is not in use. This isn't ideal, but it's the simplest way to
fix the problem.

Reported by: imp
Sponsored by: The FreeBSD Foundation

show more ...

makefs: Fix a memory leak in fs_layout_one()

Check the canmount property before building the mountpoint string.

Reported by: Coverity
Fixes: 240afd8c1fcc ("makefs: Add ZFS support")
Sponsored by:

makefs: Fix a memory leak in fs_layout_one()

Check the canmount property before building the mountpoint string.

Reported by: Coverity
Fixes: 240afd8c1fcc ("makefs: Add ZFS support")
Sponsored by: The FreeBSD Foundation

show more ...

makefs: Add ZFS support

This allows one to take a staged directory tree and create a file
consisting of a ZFS pool with one or more datasets that contain the
contents of the directory tree. This is

makefs: Add ZFS support

This allows one to take a staged directory tree and create a file
consisting of a ZFS pool with one or more datasets that contain the
contents of the directory tree. This is useful for creating virtual
machine images without using the kernel to create a pool; "zpool create"
requires root privileges and currently is not permitted in jails.
makefs -t zfs also provides reproducible images by using a fixed seed
for pseudo-random number generation, used for generating GUIDs and hash
salts. makefs -t zfs requires relatively little by way of machine
resources.

The "zpool_reguid" rc.conf setting can be used to ask a FreeBSD guest to
generate a unique pool GUID upon first boot.

A small number of pool and dataset properties are supported. The pool
is backed by a single disk vdev. Data is always checksummed using
Fletcher-4, no redundant copies are made, and no compression is used.
The manual page documents supported pool and filesystem properties.

The implementation uses a few pieces of ZFS support from with the boot
loader, especially definitions for various on-disk structures, but is
otherwise standalone and in particular doesn't depend on OpenZFS.

This feature should be treated as experimental for now, i.e., important
data shouldn't be trusted to a makefs-created pool, and the command-line
interface is subject to change.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35248

show more ...