libbe: avoid copying encryption-related props

libzfs insists that these be cloned from the origin, so avoid making a
deep copy of them ourselves to unbreak creating a new BE from a BE with
encrypted

libbe: avoid copying encryption-related props

libzfs insists that these be cloned from the origin, so avoid making a
deep copy of them ourselves to unbreak creating a new BE from a BE with
encrypted components -- in today's environment, without a loader that
does encryption, this means a deep BE setup where something underneath
the BE (e.g., home directories) are encrypted.

Reported and tested by: arrowd
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D48464

show more ...

lib/libbe/be.c: remove an extra semicolon

Signed-off-by: rilysh <nightquick@proton.me>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/959

libbe: handle destroying/renaming temporary/bootonce boot environments

When a temporary/bootonce boot environment is renamed, we need to also
update the bootenv nvlist on-disk to reflect the new nam

libbe: handle destroying/renaming temporary/bootonce boot environments

When a temporary/bootonce boot environment is renamed, we need to also
update the bootenv nvlist on-disk to reflect the new name. Additionally,
when a temporary/bootonce boot environment is destroyed, we also need to
clear out the on-disk state.

Reviewed by: kevans
Approved by: kp
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D43591

show more ...

lib: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

lib: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

libbe: recursively promote deep BE datasets

beadm will recursively promote deep BE datasets. In order to match the
beadm behavior, we need to recursively iterate over child filesystems
and promote t

libbe: recursively promote deep BE datasets

beadm will recursively promote deep BE datasets. In order to match the
beadm behavior, we need to recursively iterate over child filesystems
and promote them along the way.

This patch further refines the work from D40903, completing the fix for
promotion.

Reviewed by: kevans, rew
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D40972

show more ...

libbe: promote activated BEs all the way

This matches the beadm behavior; generally, we need to keep promoting
until the BE is no longer a clone from a snapshot. This fixes scenarios
where the data

libbe: promote activated BEs all the way

This matches the beadm behavior; generally, we need to keep promoting
until the BE is no longer a clone from a snapshot. This fixes scenarios
where the dataset associated with a BE's origin is itself a clone,
activating the BE previously would promote it to a clone of the origin's
origin.

We could keep using be_get_dataset_props here, except for two
annoyances:

1.) I couldn't find a clean way to just clear an nvlist rather than
having to re-alloc it, and I didn't want to just remove the one prop
we're inspecting out of it.

2.) That's a lot of overhead when all we want to do is fetch the origin
anyways.

Note that this is not a complete fix, but it does fix the majority of
cases; deep BE subordinates are still notably broken, pending a patch
from Christian.

Reported by: R. Christian McDonald <rcm@rcm.sh>
Reviewed by: rew
Differential Revision: https://reviews.freebsd.org/D40903

show more ...

libbe: bail out early if the zfs kmod isn't loaded

As noted in the comment, we already know the rest of libbe_init() will
fail because there's no pool imported. Avoid the side effect by
checking be

libbe: bail out early if the zfs kmod isn't loaded

As noted in the comment, we already know the rest of libbe_init() will
fail because there's no pool imported. Avoid the side effect by
checking beforehand and bailing out early.

With this, freebsd-update(8) should no longer trigger a load of the zfs
kmod just because it runs `bectl check`.

Reviewed by: jwmaag_gmail.com, rew
Differential Revision: https://reviews.freebsd.org/D36188

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

zfs: merge openzfs/zfs@d96e29576

Notable upstream pull request merges:

#11680 Add support for zpool user properties
#14145 Storage device expansion "silently" fails on degraded vdev
#14405 Cr

zfs: merge openzfs/zfs@d96e29576

Notable upstream pull request merges:

#11680 Add support for zpool user properties
#14145 Storage device expansion "silently" fails on degraded vdev
#14405 Create zap for root vdev
#14659 Allow MMP to bypass waiting for other threads
#14674 Miscellaneous FreBSD compilation bugfixes
#14692 Fix some signedness issues in arc_evict()
#14702 Fix typo in check_clones()
#14715 module: small fixes for FreeBSD/aarch64
#14716 Trim needless zeroes from checksum events
#14719 vdev: expose zfs_vdev_max_ms_shift as a module parameter
#14722 Fix "Detach spare vdev in case if resilvering does not happen"
#14723 freebsd clone range fixes
#14728 Fix BLAKE3 aarch64 assembly for FreeBSD and macOS
#14735 Fix in check_filesystem()
#14739 Fix data corruption when cloning embedded blocks
#14758 Fix VERIFY(!zil_replaying(zilog, tx)) panic
#14761 Revert "ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()"
#14774 FreeBSD .zfs fixups
#14776 FreeBSD: make zfs_vfs_held() definition consistent with declaration
#14779 powerpc64: Support ELFv2 asm on Big Endian
#14788 FreeBSD: add missing vop_fplookup assignments
#14789 PAM: support the authentication facility
#14790 Revert "Fix data race between zil_commit() and zil_suspend()"
#14795 Fix positive ABD size assertion in abd_verify()
#14798 Mark TX_COMMIT transaction with TXG_NOTHROTTLE
#14804 Correct ABD size for split block ZIOs
#14806 Use correct block pointer in block cloning case.
#14808 blake3: fix up bogus checksums in face of cpu migration

Obtained from: OpenZFS
OpenZFS commit: d96e29576c89e6e547cb82b477651d2b85ea0fed

show more ...

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Imple

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"

Obtained from: OpenZFS
OpenZFS commit: 431083f75bdd3efaee992bdd672625ec7240d252

show more ...

libbe: Avoid double printing cloning errors.

be_clone calls be_clone_cb and both call set_error on the return
error path. set_error prints the error resulting in a double print.
be_clone_cb should

libbe: Avoid double printing cloning errors.

be_clone calls be_clone_cb and both call set_error on the return
error path. set_error prints the error resulting in a double print.
be_clone_cb should just return the error code and allow be_clone
to print it.

PR: 265248
Reported by: Graham Perrin
Reviewed by: imp, kevans
Pull Request: https://github.com/freebsd/freebsd-src/pull/685

show more ...

zfs: merge openzfs/zfs@9cd71c860 (master)

Notable upstream pull request merges:
#13805 Configure zed's diagnosis engine with vdev properties
#14110 zfs list: Allow more fields in ZFS_ITER_SIMPLE

zfs: merge openzfs/zfs@9cd71c860 (master)

Notable upstream pull request merges:
#13805 Configure zed's diagnosis engine with vdev properties
#14110 zfs list: Allow more fields in ZFS_ITER_SIMPLE mode
#14121 Batch enqueue/dequeue for bqueue
#14123 arc_read()/arc_access() refactoring and cleanup
#14159 Bypass metaslab throttle for removal allocations
#14243 Implement uncached prefetch
#14251 Cache dbuf_hash() calculation
#14253 Allow reciever to override encryption property in case of replication
#14254 Restrict visibility of per-dataset kstats inside FreeBSD jails
#14255 Zero end of embedded block buffer in dump_write_embedded()
#14263 Cleanups identified by CodeQL and Coverity
#14264 Miscellaneous fixes
#14272 Change ZEVENT_POOL_GUID to ZEVENT_POOL to display pool names
#14287 FreeBSD: Remove stray debug printf
#14288 Colorize zfs diff output
#14289 deadlock between spa_errlog_lock and dp_config_rwlock
#14291 FreeBSD: Fix potential boot panic with bad label
#14292 Add tunable to allow changing micro ZAP's max size
#14293 Turn default_bs and default_ibs into ZFS_MODULE_PARAMs
#14295 zed: add hotplug support for spare vdevs
#14304 Activate filesystem features only in syncing context
#14311 zpool: do guid-based comparison in is_vdev_cb()
#14317 Pack zrlock_t by 8 bytes
#14320 Update arc_summary and arcstat outputs
#14328 FreeBSD: catch up to 1400077
#14376 Use setproctitle to report progress of zfs send
#14340 Remove some dead ARC code
#14358 Wait for txg sync if the last DRR_FREEOBJECTS might result in a hole
#14360 libzpool: fix ddi_strtoull to update nptr
#14364 Fix unprotected zfs_znode_dmu_fini
#14379 zfs_receive_one: Check for the more likely error first
#14380 Cleanup of dead code suggested by Clang Static Analyzer
#14397 Avoid passing an uninitialized index to dsl_prop_known_index
#14404 Fix reading uninitialized variable in receive_read
#14407 free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report
#14418 Introduce minimal ZIL block commit delay
#14422 x86 assembly: fix .size placement and replace .align with .balign

Obtained from: OpenZFS
OpenZFS commit: 9cd71c8604d52def22ffaddc35755712f0fb9349

show more ...

bectl: push space-in-name check down into libbe

This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing abo

bectl: push space-in-name check down into libbe

This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing about this restriction over and over again.

While we're here: `bectl rename` always succeeds, even when it doesn't.
Start returning the error.

Reported By: Christian McDonald <cmcdonald netgate com>
Reviewed by: rew, jwmaag_gmail.com (earlier version)
Differential Revision: https://reviews.freebsd.org/D34605

show more ...

When be_activate() turns on a new boot environment, it always deactivates
the current one first. And if it fails to do so, it abandons activation.
However, with the new bootonce feature, there is a l

When be_activate() turns on a new boot environment, it always deactivates
the current one first. And if it fails to do so, it abandons activation.
However, with the new bootonce feature, there is a legitimate case when
a pool doesn't have "bootfs" property set. Check for this case before
calling be_deactivate().

Reviewed by: kevans

show more ...

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, t

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

show more ...

ZFS: MFV 2.0-rc1-gfd20a8

- fixes jail delegation
- fixes raw kstat unsupported complaints
- exposes dbgmsg, dbuf, and checksum benchmark stats
- restore rename -u support

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would a

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

libbe: annotate lbh as __unused in be_is_auto_snapshot_name

lbh is included for consistency with other functions and in case
future work needs to use it, but it is currently unused. Mark it,
and a p

libbe: annotate lbh as __unused in be_is_auto_snapshot_name

lbh is included for consistency with other functions and in case
future work needs to use it, but it is currently unused. Mark it,
and a post-OpenZFS-import world will be able to raise WARNS of
libbe to the default (pending some minor changes to openzfs libzfs).

MFC after: 3 days

show more ...

Merge ^/head r356931 through r357118.

Drop "All Rights Reserved" from all libbe/bectl files

I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to d

Drop "All Rights Reserved" from all libbe/bectl files

I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.

show more ...

libbe(3): promote dependent clones when destroying an environment

When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we

libbe(3): promote dependent clones when destroying an environment

When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we found and then
remove the target environment.

This fixes the ability to destroy a boot environment when it has been used
to spawn one or more other boot environments.

PR: 242592
Submitted by: Wes Maag <jwmaag gmail com> (with changes by myself)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22953

show more ...

libbe: fix build against sysutils/openzfs, part 1

This is the half of the changes required that work as-is with both in-tree
ZFS and the new hotness, sysutils/openzfs. Highlights are less dependenc

libbe: fix build against sysutils/openzfs, part 1

This is the half of the changes required that work as-is with both in-tree
ZFS and the new hotness, sysutils/openzfs. Highlights are less dependency
on header pollution (from somewhere) and using 'mnttab' instead of
'extmnttab'. In the in-tree ZFS, the latter is a #define for the former,
but in the port extmnttab is actually a distinct struct that's a super-set
of mnttab. We really want mnttab here anyways, so just use it.

show more ...