zfs: merge openzfs/zfs@1c9a4c8cb

Notable upstream pull request merges:
#16244 acb6e71ed Added output to `zpool online` and `offline`
#16684 94a03dd1e Pack dmu_buf_impl_t by 16 bytes
#16690 6187b1

zfs: merge openzfs/zfs@1c9a4c8cb

Notable upstream pull request merges:
#16244 acb6e71ed Added output to `zpool online` and `offline`
#16684 94a03dd1e Pack dmu_buf_impl_t by 16 bytes
#16690 6187b1943 On the first vdev open ignore impossible ashift hints
#16692 673efbbf5 zdb: add extra -T flag to show histograms of BRT refcounts
#16693 2bf152021 Fix gcc uninitialized warning in FreeBSD zio_crypt.c
#16694 b16e09619 Reduce dirty records memory usage
#16701 5945676bc ZFS send should use spill block prefetched from
send_reader_thread
#16734 1c9a4c8cb Fix user properties output for zpool list

Obtained from: OpenZFS
OpenZFS commit: 1c9a4c8cb44d5f865c29e3df3f019872329554b3

show more ...

Retire MK_PROFILE infrastructure

It was disabled by default in fe52b7f60ef4. We planned to (but did not)
remove the option before FreeBSD 14. Remove it now, for FreeBSD 15.

Relnotes: Yes
Reviewed

Retire MK_PROFILE infrastructure

It was disabled by default in fe52b7f60ef4. We planned to (but did not)
remove the option before FreeBSD 14. Remove it now, for FreeBSD 15.

Relnotes: Yes
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31558

show more ...

zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#9416 -multiple zio_compress: introduce max size threshold
#10018 a10e552b9 Adding Direct IO Support
#15147 e419a63bf xattr

zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#9416 -multiple zio_compress: introduce max size threshold
#10018 a10e552b9 Adding Direct IO Support
#15147 e419a63bf xattr dataset prop: change defaults to sa
#15454 7e957fde7 send/recv: open up additional stream feature flags
#15810 0d77e738e Defer resilver only when progress is above a threshold
#15921 3cf2bfa57 Allocate zap_attribute_t from kmem instead of stack
#16483 -multiple dmu_objset: replace dnode_hash impl with cityhash4
#16485 8be2f4c3d zio_resume: log when unsuspending the pool
#16491 88433e640 sys/types32.h: Remove struct timeval32 from libspl header
#16496 f245541e2 zfs_file: implement zfs_file_deallocate for FreeBSD 14
#16511 308f7c2f1 Fix an uninitialized data access
#16529 29c9e6c32 Fix handling of DNS names with '-' in them for sharenfs
#16531 ddf5f34f0 Avoid fault diagnosis if multiple vdevs have errors
#16539 6f50f8e16 zfs_log: add flex array fields to log record structs
#16546 d40d40913 Evicting too many bytes from MFU metadata
#16551 3014dcb76 Reduce and handle EAGAIN errors on AIO label reads
#16554 80645d658 FreeBSD: restore zfs_znode_update_vfs()
#16565 832f66b21 FreeBSD: Sync taskq_cancel_id() returns with Linux
#16567 48d1be254 Properly release key in spa_keystore_dsl_key_hold_dd()
#16569 141368a4b Restrict raidz faulted vdev count
#16583 c84a37ae9 lua: add flex array field to TString type
#16584 86737c592 Avoid computing strlen() inside loops
#16587 d34d4f97a snapdir: add 'disabled' value to make .zfs inaccessible
#16593 224393a32 feature: large_microzap
#16597 412105977 Temporarily disable Direct IO by default
#16605 4ebe674d9 ARC: Cache arc_c value during arc_evict()

Backported pull request merges:
#16613 ab777f436 Return boolean_t in inline functions of
lib/libspl/include/sys/uio.h
#16616 efeb60b86 FreeBSD: ignore some includes when not building kernel
#16635 ---TBD--- zdb: fix printf format in dump_zap()

Obtained from: OpenZFS
OpenZFS commit: b109925820fb79db3e37670c159977f03edd950f
OpenZFS tag: 2.3.0-rc1

show more ...

zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry

zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry format
#15895 -multiple Fast Dedup: FDT-log feature
#16239 6be8bf555 zpool: Provide GUID to zpool-reguid(8) with -g
#16277 -multiple Fast Dedup: prune unique entries
#16316 5807de90a Fix null ptr deref when renaming a zvol with snaps and snapdev=visible
#16343 77a797a38 Enable L2 cache of all (MRU+MFU) metadata but MFU data only
#16446 83f359245 FreeBSD: fix build without kernel option MAC
#16449 963e6c9f3 Fix incorrect error report on vdev attach/replace
#16505 b10992582 spa_prop_get: require caller to supply output nvlist

Obtained from: OpenZFS
OpenZFS commit: b109925820fb79db3e37670c159977f03edd950f

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

libzpool: Disable -Wpointer-to-int-cast warnings for GCC

This warning is already disabled for zfs.ko.

zfs: merge openzfs/zfs@8f2f6cd2a

Notable upstream pull request merges:
#15887 -multiple Fast Dedup: Cleanup and documentation ahead of
integrating Fast Dedup
#15907 5600dff0e Fix

zfs: merge openzfs/zfs@8f2f6cd2a

Notable upstream pull request merges:
#15887 -multiple Fast Dedup: Cleanup and documentation ahead of
integrating Fast Dedup
#15907 5600dff0e Fixed parameter passing error when calling zfs_acl_chmod
#15908 8f2f6cd2a ddt: reduce DDT_NAMELEN

Obtained from: OpenZFS
OpenZFS commit: 8f2f6cd2ac688916adb2caf979daf95365ccb48f

show more ...

zfs: merge openzfs/zfs@887a3c533

Notable upstream pull request merges:
#15022 5caeef02f RAID-Z expansion feature
#15457 887a3c533 Increase L2ARC write rate and headroom
#15504 1c1be60fa Unbreak F

zfs: merge openzfs/zfs@887a3c533

Notable upstream pull request merges:
#15022 5caeef02f RAID-Z expansion feature
#15457 887a3c533 Increase L2ARC write rate and headroom
#15504 1c1be60fa Unbreak FreeBSD world build after 3bd4df384

Obtained from: OpenZFS
OpenZFS commit: 887a3c533b94a4b70075e310f15c45b9dee19410

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

zfs: merge openzfs/zfs@feff9dfed

Notable upstream pull request merges:
#14833 Update compatibility.d files
#14841 ZIL: Reduce scope of per-dataset zl_issuer_lock
#14863 zil: Add some more stat

zfs: merge openzfs/zfs@feff9dfed

Notable upstream pull request merges:
#14833 Update compatibility.d files
#14841 ZIL: Reduce scope of per-dataset zl_issuer_lock
#14863 zil: Add some more statistics
#14866 btree: Implement faster binary search algorithm
#14894 Fix inconsistent definition of zfs_scrub_error_blocks_per_txg
#14892 Fix concurrent resilvers initiated at same time
#14903 Fix NULL pointer dereference when doing concurrent 'send' operations
#14910 ZIL: Allow to replay blocks of any size
#14939 Fix the L2ARC write size calculating logic
#14934 Introduce zfs_refcount_(add|remove)_few()
#14946 Improve l2arc reporting in arc_summary
#14953 Finally drop long disabled vdev cache
#14954 Fix the L2ARC write size calculating logic (2)
#14955 Use list_remove_head() where possible
#14959 ZIL: Fix race introduced by f63811f0721

Obtained from: OpenZFS
OpenZFS commit: feff9dfed3df1bbae5dd74959a6ad87d11f27ffb

show more ...

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Imple

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"

Obtained from: OpenZFS
OpenZFS commit: 431083f75bdd3efaee992bdd672625ec7240d252

show more ...

pkgbase: Put zfs utilities and lib in their own package

It is useful to have zfs utilities and lib in a separate package as
it allow users to create image that can support ZFS (i.e. not with
WITHOUT

pkgbase: Put zfs utilities and lib in their own package

It is useful to have zfs utilities and lib in a separate package as
it allow users to create image that can support ZFS (i.e. not with
WITHOUT_ZFS in src.conf set) without bloating the default image with
all zfs tools (for example for jails).

Differential Revision: https://reviews.freebsd.org/D36225

show more ...

libzpool: Disable -Wuse-after-free for dbuf.c.

The debug traces for reference counting in ZFS use the pointer of the
owning object as a "tag" for references to check that when an object
drops a refe

libzpool: Disable -Wuse-after-free for dbuf.c.

The debug traces for reference counting in ZFS use the pointer of the
owning object as a "tag" for references to check that when an object
drops a reference it had actually held one. In a couple of places ZFS
drops references after freeing the owning object. In userland GCC
realizes this is a use after free. However, since only the value of
the pointer is used, and it isn't indirected, the use is harmless.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D36818

show more ...

zfs: merge openzfs/zfs@c629f0bf6

Notable upstream pull request merges:
#13725 Fix BLAKE3 tuneable and module loading on Linux and FreeBSD
#13756 FreeBSD: Organize sysctls
#13773 FreeBSD: add k

zfs: merge openzfs/zfs@c629f0bf6

Notable upstream pull request merges:
#13725 Fix BLAKE3 tuneable and module loading on Linux and FreeBSD
#13756 FreeBSD: Organize sysctls
#13773 FreeBSD: add kqfilter support for zvol cdev
#13781 Importing from cachefile can trip assertion
#13794 Apply arc_shrink_shift to ARC above arc_c_min
#13798 Improve too large physical ashift handling
#13799 Revert "Avoid panic with recordsize > 128k, raw sending and
no large_blocks"
#13802 Add zfs.sync.snapshot_rename
#13831 zfs_enter rework
#13855 zfs recv hangs if max recordsize is less than received
recordsize

Obtained from: OpenZFS
OpenZFS commit: c629f0bf62e351355716f9870d6c2e377584b016

show more ...

zfs: merge openzfs/zfs@deb121309

Notable upstream pull request merges:
#12918 Introduce BLAKE3 checksums as an OpenZFS feature
#13553 Reduce ZIO io_lock contention on sorted scrub
#13537 Impro

zfs: merge openzfs/zfs@deb121309

Notable upstream pull request merges:
#12918 Introduce BLAKE3 checksums as an OpenZFS feature
#13553 Reduce ZIO io_lock contention on sorted scrub
#13537 Improve sorted scan memory accounting
#13540 AVL: Remove obsolete branching optimizations
#13563 FreeBSD: Improve crypto_dispatch() handling

Obtained from: OpenZFS
OpenZFS commit: deb1213098e2dc10e6eee5e5c57bb40584e096a6

show more ...

zfs: merge openzfs/zfs@a86e08941 (master) into main

Notable upstream pull request merges:
#9078: log xattr=sa create/remove/update to ZIL
#11919: Cross-platform xattr user namespace compatibili

zfs: merge openzfs/zfs@a86e08941 (master) into main

Notable upstream pull request merges:
#9078: log xattr=sa create/remove/update to ZIL
#11919: Cross-platform xattr user namespace compatibility
#13014: Report dnodes with faulty bonuslen
#13016: FreeBSD: Fix zvol_cdev_open locking
#13019: spl: Don't check FreeBSD rwlocks for double initialization
#13027: Fix clearing set-uid and set-gid bits on a file when
replying a write
#13031: Add enumerated vdev names to 'zpool iostat -v' and
'zpool list -v'
#13074: Enable encrypted raw sending to pools with greater ashift
#13076: Receive checks should allow unencrypted child datasets
#13098: Avoid dirtying the final TXGs when exporting a pool
#13172: Fix ENOSPC when unlinking multiple files from full pool

Obtained from: OpenZFS
OpenZFS commit: a86e089415679cf1b98eb424a159bb36aa2c19e3

show more ...

Disable clang 14 warning about bitwise operators in zstd

Parts of zstd, used in openzfs and other places, trigger a new clang 14
-Werror warning:

```
sys/contrib/zstd/lib/decompress/huf_decompress.

Disable clang 14 warning about bitwise operators in zstd

Parts of zstd, used in openzfs and other places, trigger a new clang 14
-Werror warning:

```
sys/contrib/zstd/lib/decompress/huf_decompress.c:889:25: error: use of bitwise '&' with boolean operands [-Werror,-Wbitwise-instead-of-logical]
(BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

While the warning is benign, it should ideally be fixed upstream and
then vendor-imported, but for now silence it selectively.

MFC after: 3 days

show more ...

zfs: merge openzfs/zfs@17b2ae0b2 (master) into main

Notable upstream pull request merges:
#12766 Fix error propagation from lzc_send_redacted
#12805 Updated the lz4 decompressor
#12851 FreeBSD

zfs: merge openzfs/zfs@17b2ae0b2 (master) into main

Notable upstream pull request merges:
#12766 Fix error propagation from lzc_send_redacted
#12805 Updated the lz4 decompressor
#12851 FreeBSD: Provide correct file generation number
#12857 Verify dRAID empty sectors
#12874 FreeBSD: Update argument types for VOP_READDIR
#12896 Reduce number of arc_prune threads
#12934 FreeBSD: Fix zvol_*_open() locking
#12947 lz4: Cherrypick fix for CVE-2021-3520
#12961 FreeBSD: Fix leaked strings in libspl mnttab
#12964 Fix handling of errors from dmu_write_uio_dbuf() on FreeBSD
#12981 Introduce a flag to skip comparing the local mac when raw sending
#12985 Avoid memory allocations in the ARC eviction thread

Obtained from: OpenZFS
OpenZFS commit: 17b2ae0b24d487fdda2ef1098ec26fa7f79a61f6

show more ...

zfs: merge OpenZFS master-9312e0fd1

Notable upstream changes:
778869fa1 Fix reporting of mount progress
e7adccf7f Disable use of hardware crypto offload drivers on FreeBSD
03e02e5b5 Fix checks

zfs: merge OpenZFS master-9312e0fd1

Notable upstream changes:
778869fa1 Fix reporting of mount progress
e7adccf7f Disable use of hardware crypto offload drivers on FreeBSD
03e02e5b5 Fix checksum errors not being counted on repeated repair
64e0fe14f Restore FreeBSD resource usage accounting
11f2e9a49 Fix panic if scrubbing after removing a slog device

MFC after: 2 weeks

show more ...

zfs: merge OpenZFS master-436ab35a5

- speed up writing to ZFS pools without ZIL devices (aa755b3)
- speed up importing ZFS pools (2d8f72d, a0e0199, cf0977a)
...

MFC after: 2 weeks
Reviewed by: mj

zfs: merge OpenZFS master-436ab35a5

- speed up writing to ZFS pools without ZIL devices (aa755b3)
- speed up importing ZFS pools (2d8f72d, a0e0199, cf0977a)
...

MFC after: 2 weeks
Reviewed by: mjg (partial)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D28677

show more ...

OpenZFS: Fix issues caused by reversed commits + rebase

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, t

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

show more ...

MFV 2.0-rc2

- Fixes divide by zero for unusual hz
- remove cryptodev dependency

Use MACHINE_CPUARCH when checking for arm64

Use MACHINE_CPUARCH with arm64 (aarch64) when we build code that could run
on any 64-bit Arm instruction set. This will simplify checks in downstream
cons

Use MACHINE_CPUARCH when checking for arm64

Use MACHINE_CPUARCH with arm64 (aarch64) when we build code that could run
on any 64-bit Arm instruction set. This will simplify checks in downstream
consumers targeting prototype instruction sets.

The only place we check for MACHINE_ARCH == aarch64 is when building the
device tree blobs. As these are targeting current generation ISAs.

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26370

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)