zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry

zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry format
#15895 -multiple Fast Dedup: FDT-log feature
#16239 6be8bf555 zpool: Provide GUID to zpool-reguid(8) with -g
#16277 -multiple Fast Dedup: prune unique entries
#16316 5807de90a Fix null ptr deref when renaming a zvol with snaps and snapdev=visible
#16343 77a797a38 Enable L2 cache of all (MRU+MFU) metadata but MFU data only
#16446 83f359245 FreeBSD: fix build without kernel option MAC
#16449 963e6c9f3 Fix incorrect error report on vdev attach/replace
#16505 b10992582 spa_prop_get: require caller to supply output nvlist

Obtained from: OpenZFS
OpenZFS commit: b109925820fb79db3e37670c159977f03edd950f

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

libc: expose execvpe for Linux compat

We already implemented execvpe internally with an _ prefix in libc so
go ahead and expose it for compatibility with Linux.

This reverts c605eea952146348e5e1ad5

libc: expose execvpe for Linux compat

We already implemented execvpe internally with an _ prefix in libc so
go ahead and expose it for compatibility with Linux.

This reverts c605eea952146348e5e1ad5cab6c127d7a1bd164.

Bump __FreeBSD_version for the addition and add definitions to supress
compat shims in libzfs (zfs changes were merged from upstream).

PR: 275370 (request and exp-run (thanks antoine!))
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D42846

show more ...

zfs: merge openzfs/zfs@2e2a46e0a

Notable upstream pull request merges:
#15243 b53077a9e Add zfs_prepare_disk script for disk firmware install
#15298 2e2a46e0a Invoke zdb by guid to avoid import er

zfs: merge openzfs/zfs@2e2a46e0a

Notable upstream pull request merges:
#15243 b53077a9e Add zfs_prepare_disk script for disk firmware install
#15298 2e2a46e0a Invoke zdb by guid to avoid import errors
#15301 e5d70f467 ZIL: Avoid dbuf_read() in ztest_get_data()

Obtained from: OpenZFS
OpenZFS commit: 2e2a46e0a597b3ee606ea7dc5bc527459077322f

show more ...

zfs: merge openzfs/zfs@804414aad

Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD:

zfs: merge openzfs/zfs@804414aad

Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15076 Refactor dmu_prefetch()
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress

Obtained from: OpenZFS
OpenZFS commit: 804414aad224b432590afe3f9ec114ffb49e0f13

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Imple

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"

Obtained from: OpenZFS
OpenZFS commit: 431083f75bdd3efaee992bdd672625ec7240d252

show more ...

pkgbase: Put zfs utilities and lib in their own package

It is useful to have zfs utilities and lib in a separate package as
it allow users to create image that can support ZFS (i.e. not with
WITHOUT

pkgbase: Put zfs utilities and lib in their own package

It is useful to have zfs utilities and lib in a separate package as
it allow users to create image that can support ZFS (i.e. not with
WITHOUT_ZFS in src.conf set) without bloating the default image with
all zfs tools (for example for jails).

Differential Revision: https://reviews.freebsd.org/D36225

show more ...

zfs: merge openzfs/zfs@ec64fdb93 (master) into main

Notable upstream pull request merges:
#12392 Avoid panic in case of pool errors and missing L2ARC
#12448 skip snapshot in zfs_iter_mounted()

zfs: merge openzfs/zfs@ec64fdb93 (master) into main

Notable upstream pull request merges:
#12392 Avoid panic in case of pool errors and missing L2ARC
#12448 skip snapshot in zfs_iter_mounted()
#12516 Fix NFS and large reads on older kernels
#12533 Fail invalid incremental recursive send gracefully
#12569 FreeBSD: Really zero the zero page
#12575 Reject zfs send -RI with nonexistent fromsnap
#12602 Correct refcount_add in dmu_zfetch
#12650 zpool should call zfs_nicestrtonum() with non-NULL handle

Obtained from: OpenZFS
OpenZFS commit: ec64fdb93d144ab1884097cfd36e18b62a2db848

show more ...

zfs: unbreak build broken in 160388166

zfs: bump version and install new share files

- bump version to 2.0.0-FreeBSD_gbf156c966
- install definition files for the new "-o compatibility" option
to "zpool create"

MFC after: 2 weeks

zfs: merge OpenZFS master-436ab35a5

- speed up writing to ZFS pools without ZIL devices (aa755b3)
- speed up importing ZFS pools (2d8f72d, a0e0199, cf0977a)
...

MFC after: 2 weeks
Reviewed by: mj

zfs: merge OpenZFS master-436ab35a5

- speed up writing to ZFS pools without ZIL devices (aa755b3)
- speed up importing ZFS pools (2d8f72d, a0e0199, cf0977a)
...

MFC after: 2 weeks
Reviewed by: mjg (partial)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D28677

show more ...

Use MACHINE_CPUARCH when checking for arm64

Use MACHINE_CPUARCH with arm64 (aarch64) when we build code that could run
on any 64-bit Arm instruction set. This will simplify checks in downstream
cons

Use MACHINE_CPUARCH when checking for arm64

Use MACHINE_CPUARCH with arm64 (aarch64) when we build code that could run
on any 64-bit Arm instruction set. This will simplify checks in downstream
consumers targeting prototype instruction sets.

The only place we check for MACHINE_ARCH == aarch64 is when building the
device tree blobs. As these are targeting current generation ISAs.

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26370

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

libzfs: Add missing crypto dependency

libzfs_crypto.c uses PKCS5_PBKDF2_HMAC_SHA1 from libcrypto.

Reported by: John Kennedy
Sponsored by: iXsystems, Inc.

Move zstd sources from libzfs to libzpool

zstd is kernel code that was not supposed to be in libzfs.

libzpool provides userland shims for kernel code and is where the
zstd code needs to be included

Move zstd sources from libzfs to libzpool

zstd is kernel code that was not supposed to be in libzfs.

libzpool provides userland shims for kernel code and is where the
zstd code needs to be included.

Reported by: John Kennedy
Discussed with: mmacy
Sponsored by: iXsystems, Inc.

show more ...

Fix libzfs build failures for some

In some environments adding zstd/include to the include path
would cause zstd's stdlib.h to hide the system one.

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would a

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

show more ...

Merge ^/head r351732 through r352104.

pkgbase: Force zfs(8) and zpool(8) to be in the runtime package

Those commands are needed to repair a FreeBSD installation so add them
to the runtime package

Reviewed by: bapt, gjb
Differential Rev

pkgbase: Force zfs(8) and zpool(8) to be in the runtime package

Those commands are needed to repair a FreeBSD installation so add them
to the runtime package

Reviewed by: bapt, gjb
Differential Revision: https://reviews.freebsd.org/D21498

show more ...

MFH @ r337607, in preparation for boarding

Merge ^/head r336870 through r337285, and resolve conflicts.

Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.

Merge ^/head r323559 through r325504.

MFhead@r325119