zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry

zfs: merge openzfs/zfs@b10992582

Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry format
#15895 -multiple Fast Dedup: FDT-log feature
#16239 6be8bf555 zpool: Provide GUID to zpool-reguid(8) with -g
#16277 -multiple Fast Dedup: prune unique entries
#16316 5807de90a Fix null ptr deref when renaming a zvol with snaps and snapdev=visible
#16343 77a797a38 Enable L2 cache of all (MRU+MFU) metadata but MFU data only
#16446 83f359245 FreeBSD: fix build without kernel option MAC
#16449 963e6c9f3 Fix incorrect error report on vdev attach/replace
#16505 b10992582 spa_prop_get: require caller to supply output nvlist

Obtained from: OpenZFS
OpenZFS commit: b109925820fb79db3e37670c159977f03edd950f

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

stand: remove ZFS warning about computered but unused variable

Toss a __unused on the unexpected_errors variable. We compute it, but
never use it.

Sponsored by: Netflix

zfs: Increase compatibility for different environments

libsa uses the full OpenZFS compilation environment when we build this
included in zfs.c there. Other parts of the tree have not been adapted
t

zfs: Increase compatibility for different environments

libsa uses the full OpenZFS compilation environment when we build this
included in zfs.c there. Other parts of the tree have not been adapted
to the full OpenZFS environment yet and need these ASSERT* defines to
build properly. Since the ASSERT* macros are normally defined in
sys/debug.h in the OpenZFS compatibility spl, only define them when
ASSERT3S is not defined to cope with the parts of the loader that don't
yet use the full OpenZFS environment.

Sponsored by: Netflix
Reviewed by: tsoome, delphij
Differential Revision: https://reviews.freebsd.org/D35893

show more ...

stand: Add blake3 support to boot loader

Add the necessary glue to get blake3 building for the boot loaded as
well as connected to the ZFS system so it is useful.

On some platforms, we create refer

stand: Add blake3 support to boot loader

Add the necessary glue to get blake3 building for the boot loaded as
well as connected to the ZFS system so it is useful.

On some platforms, we create references to blake3_sse2_impl and
blake3_sse41_impl ops structs to utilize SIMD. These aren't present on
x86 (since we dind't ask for them), but are on aarch64 with no
implementation. Since we don't want SIMD in the boot loader, have these
all return 'unsupported' always. This should be fixed upstream to allow
more flexibility in this selection, but for now we use this hack to not
modify the sys/contrib/openzfs with difficult to maintain hacks while
an upstreamable solution is found.

tsoome@ did the implementation bits in sys/cddl/boot, and I did the
Makefile work and the aweful blake3_impl_hack.c.

Co-author: tsoome@freebsd.org
Sponsored by: Netflix
Reviewed by: kevans (earlier version)
Differential Revision: https://reviews.freebsd.org/D35750

show more ...

loader: add support for gzip compression

As we do have zlib code in loader, we should also support gzip
compression in zfs.

PR: 153173
Submitted by: Mikhail Zakharov <zmey20000@yahoo.com>
Reviewed

loader: add support for gzip compression

As we do have zlib code in loader, we should also support gzip
compression in zfs.

PR: 153173
Submitted by: Mikhail Zakharov <zmey20000@yahoo.com>
Reviewed by: imp, markj, delphij
Differential Revision: https://reviews.freebsd.org/D35320
MFC after: 1 month

show more ...

stand/zfs: Mark pcount as unused

pcount is unused in the sense that it's set but never used except in an
assert. But asserts are compiled out always, so just mark it as unused.

Sponsored by: Netfl

stand/zfs: Mark pcount as unused

pcount is unused in the sense that it's set but never used except in an
assert. But asserts are compiled out always, so just mark it as unused.

Sponsored by: Netflix

show more ...

Turn off zstd on aarch64

loader support for zstd and zfs doesn't work for aarch64. Disable it
to unbreak the build.

Add zstd support to the boot loader.

Add support to the _STANDALONE environment enough bits of the kernel
that we can compile it. We still have a small zstd_shim.c since there
were 3 items that were

Add zstd support to the boot loader.

Add support to the _STANDALONE environment enough bits of the kernel
that we can compile it. We still have a small zstd_shim.c since there
were 3 items that were a bit hard to nail down and may be cleaned up
in the future. These go hand in hand with a number of commits to
sys/sys in the past weeks, should this need be MFCd.

Discussed with: mmacy (in review and on IRC/Slack)
Reviewed by: freqlabs (on openzfs repo)
Differential Revision: https://reviews.freebsd.org/D26218

show more ...

Merge ^/head r358269 through r358399.

loader: replace zfs_alloc/zfs_free with malloc/free

Use common memory management.

loader: rewrite zfs vdev initialization

In some cases the pool discovery will get stuck in infinite loop while setting
up the vdev children.

To fix, we split the vdev setup into two parts, first we

loader: rewrite zfs vdev initialization

In some cases the pool discovery will get stuck in infinite loop while setting
up the vdev children.

To fix, we split the vdev setup into two parts, first we create vdevs based on
configuration we do get from pool label, then, we process pool config from MOS
and update the pool config if needed.

Testing done: confirm previously hung loader is not hung any more.

MFC after: 1 week

show more ...

Remove duplicate lz4 implementations

Port illumos change: https://www.illumos.org/issues/11667

Move lz4.c out of zfs tree to opensolaris/common/lz4, adjust it to be
usable from kernel/stand/userlan

Remove duplicate lz4 implementations

Port illumos change: https://www.illumos.org/issues/11667

Move lz4.c out of zfs tree to opensolaris/common/lz4, adjust it to be
usable from kernel/stand/userland builds, so we can use just one single
source. Add lz4.h to declare lz4_compress() and lz4_decompress().

MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22037

show more ...

loader: zio_checksum_verify should check byteswap

We do have both native and byteswap checksum callbacks in place but the
selection is not wired.

MFC after: 1 week

Mark two things as unused (since they are only sometimes used) and
toss in a DECONST to remove a const in some tricky code that would
require too extensive a change to unwind otherwise.

Sponsored by

Mark two things as unused (since they are only sometimes used) and
toss in a DECONST to remove a const in some tricky code that would
require too extensive a change to unwind otherwise.

Sponsored by: Netflix

show more ...

Fix all warnings related to geli and ZFS support on x86.

Default WARNS to 0 still, since there's still some warnings on other
architectures.

Sponsored by: Netflix
Differential Revision: https://rev

Fix all warnings related to geli and ZFS support on x86.

Default WARNS to 0 still, since there's still some warnings on other
architectures.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D13301

show more ...

Merge ^/head r304700 through r304884.

MFhead @ r304815

Bug 212114 - loader: zio_checksum_verify() must test spa for NULL pointer

The issue was introduced with adding support for salted checksums, and
was revealed by bhyve userboot.so.

During pool disco

Bug 212114 - loader: zio_checksum_verify() must test spa for NULL pointer

The issue was introduced with adding support for salted checksums, and
was revealed by bhyve userboot.so.

During pool discovery the loader is reading pool label from disks, and
at that time the spa structure is not yet set up, so the NULL pointer
is passed for spa. This condition must be checked to avoid the corruption
of the memory and NULL pointer dereference.

PR: 212114
Reported by: tsoome@freebsd.com
Reviewed by: allanjude
Approved by: allanjude (mentor)
Differential Revision: https://reviews.freebsd.org/D7634

show more ...

Merge ^/head r304236 through r304536.

Add SHA512, skein, large blocks support for loader zfs.

Updated sha512 from illumos.
Using skein from freebsd crypto tree.
Since loader itself is using 64MB memory for heap, updated zfsboot to
use s

Add SHA512, skein, large blocks support for loader zfs.

Updated sha512 from illumos.
Using skein from freebsd crypto tree.
Since loader itself is using 64MB memory for heap, updated zfsboot to
use same, and this also allows to support zfs large blocks.

Note, adding additional features does increate zfsboot code, therefore
this update does increase zfsboot code to 128k, also I have ported gptldr.S
update to zfsldr.S to support 64k+ code.

With this update, boot1.efi has almost reached the current limit of the size
set for it, so one of the future patches for boot1.efi will need to
increase the limit.

Currently known missing zfs features in boot loader are edonr and gzip support.

Reviewed by: delphij, imp
Approved by: imp (mentor)
Obtained from: sha256.c update and skein_zfs.c stub from illumos.
Differential Revision: https://reviews.freebsd.org/D7418

show more ...

IFC @r269962

Submitted by: Anish Gupta (akgupt3@gmail.com)

Merge head from 7/28

Sync to HEAD@r269943.

MFV r267565:

4757 ZFS embedded-data block pointers ("zero block compression")
4913 zfs release should not be subject to space checks

MFC after: 2 weeks