Include ccompile.h after opt_global.h.

This restores INVARIANTS enabling ZFS_DEBUG when built as module.

Discussed with: freqlabs
MFC after: 1 week

zfs: merge OpenZFS master-9312e0fd1

Notable upstream changes:
778869fa1 Fix reporting of mount progress
e7adccf7f Disable use of hardware crypto offload drivers on FreeBSD
03e02e5b5 Fix checks

zfs: merge OpenZFS master-9312e0fd1

Notable upstream changes:
778869fa1 Fix reporting of mount progress
e7adccf7f Disable use of hardware crypto offload drivers on FreeBSD
03e02e5b5 Fix checksum errors not being counted on repeated repair
64e0fe14f Restore FreeBSD resource usage accounting
11f2e9a49 Fix panic if scrubbing after removing a slog device

MFC after: 2 weeks

show more ...

ZFS: fix assertions with INVARIANTS

AFAICT, this was an oversight from
9e5787d2284e187abb5b654d924394a65772e004 (svn r364746). That revision
inadvertently disabled assertions unconditionally.

Revi

ZFS: fix assertions with INVARIANTS

AFAICT, this was an oversight from
9e5787d2284e187abb5b654d924394a65772e004 (svn r364746). That revision
inadvertently disabled assertions unconditionally.

Reviewed by: freqlabs
MFC after: 3 days
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D28256

show more ...

OpenZFS: Fix issues caused by reversed commits + rebase

openzfs: fix gcc kernel module builds

- Suppress -Wredundant-decls. Ultimately this warning is harmless in
any case, and it does not look like there is a simple way to avoid
redundant declar

openzfs: fix gcc kernel module builds

- Suppress -Wredundant-decls. Ultimately this warning is harmless in
any case, and it does not look like there is a simple way to avoid
redundant declarations in this case without a lot of header pollution
(e.g. having openzfs's shim param.h pulling in sys/kernel.h for hz).
- Suppress -Wnested-externs, which is useless anyway.

Unfortunately it was not sufficient just to modify OPENZFS_CFLAGS,
because the warning suppressions need to appear on the command line
after they are explicitly enabled by CWARNFLAGS from sys/conf/kern.mk,
but OPENZFS_CFLAGS get added before due to use of -I for the shims.

Reviewed by: markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27685

show more ...

zfs: quiet gcc -Wmissing-include-dirs

Don't tell it to look for headers in a non-existent directory.

Reviewed by: imp, mmacy
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.fre

zfs: quiet gcc -Wmissing-include-dirs

Don't tell it to look for headers in a non-existent directory.

Reviewed by: imp, mmacy
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27565

show more ...

ZFS: remove some extra defines

When merging a number of defines that are needed in the standalone
build made it in to the module makefile.

Reported by: markj@

Stop setting DEBUG_FLAGS in the zfs.ko Makefile.

Rely on system settings to decide whether or not to build with debug
symbols and extract them to a separate file.

Reported and tested by: rm

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

[PowerPC] More preemptive powerpcspe ZFS build fixes

I went through the merge and found the rest of the instances where
${MACHINE_ARCH} == "powerpc" was being used to detect 32-bit and adjusted
the

[PowerPC] More preemptive powerpcspe ZFS build fixes

I went through the merge and found the rest of the instances where
${MACHINE_ARCH} == "powerpc" was being used to detect 32-bit and adjusted
the rest of the instances to also check for powerpcspe.

mips32* will probably want to do the same.

Sponsored by: Tag1 Consulting, Inc.

show more ...

Fix zstd in OpenZFS module with CPUTYPE?=<something with BMI>

The build breaks when something adds -march=<something with BMI> to the
compiler flags, for example CPUTYPE?=native. When the arch supp

Fix zstd in OpenZFS module with CPUTYPE?=<something with BMI>

The build breaks when something adds -march=<something with BMI> to the
compiler flags, for example CPUTYPE?=native. When the arch supports BMI,
__BMI__ is defined and zstd.c tries to include immintrin.h, which is not
present when building the kernel.

Disable experimental BMI intrinsics in zstd in the OpenZFS kernel module
by explicitly undefining __BMI__ for zstd.c.

A similar fix was needed for the original zstd import, done in r327738.

Reported by: Jakob Alvermark
Discussed with: mmacy
Sponsored by: iXsystems, Inc.

show more ...

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would a

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

show more ...

Remove duplicate lz4 implementations

Port illumos change: https://www.illumos.org/issues/11667

Move lz4.c out of zfs tree to opensolaris/common/lz4, adjust it to be
usable from kernel/stand/userlan

Remove duplicate lz4 implementations

Port illumos change: https://www.illumos.org/issues/11667

Move lz4.c out of zfs tree to opensolaris/common/lz4, adjust it to be
usable from kernel/stand/userland builds, so we can use just one single
source. Add lz4.h to declare lz4_compress() and lz4_decompress().

MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22037

show more ...

Expose zlib's utility functions in Z_SOLO library when building kernel.
This allows kernel code to reuse zlib's implementation.

PR: 229763
Reviewed by: Yoshihiro Ota <ota j email ne jp>
Relnotes: y

Expose zlib's utility functions in Z_SOLO library when building kernel.
This allows kernel code to reuse zlib's implementation.

PR: 229763
Reviewed by: Yoshihiro Ota <ota j email ne jp>
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D21156

show more ...

Allow Kernel to link in both legacy libkern/zlib and new sys/contrib/zlib,
with an eventual goal to convert all legacl zlib callers to the new zlib
version:

* Move generic zlib shims that are not s

Allow Kernel to link in both legacy libkern/zlib and new sys/contrib/zlib,
with an eventual goal to convert all legacl zlib callers to the new zlib
version:

* Move generic zlib shims that are not specific to zlib 1.0.4 to
sys/dev/zlib.
* Connect new zlib (1.2.11) to the zlib kernel module, currently built
with Z_SOLO.
* Prefix the legacy zlib (1.0.4) with 'zlib104_' namespace.
* Convert sys/opencrypto/cryptodeflate.c to use new zlib.
* Remove bundled zlib 1.2.3 from ZFS and adapt it to new zlib and make
it depend on the zlib module.
* Fix Z_SOLO build of new zlib.

PR: 229763
Submitted by: Yoshihiro Ota <ota j email ne jp>
Reviewed by: markm (sys/dev/zlib/zlib_kmod.c)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D19706

show more ...

Merge ^/head r338026 through r338297, and resolve conflicts.

Make dnode definition uniform on !x86

gcc4 requires -fms-extensions to accept anonymous union members

Merge ^/head r323559 through r325504.

MFhead@r324482

MFV r323530,r323533,r323534: 7431 ZFS Channel Programs, and followups

7431 ZFS Channel Programs

illumos/illumos-gate@dfc115332c94a2f62058ac7f2bce7631fbd20b3d
https://github.com/illumos/illumos-gate

MFV r323530,r323533,r323534: 7431 ZFS Channel Programs, and followups

7431 ZFS Channel Programs

illumos/illumos-gate@dfc115332c94a2f62058ac7f2bce7631fbd20b3d
https://github.com/illumos/illumos-gate/commit/dfc115332c94a2f62058ac7f2bce7631fbd20b3d

https://www.illumos.org/issues/7431
ZFS channel programs (ZCP) adds support for performing compound ZFS
administrative actions via Lua scripts in a sandboxed environment (with time
and memory limits).
This initial commit includes both base support for running ZCP scripts, and a
small initial library of API calls which support getting properties and
listing, destroying, and promoting datasets.
Testing: in addition to the included unit tests, channel programs have been in
use at Delphix for several months for batch destroying filesystems. The
dsl_destroy_snaps_nvl() call has also been replaced with

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Chris Williamson <chris.williamson@delphix.com>

8552 ZFS LUA code uses floating point math

illumos/illumos-gate@916c8d881190bd2c3ca20d9fca919aecff504435
https://github.com/illumos/illumos-gate/commit/916c8d881190bd2c3ca20d9fca919aecff504435

https://www.illumos.org/issues/8552
In the LUA interpreter used by "zfs program", the lua format() function
accidentally includes support for '%f' and friends, which can cause compilation
problems when building on platforms that don't support floating-point math in
the kernel (e.g. sparc). Support for '%f' friends (%f %e %E %g %G) should be
removed, since there's no way to supply a floating-point value anyway (all
numbers in ZFS LUA are int64_t's).

Reviewed by: Yuri Pankov <yuripv@gmx.com>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

8590 memory leak in dsl_destroy_snapshots_nvl()

illumos/illumos-gate@e6ab4525d156c82445c116ecf6b2b874d5e9009d
https://github.com/illumos/illumos-gate/commit/e6ab4525d156c82445c116ecf6b2b874d5e9009d

https://www.illumos.org/issues/8590
In dsl_destroy_snapshots_nvl(), "snaps_normalized" is not freed after it is
added to "arg".

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

FreeBSD notes:
- zfs-program.8 manual page is taken almost as is from the vendor repository,
no FreeBSD-ification done
- fixed multiple instances of NULL being used where an integer is expected
- replaced ETIME and ECHRNG with ETIMEDOUT and EDOM respectively

This commit adds a modified version of Lua 5.2.4 under
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua, mirroring the
upstream. See README.zfs in that directory for the description of Lua
customizations.
See zfs-program.8 on how to use the new feature.

MFC after: 5 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D12528

show more ...

sys/modules: normalize .CURDIR-relative paths to SRCTOP

This simplifies make output/logic

Tested with: `cd sys/modules; make ALL_MODULES=` on amd64
MFC after: 1 month
Sponsored by: Dell EMC Isilon

Merge ^/head r314178 through r314269.

zfs: clean up unused files and definitions

MFC after: 1 month
X-MFC after: r314048

MFhead@r313243

Merge ^/head r312968 through r313054.