sys: make the kernel metadata pointer global

The way we got the kernel metadata pointer was by calling
preload_search_by_type with one of the following three: "elf kernel",
"elf32 kernel" and "elf64

sys: make the kernel metadata pointer global

The way we got the kernel metadata pointer was by calling
preload_search_by_type with one of the following three: "elf kernel",
"elf32 kernel" and "elf64 kernel". Which one(s) we used wasn't
consistent though. Sometimes we would only try "elf kernel", and other
times we would try one of the latter two if the first failed. However,
the loader only ever sets "elf kernel" as the kernel type.

Now, the kmdp is a global, preload_kmdp, and it's initialized using
preload_initkmdp in machdep.c (or machdep_boot.c on arm/64).
preload_initkmdp takes a single boolean argument that tells us whether
not finding the kmdp is fatal or not.

Reviewed by: imp, kib
Pull Request: https://github.com/freebsd/freebsd-src/pull/1394

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Complete removal of opt_compat.h

Since Linux emulation layer build options was removed there is no reason
to keep opt_compat.h.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.o

Complete removal of opt_compat.h

Since Linux emulation layer build options was removed there is no reason
to keep opt_compat.h.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D38548
MFC after: 2 weeks

show more ...

crypto: move all zeroing into crypto_getreq

Reviewed by: jhb, markj
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D36774

[skip ci] Remove obsolete references in crypto_request(9)

They were missed by 68f6800ce05c3.

Sponsored by: Axcient
MFC with: 68f6800ce05c3
Reviewed by: markj
Differential Revision: https://reviews.

[skip ci] Remove obsolete references in crypto_request(9)

They were missed by 68f6800ce05c3.

Sponsored by: Axcient
MFC with: 68f6800ce05c3
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D37239

show more ...

ddb: annotate some commands with DB_CMD_MEMSAFE

This is not completely exhaustive, but covers a large majority of
commands in the tree.

Reviewed by: markj
Sponsored by: Juniper Networks, Inc.
Spons

ddb: annotate some commands with DB_CMD_MEMSAFE

This is not completely exhaustive, but covers a large majority of
commands in the tree.

Reviewed by: markj
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35583

show more ...

crypto: Validate return values from CRYPTODEV_PROCESS()

Errors are always handled by the completion callback, so we should check
that they're not also passed back to the caller.

No functional chang

crypto: Validate return values from CRYPTODEV_PROCESS()

Errors are always handled by the completion callback, so we should check
that they're not also passed back to the caller.

No functional change intended.

Reviewed by: kp, mav, jhb
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35382

show more ...

crypto: Fix assertions for digest-only sessions with separate output.

Digest-only sessions do not generate modified payload as an output, so
don't bother asserting anything about the payload with re

crypto: Fix assertions for digest-only sessions with separate output.

Digest-only sessions do not generate modified payload as an output, so
don't bother asserting anything about the payload with respect to the
output buffer other than the payload output start being zero.

In addition, a verify request on a digest-only session doesn't
generate any output at all so should never have a separate output
buffer.

PR: 252316
Reviewed by: markj
Co-authored-by: Jeremy Faulkner <gldisater@gmail.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35578

show more ...

crypto: hide crypto_destroyreq behind a tunable

Reviewed by: jhb, markj
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D32084

crypto: Add support for the XChaCha20-Poly1305 AEAD cipher.

This cipher is a wrapper around the ChaCha20-Poly1305 AEAD cipher
which accepts a larger nonce. Part of the nonce is used along with
the

crypto: Add support for the XChaCha20-Poly1305 AEAD cipher.

This cipher is a wrapper around the ChaCha20-Poly1305 AEAD cipher
which accepts a larger nonce. Part of the nonce is used along with
the key as an input to HChaCha20 to generate a derived key used for
ChaCha20-Poly1305.

This cipher is used by WireGuard.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33523

show more ...

OCF: Add crypto_clonereq().

This function clones an existing crypto request, but associates the
new request with a specified session. The intended use case is for
drivers to be able to fall back to

OCF: Add crypto_clonereq().

This function clones an existing crypto request, but associates the
new request with a specified session. The intended use case is for
drivers to be able to fall back to software by cloning a request and
dispatch it to an internally allocated software session.

Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33607

show more ...

OCF: Hook up plain RIPEMD160 in cryptosoft and /dev/crypto.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33612

crypto: Consistently use AES instead of Rijndael128 for the AES-CBC cipher.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33486

crypto: Validate AES-GCM IV length in check_csp().

This centralizes the check for valid nonce lengths for AES-GCM.

While here, remove some duplicate checks for valid AES-GCM tag lengths
from ccp(4)

crypto: Validate AES-GCM IV length in check_csp().

This centralizes the check for valid nonce lengths for AES-GCM.

While here, remove some duplicate checks for valid AES-GCM tag lengths
from ccp(4) and ccr(4).

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33194

show more ...

crypto: Don't assert for empty output buffers.

It is always valid for crp_payload_output_start to be 0. However, if
an output buffer is empty (e.g. a decryption request with a tag but an
empty payl

crypto: Don't assert for empty output buffers.

It is always valid for crp_payload_output_start to be 0. However, if
an output buffer is empty (e.g. a decryption request with a tag but an
empty payload), the existing assertion failed since 0 is not less than
0.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33193

show more ...

crypto: Cleanup mtx_init() calls.

Don't pass the same name to multiple mutexes while using unique types
for WITNESS. Just use the unique types as the mutex names.

Reviewed by: markj
MFC after: 1 w

crypto: Cleanup mtx_init() calls.

Don't pass the same name to multiple mutexes while using unique types
for WITNESS. Just use the unique types as the mutex names.

Reviewed by: markj
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32740

show more ...

crypto: Use a single "crypto" kproc for all of the OCF kthreads.

Reported by: julian
Reviewed by: markj
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.

crypto: Use a single "crypto" kproc for all of the OCF kthreads.

Reported by: julian
Reviewed by: markj
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32739

show more ...

crypto: Support Chacha20-Poly1305 with a nonce size of 8 bytes.

This is useful for WireGuard which uses a nonce of 8 bytes rather
than the 12 bytes used for IPsec and TLS.

Note that this also fixes

crypto: Support Chacha20-Poly1305 with a nonce size of 8 bytes.

This is useful for WireGuard which uses a nonce of 8 bytes rather
than the 12 bytes used for IPsec and TLS.

Note that this also fixes a (should be) harmless bug in ossl(4) where
the counter was incorrectly treated as a 64-bit counter instead of a
32-bit counter in terms of wrapping when using a 12 byte nonce.
However, this required a single message (TLS record) longer than 64 *
(2^32 - 1) bytes (about 256 GB) to trigger.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32122

show more ...

crypto: Support multiple nonce lengths for AES-CCM.

Permit nonces of lengths 7 through 13 in the OCF framework and the
cryptosoft driver. A helper function (ccm_max_payload_length) can be
used in O

crypto: Support multiple nonce lengths for AES-CCM.

Permit nonces of lengths 7 through 13 in the OCF framework and the
cryptosoft driver. A helper function (ccm_max_payload_length) can be
used in OCF drivers to reject CCM requests which are too large for the
specified nonce length.

Reviewed by: sef
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32111

show more ...

opencrypto: Allow kern.crypto.allow_soft to be specified as a tunable

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

crypto: Constify all transform descriptors

No functional change intended.

Reviewed by: ae, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.

crypto: Constify all transform descriptors

No functional change intended.

Reviewed by: ae, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31196

show more ...

crypto: Add a new type of crypto buffer for a single mbuf.

This is intended for use in KTLS transmit where each TLS record is
described by a single mbuf that is itself queued in the socket buffer.
U

crypto: Add a new type of crypto buffer for a single mbuf.

This is intended for use in KTLS transmit where each TLS record is
described by a single mbuf that is itself queued in the socket buffer.
Using the existing CRYPTO_BUF_MBUF would result in
bus_dmamap_load_crp() walking additional mbufs in the socket buffer
that are not relevant, but generating a S/G list that potentially
exceeds the limit of the tag (while also wasting CPU cycles).

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30136

show more ...

OCF: Remove support for asymmetric cryptographic operations.

There haven't been any non-obscure drivers that supported this
functionality and it has been impossible to test to ensure that it
still w

OCF: Remove support for asymmetric cryptographic operations.

There haven't been any non-obscure drivers that supported this
functionality and it has been impossible to test to ensure that it
still works. The only known consumer of this interface was the engine
in OpenSSL < 1.1. Modern OpenSSL versions do not include support for
this interface as it was not well-documented.

Reviewed by: cem
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29736

show more ...

Add an OCF algorithm for ChaCha20-Poly1305 AEAD.

Note that this algorithm implements the mode defined in RFC 8439.

Reviewed by: cem
Sponsored by: Netflix
Differential Revision: https://reviews.free

Add an OCF algorithm for ChaCha20-Poly1305 AEAD.

Note that this algorithm implements the mode defined in RFC 8439.

Reviewed by: cem
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27836

show more ...

opencrypto: Introduce crypto_dispatch_async()

Currently, OpenCrypto consumers can request asynchronous dispatch by
setting a flag in the cryptop. (Currently only IPSec may do this.) I
think this

opencrypto: Introduce crypto_dispatch_async()

Currently, OpenCrypto consumers can request asynchronous dispatch by
setting a flag in the cryptop. (Currently only IPSec may do this.) I
think this is a bit confusing: we (conditionally) set cryptop flags to
request async dispatch, and then crypto_dispatch() immediately examines
those flags to see if the consumer wants async dispatch. The flag names
are also confusing since they don't specify what "async" applies to:
dispatch or completion.

Add a new KPI, crypto_dispatch_async(), rather than encoding the
requested dispatch type in each cryptop. crypto_dispatch_async() falls
back to crypto_dispatch() if the session's driver provides asynchronous
dispatch. Get rid of CRYPTOP_ASYNC() and CRYPTOP_ASYNC_KEEPORDER().

Similarly, add crypto_dispatch_batch() to request processing of a tailq
of cryptops, rather than encoding the scheduling policy using cryptop
flags. Convert GELI, the only user of this interface (disabled by
default) to use the new interface.

Add CRYPTO_SESS_SYNC(), which can be used by consumers to determine
whether crypto requests will be dispatched synchronously. This is just
a helper macro. Use it instead of looking at cap flags directly.

Fix style in crypto_done(). Also get rid of CRYPTO_RETW_EMPTY() and
just check the relevant queues directly. This could result in some
unnecessary wakeups but I think it's very uncommon to be using more than
one queue per worker in a given workload, so checking all three queues
is a waste of cycles.

Reviewed by: jhb
Sponsored by: Ampere Computing
Submitted by: Klara, Inc.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28194

show more ...