Make nvd vs nda choice boot-time rather than build-time

Introduce hw.nvme.use_nvd tunable. This tunable allows both nvd and
nda to be installed in the kernel, while allowing only one of them to
crea

Make nvd vs nda choice boot-time rather than build-time

Introduce hw.nvme.use_nvd tunable. This tunable allows both nvd and
nda to be installed in the kernel, while allowing only one of them to
create devices. This is an all-or-nothing setting, and you can't
change it after boot-time. However, it will allow easier A/B testing.

Differential Revision: https://reviews.freebsd.org/D11825

show more ...

Merge ^/head r311940 through r312200.

Report random flash storage as non-rotating to GEOM_DISK.

While doing it, introduce respective constants in geom_disk.h.

MFC after: 1 week

Remove unused variable from last commit.

Supporting flushing the dump before returning, and simplify/combine the
logic. Switch to a 5us delay since most NVME devices can easily do 200,000
iops.

Submitted by: imp
MFC after: 3 days
Sponsore

Supporting flushing the dump before returning, and simplify/combine the
logic. Switch to a 5us delay since most NVME devices can easily do 200,000
iops.

Submitted by: imp
MFC after: 3 days
Sponsored by: Netflix, Inc.

show more ...

Implement crashdump support on NVME

MFC after: 3 days
Sponsored by: Netflix, Inc.

MFH

Sponsored by: The FreeBSD Foundation

Revert r292074 (by smh): Limit stripesize reported from nvd(4) to 4K

I believe that this patch handled the problem from the wrong side.
Instead of making ZFS properly handle large stripe sizes, it m

Revert r292074 (by smh): Limit stripesize reported from nvd(4) to 4K

I believe that this patch handled the problem from the wrong side.
Instead of making ZFS properly handle large stripe sizes, it made
unrelated driver to lie in reported parameters to workaround that.

Alternative solution for this problem from ZFS side was committed at
r296615.

Discussed with: smh

show more ...

Merge ^/head r294961 through r295350.

MfH @r295202

Expect to see panics in routing code at least now.

MFH

Sponsored by: The FreeBSD Foundation

nvd: add hw.nvd.delete_max tunable

The NVMe specification does not define a maximum or optimal delete
size, so technically max delete size is min(full size of namespace,
2^32 - 1 LBAs). A single de

nvd: add hw.nvd.delete_max tunable

The NVMe specification does not define a maximum or optimal delete
size, so technically max delete size is min(full size of namespace,
2^32 - 1 LBAs). A single delete operation for a multi-TB NVMe
namespace though may take much longer to complete than the nvme(4)
I/O timeout period. So choose a sensible default here that is still
suitably large to minimize the number of overall delete operations.

This also fixes possible uint32_t overflow on initial TRIM operation
for zpool create operations for NVMe namespaces with >4G LBAs.

MFC after: 3 days
Sponsored by: Intel

show more ...

MFH @r294567

Merge ^/head r293280 through r293429.

MFH

Sponsored by: The FreeBSD Foundation

nvd: submit bios directly when BIO_ORDERED not set or in flight

This significantly improves parallelism in the most common case.
The taskqueue is still used whenever BIO_ORDERED bios are in flight.

nvd: submit bios directly when BIO_ORDERED not set or in flight

This significantly improves parallelism in the most common case.
The taskqueue is still used whenever BIO_ORDERED bios are in flight.

This patch is based heavily on a patch from gallatin@.

MFC after: 3 days
Sponsored by: Intel

show more ...

nvd: break out submission logic into separate function

This enables a future patch using this same logic to submit
I/O directly bypassing the taskqueue.

MFC after: 3 days
Sponsored by: Intel

nvd: skip BIO_ORDERED logic when bio fails submission

This ensures the bio flags are not read after biodone().
The ordering will still be enforced, after the bio is
submitted successfully.

MFC afte

nvd: skip BIO_ORDERED logic when bio fails submission

This ensures the bio flags are not read after biodone().
The ordering will still be enforced, after the bio is
submitted successfully.

MFC after: 3 days
Sponsored by: Intel

show more ...

nvd: do not wait for previous bios before submitting ordered bio

Still wait until all in-flight bios (including the ordered bio)
complete before processing more bios from the queue.

MFC after: 3 da

nvd: do not wait for previous bios before submitting ordered bio

Still wait until all in-flight bios (including the ordered bio)
complete before processing more bios from the queue.

MFC after: 3 days
Sponsored by: Intel

show more ...

nvd: set DISKFLAG_DIRECT_COMPLETION

Submitted by: gallatin
MFC after: 3 days

MFH r289384-r293170

Sponsored by: The FreeBSD Foundation

MFH @r292599

This includes the pluggable TCP framework and other chnages to the
netstack to track for VNET stability.

Security: The FreeBSD Foundation

Limit stripesize reported from nvd(4) to 4K

Intel NVMe controllers have a slow path for I/Os that span a 128KB stripe boundary but ZFS limits ashift, which is derived from d_stripesize, to 13 (8KB)

Limit stripesize reported from nvd(4) to 4K

Intel NVMe controllers have a slow path for I/Os that span a 128KB stripe boundary but ZFS limits ashift, which is derived from d_stripesize, to 13 (8KB) so we limit the stripesize reported to geom(8) to 4KB.

This may result in a small number of additional I/Os to require splitting in nvme(4), however the NVMe I/O path is very efficient so these additional I/Os will cause very minimal (if any) difference in performance or CPU utilisation.

This can be controller by the new sysctl kern.nvme.max_optimal_sectorsize.

MFC after: 1 week
Sponsored by: Multiplay
Differential Revision: https://reviews.freebsd.org/D4446

show more ...

Catch up with head (r291075).

Merge from head