nvmf: Add NVMF_CONNECTION_STATUS ioctl

This returns an nvlist indicating if a Fabrics host is connected and
the time of the most recent disconnection.

Reviewed by: imp
Sponsored by: Chelsio Communi

nvmf: Add NVMF_CONNECTION_STATUS ioctl

This returns an nvlist indicating if a Fabrics host is connected and
the time of the most recent disconnection.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D48219

show more ...

nvme/nvmf: Add NVME_GET_CONTROLLER_DATA ioctl to fetch cached cdata

Both nvme and nvmf cache a copy of the controller's identify data in
the softc. Add an ioctl to fetch this copy of the cdata. Th

nvme/nvmf: Add NVME_GET_CONTROLLER_DATA ioctl to fetch cached cdata

Both nvme and nvmf cache a copy of the controller's identify data in
the softc. Add an ioctl to fetch this copy of the cdata. This is
primarily useful for allowing commands like 'nvmecontrol devlist' to
work against a disconnected Fabrics host.

Reviewed by: dab, imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D48218

show more ...

nvmf: Refactor reconnection support

Save more data associated with a new association including the network
address of the remote controller. This permits reconnecting an
association without providi

nvmf: Refactor reconnection support

Save more data associated with a new association including the network
address of the remote controller. This permits reconnecting an
association without providing the address or other details. To use
this new mode, provide only an existing device ID to nvmecontrol's
reconnect command. An address can still be provided to request a
different address or other different settings for the new association.

The saved data includes an entire Discovery Log page entry to aim to
be compatible with other transports in the future. When a remote
controller is connected to via a Discovery Log page entry (nvmecontrol
connect-all), the raw entry is used. When a remote controller is
connected to via an explicit address, an entry is synthesized from the
parameters.

Note that this is a pseudo-ABI break for the ioctls used by nvmf(4) in
that the nvlists for handoff and reconnect now use a slightly
different set of elements. Since this is only present in main I did
not bother implementing compatability shims.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D48214

show more ...

nvmf: Defer the post-sync shutdown handler to SHUTDOWN_PRI_LAST

nda(4) has its own shutdown handler that runs at SHUTDOWN_PRI_DEFAULT
that calls ndaflush() that could run after the nvmf handler. In

nvmf: Defer the post-sync shutdown handler to SHUTDOWN_PRI_LAST

nda(4) has its own shutdown handler that runs at SHUTDOWN_PRI_DEFAULT
that calls ndaflush() that could run after the nvmf handler. Instead,
give a the flush a chance to run before the graceful shutdown of the
controller.

While here, be a bit more defensive in the post-sync case and shutdown
the consumers (sim and /dev/nvmeXnY devices) before destroying the
queue pairs so that if any requests are submitted after the post-sync
handler they fail gracefully instead of trying to use a destroyed
queue pair.

Reported by: Sony Arpita Das <sonyarpitad@chelsio.com>
Sponsored by: Chelsio Communications

show more ...

nvmf: Switch several ioctls to using nvlists

For requests that handoff queues from userspace to the kernel as well
as the request to fetch reconnect parameters from the kernel, switch
from using fla

nvmf: Switch several ioctls to using nvlists

For requests that handoff queues from userspace to the kernel as well
as the request to fetch reconnect parameters from the kernel, switch
from using flat structures to nvlists. In particular, this will
permit adding support for additional transports in the future without
breaking the ABI of the structures.

Note that this is an ABI break for the ioctls used by nvmf(4) and
nvmft(4). Since this is only present in main I did not bother
implementing compatability shims.

Inspired by: imp (suggestion on a different review)
Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D48230

show more ...

nvmf: Schedule requests across multiple I/O queues

Similar to nvme(4), use the current CPU to select which I/O queue to
use. The assignment in nvmf_attach() had to be moved down since
sc->num_io_qu

nvmf: Schedule requests across multiple I/O queues

Similar to nvme(4), use the current CPU to select which I/O queue to
use. The assignment in nvmf_attach() had to be moved down since
sc->num_io_queues is initialized in nvmf_establish_connection().

Note that nvmecontrol(8) still defaults to using a single I/O queue
for an association.

Sponsored by: Chelsio Communications

show more ...

nvmf: Fix an off by one error when scanning active namespace IDs

The active namespace list query fetches namespaces greater than the
passed in namespace ID, not greater than or equal to the passed i

nvmf: Fix an off by one error when scanning active namespace IDs

The active namespace list query fetches namespaces greater than the
passed in namespace ID, not greater than or equal to the passed in
namespace ID. Thus, a multi-page request should start with the last
namespace ID from the previous page, not that ID plus 1.

While here, make use of NVME_GLOBAL_NAMESPACE_TAG instead of a magic
number to handle the edge case that the last namespace ID in a page is
the largest valid namespace ID.

Reviewed by: chuck
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D47393

show more ...

nvmf: Deregister the post_sync eventhandler correctly during detach

Previously the handler was removed from the wrong eventhandler list.

Fixes: f46d4971b5af nvmf: Handle shutdowns more gracefully

nvmf: Deregister the post_sync eventhandler correctly during detach

Previously the handler was removed from the wrong eventhandler list.

Fixes: f46d4971b5af nvmf: Handle shutdowns more gracefully
Sponsored by: Chelsio Communications

show more ...

nvmf: Add sysctl nodes for each queue pair

These report the queue size, queue head, queue tail, and the number of
commands submitted.

Sponsored by: Chelsio Communications

nvmf: Fail pass through commands while a controller is not associated

Previously this just dereferenced NULL qp pointers and panicked.
Instead, use a shared lock on the connection lock to protect ac

nvmf: Fail pass through commands while a controller is not associated

Previously this just dereferenced NULL qp pointers and panicked.
Instead, use a shared lock on the connection lock to protect access to
the qp pointers and allocate a request. If the controller is not
associated, fail the request with ECONNABORTED.

Possibly this should be honoring kern.nvmf.fail_on_disconnection and
block waiting for a reconnect request while disconnected if that
tunable is false.

Reported by: Suhas Lokesha <suhas@chelsio.com>
Sponsored by: Chelsio Communications

show more ...

nvmf: Use device_set_descf()

No functional change intended.

MFC after: 1 week

nvmf: Handle shutdowns more gracefully

If an association is disconnected during a clean shutdown, abort all
pending and future I/O requests with an error to avoid hangs either due
to filesystem unmo

nvmf: Handle shutdowns more gracefully

If an association is disconnected during a clean shutdown, abort all
pending and future I/O requests with an error to avoid hangs either due
to filesystem unmounts or a stuck GEOM event.

If an association is connected during a clean shutdown, gracefully
disconnect from the remote controller and close the open queues.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45462

show more ...

nvmf: Permit failing I/O requests while disconnected

Add a kern.nvmf.fail_on_disconnection sysctl similar to the
kern.iscsi.fail_on_disconnection sysctl. This causes pending I/O
requests to fail wi

nvmf: Permit failing I/O requests while disconnected

Add a kern.nvmf.fail_on_disconnection sysctl similar to the
kern.iscsi.fail_on_disconnection sysctl. This causes pending I/O
requests to fail with an error if an association is disconnected
instead of requeueing to be retried once the association is
reconnected. As with iSCSI, the default is to queue and retry
operations.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45308

show more ...

nvmf: Rescan namespaces after reconnecting

While a host was disconnected from a remote controller, namespaces
might have been added, removed, or altered properties. Rescan the
namespaces after reco

nvmf: Rescan namespaces after reconnecting

While a host was disconnected from a remote controller, namespaces
might have been added, removed, or altered properties. Rescan the
namespaces after reconnecting to detect any such changes.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45461

show more ...

nvmf: Rescan all namespaces if the changed NS log page is too large

Previously this just punted with a warning message.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision:

nvmf: Rescan all namespaces if the changed NS log page is too large

Previously this just punted with a warning message.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45460

show more ...

nvmf: Factor out most of nvmf_rescan_ns into a helper routine

This function accepts a namespace ID and associated namespace data
from IDENTIFY and takes care of updating nvmeXnY and ndaZ.

Reviewed

nvmf: Factor out most of nvmf_rescan_ns into a helper routine

This function accepts a namespace ID and associated namespace data
from IDENTIFY and takes care of updating nvmeXnY and ndaZ.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45459

show more ...

nvmf: Refactor nvmf_add_namespaces to be more generic

Rename to nvmf_scan_active_namespaces and accept an additional
callback function and callback argument. The callback is invoked on
each active

nvmf: Refactor nvmf_add_namespaces to be more generic

Rename to nvmf_scan_active_namespaces and accept an additional
callback function and callback argument. The callback is invoked on
each active namespace enumerated by the active namespace list from the
IDENTIFY command.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45458

show more ...

nvme/f: Use strlcpy instead of strncpy + manual string termination

Reviewed by: dab, imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45153

nvmf: The in-kernel NVMe over Fabrics host

This is the client (initiator in SCSI terms) for NVMe over Fabrics.
Userland is responsible for creating a set of queue pairs and then
handing them off via

nvmf: The in-kernel NVMe over Fabrics host

This is the client (initiator in SCSI terms) for NVMe over Fabrics.
Userland is responsible for creating a set of queue pairs and then
handing them off via an ioctl to this driver, e.g. via the 'connect'
command from nvmecontrol(8). An nvmeX new-bus device is created
at the top-level to represent the remote controller similar to PCI
nvmeX devices for PCI-express controllers.

As with nvme(4), namespace devices named /dev/nvmeXnsY are created and
pass through commands can be submitted to either the namespace devices
or the controller device. For example, 'nvmecontrol identify nvmeX'
works for a remote Fabrics controller the same as for a PCI-express
controller.

nvmf exports remote namespaces via nda(4) devices using the new NVMF
CAM transport. nvmf does not support nvd(4), only nda(4).

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44714

show more ...