1.\" 2.\" Copyright (c) 2017 Netflix, Inc. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.Dd June 6, 2020 27.Dt NDA 4 28.Os 29.Sh NAME 30.Nm nda 31.Nd NVMe Direct Access device driver 32.Sh SYNOPSIS 33.Cd device nvme 34.Cd device scbus 35.Sh DESCRIPTION 36The 37.Nm 38driver provides support for direct access devices, implementing the 39.Tn NVMe 40command protocol, that are attached to the system through a host adapter 41supported by the CAM subsystem. 42.Sh SYSCTL VARIABLES 43The following variables are available as both 44.Xr sysctl 8 45variables and 46.Xr loader 8 47tunables: 48.Bl -tag -width 12 49.It Va hw.nvme.use_nvd 50The 51.Xr nvme 4 52driver will create 53.Nm 54device nodes for block storage when set to 0. 55Create 56.Xr nvd 4 57device nodes for block storage when set to 1. 58See 59.Xr nvd 4 60when set to 1. 61.It Va kern.cam.nda.nvd_compat 62When set to 1, 63.Xr nvd 4 64aliases will be created for all 65.Nm 66devices, including partitions and other 67.Xr geom 4 68providers that take their names from the disk's name. 69.Xr nvd 4 70devices will not, however, be reported in the 71.Va kern.disks 72.Xr sysctl 8 . 73.It Va kern.cam.nda.sort_io_queue 74This variable determines whether the software queued entries are 75sorted in LBA order or not. 76Sorting is almost always a waste of time. 77The default is to not sort. 78.It Va kern.cam.nda.enable_biospeedup 79This variable determines if the 80.Nm 81devices participate in the speedup protocol. 82When the device participates in the speedup, then when the upper layers 83send a 84.Va BIO_SPEEDUP , 85all current 86.Va BIO_DELETE 87requests not yet sent to the hardware are completed successfully immediate 88without sending them to the hardware. 89Used in low disk space scenarios when the filesystem encounters 90a critical shortage and needs blocks immediately. 91Since trims have maximum benefit when the LBA is unused for a long time, 92skipping the trim when space is needed for immediate writes results in little to 93no excess wear. 94When participation is disabled, 95.Va BIO_SPEEDUP 96requests are ignored. 97.It Va kern.cam.nda.max_trim 98The maximum number of LBA ranges to be collected together for each DSM trims 99send to the hardware. 100Defaults to 256, which is the maximum number of ranges the protocol supports. 101Sometimes poor trim performance can be mitigated by limiting the number of 102ranges sent to the device. 103This value must be between 1 and 256 inclusive. 104.El 105.Pp 106The following report per-device settings, and are read-only unless 107otherwise indicated. 108Replace 109.Va N 110with the device unit number. 111.Bl -tag -width 12 112.It Va kern.cam.nda.N.rotating 113This variable reports whether the storage volume is spinning or 114flash. 115Its value is hard coded to 0 indicating flash. 116.It Va kern.cam.nda.N.unmapped_io 117This variable reports whether the 118.Nm 119driver accepts unmapped I/O for this unit. 120.It Va kern.cam.nda.N.flags 121This variable reports the current flags. 122.Bl -tag -width 12 123.It Va OPEN 124The device is open. 125.It Va DIRTY 126Set when a write to the drive is scheduled. 127Cleared after flush commands. 128.It Va SCTX_INIT 129Internal flag set after 130.Xr sysctl 8 131nodes have been created. 132.El 133.It Va kern.cam.nda.N.sort_io_queue 134Same as the 135.Va kern.cam.nda.sort_io_queue 136tunable. 137.It Va kern.cam.nda.N.trim_ticks 138Writable. 139When greater than zero, hold trims for up to this many ticks before sending 140to the drive. 141Sometimes waiting a little bit to collect more trims to send at one time 142improves trim performance. 143When 0, no delaying of trims are done. 144.It Va kern.cam.nda.N.trim_goal 145Writable. 146When delaying a bit to collect multiple trims, send the accumulated DSM TRIM to 147the drive. 148.It Va kern.cam.nda.N.trim_lbas 149Total number of LBAs that have been trimmed. 150.It Va kern.cam.nda.N.trim_ranges 151Total number of LBA ranges that have been trimmed. 152.It Va kern.cam.nda.N.trim_count 153Total number of trims sent to the hardware. 154.It Va kern.cam.nda.N.deletes 155Total number of 156.Va BIO_DELETE 157requests queued to the device. 158.El 159.Sh NAMESPACE MAPPING 160Each 161.Xr nvme 4 162drive has one or more namespaces associated with it. 163One instance of the 164.Nm 165driver will be created for each of the namespaces on 166the drive. 167All the 168.Nm 169nodes for a 170.Xr nvme 4 171device are at target 0. 172However, the namespace ID maps to the CAM lun, as reported 173in kernel messages and in the 174.Va devlist 175sub command of 176.Xr camcontrol 8 . 177.Pp 178Namespaces are managed with the 179.Va ns 180sub command of 181.Xr nvmecontrol 8 . 182Not all drives support namespace management, 183but all drives support at least one namespace. 184Device nodes for 185.Nm 186will be created and destroyed dynamically as 187namespaces are activated or detached. 188.Sh FILES 189.Bl -tag -width ".Pa /dev/nda*" -compact 190.It Pa /dev/nda* 191NVMe storage device nodes 192.El 193.Sh SEE ALSO 194.Xr cam 4 , 195.Xr geom 4 , 196.Xr nvd 4 , 197.Xr nvme 4 , 198.Xr gpart 8 199.Sh HISTORY 200The 201.Nm 202driver first appeared in 203.Fx 12.0 . 204.Sh AUTHORS 205.An Warner Losh Aq Mt imp@FreeBSD.org 206