1.\" 2.\" Copyright (c) 2012-2014 Intel Corporation 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions, and the following disclaimer, 10.\" without modification. 11.\" 2. Redistributions in binary form must reproduce at minimum a disclaimer 12.\" substantially similar to the "NO WARRANTY" disclaimer below 13.\" ("Disclaimer") and any redistribution must be conditioned upon 14.\" including a substantially similar Disclaimer requirement for further 15.\" binary redistribution. 16.\" 17.\" NO WARRANTY 18.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR 21.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22.\" HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 23.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 24.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 25.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 26.\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING 27.\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 28.\" POSSIBILITY OF SUCH DAMAGES. 29.\" 30.\" nvme driver man page. 31.\" 32.\" Author: Jim Harris <jimharris@FreeBSD.org> 33.\" 34.\" $FreeBSD$ 35.\" 36.Dd March 18, 2014 37.Dt NVME 4 38.Os 39.Sh NAME 40.Nm nvme 41.Nd NVM Express core driver 42.Sh SYNOPSIS 43To compile this driver into your kernel, 44place the following line in your kernel configuration file: 45.Bd -ragged -offset indent 46.Cd "device nvme" 47.Ed 48.Pp 49Or, to load the driver as a module at boot, place the following line in 50.Xr loader.conf 5 : 51.Bd -literal -offset indent 52nvme_load="YES" 53.Ed 54.Pp 55Most users will also want to enable 56.Xr nvd 4 57to surface NVM Express namespaces as disk devices which can be 58partitioned. 59Note that in NVM Express terms, a namespace is roughly equivalent to a 60SCSI LUN. 61.Sh DESCRIPTION 62The 63.Nm 64driver provides support for NVM Express (NVMe) controllers, such as: 65.Bl -bullet 66.It 67Hardware initialization 68.It 69Per-CPU IO queue pairs 70.It 71API for registering NVMe namespace consumers such as 72.Xr nvd 4 73.It 74API for submitting NVM commands to namespaces 75.It 76Ioctls for controller and namespace configuration and management 77.El 78.Pp 79The 80.Nm 81driver creates controller device nodes in the format 82.Pa /dev/nvmeX 83and namespace device nodes in 84the format 85.Pa /dev/nvmeXnsY . 86Note that the NVM Express specification starts numbering namespaces at 1, 87not 0, and this driver follows that convention. 88.Sh CONFIGURATION 89By default, 90.Nm 91will create an I/O queue pair for each CPU, provided enough MSI-X vectors 92can be allocated. 93To force a single I/O queue pair shared by all CPUs, set the following 94tunable value in 95.Xr loader.conf 5 : 96.Bd -literal -offset indent 97hw.nvme.per_cpu_io_queues=0 98.Ed 99.Pp 100To force legacy interrupts for all 101.Nm 102driver instances, set the following tunable value in 103.Xr loader.conf 5 : 104.Bd -literal -offset indent 105hw.nvme.force_intx=1 106.Ed 107.Pp 108Note that use of INTx implies disabling of per-CPU I/O queue pairs. 109.Sh SYSCTL VARIABLES 110The following controller-level sysctls are currently implemented: 111.Bl -tag -width indent 112.It Va dev.nvme.0.int_coal_time 113(R/W) Interrupt coalescing timer period in microseconds. 114Set to 0 to disable. 115.It Va dev.nvme.0.int_coal_threshold 116(R/W) Interrupt coalescing threshold in number of command completions. 117Set to 0 to disable. 118.El 119.Pp 120The following queue pair-level sysctls are currently implemented. 121Admin queue sysctls take the format of dev.nvme.0.adminq and I/O queue sysctls 122take the format of dev.nvme.0.ioq0. 123.Bl -tag -width indent 124.It Va dev.nvme.0.ioq0.num_entries 125(R) Number of entries in this queue pair's command and completion queue. 126.It Va dev.nvme.0.ioq0.num_tr 127(R) Number of nvme_tracker structures currently allocated for this queue pair. 128.It Va dev.nvme.0.ioq0.num_prp_list 129(R) Number of nvme_prp_list structures currently allocated for this queue pair. 130.It Va dev.nvme.0.ioq0.sq_head 131(R) Current location of the submission queue head pointer as observed by 132the driver. 133The head pointer is incremented by the controller as it takes commands off 134of the submission queue. 135.It Va dev.nvme.0.ioq0.sq_tail 136(R) Current location of the submission queue tail pointer as observed by 137the driver. 138The driver increments the tail pointer after writing a command 139into the submission queue to signal that a new command is ready to be 140processed. 141.It Va dev.nvme.0.ioq0.cq_head 142(R) Current location of the completion queue head pointer as observed by 143the driver. 144The driver increments the head pointer after finishing 145with a completion entry that was posted by the controller. 146.It Va dev.nvme.0.ioq0.num_cmds 147(R) Number of commands that have been submitted on this queue pair. 148.It Va dev.nvme.0.ioq0.dump_debug 149(W) Writing 1 to this sysctl will dump the full contents of the submission 150and completion queues to the console. 151.El 152.Sh SEE ALSO 153.Xr nvd 4 , 154.Xr pci 4 , 155.Xr nvmecontrol 8 , 156.Xr disk 9 157.Sh HISTORY 158The 159.Nm 160driver first appeared in 161.Fx 9.2 . 162.Sh AUTHORS 163.An -nosplit 164The 165.Nm 166driver was developed by Intel and originally written by 167.An Jim Harris Aq Mt jimharris@FreeBSD.org , 168with contributions from 169.An Joe Golio 170at EMC. 171.Pp 172This man page was written by 173.An Jim Harris Aq Mt jimharris@FreeBSD.org . 174