1.\" 2.\" Copyright (c) 2020 Warner Losh <imp@FreeBSD.org> 3.\" Copyright (c) 2018-2019 Alexander Motin <mav@FreeBSD.org> 4.\" Copyright (c) 2012 Intel Corporation 5.\" All rights reserved. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions, and the following disclaimer, 12.\" without modification. 13.\" 2. Redistributions in binary form must reproduce at minimum a disclaimer 14.\" substantially similar to the "NO WARRANTY" disclaimer below 15.\" ("Disclaimer") and any redistribution must be conditioned upon 16.\" including a substantially similar Disclaimer requirement for further 17.\" binary redistribution. 18.\" 19.\" NO WARRANTY 20.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 21.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 22.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR 23.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 24.\" HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 26.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 27.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 28.\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING 29.\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 30.\" POSSIBILITY OF SUCH DAMAGES. 31.\" 32.\" nvmecontrol man page. 33.\" 34.\" Author: Jim Harris <jimharris@FreeBSD.org> 35.\" 36.\" $FreeBSD$ 37.\" 38.Dd December 19, 2020 39.Dt NVMECONTROL 8 40.Os 41.Sh NAME 42.Nm nvmecontrol 43.Nd NVM Express control utility 44.Sh SYNOPSIS 45.Nm 46.Ic devlist 47.Nm 48.Ic identify 49.Op Fl v 50.Op Fl x 51.Op Fl n Ar nsid 52.Aq Ar device-id | Ar namespace-id 53.Nm 54.Ic perftest 55.Aq Fl n Ar num_threads 56.Aq Fl o Ar read|write 57.Op Fl p 58.Aq Fl s Ar size_in_bytes 59.Aq Fl t Ar time_in_sec 60.Aq Ar namespace-id 61.Nm 62.Ic reset 63.Aq Ar device-id 64.Nm 65.Ic logpage 66.Aq Fl p Ar page_id 67.Op Fl x 68.Op Fl v Ar vendor-string 69.Op Fl b 70.Op Fl f Ar LSP 71.Op Fl i Ar LSI 72.Op Fl r 73.Aq Ar device-id | Ar namespace-id 74.Nm 75.Ic ns active 76.Aq Ar device-id 77.Nm 78.Ic ns allocated 79.Aq Ar device-id 80.Nm 81.Ic ns attach 82.Aq Fl n Ar nsid 83.Aq Fl c Ar cntid 84.Aq Ar device-id 85.Nm 86.Ic ns attached 87.Aq Fl n Ar nsid 88.Aq Ar device-id 89.Nm 90.Ic ns controllers 91.Aq Ar device-id 92.Nm 93.Ic ns create 94.Aq Fl s Ar nsze 95.Op Fl c Ar ncap 96.Op Fl f Ar lbaf 97.Op Fl m Ar mset 98.Op Fl n Ar nmic 99.Op Fl p Ar pi 100.Op Fl l Ar pil 101.Op Fl L Ar flbas 102.Op Fl d Ar dps 103.Aq Ar device-id 104.Nm 105.Ic ns delete 106.Aq Fl n Ar nsid 107.Aq Ar device-id 108.Nm 109.Ic ns detach 110.Aq Fl n Ar nsid 111.Aq Fl c Ar cntid 112.Aq Ar device-id 113.Nm 114.Ic ns identify 115.Op Fl v 116.Op Fl x 117.Aq Fl n Ar nsid 118.Aq Ar device-id 119.Nm 120.Ic nsid 121.Aq Ar device-id | Ar namespace-id 122.Nm 123.Ic resv acquire 124.Aq Fl c Ar crkey 125.Op Fl p Ar prkey 126.Aq Fl t Ar rtype 127.Aq Fl a Ar racqa 128.Aq Ar namespace-id 129.Nm 130.Ic resv register 131.Op Fl c Ar crkey 132.Aq Fl k Ar nrkey 133.Aq Fl r Ar rrega 134.Op Fl i Ar iekey 135.Op Fl p Ar cptpl 136.Aq Ar namespace-id 137.Nm 138.Ic resv release 139.Aq Fl c Ar crkey 140.Aq Fl t Ar rtype 141.Aq Fl a Ar rrela 142.Aq Ar namespace-id 143.Nm 144.Ic resv report 145.Op Fl e 146.Op Fl v 147.Op Fl x 148.Aq Ar namespace-id 149.Nm 150.Ic firmware 151.Op Fl s Ar slot 152.Op Fl f Ar path_to_firmware 153.Op Fl a 154.Aq Ar device-id 155.Nm 156.Ic format 157.Op Fl f Ar fmt 158.Op Fl m Ar mset 159.Op Fl o Ar pi 160.Op Fl l Ar pil 161.Op Fl E 162.Op Fl C 163.Aq Ar device-id | Ar namespace-id 164.Nm 165.Ic sanitize 166.Aq Fl a Ar sanact 167.Op Fl c Ar owpass 168.Op Fl d 169.Op Fl p Ar ovrpat 170.Op Fl r 171.Op Fl I 172.Op Fl U 173.Aq Ar device-id 174.Nm 175.Ic power 176.Op Fl l 177.Op Fl p power_state 178.Op Fl w workload_hint 179.Nm 180.Ic selftest 181.Aq Fl c Ar code 182.Aq Ar device-id | Ar namespace-id 183.Nm 184.Ic wdc cap-diag 185.Op Fl o path_template 186.Aq Ar device-id 187.Nm 188.Ic wdc drive-log 189.Op Fl o path_template 190.Aq Ar device-id 191.Nm 192.Ic wdc get-crash-dump 193.Op Fl o path_template 194.Aq Ar device-id 195.\" .Nm 196.\" .Ic wdc purge 197.\" .Aq device-id 198.\" .Nm 199.\" .Ic wdc purge-monitor 200.\" .Aq device-id 201.Nm 202.Ic admin-passthru 203.Op args 204.Aq Ar device-id 205.Nm 206.Ic io-passthru 207.Op args 208.Aq Ar namespace-id 209.Sh DESCRIPTION 210NVM Express (NVMe) is a storage protocol standard, for SSDs and other 211high-speed storage devices over PCI Express. 212.Ss identify 213The identify commands reports information from the drive's 214.Dv IDENTIFY_CONTROLLER 215if a 216.Ar device-id 217is specified. 218It reports 219.Dv IDENTIFY_NAMESPACE 220data if a 221.Ar namespace-id 222is specified. 223When used with disk names, the 224.Dv IDENTIFY_NAMESPACE 225data is reported, unless the namespace 226.Ar nsid 227is overridden with the 228.Fl n 229flag. 230Then that namespace's data is reported, if it exists. 231The command accepts the following parameters: 232.Bl -tag -width 6n 233.It Fl n 234The namespace 235.Aq nsid 236to use instead of the namespace associated with the device. 237A 238.Ar nsid 239of 240.Dq 0 241is used to retrieve the 242.Dv IDENTIFY_CONTROLLER 243data associated with that drive. 244.El 245.Ss logpage 246The logpage command knows how to print log pages of various types. 247It also knows about vendor specific log pages from hgst/wdc and intel. 248Note that some vendors use the same log page numbers for different data. 249.Pp 250.Bl -tag -compact -width "Page 0x00" 251.It Dv Page 0x01 252Drive Error Log 253.It Dv Page 0x02 254Health/SMART Data 255.It Dv Page 0x03 256Firmware Information 257.It Dv Page 0x04 258Changed Namespace List 259.It Dv Page 0x05 260Commands Supported and Effects 261.It Dv Page 0x06 262Device Self-test 263.It Dv Page 0x80 264Reservation Notification 265.It Dv Page 0x81 266Sanitize Status 267.It Dv Page 0xc1 268Advanced SMART information (WDC/HGST) 269.It Dv Page 0xc1 270Read latency stats (Intel) 271.It Dv Page 0xc2 272Wite latency stats (Intel) 273.It Dv Page 0xc5 274Temperature stats (Intel) 275.It Dv Page 0xca 276Advanced SMART information (Intel) 277.El 278.Pp 279Specifying 280.Fl v 281.Ic help 282will list all valid vendors and pages. 283.Fl x 284will print the page as hex. 285.Fl b 286will print the binary data for the page. 287.Fl s 288will set Log Specific Field. 289.Fl i 290will set Log Specific Identifier. 291.Fl r 292will set Retain Asynchronous Event. 293.Ss ns 294Various namespace management commands. 295If namespace management is supported by device, allow list, create and delete 296namespaces, list, attach and detach controllers to namespaces. 297.Ss nsid 298Reports the namespace id and controller device associated with the 299.Aq Ar namespace-id 300or 301.Aq Ar device-id 302argument. 303.Ss resv acquire 304Acquire or preempt namespace reservation, using specified parameters: 305.Bl -tag -width 6n 306.It Fl a 307Acquire action: 308.Bl -tag -compact -width 6n 309.It Dv 0 310Acquire 311.It Dv 1 312Preempt 313.It Dv 2 314Preempt and abort 315.El 316.It Fl c 317Current reservation key. 318.It Fl p 319Preempt reservation key. 320.It Fl t 321Reservation type: 322.Bl -tag -compact -width 6n 323.It Dv 1 324Write Exclusive 325.It Dv 2 326Exclusive Access 327.It Dv 3 328Write Exclusive - Registrants Only 329.It Dv 4 330Exclusive Access - Registrants Only 331.It Dv 5 332Write Exclusive - All Registrants 333.It Dv 6 334Exclusive Access - All Registrants 335.El 336.El 337.Ss resv register 338Register, unregister or replace reservation key, using specified parameters: 339.Bl -tag -width 6n 340.It Fl c 341Current reservation key. 342.It Fl k 343New reservation key. 344.It Fl r 345Register action: 346.Bl -tag -compact -width 6n 347.It Dv 0 348Register 349.It Dv 1 350Unregister 351.It Dv 2 352Replace 353.El 354.It Fl i 355Ignore Existing Key 356.It Fl p 357Change Persist Through Power Loss State: 358.Bl -tag -compact -width 6n 359.It Dv 0 360No change to PTPL state 361.It Dv 2 362Set PTPL state to ‘0’. 363Reservations are released and registrants are cleared on a power on. 364.It Dv 3 365Set PTPL state to ‘1’. 366Reservations and registrants persist across a power loss. 367.El 368.El 369.Ss resv release 370Release or clear reservation, using specified parameters: 371.Bl -tag -width 6n 372.It Fl c 373Current reservation key. 374.It Fl t 375Reservation type. 376.It Fl a 377Release action: 378.Bl -tag -compact -width 6n 379.It Dv 0 380Release 381.It Dv 1 382Clean 383.El 384.El 385.Ss resv report 386Print reservation status, using specified parameters: 387.Bl -tag -width 6n 388.It Fl x 389Print reservation status in hex. 390.It Fl e 391Use Extended Data Structure. 392.El 393.Ss format 394Format either specified namespace, or all namespaces of specified controller, 395using specified parameters: 396.Ar fmt 397LBA Format, 398.Ar mset 399Metadata Settings, 400.Ar pi 401Protection Information, 402.Ar pil 403Protection Information Location. 404When formatting specific namespace, existing values are used as defaults. 405When formatting all namespaces, all parameters should be specified. 406Some controllers may not support formatting or erasing specific or all 407namespaces. 408Option 409.Fl E 410enables User Data Erase during format. 411Option 412.Fl C 413enables Cryptographic Erase during format. 414.Ss sanitize 415Sanitize NVM subsystem of specified controller, 416using specified parameters: 417.Bl -tag -width 6n 418.It Fl a Ar operation 419Specify the sanitize operation to perform. 420.Bl -tag -width 16n 421.It overwrite 422Perform an overwrite operation by writing a user supplied 423data pattern to the device one or more times. 424The pattern is given by the 425.Fl p 426argument. 427The number of times is given by the 428.Fl c 429argument. 430.It block 431Perform a block erase operation. 432All the device's blocks are set to a vendor defined 433value, typically zero. 434.It crypto 435Perform a cryptographic erase operation. 436The encryption keys are changed to prevent the decryption 437of the data. 438.It exitfailure 439Exits a previously failed sanitize operation. 440A failed sanitize operation can only be exited if it was 441run in the unrestricted completion mode, as provided by the 442.Fl U 443argument. 444.El 445.It Fl c Ar passes 446The number of passes when performing an 447.Sq overwrite 448operation. 449Valid values are between 1 and 16. 450The default is 1. 451.It Fl d 452No Deallocate After Sanitize. 453.It Fl I 454When performing an 455.Sq overwrite 456operation, the pattern is inverted between consecutive passes. 457.It Fl p Ar pattern 45832 bits of pattern to use when performing an 459.Sq overwrite 460operation. 461The pattern is repeated as needed to fill each block. 462.It Fl U 463Perform the sanitize in the unrestricted completion mode. 464If the operation fails, it can later be exited with the 465.Sq exitfailure 466operation. 467.It Fl r 468Run in 469.Dq report only 470mode. 471This will report status on a sanitize that is already running on the drive. 472.El 473.Ss power 474Manage the power modes of the NVMe controller. 475.Bl -tag -width 6n 476.It Fl l 477List all supported power modes. 478.It Fl p Ar mode 479Set the power mode to 480.Ar mode . 481This must be a mode listed with the 482.Dl nvmecontrol power -l 483command. 484.It Fl w Ar hint 485Set the workload hint for automatic power mode control. 486.Bl -tag -compact -width 6n 487.It 0 488No workload hint is provided. 489.It 1 490Extended idle period workload. 491The device is often idle for minutes at a time. 492A burst of write commands comes in over a period of seconds. 493Then the device returns to being idle. 494.It 2 495Heavy sequential writes. 496A huge number of sequential writes will be submitted, filling the submission queues. 497.It Other 498All other values are reserved and have no standard meaning. 499.El 500Please see the 501.Dq NVM Subsystem Workloads 502section of the relevant NVM Express Base Standard for details. 503.El 504.Ss selftest 505Start the specified device self-test: 506.Bl -tag -width 6n 507.It Fl c Ar code 508Specify the device self-test command code. 509Common codes are: 510.Bl -tag -compact -width 6n 511.It Dv 0x1 512Start a short device self-test operation 513.It Dv 0x2 514Start an extended device self-test operation 515.It Dv 0xe 516Start a vendor specific device self-test operation 517.It Dv 0xf 518Abort the device self-test operation 519.El 520.El 521.Ss wdc 522The various wdc command retrieve log data from the wdc/hgst drives. 523The 524.Fl o 525flag specifies a path template to use to output the files. 526Each file takes the path template (which defaults to nothing), appends 527the drive's serial number and the type of dump it is followed 528by .bin. 529These logs must be sent to the vendor for analysis. 530This tool only provides a way to extract them. 531.Ss passthru 532The 533.Dq admin-passthru 534and 535.Dq io-passthru 536commands send NVMe commands to 537either the administrative or the data part of the device. 538These commands are expected to be compatible with nvme-cli. 539Please see the NVM Express Base Standard for details. 540.Bl -tag -width 16n 541.It Fl o -opcode Ar opcode 542Opcode to send. 543.It Fl 2 -cdw2 Ar value 54432-bit value for CDW2. 545.It Fl 3 -cdw3 Ar value 54632-bit value for CDW3. 547.It Fl 4 -cdw10 Ar value 54832-bit value for CDW10. 549.It Fl 5 -cdw11 Ar value 55032-bit value for CDW11. 551.It Fl 6 -cdw12 Ar value 55232-bit value for CDW12. 553.It Fl 7 -cdw13 Ar value 55432-bit value for CDW13. 555.It Fl 8 -cdw14 Ar value 55632-bit value for CDW14. 557.It Fl 9 -cdw15 Ar value 55832-bit value for CDW15. 559.It Fl l -data-len 560Length of the data for I/O (bytes). 561.It Fl m -metadata-len 562Length of the metadata segment for command (bytes). 563This is ignored and not implemented in 564.Xr nvme 4 . 565.It Fl f -flags 566Nvme command flags. 567.It Fl n -namespace-id 568Namespace ID for command (Ignored). 569.It Fl p -prefill 570Value to prefill payload with. 571.It Fl b -raw-binary 572Output in binary format (otherwise a hex dump is produced). 573.It Fl d -dry-run 574Do not actually execute the command, but perform sanity checks on it. 575.It Fl r -read 576Command reads data from the device. 577.It Fl s -show-command 578Show all the command values on stdout. 579.It Fl w -write 580Command writes data to the device. 581.El 582Send arbitrary commands to the device. 583Can be used to extract vendor specific logs. 584Transfers to/from the device possible, but limited to 585.Dv MAXPHYS 586bytes. 587Commands either read data or write it, but not both. 588Commands needing metadata are not supported by the 589.Xr nvme 4 590drive. 591.Sh DEVICE NAMES 592Where 593.Aq Ar namespace-id 594is required, you can use either the 595.Pa nvmeXnsY 596device, or the disk device such as 597.Pa ndaZ 598or 599.Pa nvdZ . 600The leading 601.Pa /dev/ 602is omitted. 603Where 604.Aq Ar device-id 605is required, you can use either the 606.Pa nvmeX 607device, or the disk device such as 608.Pa nda Z 609or 610.Pa nvdZ . 611For commands that take an optional 612.Aq nsid 613you can use it to get information on other namespaces, or to query the 614drive itself. 615A 616.Aq nsid 617of 618.Dq 0 619means query the drive itself. 620.Sh EXAMPLES 621.Dl nvmecontrol devlist 622.Pp 623Display a list of NVMe controllers and namespaces along with their device nodes. 624.Pp 625.Dl nvmecontrol identify nvme0 626.Dl nvmecontrol identify -n 0 nvd0 627.Pp 628Display a human-readable summary of the nvme0 629.Dv IDENTIFY_CONTROLLER 630data. 631In this example, nvd0 is connected to nvme0. 632.Pp 633.Dl nvmecontrol identify -x -v nvme0ns1 634.Dl nvmecontrol identify -x -v -n 1 nvme0 635.Pp 636Display an hexadecimal dump of the nvme0 637.Dv IDENTIFY_NAMESPACE 638data for namespace 1. 639.Pp 640.Dl nvmecontrol perftest -n 32 -o read -s 512 -t 30 nvme0ns1 641.Pp 642Run a performance test on nvme0ns1 using 32 kernel threads for 30 seconds. 643Each thread will issue a single 512 byte read command. 644Results are printed to stdout when 30 seconds expires. 645.Pp 646.Dl nvmecontrol reset nvme0 647.Dl nvmecontrol reset nda4 648.Pp 649Perform a controller-level reset of the nvme0 controller. 650In this example, nda4 is wired to nvme0. 651.Pp 652.Dl nvmecontrol logpage -p 1 nvme0 653.Pp 654Display a human-readable summary of the nvme0 controller's Error Information Log. 655Log pages defined by the NVMe specification include Error Information Log (ID=1), 656SMART/Health Information Log (ID=2), and Firmware Slot Log (ID=3). 657.Pp 658.Dl nvmecontrol logpage -p 0xc1 -v wdc nvme0 659.Pp 660Display a human-readable summary of the nvme0's wdc-specific advanced 661SMART data. 662.Pp 663.Dl nvmecontrol logpage -p 1 -x nvme0 664.Pp 665Display a hexadecimal dump of the nvme0 controller's Error Information Log. 666.Pp 667.Dl nvmecontrol logpage -p 0xcb -b nvme0 > /tmp/page-cb.bin 668.Pp 669Print the contents of vendor specific page 0xcb as binary data on 670standard out. 671Redirect it to a temporary file. 672.Pp 673.Dl nvmecontrol firmware -s 2 -f /tmp/nvme_firmware nvme0 674.Pp 675Download the firmware image contained in "/tmp/nvme_firmware" to slot 2 of the 676nvme0 controller, but do not activate the image. 677.Pp 678.Dl nvmecontrol firmware -s 4 -a nvme0 679.Pp 680Activate the firmware in slot 4 of the nvme0 controller on the next reset. 681.Pp 682.Dl nvmecontrol firmware -s 7 -f /tmp/nvme_firmware -a nvme0 683.Pp 684Download the firmware image contained in "/tmp/nvme_firmware" to slot 7 of the 685nvme0 controller and activate it on the next reset. 686.Pp 687.Dl nvmecontrol power -l nvme0 688.Pp 689List all the current power modes. 690.Pp 691.Dl nvmecontrol power -p 3 nvme0 692.Pp 693Set the current power mode. 694.Pp 695.Dl nvmecontrol power nvme0 696.Pp 697Get the current power mode. 698.Pp 699.Dl nvmecontrol identify -n 0 nda0 700.Pp 701Identify the drive data associated with the 702.Pa nda0 703device. 704The corresponding 705.Pa nvmeX 706devices is used automatically. 707.Pp 708.Dl nvmecontrol identify nda0 709.Pp 710Get the namespace parameters associated with the 711.Pa nda0 712device. 713The corresponding 714.Pa nvmeXnsY 715device is used automatically. 716.Sh DYNAMIC LOADING 717The directories 718.Pa /lib/nvmecontrol 719and 720.Pa /usr/local/lib/nvmecontrol 721are scanned for any .so files. 722These files are loaded. 723The members of the 724.Va top 725linker set are added to the top-level commands. 726The members of the 727.Va logpage 728linker set are added to the logpage parsers. 729.Sh SEE ALSO 730.Rs 731.%T The NVM Express Base Specification 732.%D June 10, 2019 733.%U https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4-2019.06.10-Ratified.pdf 734.Re 735.Sh HISTORY 736The 737.Nm 738utility appeared in 739.Fx 9.2 . 740.Sh AUTHORS 741.An -nosplit 742.Nm 743was developed by Intel and originally written by 744.An Jim Harris Aq Mt jimharris@FreeBSD.org . 745.Pp 746This man page was written by 747.An Jim Harris Aq Mt jimharris@FreeBSD.org . 748