1.\" 2.\" Copyright (c) 2020 Warner Losh <imp@FreeBSD.org> 3.\" Copyright (c) 2018-2019 Alexander Motin <mav@FreeBSD.org> 4.\" Copyright (c) 2012 Intel Corporation 5.\" All rights reserved. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions, and the following disclaimer, 12.\" without modification. 13.\" 2. Redistributions in binary form must reproduce at minimum a disclaimer 14.\" substantially similar to the "NO WARRANTY" disclaimer below 15.\" ("Disclaimer") and any redistribution must be conditioned upon 16.\" including a substantially similar Disclaimer requirement for further 17.\" binary redistribution. 18.\" 19.\" NO WARRANTY 20.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 21.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 22.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR 23.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 24.\" HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 26.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 27.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 28.\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING 29.\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 30.\" POSSIBILITY OF SUCH DAMAGES. 31.\" 32.\" nvmecontrol man page. 33.\" 34.\" Author: Jim Harris <jimharris@FreeBSD.org> 35.\" 36.Dd December 14, 2022 37.Dt NVMECONTROL 8 38.Os 39.Sh NAME 40.Nm nvmecontrol 41.Nd NVM Express control utility 42.Sh SYNOPSIS 43.Nm 44.Ic devlist 45.Op Fl h 46.Nm 47.Ic identify 48.Op Fl v 49.Op Fl x 50.Op Fl n Ar nsid 51.Aq Ar device-id | Ar namespace-id 52.Nm 53.Ic perftest 54.Aq Fl n Ar num_threads 55.Aq Fl o Ar read|write 56.Op Fl p 57.Aq Fl s Ar size_in_bytes 58.Aq Fl t Ar time_in_sec 59.Aq Ar namespace-id 60.Nm 61.Ic reset 62.Aq Ar device-id 63.Nm 64.Ic logpage 65.Aq Fl p Ar page_id 66.Op Fl x 67.Op Fl v Ar vendor-string 68.Op Fl b 69.Op Fl f Ar LSP 70.Op Fl i Ar LSI 71.Op Fl r 72.Aq Ar device-id | Ar namespace-id 73.Nm 74.Ic ns active 75.Aq Ar device-id 76.Nm 77.Ic ns allocated 78.Aq Ar device-id 79.Nm 80.Ic ns attach 81.Aq Fl n Ar nsid 82.Aq Fl c Ar cntid 83.Aq Ar device-id 84.Nm 85.Ic ns attached 86.Aq Fl n Ar nsid 87.Aq Ar device-id 88.Nm 89.Ic ns controllers 90.Aq Ar device-id 91.Nm 92.Ic ns create 93.Aq Fl s Ar nsze 94.Op Fl c Ar ncap 95.Op Fl f Ar lbaf 96.Op Fl m Ar mset 97.Op Fl n Ar nmic 98.Op Fl p Ar pi 99.Op Fl l Ar pil 100.Op Fl L Ar flbas 101.Op Fl d Ar dps 102.Aq Ar device-id 103.Nm 104.Ic ns delete 105.Aq Fl n Ar nsid 106.Aq Ar device-id 107.Nm 108.Ic ns detach 109.Aq Fl n Ar nsid 110.Aq Fl c Ar cntid 111.Aq Ar device-id 112.Nm 113.Ic ns identify 114.Op Fl v 115.Op Fl x 116.Aq Fl n Ar nsid 117.Aq Ar device-id 118.Nm 119.Ic nsid 120.Aq Ar device-id | Ar namespace-id 121.Nm 122.Ic resv acquire 123.Aq Fl c Ar crkey 124.Op Fl p Ar prkey 125.Aq Fl t Ar rtype 126.Aq Fl a Ar racqa 127.Aq Ar namespace-id 128.Nm 129.Ic resv register 130.Op Fl i 131.Op Fl c Ar crkey 132.Aq Fl k Ar nrkey 133.Aq Fl r Ar rrega 134.Op Fl p Ar cptpl 135.Aq Ar namespace-id 136.Nm 137.Ic resv release 138.Aq Fl c Ar crkey 139.Aq Fl t Ar rtype 140.Aq Fl a Ar rrela 141.Aq Ar namespace-id 142.Nm 143.Ic resv report 144.Op Fl e 145.Op Fl v 146.Op Fl x 147.Aq Ar namespace-id 148.Nm 149.Ic firmware 150.Op Fl s Ar slot 151.Op Fl f Ar path_to_firmware 152.Op Fl a 153.Aq Ar device-id 154.Nm 155.Ic format 156.Op Fl f Ar fmt 157.Op Fl m Ar mset 158.Op Fl o Ar pi 159.Op Fl l Ar pil 160.Op Fl E 161.Op Fl C 162.Aq Ar device-id | Ar namespace-id 163.Nm 164.Ic sanitize 165.Aq Fl a Ar sanact 166.Op Fl c Ar owpass 167.Op Fl d 168.Op Fl p Ar ovrpat 169.Op Fl r 170.Op Fl I 171.Op Fl U 172.Aq Ar device-id 173.Nm 174.Ic power 175.Op Fl l 176.Op Fl p power_state 177.Op Fl w workload_hint 178.Nm 179.Ic selftest 180.Aq Fl c Ar code 181.Aq Ar device-id | Ar namespace-id 182.Nm 183.Ic wdc cap-diag 184.Op Fl o path_template 185.Aq Ar device-id 186.Nm 187.Ic wdc drive-log 188.Op Fl o path_template 189.Aq Ar device-id 190.Nm 191.Ic wdc get-crash-dump 192.Op Fl o path_template 193.Aq Ar device-id 194.\" .Nm 195.\" .Ic wdc purge 196.\" .Aq device-id 197.\" .Nm 198.\" .Ic wdc purge-monitor 199.\" .Aq device-id 200.Nm 201.Ic admin-passthru 202.Op args 203.Aq Ar device-id 204.Nm 205.Ic io-passthru 206.Op args 207.Aq Ar namespace-id 208.Sh DESCRIPTION 209NVM Express (NVMe) is a storage protocol standard, for SSDs and other 210high-speed storage devices over PCI Express. 211.Ss devlist 212List all NVMe controllers and namespaces along with their device nodes. 213With the 214.Fl h 215argument, use unit suffixes: Byte, Kibibyte, Mebibyte, Gibibyte, Tebibyte 216and Pebibyte (based on powers of 1024) when showing the disk space. 217By default, uses Mebibyte. 218.Ss identify 219The identify commands reports information from the drive's 220.Dv IDENTIFY_CONTROLLER 221if a 222.Ar device-id 223is specified. 224It reports 225.Dv IDENTIFY_NAMESPACE 226data if a 227.Ar namespace-id 228is specified. 229When used with disk names, the 230.Dv IDENTIFY_NAMESPACE 231data is reported, unless the namespace 232.Ar nsid 233is overridden with the 234.Fl n 235flag. 236Then that namespace's data is reported, if it exists. 237The command accepts the following parameters: 238.Bl -tag -width 6n 239.It Fl n 240The namespace 241.Aq nsid 242to use instead of the namespace associated with the device. 243A 244.Ar nsid 245of 246.Dq 0 247is used to retrieve the 248.Dv IDENTIFY_CONTROLLER 249data associated with that drive. 250.El 251.Ss logpage 252The logpage command knows how to print log pages of various types. 253It also knows about vendor specific log pages from hgst/wdc, samsung and intel. 254Note that some vendors use the same log page numbers for different data. 255.Pp 256.Bl -tag -compact -width "Page 0x00" 257.It Dv Page 0x01 258Drive Error Log 259.It Dv Page 0x02 260Health/SMART Data 261.It Dv Page 0x03 262Firmware Information 263.It Dv Page 0x04 264Changed Namespace List 265.It Dv Page 0x05 266Commands Supported and Effects 267.It Dv Page 0x06 268Device Self-test 269.It Dv Page 0x80 270Reservation Notification 271.It Dv Page 0x81 272Sanitize Status 273.It Dv Page 0xc1 274Advanced SMART information (WDC/HGST) 275.It Dv Page 0xc1 276Read latency stats (Intel) 277.It Dv Page 0xc2 278Wite latency stats (Intel) 279.It Dv Page 0xc5 280Temperature stats (Intel) 281.It Dv Page 0xca 282Advanced SMART information (Intel) 283.It Dv Page 0xca 284Extended SMART information (Samsung) 285.El 286.Pp 287Specifying 288.Fl v 289.Ic help 290will list all valid vendors and pages. 291.Fl x 292will print the page as hex. 293.Fl b 294will print the binary data for the page. 295.Fl s 296will set Log Specific Field. 297.Fl i 298will set Log Specific Identifier. 299.Fl r 300will set Retain Asynchronous Event. 301.Ss ns 302Various namespace management commands. 303If namespace management is supported by device, allow list, create and delete 304namespaces, list, attach and detach controllers to namespaces. 305.Ss nsid 306Reports the namespace id and controller device associated with the 307.Aq Ar namespace-id 308or 309.Aq Ar device-id 310argument. 311.Ss resv acquire 312Acquire or preempt namespace reservation, using specified parameters: 313.Bl -tag -width 6n 314.It Fl a 315Acquire action: 316.Bl -tag -compact -width 6n 317.It Dv 0 318Acquire 319.It Dv 1 320Preempt 321.It Dv 2 322Preempt and abort 323.El 324.It Fl c 325Current reservation key. 326.It Fl p 327Preempt reservation key. 328.It Fl t 329Reservation type: 330.Bl -tag -compact -width 6n 331.It Dv 1 332Write Exclusive 333.It Dv 2 334Exclusive Access 335.It Dv 3 336Write Exclusive - Registrants Only 337.It Dv 4 338Exclusive Access - Registrants Only 339.It Dv 5 340Write Exclusive - All Registrants 341.It Dv 6 342Exclusive Access - All Registrants 343.El 344.El 345.Ss resv register 346Register, unregister or replace reservation key, using specified parameters: 347.Bl -tag -width 6n 348.It Fl c 349Current reservation key. 350.It Fl k 351New reservation key. 352.It Fl r 353Register action: 354.Bl -tag -compact -width 6n 355.It Dv 0 356Register 357.It Dv 1 358Unregister 359.It Dv 2 360Replace 361.El 362.It Fl i 363Ignore Existing Key 364.It Fl p 365Change Persist Through Power Loss State: 366.Bl -tag -compact -width 6n 367.It Dv 0 368No change to PTPL state 369.It Dv 2 370Set PTPL state to ‘0’. 371Reservations are released and registrants are cleared on a power on. 372.It Dv 3 373Set PTPL state to ‘1’. 374Reservations and registrants persist across a power loss. 375.El 376.El 377.Ss resv release 378Release or clear reservation, using specified parameters: 379.Bl -tag -width 6n 380.It Fl c 381Current reservation key. 382.It Fl t 383Reservation type. 384.It Fl a 385Release action: 386.Bl -tag -compact -width 6n 387.It Dv 0 388Release 389.It Dv 1 390Clean 391.El 392.El 393.Ss resv report 394Print reservation status, using specified parameters: 395.Bl -tag -width 6n 396.It Fl x 397Print reservation status in hex. 398.It Fl e 399Use Extended Data Structure. 400.El 401.Ss format 402Format either specified namespace, or all namespaces of specified controller, 403using specified parameters: 404.Ar fmt 405LBA Format, 406.Ar mset 407Metadata Settings, 408.Ar pi 409Protection Information, 410.Ar pil 411Protection Information Location. 412When formatting specific namespace, existing values are used as defaults. 413When formatting all namespaces, all parameters should be specified. 414Some controllers may not support formatting or erasing specific or all 415namespaces. 416Option 417.Fl E 418enables User Data Erase during format. 419Option 420.Fl C 421enables Cryptographic Erase during format. 422.Ss sanitize 423Sanitize NVM subsystem of specified controller, 424using specified parameters: 425.Bl -tag -width 6n 426.It Fl a Ar operation 427Specify the sanitize operation to perform. 428.Bl -tag -width 16n 429.It overwrite 430Perform an overwrite operation by writing a user supplied 431data pattern to the device one or more times. 432The pattern is given by the 433.Fl p 434argument. 435The number of times is given by the 436.Fl c 437argument. 438.It block 439Perform a block erase operation. 440All the device's blocks are set to a vendor defined 441value, typically zero. 442.It crypto 443Perform a cryptographic erase operation. 444The encryption keys are changed to prevent the decryption 445of the data. 446.It exitfailure 447Exits a previously failed sanitize operation. 448A failed sanitize operation can only be exited if it was 449run in the unrestricted completion mode, as provided by the 450.Fl U 451argument. 452.El 453.It Fl c Ar passes 454The number of passes when performing an 455.Sq overwrite 456operation. 457Valid values are between 1 and 16. 458The default is 1. 459.It Fl d 460No Deallocate After Sanitize. 461.It Fl I 462When performing an 463.Sq overwrite 464operation, the pattern is inverted between consecutive passes. 465.It Fl p Ar pattern 46632 bits of pattern to use when performing an 467.Sq overwrite 468operation. 469The pattern is repeated as needed to fill each block. 470.It Fl U 471Perform the sanitize in the unrestricted completion mode. 472If the operation fails, it can later be exited with the 473.Sq exitfailure 474operation. 475.It Fl r 476Run in 477.Dq report only 478mode. 479This will report status on a sanitize that is already running on the drive. 480.El 481.Ss power 482Manage the power modes of the NVMe controller. 483.Bl -tag -width 6n 484.It Fl l 485List all supported power modes. 486.It Fl p Ar mode 487Set the power mode to 488.Ar mode . 489This must be a mode listed with the 490.Dl nvmecontrol power -l 491command. 492.It Fl w Ar hint 493Set the workload hint for automatic power mode control. 494.Bl -tag -compact -width 6n 495.It 0 496No workload hint is provided. 497.It 1 498Extended idle period workload. 499The device is often idle for minutes at a time. 500A burst of write commands comes in over a period of seconds. 501Then the device returns to being idle. 502.It 2 503Heavy sequential writes. 504A huge number of sequential writes will be submitted, filling the submission queues. 505.It Other 506All other values are reserved and have no standard meaning. 507.El 508Please see the 509.Dq NVM Subsystem Workloads 510section of the relevant NVM Express Base Standard for details. 511.El 512.Ss selftest 513Start the specified device self-test: 514.Bl -tag -width 6n 515.It Fl c Ar code 516Specify the device self-test command code. 517Common codes are: 518.Bl -tag -compact -width 6n 519.It Dv 0x1 520Start a short device self-test operation 521.It Dv 0x2 522Start an extended device self-test operation 523.It Dv 0xe 524Start a vendor specific device self-test operation 525.It Dv 0xf 526Abort the device self-test operation 527.El 528.El 529.Ss wdc 530The various wdc command retrieve log data from the wdc/hgst drives. 531The 532.Fl o 533flag specifies a path template to use to output the files. 534Each file takes the path template (which defaults to nothing), appends 535the drive's serial number and the type of dump it is followed 536by .bin. 537These logs must be sent to the vendor for analysis. 538This tool only provides a way to extract them. 539.Ss passthru 540The 541.Dq admin-passthru 542and 543.Dq io-passthru 544commands send NVMe commands to 545either the administrative or the data part of the device. 546These commands are expected to be compatible with nvme-cli. 547Please see the NVM Express Base Standard for details. 548.Bl -tag -width 16n 549.It Fl o -opcode Ar opcode 550Opcode to send. 551.It Fl 2 -cdw2 Ar value 55232-bit value for CDW2. 553.It Fl 3 -cdw3 Ar value 55432-bit value for CDW3. 555.It Fl 4 -cdw10 Ar value 55632-bit value for CDW10. 557.It Fl 5 -cdw11 Ar value 55832-bit value for CDW11. 559.It Fl 6 -cdw12 Ar value 56032-bit value for CDW12. 561.It Fl 7 -cdw13 Ar value 56232-bit value for CDW13. 563.It Fl 8 -cdw14 Ar value 56432-bit value for CDW14. 565.It Fl 9 -cdw15 Ar value 56632-bit value for CDW15. 567.It Fl l -data-len 568Length of the data for I/O (bytes). 569.It Fl m -metadata-len 570Length of the metadata segment for command (bytes). 571This is ignored and not implemented in 572.Xr nvme 4 . 573.It Fl f -flags 574Nvme command flags. 575.It Fl n -namespace-id 576Namespace ID for command (Ignored). 577.It Fl p -prefill 578Value to prefill payload with. 579.It Fl b -raw-binary 580Output in binary format (otherwise a hex dump is produced). 581.It Fl d -dry-run 582Do not actually execute the command, but perform sanity checks on it. 583.It Fl r -read 584Command reads data from the device. 585.It Fl s -show-command 586Show all the command values on stdout. 587.It Fl w -write 588Command writes data to the device. 589.El 590.Pp 591Send arbitrary commands to the device. 592Can be used to extract vendor specific logs. 593Transfers to/from the device possible, but limited to 594.Dv MAXPHYS 595bytes. 596Commands either read data or write it, but not both. 597Commands needing metadata are not supported by the 598.Xr nvme 4 599drive. 600.Sh DEVICE NAMES 601Where 602.Aq Ar namespace-id 603is required, you can use either the 604.Pa nvmeXnsY 605device, or the disk device such as 606.Pa ndaZ 607or 608.Pa nvdZ . 609The leading 610.Pa /dev/ 611is omitted. 612Where 613.Aq Ar device-id 614is required, you can use either the 615.Pa nvmeX 616device, or the disk device such as 617.Pa ndaZ 618or 619.Pa nvdZ . 620For commands that take an optional 621.Aq nsid 622you can use it to get information on other namespaces, or to query the 623drive itself. 624A 625.Aq nsid 626of 627.Dq 0 628means query the drive itself. 629.Sh EXAMPLES 630.Dl nvmecontrol devlist 631.Pp 632Display a list of NVMe controllers and namespaces along with their device nodes. 633.Pp 634.Dl nvmecontrol identify nvme0 635.Dl nvmecontrol identify -n 0 nvd0 636.Pp 637Display a human-readable summary of the nvme0 638.Dv IDENTIFY_CONTROLLER 639data. 640In this example, nvd0 is connected to nvme0. 641.Pp 642.Dl nvmecontrol identify -x -v nvme0ns1 643.Dl nvmecontrol identify -x -v -n 1 nvme0 644.Pp 645Display an hexadecimal dump of the nvme0 646.Dv IDENTIFY_NAMESPACE 647data for namespace 1. 648.Pp 649.Dl nvmecontrol perftest -n 32 -o read -s 512 -t 30 nvme0ns1 650.Pp 651Run a performance test on nvme0ns1 using 32 kernel threads for 30 seconds. 652Each thread will issue a single 512 byte read command. 653Results are printed to stdout when 30 seconds expires. 654.Pp 655.Dl nvmecontrol reset nvme0 656.Dl nvmecontrol reset nda4 657.Pp 658Perform a controller-level reset of the nvme0 controller. 659In this example, nda4 is wired to nvme0. 660.Pp 661.Dl nvmecontrol logpage -p 1 nvme0 662.Pp 663Display a human-readable summary of the nvme0 controller's Error Information Log. 664Log pages defined by the NVMe specification include Error Information Log (ID=1), 665SMART/Health Information Log (ID=2), and Firmware Slot Log (ID=3). 666.Pp 667.Dl nvmecontrol logpage -p 0xc1 -v wdc nvme0 668.Pp 669Display a human-readable summary of the nvme0's wdc-specific advanced 670SMART data. 671.Pp 672.Dl nvmecontrol logpage -p 1 -x nvme0 673.Pp 674Display a hexadecimal dump of the nvme0 controller's Error Information Log. 675.Pp 676.Dl nvmecontrol logpage -p 0xcb -b nvme0 > /tmp/page-cb.bin 677.Pp 678Print the contents of vendor specific page 0xcb as binary data on 679standard out. 680Redirect it to a temporary file. 681.Pp 682.Dl nvmecontrol firmware -s 2 -f /tmp/nvme_firmware nvme0 683.Pp 684Download the firmware image contained in "/tmp/nvme_firmware" to slot 2 of the 685nvme0 controller, but do not activate the image. 686.Pp 687.Dl nvmecontrol firmware -s 4 -a nvme0 688.Pp 689Activate the firmware in slot 4 of the nvme0 controller on the next reset. 690.Pp 691.Dl nvmecontrol firmware -s 7 -f /tmp/nvme_firmware -a nvme0 692.Pp 693Download the firmware image contained in "/tmp/nvme_firmware" to slot 7 of the 694nvme0 controller and activate it on the next reset. 695.Pp 696.Dl nvmecontrol power -l nvme0 697.Pp 698List all the current power modes. 699.Pp 700.Dl nvmecontrol power -p 3 nvme0 701.Pp 702Set the current power mode. 703.Pp 704.Dl nvmecontrol power nvme0 705.Pp 706Get the current power mode. 707.Pp 708.Dl nvmecontrol identify -n 0 nda0 709.Pp 710Identify the drive data associated with the 711.Pa nda0 712device. 713The corresponding 714.Pa nvmeX 715devices is used automatically. 716.Pp 717.Dl nvmecontrol identify nda0 718.Pp 719Get the namespace parameters associated with the 720.Pa nda0 721device. 722The corresponding 723.Pa nvmeXnsY 724device is used automatically. 725.Sh DYNAMIC LOADING 726The directories 727.Pa /lib/nvmecontrol 728and 729.Pa /usr/local/lib/nvmecontrol 730are scanned for any .so files. 731These files are loaded. 732The members of the 733.Va top 734linker set are added to the top-level commands. 735The members of the 736.Va logpage 737linker set are added to the logpage parsers. 738.Sh SEE ALSO 739.Rs 740.%T The NVM Express Base Specification 741.%D June 10, 2019 742.%U https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4-2019.06.10-Ratified.pdf 743.Re 744.Sh HISTORY 745The 746.Nm 747utility appeared in 748.Fx 9.2 . 749.Sh AUTHORS 750.An -nosplit 751.Nm 752was developed by Intel and originally written by 753.An Jim Harris Aq Mt jimharris@FreeBSD.org . 754.Pp 755This man page was written by 756.An Jim Harris Aq Mt jimharris@FreeBSD.org . 757