1.\" 2.\" This file and its contents are supplied under the terms of the 3.\" Common Development and Distribution License ("CDDL"), version 1.0. 4.\" You may only use this file in accordance with the terms of version 5.\" 1.0 of the CDDL. 6.\" 7.\" A full copy of the text of the CDDL should have accompanied this 8.\" source. A copy of the CDDL is also available via the Internet at 9.\" http://www.illumos.org/license/CDDL. 10.\" 11.\" 12.\" Copyright 2021 Oxide Computer Company 13.\" Copyright 2022 Tintri by DDN, Inc. All rights reserved. 14.\" 15.Dd April 25, 2022 16.Dt NVMEADM 8 17.Os 18.Sh NAME 19.Nm nvmeadm 20.Nd NVMe administration utility 21.Sh SYNOPSIS 22.Nm 23.Fl h 24.Op Ar command 25.Nm 26.Op Fl dv 27.Cm list 28.Oo 29.Fl p 30.Fl o Ar field Ns [,...] 31.Oc 32.Op Ar ctl[/ns] Ns [,...] 33.Nm 34.Op Fl dv 35.Cm identify 36.Op Fl C | c | d | Oo Fl a Oc Fl n 37.Ar ctl[/ns] Ns [,...] 38.Nm 39.Op Fl dv 40.Cm identify-controller 41.Op Fl C | c | Oo Fl a Oc Fl n 42.Ar ctl Ns [,...] 43.Nm 44.Op Fl dv 45.Cm identify-namespace 46.Op Fl c | d 47.Ar ctl/ns Ns [,...] 48.Nm 49.Op Fl dv 50.Cm get-logpage 51.Ar ctl[/ns] Ns [,...] 52.Ar logpage 53.Nm 54.Op Fl dv 55.Cm get-features 56.Ar ctl[/ns] Ns [,...] 57.Op Ar feature-list 58.Nm 59.Op Fl dv 60.Cm format 61.Ar ctl[/ns] 62.Op Ar lba-format 63.Nm 64.Op Fl dv 65.Cm secure-erase 66.Op Fl c 67.Ar ctl[/ns] 68.Nm 69.Op Fl dv 70.Cm detach 71.Ar ctl[/ns] 72.Nm 73.Op Fl dv 74.Cm attach 75.Ar ctl[/ns] 76.Nm 77.Op Fl dv 78.Cm list-firmware 79.Ar ctl 80.Nm 81.Op Fl dv 82.Cm load-firmware 83.Ar ctl 84.Ar firmware-file 85.Op Ar offset 86.Nm 87.Op Fl dv 88.Cm commit-firmware 89.Ar ctl 90.Ar slot 91.Nm 92.Op Fl dv 93.Cm activate-firmware 94.Ar ctl 95.Ar slot 96.Sh DESCRIPTION 97The 98.Nm 99utility can be used to enumerate the NVMe controllers and their 100namespaces, query hardware information from a NVMe controller or 101namespace, and to format or secure-erase a NVMe controller or 102namespace. 103.Pp 104The information returned by the hardware is printed by 105.Nm 106in a human-readable form were applicable. 107Generally all 0-based counts are normalized and values may be 108converted to human-readable units such as MB (megabytes), W (watts), 109or C (degrees Celsius). 110.Sh OPTIONS 111The following options are supported: 112.Bl -tag -width Ds 113.It Fl h 114Print a short help text for 115.Nm , 116or for an optionally specified 117.Nm 118command. 119.It Fl d 120Enable debugging output. 121.It Fl v 122Enable verbose output. 123.El 124.Sh ARGUMENTS 125.Nm 126expects the following kinds of arguments: 127.Bl -tag -width "ctl[/ns]" 128.It Ar command 129Any command 130.Nm 131understands. 132See section 133.Sx COMMANDS . 134.It Ar ctl[/ns] 135Specifies a NVMe controller and optionally a namespace within that 136controller. 137The controller name consists of the driver name 138.Qq nvme 139followed by an instance number. 140A namespace is specified by appending a single 141.Qq / 142followed by the namespace ID to the controller name. 143The namespace ID is the EUI64 of the namespace, or a positive non-zero 144decimal number if the namespace doesn't have an EUI64. 145For commands that don't change the device state multiple controllers 146and namespaces can be specified as a comma-separated list. 147.Pp 148The list of controllers and namespaces present in the system can be 149queried with the 150.Cm list 151command without any arguments. 152.It Ar logpage 153Specifies the log page name for the 154.Cm get-logpage 155command. 156.It Ar feature-list 157A comma-separated list of feature names for the 158.Cm get-features 159command. 160Feature names can be specified in upper or lower case and can be 161shortened the shortest unique name. 162Some features may also have an alternative short name. 163.It Ar lba-format 164A non-zero integer specifying the LBA format for the 165.Cm format 166command. 167The list of supported LBA formats on a namespace can be retrieved 168with the 169.Nm 170.Cm identify 171command. 172.It Ar firmware-file 173Specifies the name of a firmware file to be loaded into the controller 174using the 175.Cm load-firmware 176command. 177.It Ar offset 178Specifies the byte offset at which to load 179.Ar firmware-file 180within the controller's upload buffer. 181Vendors may require multiple images to be loaded at different offsets 182before a firmware set is committed to a 183.Ar slot . 184.It Ar slot 185Specifies the firmware slot into which a firmware set is committed 186using the 187.Cm commit-firmware 188command, and subsequently activated with the 189.Cm activate-firmware 190command. 191Slots and their contents can be printed using the 192.Nm 193.Cm list-firmware 194command. 195.El 196.Sh COMMANDS 197.Bl -tag -width "" 198.It Xo 199.Nm 200.Cm list 201.Oo 202.Fl p 203.Fl o Ar field Ns [,...] 204.Oc 205.Op Ar ctl[/ns] Ns [,...] 206.Xc 207Lists the NVMe controllers and their namespaces in the system and 208prints a 1-line summary of their basic properties for each. 209If a list of controllers and/or namespaces is given then the listing 210is limited to those devices. 211By default, output is human-readable; however, a parsable form can 212controlled by using the following options: 213.Bl -tag -width Fl 214.It Fl p 215Rather than printing human-readable output, outputs an entry for each of 216the specified controllers and namespaces. 217If no controllers or namespaces are given as arguments, then the primary 218namespace of each controller is listed and if the 219.Fl v 220option is specified, then all of the namespaces for a controller are 221listed. 222This option requires that output fields be selected with the 223.Fl o 224option. 225.It Fl o Ar field Ns [,...] 226A comma-separated list of one or more output fields to be used. 227Fields are listed below and the name is case insensitive. 228.El 229.Pp 230The following fields can be specified when using the parsable form: 231.Bl -tag -width CAPACITY 232.It Sy MODEL 233The model number of the device, generally containing information about 234both the manufacturer and the product. 235.It Sy SERIAL 236The NVMe controller's serial number. 237.It Sy FWREV 238The controller's firmware revision. 239.It Sy VERSION 240The version of the NVMe specification the controller supports. 241.It Sy SIZE 242The logical size in bytes of the namespace. 243.It Sy CAPACITY 244The amount of logical bytes that the namespace may actually have allocated at 245any time. 246This may be different than size due to the use of thin provisioning or due to 247administrative action. 248.It Sy USED 249The number of bytes used in the namespace. 250.It Sy INSTANCE 251The name of the device node and instance of it. 252.It Sy NAMESPACE 253The numerical value of the namespace which can be used as part of other 254.Nm 255operations. 256.It Sy DISK 257The name of the disk device that corresponds to the namespace, if any. 258.El 259.It Xo 260.Nm 261.Cm identify-controller 262.Op Fl C | c | Oo Fl a Oc Fl n 263.Ar ctl Ns [,...] 264.Xc 265Print detailed information about the specified controllers. 266For an explanation of the data printed by this command refer to the description 267of the 268.Qq IDENTIFY 269admin command in the NVMe specification. 270.Pp 271By default, a relevant subset of the 272.Qq IDENTIFY CONTROLLER 273data structure is printed. 274The full data structure is only printed when verbose output is requested. 275.Pp 276The following options can be used to print other 277.Qq IDENTIFY 278information: 279.Bl -tag -width Fl 280.It Fl C 281Print the Common Namespace Identification of the controller. 282.It Fl a 283Alter the output of the 284.Fl n 285option to print the list allocated namespace identifiers. 286Can only be specified together with the 287.Fl n 288option. 289.It Fl c 290Print the list of all unique controller identifiers in the NVMe subsystem the 291specified controller belongs to. 292.It Fl n 293Print the list of active namespace identifiers of the controller. 294.El 295.It Xo 296.Nm 297.Cm identify-namespace 298.Op Fl c | d 299.Ar ctl/ns Ns [,...] 300.Xc 301Print detailed information about the specified namespace. 302For an explanation of the data printed by this command refer to the description 303of the 304.Qq IDENTIFY 305admin command in the NVMe specification. 306.Pp 307By default, a relevant subset of the 308.Qq IDENTIFY NAMESPACE 309data structure is printed. 310The full data structure is only printed when verbose output is requested. 311.Pp 312The following options can be used to print other 313.Qq IDENTIFY 314information: 315.Bl -tag -width Fl 316.It Fl c 317Print the list of all unique controller identifiers in the NVMe subsystem the 318specified namespace belongs to and which are currently attached to this 319namespace. 320.It Fl d 321Print the list of namespace identification descriptors of the namespace. 322.El 323.It Xo 324.Nm 325.Cm identify 326.Op Fl C | c | d | Oo Fl a Oc Fl n 327.Ar ctl[/ns] Ns [,...] 328.Xc 329Short-hand for the 330.Cm identify-controller 331and 332.Cm identify-namespace 333commands, prints the same information about the specified controllers and/or 334namespaces, depending on whether a controller or a namespace was specified. 335.Pp 336For a description of the various optional flags refer to the above description 337of the 338.Cm identify-controller 339and 340.Cm identify-namespace 341commands. 342.It Xo 343.Nm 344.Cm get-logpage 345.Ar ctl[/ns] Ns [,...] 346.Ar logpage 347.Xc 348Print the specified log page of the specified controllers and/or namespaces. 349Most log pages are only available on a per-controller basis. 350Known log pages are: 351.Bl -tag -width "firmware" 352.It error 353Error Information 354.It health 355SMART/Health Information. 356A controller may support this log page on a per-namespace basis. 357.It firmware 358Firmware Slot Information 359.El 360.Pp 361For an explanation of the contents of the log pages refer to the 362description of the 363.Qq GET LOGPAGE 364admin command in the NVMe specification. 365.It Xo 366.Nm 367.Cm get-features 368.Ar ctl[/ns] Ns [,...] 369.Op Ar feature-list 370.Xc 371Prints information about the specified features, or all features if 372none are given, of the specified controllers and/or namespaces. 373Feature names are case-insensitive, and they can be shortened as long 374as they remain unique. 375Some features also have alternative short names to which the same 376rules apply. 377The following features are supported: 378.Pp 379.TS 380tab(:); 381l l l. 382FULL NAME:SHORT NAME:CONTROLLER/NAMESPACE 383Arbitration::controller 384Power Management::controller 385LBA Range Type:range:namespace 386Temperature Threshold::controller 387Error Recovery::controller 388Volatile Write Cache:cache:controller 389Number of Queues:queues:controller 390Interrupt Coalescing:coalescing:controller 391Interrupt Vector Configuration:vector:controller 392Write Atomicity:atomicity:controller 393Asynchronous Event Configuration:event:controller 394Autonomous Power State Transition::controller 395Software Progress Marker:progress:controller 396.TE 397.Pp 398For an explanation of the individual features refer to the description 399of the 400.Qq SET FEATURES 401admin command in the NVMe specification. 402.It Xo 403.Nm 404.Cm format 405.Ar ctl[/ns] 406.Op Ar lba-format 407.Xc 408Formats the specified namespace or all namespaces of the specified 409controller. 410This command implies a 411.Nm 412.Cm detach 413and subsequent 414.Nm 415.Cm attach 416of the specified namespace(s), which will cause a changed LBA format 417to be detected. 418If no LBA format is specified the LBA format currently used by the 419namespace will be used. 420When formatting all namespaces without specifying a LBA format the LBA 421format of namespace 1 will be used. 422A list of LBA formats supported by a namespace can be queried with the 423.Nm 424.Cm identify 425command. 426.Pp 427Note that not all devices support formatting individual or all 428namespaces, or support formatting at all. 429.Pp 430LBA formats using a non-zero metadata size are not supported by 431.Nm 432or 433.Xr nvme 4D . 434.Pp 435The list of supported LBA formats on a namespace can be retrieved 436with the 437.Nm 438.Cm identify 439command. 440.It Xo 441.Nm 442.Cm secure-erase 443.Op Fl c 444.Ar ctl[/ns] 445.Xc 446Erases the specified namespace or all namespaces of the controller. 447The flag 448.Fl c 449will cause a cryptographic erase instead of a normal erase. 450This command implies a 451.Nm 452.Cm detach 453and 454.Nm 455.Cm attach 456of the specified namespace(s). 457.Pp 458Note that not all devices support erasing individual or all 459namespaces, or support erasing at all. 460.It Xo 461.Nm 462.Cm detach 463.Ar ctl[/ns] 464.Xc 465Temporarily detaches the 466.Xr blkdev 4D 467instance from the specified namespace or all namespaces of the controller. 468This will prevent I/O access to the affected namespace(s). 469Detach will only succeed if the affected namespace(s) are not 470currently opened. 471The detached state will not persist across reboots or reloads of the 472.Xr nvme 4D 473driver. 474.Pp 475It is not an error to detach a namespace that is already detached, any such 476request will be silently ignored. 477.It Xo 478.Nm 479.Cm attach 480.Ar ctl[/ns] 481.Xc 482Attaches the 483.Xr blkdev 4D 484instance to the specified namespace or all namespaces of the controller. 485This will make I/O accesses to the namespace(s) possible again after a 486previous 487.Nm 488.Cm detach 489command. 490.Pp 491It is not an error to attach a namespace that is already attached, any such 492request will be silently ignored. 493.It Xo 494.Nm 495.Cm list-firmware 496.Ar ctl 497.Xc 498List currently active firmware slot, the next active firmware slot, and the 499current contents of all firmware slots of an NVMe controller. 500This is a synonym for the 501.Nm 502.Cm get-logpage 503.Ar ctl 504.Cm firmware 505command. 506.It Xo 507.Nm 508.Cm load-firmware 509.Ar ctl 510.Ar firmware-file 511.Op Ar offset 512.Xc 513Loads 514.Ar firmware-file 515into the controller's upload memory at 516.Ar offset , 517the default is 0. A vendor may require multiple files to be loaded 518at different offsets before the firmware is committed to a 519.Ar slot . 520.It Xo 521.Nm 522.Cm commit-firmware 523.Ar ctl 524.Ar slot 525.Xc 526Commits firmware previously loaded by the 527.Cm load-firmware 528command to 529.Ar slot . 530.It Xo 531.Nm 532.Cm activate-firmware 533.Ar ctl 534.Ar slot 535.Xc 536Activates the firmware in slot 537.Ar slot . 538The firmware image in 539.Ar slot 540is activated at the next NVM controller reset. 541.El 542.Sh EXIT STATUS 543.Ex -std 544.Sh EXAMPLES 545.Bl -tag -width "" 546.It Sy Example 1: List all NVMe controllers and namespaces 547.Bd -literal 548# nvmeadm list 549nvme1: model: INTEL SSDPEDMD800G4, serial: CVFT4134001R800CGN, FW rev: 8DV10049, NVMe v1.0 550 nvme1/1 (c1t1d0): Size = 763097 MB, Capacity = 763097 MB, Used = 763097 MB 551nvme4: model: SAMSUNG MZVPV128HDGM-00000, serial: S1XVNYAGA00640, FW rev: BXW7300Q, NVMe v1.1 552 nvme4/1 (c2t2d0): Size = 122104 MB, Capacity = 122104 MB, Used = 5127 MB 553.Ed 554.It Sy Example 2: Identify a namespace 555.Bd -literal 556# nvmeadm identify nvme4/1 557nvme4/1: Identify Namespace 558 Namespace Capabilities and Features 559 Namespace Size: 122104MB 560 Namespace Capacity: 122104MB 561 Namespace Utilization: 5127MB 562 Namespace Features 563 Thin Provisioning: unsupported 564 Number of LBA Formats: 1 565 Formatted LBA Size 566 LBA Format: 1 567 Extended Data LBA: no 568 Metadata Capabilities 569 Extended Data LBA: unsupported 570 Separate Metadata: unsupported 571 End-to-End Data Protection Capabilities 572 Protection Information Type 1: unsupported 573 Protection Information Type 2: unsupported 574 Protection Information Type 3: unsupported 575 Protection Information first: unsupported 576 Protection Information last: unsupported 577 End-to-End Data Protection Settings 578 Protection Information: disabled 579 Protection Information in Metadata: last 8 bytes 580 LBA Format 1 581 Metadata Size: 0 bytes 582 LBA Data Size: 512 bytes 583 Relative Performance: Best 584.Ed 585.It Sy Example 3: Get SMART/Health information (verbose) 586.Bd -literal 587# nvmeadm -v get-logpage nvme4/1 health 588nvme4/1: SMART/Health Information 589 Critical Warnings 590 Available Space: OK 591 Temperature: OK 592 Device Reliability: OK 593 Media: OK 594 Volatile Memory Backup: OK 595 Temperature: 37C 596 Available Spare Capacity: 100% 597 Available Spare Threshold: 10% 598 Device Life Used: 0% 599 Data Read: 0GB 600 Data Written: 64GB 601 Read Commands: 52907 602 Write Commands: 567874 603 Controller Busy: 1min 604 Power Cycles: 6 605 Power On: 141h 606 Unsafe Shutdowns: 1 607 Uncorrectable Media Errors: 0 608 Errors Logged: 1 609.Ed 610.It Sy Example 4: Get Asynchronous Event Configuration information 611.Bd -literal 612# nvmeadm get-features nvme0,nvme4 event,power 613nvme0: Get Features 614 Asynchronous Event Configuration 615 Available Space below threshold: disabled 616 Temperature above threshold: disabled 617 Device Reliability compromised: disabled 618 Media read-only: disabled 619 Power Management 620 Power State: 0 621nvme4: Get Features 622 Asynchronous Event Configuration 623 Available Space below threshold: disabled 624 Temperature above threshold: disabled 625 Device Reliability compromised: disabled 626 Media read-only: disabled 627 Volatile Memory Backup failed: disabled 628 Power Management 629 Power State: 0 630.Ed 631.It Sy Example 5: Load and activate firmware 632.Bd -literal 633# nvmeadm list-firmware nvme3 634nvme3: Firmware Slot Information 635 Active Firmware Slot: 4 636 Next Firmware Slot: 4 637 Firmware Revision for Slot 1: KNGND110 (read-only) 638 Firmware Revision for Slot 2: KNGND110 639 Firmware Revision for Slot 3: KNGND110 640 Firmware Revision for Slot 4: KNGND112 641 Firmware Revision for Slot 5: KNGND110 642 643# nvmeadm -v load-firmware nvme3 KNGND113.bin 6441740544 bytes downloaded. 645 646# nvmeadm -v commit-firmware nvme3 5 647Firmware committed to slot 5. 648 649# nvmeadm -v activate-firmware nvme3 5 650Slot 5 activated: NVM subsystem reset required - power cycle your system. 651 652# nvmeadm list-firmware nvme3 653nvme3: Firmware Slot Information 654 Active Firmware Slot: 4 655 Next Firmware Slot: 5 656 Firmware Revision for Slot 1: KNGND110 (read-only) 657 Firmware Revision for Slot 2: KNGND110 658 Firmware Revision for Slot 3: KNGND110 659 Firmware Revision for Slot 4: KNGND112 660 Firmware Revision for Slot 5: KNGND113 661.Ed 662.El 663.Sh INTERFACE STABILITY 664The command line interface of 665.Nm 666is 667.Sy Evolving . 668The output of 669.Nm 670is 671.Sy Not-an-Interface 672and may change any time. 673.Sh SEE ALSO 674.Xr nvme 4D 675.Pp 676.Lk http://www.nvmexpress.org/specifications/ "NVMe specifications" 677