#
106329ef |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: Add AER support to NVMe emulation
This adds support to bhyve's NVMe device emulation for processing Async Event Requests but not returning them (i.e. Async Event Notifications).
Fixes UNH Te
bhyve: Add AER support to NVMe emulation
This adds support to bhyve's NVMe device emulation for processing Async Event Requests but not returning them (i.e. Async Event Notifications).
Fixes UNH Test 5.5.2
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24896
show more ...
|
#
8bba8666 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: validate the NVMe LBA start and count
Add checks that the combination of Starting LBA and Number of Logical Blocks in a command will not exceed the range of the underlying storage.
Note that
bhyve: validate the NVMe LBA start and count
Add checks that the combination of Starting LBA and Number of Logical Blocks in a command will not exceed the range of the underlying storage.
Note that because NVMe specifices the Starting LBA as a uint64_t, care must be taken when converting it and the block count to avoid an integer overflow.
Fixes UNH Tests 2.2.3, 2.3.2, and 2.4.2
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24895
show more ...
|
#
7d248cff |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: implement NVMe SMART data I/O statistics
SMART data in NVMe includes statistics for number of read and write commands issued as well as the number of "data units" read and written. NVMe defin
bhyve: implement NVMe SMART data I/O statistics
SMART data in NVMe includes statistics for number of read and write commands issued as well as the number of "data units" read and written. NVMe defines "data unit" as thousands of 512 byte blocks (e.g. 1 data unit is 1-1,000 512 byte blocks, 3 data units are 2,001-3,000 512 byte blocks).
This patch implements counters for: - Data Units Read - Data Units Written - Host Read Commands - Host Write Commands and exposes the values when the guest reads the SMART/Health Log Page.
Fixes UNH Test 1.3.8
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24894
show more ...
|
#
ae638f2b |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: validate NVMe deallocate range values
For NVMe emulation, validate the Data Set Management LBA ranges do not exceed the capacity of the backing storage. If they do, return an "LBA Out of Rang
bhyve: validate NVMe deallocate range values
For NVMe emulation, validate the Data Set Management LBA ranges do not exceed the capacity of the backing storage. If they do, return an "LBA Out of Range" error.
Fixes UNH Test 2.2.3
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24893
show more ...
|
#
73cd73c0 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: base pci_nvme_ioreq size on advertised MDTS
NVMe controllers advertise their Max Data Transfer Size (MDTS) to limit the number of page descriptors in an I/O request. Take advantage of this an
bhyve: base pci_nvme_ioreq size on advertised MDTS
NVMe controllers advertise their Max Data Transfer Size (MDTS) to limit the number of page descriptors in an I/O request. Take advantage of this and size the struct pci_nvme_ioreq accordingly.
Ensuring these values match both future-proofs the code and allows removing some complexity which only exists to handle this possibility.
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24891
show more ...
|
#
206edceb |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: refactor NVMe I/O read/write
Split the NVM I/O function (i.e. nvme_opc_write_read) into separate functions - one for RAM based backing-store and another for disk based backing-store for easie
bhyve: refactor NVMe I/O read/write
Split the NVM I/O function (i.e. nvme_opc_write_read) into separate functions - one for RAM based backing-store and another for disk based backing-store for easier maintenance. No functional changes.
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24890
show more ...
|
#
a0900f46 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: implement NVMe Format NVM command
The Format NVM command mainly allows the host to specify the block size and protection information used for the Namespace. As the bhyve implementation simply
bhyve: implement NVMe Format NVM command
The Format NVM command mainly allows the host to specify the block size and protection information used for the Namespace. As the bhyve implementation simply maps the capabilities of the backing storage through to the guest, there isn't anything to implement. But a side effect of the format is the NVMe Controller shall not return any data previously written (i.e. erase previously written data). This patch implements this later behavior to provide a compliant implementation.
Fixes UNH Test 1.6
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24889
show more ...
|
#
45cf8268 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: make unsupported NVMe commands a debug message
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24888
|
#
e3ebd421 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: add more compliant NVMe Get/Set Features
Create a generic Get/Set Features by saving off the contents of CDW11 from the Set command and returning the saved value in the completion of the Get
bhyve: add more compliant NVMe Get/Set Features
Create a generic Get/Set Features by saving off the contents of CDW11 from the Set command and returning the saved value in the completion of the Get command. Implementation allows providing optional implementation for both Set and Get.
Add infrastructure to determine which feature ID's are namespace specific and flag violations of this category of error.
Also adds the feature specific behavior of Set Features, Number of Queues to only allow this command once per Controller reset.
Fixes UNH Tests 1.2, 5.4, and 5.5.6
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24887
show more ...
|
#
d708ced6 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: fix NVMe queue creation and deletion
Add checks for various types of invalid I/O Queue Create and Delete command parameters, including: - QID=0 - QID>MAX - QID already in use - Delete an
bhyve: fix NVMe queue creation and deletion
Add checks for various types of invalid I/O Queue Create and Delete command parameters, including: - QID=0 - QID>MAX - QID already in use - Delete an Active CQ - Invalid QSIZE - Invalid CQID (SQ creation) - Invalid interrupt vector (CQ creation)
Fixes UNH Tests 1.4.2-5,7-8
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24886
show more ...
|
#
f6f02911 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: fix NVMe Get Log Page command
Fix the logic in nvme_opc_get_log_page to calculate the number of DWORDS (uint32_t) instead of WORDS (uint16_t) for the byte length. And only return the allowed
bhyve: fix NVMe Get Log Page command
Fix the logic in nvme_opc_get_log_page to calculate the number of DWORDS (uint32_t) instead of WORDS (uint16_t) for the byte length. And only return the allowed number of Log Page bytes as determined by the user request and actual size of the requested log page.
Fixes UNH Test 1.3
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24885
show more ...
|
#
f8fa7467 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: implement NVMe Namespace Identification Descriptor
NVMe 1.3 compliant controllers must implement the Namespace Identification Descriptor structure (i.e. CNS=3). Previously this was unimplemen
bhyve: implement NVMe Namespace Identification Descriptor
NVMe 1.3 compliant controllers must implement the Namespace Identification Descriptor structure (i.e. CNS=3). Previously this was unimplemented.
Fixes UNH Test 1.1.4-0
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24884
show more ...
|
#
064ca48f |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: Consolidate NVMe CQ update
Consolidate the code which writes Completion Queue entries and updates the CQ doorbell value. While in the neighborhood, convert the "toggle CQ phase bit" code to u
bhyve: Consolidate NVMe CQ update
Consolidate the code which writes Completion Queue entries and updates the CQ doorbell value. While in the neighborhood, convert the "toggle CQ phase bit" code to use an XOR operation instead of an "if/else" branch.
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24882
show more ...
|
#
d7e180fe |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: add locks around NVMe queue accesses
The NVMe code attempted to ensure thread safety through a combination of using atomics and a "busy" flag. But this approach leads to unavoidable race cond
bhyve: add locks around NVMe queue accesses
The NVMe code attempted to ensure thread safety through a combination of using atomics and a "busy" flag. But this approach leads to unavoidable race conditions.
Fix is to use per-queue mutex locks to ensure thread safety within the queue processing code. While in the neighborhood, move all the queue initialization code to a common function.
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D19841
show more ...
|
#
cf20131a |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: add a comment explaining NVME dsm option
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24881
|
#
9963f180 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: implement NVMe Flush command
This adds support for the NVMe I/O command Flush. For block-based devices, submit a DIOCGFLUSH to the backing storage. Otherwise, command is treated like a NOP an
bhyve: implement NVMe Flush command
This adds support for the NVMe I/O command Flush. For block-based devices, submit a DIOCGFLUSH to the backing storage. Otherwise, command is treated like a NOP and completes with a Successful status.
Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24880
show more ...
|
#
a43ab8d2 |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: refactor NVMe IO command handling
This refactors the NVMe I/O command processing function to make adding new commands easier. The main change is to move command specific processing (i.e. Read
bhyve: refactor NVMe IO command handling
This refactors the NVMe I/O command processing function to make adding new commands easier. The main change is to move command specific processing (i.e. Read/Write) to separate functions for each NVMe I/O command and leave the common per-command processing in the existing pci_nvme_handle_io_cmd() function.
While here, add checks for some common errors (invalid Namespace ID, invalid opcode, LBA out of range).
Add myself to the Copyright holders
Reviewed by: imp Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24879
show more ...
|
#
0220a2ae |
| 29-Jun-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: convert NVMe logging statements
Convert the debug and warning logging macros to be parameterized and correctly use bhyve's PRINTLN macro.
Reviewed by: imp Tested by: Jason Tubnor MFC after:
bhyve: convert NVMe logging statements
Convert the debug and warning logging macros to be parameterized and correctly use bhyve's PRINTLN macro.
Reviewed by: imp Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24878
show more ...
|
#
1264a2b9 |
| 27-Mar-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: fix NVMe emulation update of SQHD
The SQHD field of a Completion Queue entry indicates the current Submission Queue head pointer value. The head pointer represents the next entry to be consum
bhyve: fix NVMe emulation update of SQHD
The SQHD field of a Completion Queue entry indicates the current Submission Queue head pointer value. The head pointer represents the next entry to be consumed and is updated after consuming the current entry.
In the Admin queue processing, the current code updates the head pointer after reporting the value to the host via the SQHD. This gives the impression that the Controller is perpetually one command behind in its processing of the Admin SQ. And while this doesn't appear to bother some initiators, it is wrong.
Fix is to update the SQ head pointer prior to writing the SQHD value in the completion.
While here, fix missed update of dword 0 (cdw0) in the completion message.
Reported by: khng300 Reviewed by: jhb, imp Approved by: jhb (maintainer) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24083
show more ...
|
#
961be12f |
| 27-Mar-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: fix NVMe emulation missed interrupts
The bhyve NVMe emulation has a race in the logic which generates command completion interrupts. On FreeBSD guests, this manifests as kernel log messages s
bhyve: fix NVMe emulation missed interrupts
The bhyve NVMe emulation has a race in the logic which generates command completion interrupts. On FreeBSD guests, this manifests as kernel log messages similar to: nvme0: Missing interrupt
The NVMe emulation code sets a per-submission queue "busy" flag while processing the submission queue, and only generates an interrupt when the submission queue is not busy.
Aside from being counter to the NVMe design (i.e. interrupt properties are tied to the completion queue) and adding complexity (e.g. exceptions to not generating an interrupt when "busy"), it causes a race condition under the following conditions: - guest OS has no outstanding interrupts - guest OS submits a single NVMe IO command - bhyve emulation processes the SQ and sets the "busy" flag - bhyve emulation submits the asynchronous IO to the backing storage - IO request to the backing storage completes before the SQ processing loop exits and doesn't generate an interrupt because the SQ is "busy" - bhyve emulation finishes processing the SQ and clears the "busy" flag
Fix is to remove the "busy" flag and generate an interrupt when the CQ head and tail pointers do not match.
Reported by: khng300 Reviewed by: jhb, imp Approved by: jhb (maintainer) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24082
show more ...
|
#
f3e46ff9 |
| 27-Mar-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: use STAILQ in NVMe emulation
Use the standard queue(3) macros instead of hand-crafted linked list code.
Reviewed by: imp, jhb Approved by: jhb (maintainer) MFC after: 2 weeks Differential Re
bhyve: use STAILQ in NVMe emulation
Use the standard queue(3) macros instead of hand-crafted linked list code.
Reviewed by: imp, jhb Approved by: jhb (maintainer) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24081
show more ...
|
#
cd65e089 |
| 27-Mar-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: implement NVMe deallocate command
This adds support for the Dataset Management (DSM) command to the NVMe emulation in general, and more specifically, for the deallocate attribute (a.k.a. trim
bhyve: implement NVMe deallocate command
This adds support for the Dataset Management (DSM) command to the NVMe emulation in general, and more specifically, for the deallocate attribute (a.k.a. trim in the ATA protocol). If the backing storage for the namespace supports delete (i.e. deallocate), setting the deallocate attribute in a DSM will trim/delete the requested LBA ranges in the underlying storage.
Reviewed by: jhb, araujo, imp Approved by: jhb (maintainer) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D21839
show more ...
|
#
d31d525e |
| 27-Mar-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: refactor NVMe namespace initialization
Pass the struct pci_nvme_blockstore pointer for this namespace to the namespace initialization function instead of only the desired eui64 value.
Minor
bhyve: refactor NVMe namespace initialization
Pass the struct pci_nvme_blockstore pointer for this namespace to the namespace initialization function instead of only the desired eui64 value.
Minor functional change in that the code updates the eui64 value in the blockstore.
Reviewed by: jhb, araujo Approved by: jhb (maintainer) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D21838
show more ...
|
#
da8de3e9 |
| 27-Mar-2020 |
Chuck Tuffli <chuck@FreeBSD.org> |
bhyve: refactor NVMe PRP memcpy
Add a "copy direction" parameter to nvme_prp_memcpy such that data can be copied to the memory specified by the PRP entries (current behavior) or copied from the PRP
bhyve: refactor NVMe PRP memcpy
Add a "copy direction" parameter to nvme_prp_memcpy such that data can be copied to the memory specified by the PRP entries (current behavior) or copied from the PRP entries (new behavior). The upcoming deallocate functionality will use the copy from capability.
Reviewed by: jhb, araujo Approved by: jhb (maintainer) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D21837
show more ...
|
#
332eff95 |
| 08-Jan-2020 |
Vincenzo Maffione <vmaffione@FreeBSD.org> |
bhyve: add wrapper for debug printf statements
Add printf() wrapper to use CR/CRLF terminators depending on whether stdio is mapped to a tty open in raw mode. Try to use the wrapper everywhere. For
bhyve: add wrapper for debug printf statements
Add printf() wrapper to use CR/CRLF terminators depending on whether stdio is mapped to a tty open in raw mode. Try to use the wrapper everywhere. For now we leave the custom DPRINTF/WPRINTF defined by device models, but we may remove them in the future.
Reviewed by: grehan, jhb MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D22657
show more ...
|