Use bitfield for storing global ENA device states

As the ENA can have multiple states turned on/off, it is more convenient
to store them in single bitfield instead of multiple boolean variables.

Th

Use bitfield for storing global ENA device states

As the ENA can have multiple states turned on/off, it is more convenient
to store them in single bitfield instead of multiple boolean variables.

The bitset FreeBSD API was used for the bitfield implementation, as it
provides flexible structure together with API which also supports atomic
bitfield operations.

For better readability basic macros from API were wrapped into custom
ENA_FLAG_* macros, which are filling up common parameters for all calls.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Fix error handling when ENA reset fails

Before the patch, error handling was not releasing all resources and
was not issuing device reset if the reset task failed.

That could cause memory leak and

Fix error handling when ENA reset fails

Before the patch, error handling was not releasing all resources and
was not issuing device reset if the reset task failed.

That could cause memory leak and fault of the device.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Fill bdf field of the host_info structure in ENA

The host info bdf field is the abbreviation for the bus, device,
function of the PCI on which the device is being attached to.

Now the driver is fil

Fill bdf field of the host_info structure in ENA

The host info bdf field is the abbreviation for the bus, device,
function of the PCI on which the device is being attached to.

Now the driver is filling information about that using FreeBSD RID
resource.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Add additional doorbells on ENA Tx path

The new ENA HAL is introducing API, which can determine on Tx path if
the doorbell is needed.

That way, it can tell the driver, that it should call an doorbe

Add additional doorbells on ENA Tx path

The new ENA HAL is introducing API, which can determine on Tx path if
the doorbell is needed.

That way, it can tell the driver, that it should call an doorbell.
The old threshold value wasn't removed, as not all HW is supporting this
feature - so it was reworked to also work with the new API.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Limit maximum size of Rx refill threshold in ENA

The Rx ring size can be as high as 8k. Because of that we want to limit
the cleanup threshold by maximum value of 256.

Submitted by: Michal Krawczy

Limit maximum size of Rx refill threshold in ENA

The Rx ring size can be as high as 8k. Because of that we want to limit
the cleanup threshold by maximum value of 256.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Add support for the LLQv2 and WC in ENA

LLQ (Low Latency Queue) is the feature, that allows pushing header
directly to the device through PCI before even DMA is triggered.

It reduces latency, becau

Add support for the LLQv2 and WC in ENA

LLQ (Low Latency Queue) is the feature, that allows pushing header
directly to the device through PCI before even DMA is triggered.

It reduces latency, because device can start preparing packet before
payload is sent through DMA.

To speed up sending data through PCI, the Write Combining is enabled,
which allows hardware to buffer data before sending them on the PCI - it
allows to reduce number of PCI IO operations.

ENAv2 is using special descriptor for the negotiation of the LLQ.
Currently, only the default configuration is supported.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Lock optimization in ENA

Handle IO interrupts using filter routine. That way, the main cleanup
task could be moved to the separate thread using taskqueue.

The deferred Rx cleanup task was removed,

Lock optimization in ENA

Handle IO interrupts using filter routine. That way, the main cleanup
task could be moved to the separate thread using taskqueue.

The deferred Rx cleanup task was removed, and now the cleanup task is
begin called instead. That way, the Rx lock could be removed.

In addition, Queue management (wake up and stop TX ring) was added, so
the TX cleanup task can be performed mostly lockless.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Add tuneable drbr ring size and hw queues depth for ENA

The driver now supports per adapter tuning of buffer ring size and HW Rx
ring size.

It can be achieved using sysctl node dev.ena.X.

Submitte

Add tuneable drbr ring size and hw queues depth for ENA

The driver now supports per adapter tuning of buffer ring size and HW Rx
ring size.

It can be achieved using sysctl node dev.ena.X.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Fix error in validate_tx_req_id() in ENA

If the requested ID was out of range, the tx_info structure was NULL and
the function was trying to access the field of the NULL object.

Submitted by: Mich

Fix error in validate_tx_req_id() in ENA

If the requested ID was out of range, the tx_info structure was NULL and
the function was trying to access the field of the NULL object.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Change attach order to prevent crash upon failure in ENA

The if_detach was causing crash if the MSI-x configuration in the attach
failed. To prevent this issue, the ifnet is being configured at the

Change attach order to prevent crash upon failure in ENA

The if_detach was causing crash if the MSI-x configuration in the attach
failed. To prevent this issue, the ifnet is being configured at the end
of the attach function.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Change order of ifp release on ENA detach

In rare case, when the ifconfig is called just before kldunload, it is
possible, that ena_up routine will be called after queue locks are
released.

To prev

Change order of ifp release on ENA detach

In rare case, when the ifconfig is called just before kldunload, it is
possible, that ena_up routine will be called after queue locks are
released.

To prevent that, ifp is detached before the last ena_down is called and
further, the ifp is freed at the end of the function.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Check for number of MSI-x upon partial allocation in ENA

The ENA driver needs at least 2 MSI-x - one for admin queue, and one for
IO queues pair. If there were not enough resources to allocate more

Check for number of MSI-x upon partial allocation in ENA

The ENA driver needs at least 2 MSI-x - one for admin queue, and one for
IO queues pair. If there were not enough resources to allocate more than
one MSI-x, the device should not be attached.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Set error value when allocation of IO irq fails in ENA

bus_alloc_resource_any() is not returning error value in case of an
error.
If the function call fails, the error value was not passed to the
en

Set error value when allocation of IO irq fails in ENA

bus_alloc_resource_any() is not returning error value in case of an
error.
If the function call fails, the error value was not passed to the
ena_up() function.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Set vaddr and paddr as NULL when DMA alloc fails in ENA

To prevent errors from assigning values from the DMA structure in case
of an error, zero the vaddr and paddr values upon failure.

Submitted b

Set vaddr and paddr as NULL when DMA alloc fails in ENA

To prevent errors from assigning values from the DMA structure in case
of an error, zero the vaddr and paddr values upon failure.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Fix DMA synchronization in the ENA driver Tx and Rx paths

The DMA in FreeBSD requires explicit synchronization. ENA driver was
only doing PREREAD and PREWRITE synchronizations. Missing
bus_dmamap_sy

Fix DMA synchronization in the ENA driver Tx and Rx paths

The DMA in FreeBSD requires explicit synchronization. ENA driver was
only doing PREREAD and PREWRITE synchronizations. Missing
bus_dmamap_sync() calls were added.

It is also required to synchronize DMA engine before unloading DMA map.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Check for missing MSI-x and Tx completions in ENA

If the first MSI-x won't be executed, then the timer service will detect
that and trigger device reset.

The checking for missing Tx completion was

Check for missing MSI-x and Tx completions in ENA

If the first MSI-x won't be executed, then the timer service will detect
that and trigger device reset.

The checking for missing Tx completion was reworked, so it will also
check for missing interrupts. Checking number of missing Tx completions
can be performed after loop, instead of checking it every iteration.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Fill number of CPUs field on ENA host_info structure

The new ena_com allows the number of CPUs to be passed to the device in
the host info structure as a hint.

Submitted by: Michal Krawczyk <mk@se

Fill number of CPUs field on ENA host_info structure

The new ena_com allows the number of CPUs to be passed to the device in
the host info structure as a hint.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Print ENA Tx error conditionally

Information about Tx error should be only displayed, if packet
preparation failed due to error other than out of memory.

Submitted by: Michal Krawczyk <mk@semihalf

Print ENA Tx error conditionally

Information about Tx error should be only displayed, if packet
preparation failed due to error other than out of memory.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Trigger reset in ENA if there are too many Rx descriptors

Whenever the driver will receive too many descriptors from the device,
it should trigger the device reset, as it is indicating that the devi

Trigger reset in ENA if there are too many Rx descriptors

Whenever the driver will receive too many descriptors from the device,
it should trigger the device reset, as it is indicating that the device
is in invalid state.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Remove RSS support in ENA

Receive Side Scaling is optional feature that could be enabled in kernel
configuration by defining flag RSS.

Kernel uses hash to store and find protocol control block whic

Remove RSS support in ENA

Receive Side Scaling is optional feature that could be enabled in kernel
configuration by defining flag RSS.

Kernel uses hash to store and find protocol control block which is
stored in hash tables.
Kernel and NIC hash functions must be consistent. Otherwise case lookup
fails.

To achieve this kernel provides API to set proper hash key to NIC.
As it is not possible to change key for virtual ENA NIC, this driver
cannot support RSS function.

ENA is designed to work in virtual environments so supporting hardware
version of this card is unnecessary.

Submitted by: Rafal Kozik <rk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Add notification AENQ handler for ENA

Notification AENQ handler is responsible for handling requests from ENA
device. Missing Tx threshold, Tx timeout and keep alive timeout can be
set using hints f

Add notification AENQ handler for ENA

Notification AENQ handler is responsible for handling requests from ENA
device. Missing Tx threshold, Tx timeout and keep alive timeout can be
set using hints from the aenq descriptor which can be delivered in the
ENA admin notification.

The queue suspending and resuming tasks are not supported by the
driver.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Print information when ENA admin error occurs

ENA_ADMIN_FATAL_ERROR and ENA_ADMIN_WARNING aenq groups were indicated
as supported, so the unimplemented_aenq_handler() will print out error
message, w

Print information when ENA admin error occurs

ENA_ADMIN_FATAL_ERROR and ENA_ADMIN_WARNING aenq groups were indicated
as supported, so the unimplemented_aenq_handler() will print out error
message, whenever an error will occur within the ENA admin context.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Do not specify active media type in ENA

As the ENA is working only in virtualized environment, the active media
is not specified. Instead, the active link type is set as unknown.

Submitted by: Mic

Do not specify active media type in ENA

As the ENA is working only in virtualized environment, the active media
is not specified. Instead, the active link type is set as unknown.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Adjust ENA driver to the new ena-com

Recent HAL change preparing to support ENAv2 required minor driver
modifications.

The ena_com_sq_empty_space() is not available in this ena-com, so it had
to be

Adjust ENA driver to the new ena-com

Recent HAL change preparing to support ENAv2 required minor driver
modifications.

The ena_com_sq_empty_space() is not available in this ena-com, so it had
to be replaced with ena_com_free_desc().

Moreover, the ena_com_admin_init() is no longer using 3rd argument
indicating if the spin lock should be initialized, so it was removed.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

MFHead@r345677