Allow partial MSI-x allocation in ENA driver

The situation, where part of the MSI-x was not configured properly, was
not properly handled. Now, the driver reduces number of queues to
reflect number

Allow partial MSI-x allocation in ENA driver

The situation, where part of the MSI-x was not configured properly, was
not properly handled. Now, the driver reduces number of queues to
reflect number of existing and properly configured MSI-x vectors.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12863

show more ...

Remove deprecated and unused counters in ENA driver

Few counters were imported from the Linux driver and never used,
because of differences between the Linux and FreeBSD APIs.

Queue stops and resum

Remove deprecated and unused counters in ENA driver

Few counters were imported from the Linux driver and never used,
because of differences between the Linux and FreeBSD APIs.

Queue stops and resumes are no longer supported by the driver and
counters were incremented indicating false events.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: rlibby
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12862

show more ...

Cover ENA driver code with branch predictioning statements

The driver was using it in only few places, so the rest of the code
was covered with those statement.

Submitted by: Michal Krawczyk <mk@se

Cover ENA driver code with branch predictioning statements

The driver was using it in only few places, so the rest of the code
was covered with those statement.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: rlibby
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12861

show more ...

Refactor style of the ENA driver

* Change all conditional checks in "if" statement to boolean expressions
* Initialize variables with too complex values outside the declaration
* Fix indentations
*

Refactor style of the ENA driver

* Change all conditional checks in "if" statement to boolean expressions
* Initialize variables with too complex values outside the declaration
* Fix indentations
* Move code associated with sysctls to ena_sysctl.c file
* For consistency, remove unnecesary "return" from void functions
* Use if_getdrvflags() function instead of accesing variable directly

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12860

show more ...

Fix error handling in the ENA driver and lock drbr_free() call

Some goto tags were renamed for consistency, and few error handling
routines were reworked.

The drbr_free() must be locked just in cas

Fix error handling in the ENA driver and lock drbr_free() call

Some goto tags were renamed for consistency, and few error handling
routines were reworked.

The drbr_free() must be locked just in case code will change in the
future - for now, it should never be an issue, because drbr is being
flushed in the ena_down() call, and the lock is required only when there
are some mbufs inside.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12859

show more ...

Destroy admin queue after freeing interrupts in ENA driver

On heavy load, when interrupt handling routine was slowed down, there
could appear memory corruption, because resources were destroyed and

Destroy admin queue after freeing interrupts in ENA driver

On heavy load, when interrupt handling routine was slowed down, there
could appear memory corruption, because resources were destroyed and
interrupt was still being handled.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12858

show more ...

Split function checking for missing TX completion in ENA driver

Pure cosmetic change for better readability of the driver.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmai

Split function checking for missing TX completion in ENA driver

Pure cosmetic change for better readability of the driver.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12857

show more ...

Check for Rx ring state to prevent from stall in the ENA driver

In case when Rx ring is full and driver will fail to allocate Rx mbufs,
the ring could be stalled.

Keep alive is checking every secon

Check for Rx ring state to prevent from stall in the ENA driver

In case when Rx ring is full and driver will fail to allocate Rx mbufs,
the ring could be stalled.

Keep alive is checking every second for Rx ring state, and if it is full
for two cycles, then trigger rx_cleanup routine in another thread.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12856

show more ...

Add RX OOO completion feature

The RX out of order completion feature, allows to complete RX
descriptors out of order, by keeping trace of all free descriptors in
the separate array.

Submitted by: M

Add RX OOO completion feature

The RX out of order completion feature, allows to complete RX
descriptors out of order, by keeping trace of all free descriptors in
the separate array.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12855

show more ...

Change function validate_tx_req_id() to inline in ENA driver

The function is in hot path of the driver (TX) and asking compiler for
making this function inline was changed for consistency and higher

Change function validate_tx_req_id() to inline in ENA driver

The function is in hot path of the driver (TX) and asking compiler for
making this function inline was changed for consistency and higher
readability.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: rlibby, byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12854

show more ...

Fix ENA driver error handling in attach and basic style fixes

The patch contains following changes:

* In conditional checks, always check for NULL or 0 instead of negating values
* Use malloc and f

Fix ENA driver error handling in attach and basic style fixes

The patch contains following changes:

* In conditional checks, always check for NULL or 0 instead of negating values
* Use malloc and free explicitely, instead of ENA_MEM_FREE and ENA_MEM_FREE (the
dmadev passed to macro is never used, and could be a little misleading)
* Always check for NULL after calling malloc (few checks were missing)
* Rework naming of the goto tags in ena_attach() for consistency
* Fix error handling in ena_attach() - few goto instructions were leading to the
wrong tag
* Destroy MMIO req read request if attach failed
* Remove checking for NULL after calling malloc with M_WAITOK flag

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon.com, Inc.
Differential Revision: https://reviews.freebsd.org/D12853

show more ...

Merge ^/head r323559 through r325504.

MFhead@r325383

Rework counting of hardware statistics in ENA driver

Do not read all statistics from the device, instead count them in the
driver except from RX drops - they are received directly from the NIC
in th

Rework counting of hardware statistics in ENA driver

Do not read all statistics from the device, instead count them in the
driver except from RX drops - they are received directly from the NIC
in the AENQ descriptor.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: imp
Obtained from: Semihalf
Sponsored by: Amazon.com, Inc.
Differential Revision: https://reviews.freebsd.org/D12852

show more ...

Update ena-com HAL to v1.1.4.3 and update driver accordingly

The newest ena-com HAL supports LLQv2 and introduces
API changes. In order not to break the driver compilation
it was updated/fixed in a

Update ena-com HAL to v1.1.4.3 and update driver accordingly

The newest ena-com HAL supports LLQv2 and introduces
API changes. In order not to break the driver compilation
it was updated/fixed in a following way:

* Change version of the driver to 0.8.0
* Provide reset cause when triggering reset of the device
* Reset device after attach fails
* In the reset task free management irq after calling ena_down. Admin
queue can still be used before ena_down is called, or when it is
being handled
* Do not reset device if ena_reset_task fails
* Move call of the ena_com_dev_reset to the ena_down() routine - it
should be called only if interface was up
* Use different function for checking empty space on the sq ring
(ena-com API change)
* Fix typo on ENA_TX_CLEANUP_THRESHOLD
* Change checking for EPERM with EOPNOTSUPP - change in the ena-com API
* Minor style fixes

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Amazon.com, Inc.
Semihalf
Sponsored by: Amazon.com, Inc.
Differential Revision: https://reviews.freebsd.org/D12143

show more ...

MFhead@r320180

Merge ^/head r319973 through 321382.

Merge ^/head r320573 through r320970.

Fix error check for Rx mbuf allocation in ENA driver

ena_alloc_rx_mbuf() will return positive error code
on failure. Act accordingly.

Submitted by: Krishna Yenduri <kyenduri at brkt.com>

Replace mbuf defragmentation with collapse

Collapse should be more effective than defragmentation.
Added missing declaration of ena_check_and_collapse_mbuf().

Submitted by: Michal Krawczyk <mk@se

Replace mbuf defragmentation with collapse

Collapse should be more effective than defragmentation.
Added missing declaration of ena_check_and_collapse_mbuf().

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon.com Inc.

show more ...

Fix creation of dma tags and TSO settings

TSO settings were not reflecting real HW capabilities.

DMA tags were created with wrong window - high address was the same as
low, so excluding window was

Fix creation of dma tags and TSO settings

TSO settings were not reflecting real HW capabilities.

DMA tags were created with wrong window - high address was the same as
low, so excluding window was not working.

Capabilities of TX dma transaction were not set properly - TSO max size
had been increased and size of one segment had been adjusted.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon.com Inc.

show more ...

Remove RX mtx from ENA driver

RX lock is no longer required. There can only be one RX cleanup task
running at a time, RX cleanup cannot be executed if interface is not
yet initialized and ena_down()

Remove RX mtx from ENA driver

RX lock is no longer required. There can only be one RX cleanup task
running at a time, RX cleanup cannot be executed if interface is not
yet initialized and ena_down() will not free any RX resources if any io
interrupt is being handled - RX cleanup task is only called from an
interrupt handler.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon.com Inc.

show more ...

Call drbr_advance() before leaving TX routine

If drbr_advance() is not called before doing cleanup and packet is
already enqueued for sending (tx_info is holding pointer to mbuf), then
mbuf is clean

Call drbr_advance() before leaving TX routine

If drbr_advance() is not called before doing cleanup and packet is
already enqueued for sending (tx_info is holding pointer to mbuf), then
mbuf is cleaned both in drbr_flush() and in cleanup routine, when all
mbufs hold by tx_buffer_info are being released.

This causes panic, because mbuf is released twice.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon.com Inc.

show more ...

Unmask all IO irqs after driver state is set as running

If driver left MSI-x handlling routine because interface was put down,
it is not unmasking IRQs, so any requesting interrupt will be awaiting

Unmask all IO irqs after driver state is set as running

If driver left MSI-x handlling routine because interface was put down,
it is not unmasking IRQs, so any requesting interrupt will be awaiting
for unmasking.

On ena_up() routine all interrupts are being unmasked and any awaiting
interrupt will be handled right away.

If handler was executed before driver state was set as running, handling
routine is being ended immediately, leaving IO irqs for given queue
masked.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon.com Inc.

show more ...

Acquire locks before calling drbr_flush()

It is required to hold lock that is associated with buffer ring before
flushing drbr.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Sem

Acquire locks before calling drbr_flush()

It is required to hold lock that is associated with buffer ring before
flushing drbr.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon.com Inc.

show more ...