| 605e699c | 14-Jul-2026 |
Arthur Kiyanovski <akiyano@amazon.com> |
ena: Update driver version to v2.8.4
Bug Fixes:
* Fix false 'missing TX completions' warnings due to timestamp race
* Put taskqueues into correct NUMA domain if !RSS
Minor Changes:
* Batch RX stati
ena: Update driver version to v2.8.4
Bug Fixes:
* Fix false 'missing TX completions' warnings due to timestamp race
* Put taskqueues into correct NUMA domain if !RSS
Minor Changes:
* Batch RX statistics updates
* Swap RX/TX completions cleanup order
Submitted by: Arthur Kiyanovski <akiyano@amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58242
show more ...
|
| 74bcb115 | 21-Jun-2026 |
Gilad Ben Yakov <giladben@amazon.com> |
ena: Fix false 'missing TX completions' warnings due to timestamp race
Sporadic 'Found a Tx that wasn't completed on time' warnings appear
under sustained TX load, always reporting '1 msecs since la
ena: Fix false 'missing TX completions' warnings due to timestamp race
Sporadic 'Found a Tx that wasn't completed on time' warnings appear
under sustained TX load, always reporting '1 msecs since last cleanup'
despite the 5-second timeout threshold.
The per-packet TX timestamp uses struct bintime (128 bits: two 64-bit
fields sec and frac) which is read and written non-atomically. A race
exists between the missing TX completion check
(check_missing_comp_in_tx_queue reading the timestamp) and the TX
submit path or cleanup path writing it on another CPU. Since the two
fields are not updated atomically, the check can observe a partially
written timestamp - one field from the old value and one from the new.
This can produce a timestamp with {sec=0, frac=valid}, causing the
check to compute a time offset equal to system uptime and falsely
exceeding the 5-second timeout.
Confirmed by instrumentation showing all occurrences had sec=0 with
valid frac/mbuf, cleanup_running=0, and ticks==last_cleanup_ticks.
Replace struct bintime with sbintime_t (a single 64-bit value) for
tx_buf->timestamp. An aligned 64-bit store/load cannot be torn on
64-bit architectures. Additionally, snapshot the timestamp into a
local variable in the check path to prevent a read-then-read race
where the timestamp could be zeroed between the zero-check and the
offset calculation.
Testing:
On m6i.large (FreeBSD 15.0-RELEASE-p6 amd64, 2 IO queues), two
instances with MTU 1500. Ran iperf -P 20 -u -b 320kpps (CPU
saturated at ~7 Gbps aggregate).
Without the fix: 8 warnings in 6 hours (first at ~72 min).
With the fix: 0 warnings after 20+ hours under identical conditions.
Fixes: 9b8d05b8ac78 ("Add support for Amazon Elastic Network Adapter (ENA) NIC")
Submitted by: Gilad Ben Yakov <giladben@amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58241
show more ...
|
| 3ba01cb4 | 16-Apr-2026 |
David Arinzon <darinzon@amazon.com> |
ena: Batch RX statistics updates
Move per-packet counter_enter/counter_exit pairs out of the RX
processing loop and batch them into a single update after the
loop completes.
Previously, each receiv
ena: Batch RX statistics updates
Move per-packet counter_enter/counter_exit pairs out of the RX
processing loop and batch them into a single update after the
loop completes.
Previously, each received packet triggered two separate
counter_enter/counter_exit blocks -- one for bytes and one for
packet count. This commit accumulates totals in local variables
and updates all four counters (ring and hw stats for both packets
and bytes) in a single counter_enter/counter_exit block after the
loop.
Also move the stats update to after the refill and LRO flush
so that the error path (goto update_stats) and the normal path
converge at the same label, avoiding code duplication.
Submitted by: David Arinzon <darinzon@amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58240
show more ...
|
| af7911d3 | 25-Apr-2026 |
Arthur Kiyanovski <akiyano@amazon.com> |
ena: Update driver version to v2.8.3
Features:
* Report RX overrun errors via sysctl hw stats
Bug Fixes:
* Budget rx descriptors, not packets, to fix jumbo frame throughput
Minor Changes:
* pmap_c
ena: Update driver version to v2.8.3
Features:
* Report RX overrun errors via sysctl hw stats
Bug Fixes:
* Budget rx descriptors, not packets, to fix jumbo frame throughput
Minor Changes:
* pmap_change_attr void * API change for FreeBSD 16.0+
* Adjust ena_[rt]x_cleanup to return bool
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D56641
show more ...
|
| 0f7b8f79 | 17-Apr-2026 |
Colin Percival <cperciva@FreeBSD.org> |
ena: Budget rx descriptors, not packets
We had ENA_RX_BUDGET = 256 in order to allow up to 256 received
packets to be processed before we do other cleanups (handling tx
packets and, critically, refi
ena: Budget rx descriptors, not packets
We had ENA_RX_BUDGET = 256 in order to allow up to 256 received
packets to be processed before we do other cleanups (handling tx
packets and, critically, refilling the rx buffer ring). Since the
ring holds 1024 buffers by default, this was fine for normal packets:
We refill the ring when it falls below 7/8 full, and even with a large
burst of incoming packets allowing it to fall by another 1/4 before we
consider refilling the ring still leaves it at 7/8 - 1/4 = 5/8 full.
With jumbos, the story is different: A 9k jumbo (as is used by default
within the EC2 network) consumes 3 descriptors, so a single rx cleanup
pass can consume 3/4 of the default-sized rx ring; if the rx buffer
ring wasn't completely full before a packet burst arrives, this puts
us perilously close to running out of rx buffers.
This precise failure mode has been observed on some EC2 instance types
within a Cluster Placement Group, resulting in the nominal 10 Gbps
single-flow throughput between instances dropping to ~100 Mbps as a
result of repeated rx overruns causing packet loss and ultimately
retransmission timeouts.
To correct this, switch from processing up to ENA_RX_BUDGET (256)
packets to processing up to ENA_RX_DESC_BUDGET (256) descriptors (or
slightly more, if we hit the limit in the middle of a packet). This
ensures that, even with jumbos, we refill the ring before processing
most of a ring worth of descriptors, and returns the throughput to
expected levels.
Note that theoretically up to ENA_PKT_MAX_BUFS (19) descriptors can be
used for a single packet, in which case even 54 packets would exhaust
the default rx buffer ring; it's not clear if this ever occurs in
practice, but this fix will address that case as well.
Reviewed by: akiyano
Sponsored by: Amazon
MFC after: 6 days
Differential Revision: https://reviews.freebsd.org/D56479
show more ...
|
| 96c5eaf0 | 13-Feb-2026 |
Arthur Kiyanovski <akiyano@amazon.com> |
ena: Update driver version to v2.8.2
Bug Fixes:
* Verify that an ENA ring is in netmap only in native mode
Minor Changes:
* Move parenthesis to correct place in switch
* Add comment
* Reorder defin
ena: Update driver version to v2.8.2
Bug Fixes:
* Verify that an ENA ring is in netmap only in native mode
Minor Changes:
* Move parenthesis to correct place in switch
* Add comment
* Reorder define
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D55698
show more ...
|
| 97e84c58 | 05-Feb-2026 |
David Arinzon <darinzon@amazon.com> |
ena: Verify that an ENA ring is in netmap only in native mode
netmap operates in two modes:
1) Emulated - netmap handling is done by the network stack, the
NIC driver operates transparently to netma
ena: Verify that an ENA ring is in netmap only in native mode
netmap operates in two modes:
1) Emulated - netmap handling is done by the network stack, the
NIC driver operates transparently to netmap.
2) Native - netmap management is done by the NIC driver.
When checking whether a specific ENA ring is running in netmap
mode, only the following checks were done:
1. IFCAP_NETMAP - Check whether netmap capability is enabled on
the device.
2. NKR_NETMAP_ON - Check whether netmap is actively using this
ring.
The above checks implied that the netmap mode is native and the
ENA driver needs to handle the netmap logic.
The code was missing an explicit check on whether native mode
is actually on (NAF_NATIVE).
This led to a case where though emulated mode was used and
a netmap application was turned on, the ENA driver still managed
netmap logic partially and caused missing buffers and lack of
refill as part of the datapath.
Note: Enabling netmap emulated mode is insufficient and there's
a need to load a netmap program in order to trigger this use-case.
Add an explicit check of whether NAF_NATIVE mode is set.
The issue was reported in [1].
[1]: https://github.com/amzn/amzn-drivers/issues/361
Fixes: 358bcc4c6cde ("Add support for ENA NETMAP partial initialization")
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D55697
show more ...
|
| 59b30c1a | 25-Apr-2025 |
Arthur Kiyanovski <akiyano@amazon.com> |
ena: Bump driver version to v2.8.1
Changes since 2.8.0:
Bug Fixes:
* Fix LLQ normal width misconfiguration
* Check for errors when detaching children first, not last
Minor Changes:
* Remove \n fro
ena: Bump driver version to v2.8.1
Changes since 2.8.0:
Bug Fixes:
* Fix LLQ normal width misconfiguration
* Check for errors when detaching children first, not last
Minor Changes:
* Remove \n from sysctl description
Approved by: cperciva (mentor)
MFC after: 3 days
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D50041
show more ...
|
| ce4cc746 | 07-Aug-2024 |
osamaabb <osamaabb@amazon.com> |
ena: Update driver version to v2.8.0
Features:
* Add support for device request reset message over AENQ
* Support LLQ entry size recommendation from device
* Support max large LLQ depth from the dev
ena: Update driver version to v2.8.0
Features:
* Add support for device request reset message over AENQ
* Support LLQ entry size recommendation from device
* Support max large LLQ depth from the device
* Expand PHC infrastructures
* Configuration notification support
Bug Fixes:
* Fix leaking ifmedia resources on detach
* Fix netmap socket chain unmapping issue
* Properly reinit netmap structs upon sysctl changes
* Correctly count missing TX completions
Minor Changes:
* Add reset reason for corrupted TX/RX completion descriptors
* Add reset reason for missing admin interrupts
* Improve reset reason statistics
* Update licenses
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
show more ...
|
| 449496eb | 07-Aug-2024 |
Osama Abboud <osamaabb@amazon.com> |
ena: Fix leaking ifmedia resources on detach
ifmedia_add() allocates an ifmedia_entry during ena_attach.
Current code doesn't release this memory during ena_detach()
This commit calls ifmedia_remov
ena: Fix leaking ifmedia resources on detach
ifmedia_add() allocates an ifmedia_entry during ena_attach.
Current code doesn't release this memory during ena_detach()
This commit calls ifmedia_removeall() to properly free the
allocated memory during ena_detach().
Also, in case ena_attach fails, we need to detach ifmedia
which was allocated within ena_setup_ifnet().
This bug was first described in:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278100
Reviewed by: zlei
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
show more ...
|
| d0419551 | 07-Aug-2024 |
Osama Abboud <osamaabb@amazon.com> |
ena: Support max large LLQ depth from the device
Large LLQ depth size is currently calculated by dividing the maximum
possible size of LLQ by 2.
In newer paltforms, starting from r8g the size of BAR
ena: Support max large LLQ depth from the device
Large LLQ depth size is currently calculated by dividing the maximum
possible size of LLQ by 2.
In newer paltforms, starting from r8g the size of BAR2,
which contains LLQ, will be increased, and the maximum depth of
wide LLQ will be set according to a value set by the device, instead of
hardcoded division by 2.
The new value will be stored by the device in max_wide_llq_depth field
for drivers that expose ENA_ADMIN_LLQ_FEATURE_VERSION_1 or higher to
the device.
There is an assumption that max_llq_depth >= max_wide_llq_depth, since
they both use the same bar, and if it is possible to have a wide LLQ
of size max_wide_llq_depth, it is possible to have a normal LLQ of the
same size, since it will occupy half of the space.
Also moved the large LLQ case calculation of max_tx_queue_size
before its rounddown.
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
show more ...
|
| b1c38df0 | 07-Aug-2024 |
Osama Abboud <osamaabb@amazon.com> |
ena: Support LLQ entry size recommendation from device
This commit adds support for receiving LLQ entry size recommendation
from the device. The driver will use the recommended entry size, unless
th
ena: Support LLQ entry size recommendation from device
This commit adds support for receiving LLQ entry size recommendation
from the device. The driver will use the recommended entry size, unless
the user specifically chooses to use regular or large LLQ entry.
Also added enum ena_llq_header_size_policy_t and llq_plociy field in
order to support the new feature.
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
show more ...
|
| 70587942 | 07-Aug-2024 |
Osama Abboud <osamaabb@amazon.com> |
ena: Add support for device request reset message over AENQ
This commit adds a handler for the new aenq message
ENA_ADMIN_DEVICE_REQUEST_RESET,
which in turn causes the driver to trigger reset of a
ena: Add support for device request reset message over AENQ
This commit adds a handler for the new aenq message
ENA_ADMIN_DEVICE_REQUEST_RESET,
which in turn causes the driver to trigger reset of a new type:
ENA_REGS_RESET_DEVICE_REQUEST. Also adds counting of such occurrences in
a new statistic for it.
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
show more ...
|
| f9c9c01d | 07-Aug-2024 |
Osama Abboud <osamaabb@amazon.com> |
ena: Reinit netmap adapter struct upon sysctl changes
When attaching ENA driver, ena_netmap_attach() is invoked which, in turn
calls netmap_attach which, initializes a struct netmap_adapter,
allocat
ena: Reinit netmap adapter struct upon sysctl changes
When attaching ENA driver, ena_netmap_attach() is invoked which, in turn
calls netmap_attach which, initializes a struct netmap_adapter,
allocating the struct's netmap_ring and the struct selinfo.
When we change the interface number of queues we need to reinit the
netmap adapter struct as well, so we need to detach it in order to free
the memory allocated by netmap_attach and allocate new memory based on
the new parameters like number of rings, ring size etc...
Without detaching and attaching the netmap interface, if we're to change
the number of queues from 8 to 2 for example and try to enable netmap,
the kernel will panic since the original netmap struct within the
kernel's possession still thinks that the driver has 8 queues which will
eventually cause a non-allocated virtual address access fault.
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
show more ...
|
| 2f17afd1 | 07-Aug-2024 |
Osama Abboud <osamaabb@amazon.com> |
ena: Clear NS_MOREFRAG flag for last netmap slot
When processing packets within the rx-flow
ena_netmap_rx_load_desc doesn't know the number of descriptors, so it
sets NS_MOREFRAG to all the slots to
ena: Clear NS_MOREFRAG flag for last netmap slot
When processing packets within the rx-flow
ena_netmap_rx_load_desc doesn't know the number of descriptors, so it
sets NS_MOREFRAG to all the slots to indicate that there are more
fragments for this packet.
The code calls ena_netmap_rx_load_desc() for every descriptor in
this packet to map the relevant buffer into the netmap shared memory.
After ena_netmap_rx_load_desc() calls, we need to unset the NS_MOREFRAG
for the last fragment to indicate that this is the last fragment,
so we explicitly turn off NS_MOREFRAG flag.
Current code overrides all other flags and sets NS_BUF_CHANGED.
This patch unsets the relevant flag only.
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
show more ...
|