Check for errors when detaching children first, not last

These detach routines in these drivers all ended with 'return
(bus_generic_detach())' meaning that if any child device failed to
detach, the

Check for errors when detaching children first, not last

These detach routines in these drivers all ended with 'return
(bus_generic_detach())' meaning that if any child device failed to
detach, the parent driver was left in a mostly destroyed state, but
still marked attached. Instead, bus drivers should detach child
drivers first and return errors before destroying driver state in the
parent.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D47387

show more ...

ena: Fix leaking ifmedia resources on detach

ifmedia_add() allocates an ifmedia_entry during ena_attach.
Current code doesn't release this memory during ena_detach()

This commit calls ifmedia_remov

ena: Fix leaking ifmedia resources on detach

ifmedia_add() allocates an ifmedia_entry during ena_attach.
Current code doesn't release this memory during ena_detach()

This commit calls ifmedia_removeall() to properly free the
allocated memory during ena_detach().

Also, in case ena_attach fails, we need to detach ifmedia
which was allocated within ena_setup_ifnet().

This bug was first described in:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278100

Reviewed by: zlei
Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Support max large LLQ depth from the device

Large LLQ depth size is currently calculated by dividing the maximum
possible size of LLQ by 2.
In newer paltforms, starting from r8g the size of BAR

ena: Support max large LLQ depth from the device

Large LLQ depth size is currently calculated by dividing the maximum
possible size of LLQ by 2.
In newer paltforms, starting from r8g the size of BAR2,
which contains LLQ, will be increased, and the maximum depth of
wide LLQ will be set according to a value set by the device, instead of
hardcoded division by 2.

The new value will be stored by the device in max_wide_llq_depth field
for drivers that expose ENA_ADMIN_LLQ_FEATURE_VERSION_1 or higher to
the device.

There is an assumption that max_llq_depth >= max_wide_llq_depth, since
they both use the same bar, and if it is possible to have a wide LLQ
of size max_wide_llq_depth, it is possible to have a normal LLQ of the
same size, since it will occupy half of the space.

Also moved the large LLQ case calculation of max_tx_queue_size
before its rounddown.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Support LLQ entry size recommendation from device

This commit adds support for receiving LLQ entry size recommendation
from the device. The driver will use the recommended entry size, unless
th

ena: Support LLQ entry size recommendation from device

This commit adds support for receiving LLQ entry size recommendation
from the device. The driver will use the recommended entry size, unless
the user specifically chooses to use regular or large LLQ entry.

Also added enum ena_llq_header_size_policy_t and llq_plociy field in
order to support the new feature.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add support for device request reset message over AENQ

This commit adds a handler for the new aenq message
ENA_ADMIN_DEVICE_REQUEST_RESET,
which in turn causes the driver to trigger reset of a

ena: Add support for device request reset message over AENQ

This commit adds a handler for the new aenq message
ENA_ADMIN_DEVICE_REQUEST_RESET,
which in turn causes the driver to trigger reset of a new type:
ENA_REGS_RESET_DEVICE_REQUEST. Also adds counting of such occurrences in
a new statistic for it.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Reinit netmap adapter struct upon sysctl changes

When attaching ENA driver, ena_netmap_attach() is invoked which, in turn
calls netmap_attach which, initializes a struct netmap_adapter,
allocat

ena: Reinit netmap adapter struct upon sysctl changes

When attaching ENA driver, ena_netmap_attach() is invoked which, in turn
calls netmap_attach which, initializes a struct netmap_adapter,
allocating the struct's netmap_ring and the struct selinfo.

When we change the interface number of queues we need to reinit the
netmap adapter struct as well, so we need to detach it in order to free
the memory allocated by netmap_attach and allocate new memory based on
the new parameters like number of rings, ring size etc...

Without detaching and attaching the netmap interface, if we're to change
the number of queues from 8 to 2 for example and try to enable netmap,
the kernel will panic since the original netmap struct within the
kernel's possession still thinks that the driver has 8 queues which will
eventually cause a non-allocated virtual address access fault.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add differentiation for missing TX completions reset

This commit adds differentiation for a reset caused by missing tx
completions, by verifying if the driver didn't receive tx
completions caus

ena: Add differentiation for missing TX completions reset

This commit adds differentiation for a reset caused by missing tx
completions, by verifying if the driver didn't receive tx
completions caused by missing interrupts.
The cleanup_running field was added to ena_ring because
cleanup_task.ta_pending is zeroed before ena_cleanup() runs.

Also ena_increment_reset_counter() API was added in order to support
only incrementing the reset counter.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Set ena_min_poll_delay_us default value

This commit sets the default value for ena_min_poll_delay_us to 100.

This commit does not change the behavior of the driver, the delay is
calculated as

ena: Set ena_min_poll_delay_us default value

This commit sets the default value for ena_min_poll_delay_us to 100.

This commit does not change the behavior of the driver, the delay is
calculated as MAX(ENA_MIN_ADMIN_POLL_US, delay_us), where the first
field is already defined as 100.
The second parameter, delay_us is taken from ena_min_poll_delay_us
which is currently unset - 0.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add reset reason for missing admin interrupt

There can be cases when we trigger reset if an admin interrupt
is missing.
In order to identify this use-case specifically,
this commit adds a new r

ena: Add reset reason for missing admin interrupt

There can be cases when we trigger reset if an admin interrupt
is missing.
In order to identify this use-case specifically,
this commit adds a new reset reason.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add reset reason for corrupted TX cdescs

TX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit

ena: Add reset reason for corrupted TX cdescs

TX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit reset reason
ENA_REGS_RESET_TX_DESCRIPTOR_MALFORMED.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Improve reset reason statistics

The driver uses different reset reasons.
Some of them are counted and presented in the driver statistics.
There are cases where statistics are counted on a ring

ena: Improve reset reason statistics

The driver uses different reset reasons.
Some of them are counted and presented in the driver statistics.
There are cases where statistics are counted on a ring level,
but these are zeroed after a reset procedure takes place.

This commit makes the following changes:
1. Add statistics for the unrepresented reset reasons.
2. Add reset reasons which are counted on a ring level,
to be also global for better tracking.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Update license signatures to 2024

This commit updates all the license signatures to 2024.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: Add configuration notifications interface support

This commit is part of the effort of notifying the user of non-optimal
or performance impacting practices.
A new interface is serving as a comm

ena: Add configuration notifications interface support

This commit is part of the effort of notifying the user of non-optimal
or performance impacting practices.
A new interface is serving as a communication channel
between the device and the driver. One of the goals of this channel is
to create a new mechanism of notifying the driver and user in case of
sub-optimal configuration using a bitmap.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Count all currently missing TX completions in check

Currently we count all of the newly added and already existing
missing tx completions in each iteration of
check_missing_comp_in_tx_queue() c

ena: Count all currently missing TX completions in check

Currently we count all of the newly added and already existing
missing tx completions in each iteration of
check_missing_comp_in_tx_queue() causing duplicate counts
to missing_tx_comp stat.

This commit adds a new counter new_missed_tx within the relevant
function which only counts the newly added missing tx completions
in each iteration of check_missing_comp_in_tx_queue().
This will allow us to update missing_tx_comp stat accurately without
counting duplicates.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Fix customer metrics deallocation statement place

Upstream commit [1] made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().

Upstream comm

ena: Fix customer metrics deallocation statement place

Upstream commit [1] made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().

Upstream commit [2] removed the NULL check conducted by the driver.
This commit also removes err_customer_metrics_alloc goto label.

Commit [2] leaves behind a floating free() statement that
deallocates customer_metrics_array. This commit places the
deallocation statement where it belongs.

[1] commit 4787572d0580 ("ifnet: make if_alloc_domain() never fail")
[2] commit aa3860851b9f ("net: Remove unneeded NULL check for the allocated ifnet")

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

net: Remove unneeded NULL check for the allocated ifnet

Change 4787572d0580 made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().

No functiona

net: Remove unneeded NULL check for the allocated ifnet

Change 4787572d0580 made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().

No functional change intended.

Reviewed by: kp, imp, glebius, stevek
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D45740

show more ...

ena: Update the license dating to 2023

Some of the files are using outdated linceses.
Update the license to be 2023.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: Support srd metrics with sysctl

This commit introduces SRD metrics through sysctl.
The metrics can be queried using the following sysctl node:
sysctl dev.ena.<device index>.ena_srd_info

Approv

ena: Support srd metrics with sysctl

This commit introduces SRD metrics through sysctl.
The metrics can be queried using the following sysctl node:
sysctl dev.ena.<device index>.ena_srd_info

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Support customer metric with sysctl

This commit adds sysctl support for customer metrics.
Different customer metrics can be found in the following sysctl node:
sysctl dev.ena.<device index>.cus

ena: Support customer metric with sysctl

This commit adds sysctl support for customer metrics.
Different customer metrics can be found in the following sysctl node:
sysctl dev.ena.<device index>.customer_metrics

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Introduce shared sample interval for all stats

Rename sample_interval node to stats_sample_interval and move
it up in the sysctl tree to make it clear that it's relevant for
all the stats and n

ena: Introduce shared sample interval for all stats

Rename sample_interval node to stats_sample_interval and move
it up in the sysctl tree to make it clear that it's relevant for
all the stats and not only ENI metrics (Currently, sample interval node
is found under eni_metrics node).

Path to node:
dev.ena.<device_index>.stats_sample_interval

Once this parameter is set it will set the sample interval for all the
stats node including SRD/customer metrics.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Update ena_com_update_intr_reg API usage

This commit fixes the usage of this function to be compatible with the
new API introduced by ena-com update to v2.7.0

Approved by: cperciva (mentor)
MF

ena: Update ena_com_update_intr_reg API usage

This commit fixes the usage of this function to be compatible with the
new API introduced by ena-com update to v2.7.0

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Change measurement unit of time since last tx cleanup to ms

This commit:
1. Sets the time since last cleanup to milliseconds.
2. Fixes incorrect indentations.

Approved by: cperciva (mentor)
MF

ena: Change measurement unit of time since last tx cleanup to ms

This commit:
1. Sets the time since last cleanup to milliseconds.
2. Fixes incorrect indentations.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add sysctl support for spreading IRQs

This commit allows spreading IO IRQs over different CPUs through sysctl.
Two sysctl nodes are introduced:
1- base_cpu: servers as the first CPU to which th

ena: Add sysctl support for spreading IRQs

This commit allows spreading IO IRQs over different CPUs through sysctl.
Two sysctl nodes are introduced:
1- base_cpu: servers as the first CPU to which the first IO IRQ
will be bound.
2- cpu_stride: sets the distance between every two CPUs to which every
two consecutive IO IRQs are bound.

For example for doing the following IO IRQs / CPU binding:

IRQ idx | CPU
----------------
1 | 0
2 | 2
3 | 4
4 | 6

Run the following commands:
sysctl dev.ena.<device index>.irq_affinity.base_cpu=0
sysctl dev.ena.<device_index>.irq_affinity.cpu_stride=2

Also introduced rss_enabled field, which is intended to replace
'#ifdef RSS' in multiple places, in order to prevent code duplication.

We want to bind interrupts to CPUs in case of rss set OR in case
the newly defined sysctl paremeter is set. This requires to remove a
couple of '#ifdef RSS' as well in the structs, since we'll be using the
relevant parameters in the CPU binding code.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

ena: Fix driver unload crash

When ena_detach is called, we first call ether_ifdetach(),
which destroys internal addresses of ifp. One such address
is ifp->if_addr->ifa_addr. Then during ena_destroy_

ena: Fix driver unload crash

When ena_detach is called, we first call ether_ifdetach(),
which destroys internal addresses of ifp. One such address
is ifp->if_addr->ifa_addr. Then during ena_destroy_device(),
if_link_state_change() is called, eventually trying to access
ifp->if_addr->ifa_addr->sa_family. This causes an access
to garbage memory and crashes the kernel.

Ticket [1] was opened to the FreeBSD community to add null
check in the code of if_link_state_change().
A fix was submitted in commit [2], however it was noted
that it is our driver's responsibilty to not call
if_link_state_change() after calling ether_ifdetach().

This commit makes sure if_link_state_change() is not called
after ether_ifdetach().

[1]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270813
[2]: https://reviews.freebsd.org/D39614

Fixes: 32f63fa7f975 ("Split ENA reset routine into restore and destroy stages")
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...