cxgbe(4): Allow t4_tom to be unloaded safely.

* Disable IFCAP_TOE automatically on all ifnets on all adapters during
unload. This is user-friendly and avoids panics due to stale ifnet
state aft

cxgbe(4): Allow t4_tom to be unloaded safely.

* Disable IFCAP_TOE automatically on all ifnets on all adapters during
unload. This is user-friendly and avoids panics due to stale ifnet
state after t4_tom is unloaded.
* Do not allow unload if tids are in use by the TOE on any adapter.

Reported by: Bimal Abraham @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Export the core suspend/resume functionality to other modules.

MFC after: 1 week
Sponsored by: Chelsio Communications

cxgbe(4): Stop work request queues in a reliable manner.

Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.

MFC after: 1 week
Sp

cxgbe(4): Stop work request queues in a reliable manner.

Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.

The suspend/resume/reset implementation in the base driver (LLD)
currently works when only stateless features are in use. Th

cxgbe(4): Basic infrastructure for ULDs to participate in adapter reset.

The suspend/resume/reset implementation in the base driver (LLD)
currently works when only stateless features are in use. This commit
adds basic infrastructure for stateful upper layer drivers (ULDs) to
participate in suspend/resume/reset.

* Add a uld_restart to indicate that the adapter has been restarted
after a stop and the ULD should resume operations.
* Move the existing functionality in t4_suspend/t4_resume to stop_lld and
restart_lld. Use these and the new uld restart routines everywhere the
adapter has to be stopped abruptly and restarted, namely:
1. PCIE bus suspend/resume/reset methods invoked by the kernel.
2. Manual internal-reset using driver sysctl.
3. Automatic internal-reset on a fatal error.
* Implement an alternate internal-reset for use in VMs and for testing.

Typical reset sequence is:
stop_adapter(sc);
stop_lld(sc);
stop_all_uld(sc);
set_adapter_hwstatus(sc, false);

/* hw reset takes place here. */

restart_adapter(sc);
restart_lld(sc);
set_adapter_hwstatus(sc, true);
restart_all_uld(sc);

MFC after: 1 month
Sponsored by: Chelsio Communications

show more ...

cxgbev(4): Pay attention to the VLAN configuration for the VF.

Make sure that the transmit traffic is tagged correctly or else the
firmware will refuse to transmit and will report an ACL violation.

cxgbev(4): Pay attention to the VLAN configuration for the VF.

Make sure that the transmit traffic is tagged correctly or else the
firmware will refuse to transmit and will report an ACL violation.

On receive the hardware will make sure that tagged traffic is delivered
to the appropriate VM. The driver only asserts that the VLAN id that
was extracted from the wire traffic matches the VF's configuration.

All this works when associating a specific VLAN id with a VF. The
'trunk' setting likely needs more work.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): New knob to limit driver to the specified types of doorbells.

hw.cxgbe.doorbells_allowed="0xf"

The adapter's doorbells bitmap is clipped to the value specified in the
tunable, which is me

cxgbe(4): New knob to limit driver to the specified types of doorbells.

hw.cxgbe.doorbells_allowed="0xf"

The adapter's doorbells bitmap is clipped to the value specified in the
tunable, which is meant for debug and workarounds only. There is no
change in default behavior.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Rename rx_c_chan to rx_chan.

It is the equivalent of tx_chan but for receive so rx_chan is a better
name. Initialize both using helper functions and make sure both are
displayed in the sy

cxgbe(4): Rename rx_c_chan to rx_chan.

It is the equivalent of tx_chan but for receive so rx_chan is a better
name. Initialize both using helper functions and make sure both are
displayed in the sysctl MIB.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Allocate a taskqueue per port instead of per channel.

All the channels are not used on all boards and there's no point
allocating taskqueues that will never be used.

MFC after: 1 week
Spo

cxgbe(4): Allocate a taskqueue per port instead of per channel.

All the channels are not used on all boards and there's no point
allocating taskqueues that will never be used.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe: Support TCP_USE_DDP on offloaded TOE connections

When this socket option is enabled, relatively large contiguous
buffers are allocated and used to receive data from the remote
connection. Wh

cxgbe: Support TCP_USE_DDP on offloaded TOE connections

When this socket option is enabled, relatively large contiguous
buffers are allocated and used to receive data from the remote
connection. When data is received a wrapper M_EXT mbuf is queued to
the socket's receive buffer. This reduces the length of the linked
list of received mbufs and allows consumers to consume receive data in
larger chunks.

To minimize reprogramming the page pods in the adapter, receive
buffers for a given connection are recycled. When a buffer has been
fully consumed by the receiver and freed, the buffer is placed on a
per-connection free buffers list.

The size of the receive buffers defaults to 256k and can be set via
the hw.cxgbe.toe.ddp_rcvbuf_len sysctl. The
hw.cxgbe.toe.ddp_rcvbuf_cache sysctl (defaults to 4) determines the
maximum number of free buffers cached per connection. Note that this
limit does not apply to "in-flight" receive buffers that are
associated with mbufs in the socket's receive buffer.

Co-authored-by: Navdeep Parhar <np@FreeBSD.org>
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44001

show more ...

cxgbe: Add counters for POSIX async I/O requests handled by the driver

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43668

cxgbe(4): Fix virtual interface reattach.

Replace the DOOMED flag with a transient DETACHING flag that is cleared
when VI is detached. This fixes VI reattach when only the VI and not
the parent nex

cxgbe(4): Fix virtual interface reattach.

Replace the DOOMED flag with a transient DETACHING flag that is cleared
when VI is detached. This fixes VI reattach when only the VI and not
the parent nexus is detached. The old flag was never cleared and
prevented subsequent synch op's related to the VI.

PR: 275260
Reviewed by: jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43287
Sponsored by: Chelsio Communications

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

Mechanically convert cxgb(4) and cxgbe(4) to IfAPI

Reviewed by: np
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38597

cxgbe: Queue NIC TLS mbufs internally.

Set internal mbuf properties and queue NIC TLS mbuf chains to the
connection's TX queue directly.

Reviewed by: np
Sponsored by: Chelsio Communications
Differe

cxgbe: Queue NIC TLS mbufs internally.

Set internal mbuf properties and queue NIC TLS mbuf chains to the
connection's TX queue directly.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D38579

show more ...

cxgbe: Move helper functions for mbuf metadata to adapter.h.

Previously private to t4_sge.c, this allows other parts of the driver
(such as NIC TLS) to use these helpers directly.

Reviewed by: np
S

cxgbe: Move helper functions for mbuf metadata to adapter.h.

Previously private to t4_sge.c, this allows other parts of the driver
(such as NIC TLS) to use these helpers directly.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D38578

show more ...

cxgbe: Allow parse_pkt to internally queue a packet.

If parse_pkt returns EINPROGRESS, return from cxgbe_transmit
without queueing the packet in a txq. Use this to move the call
to ethofld_transmit

cxgbe: Allow parse_pkt to internally queue a packet.

If parse_pkt returns EINPROGRESS, return from cxgbe_transmit
without queueing the packet in a txq. Use this to move the call
to ethofld_transmit for packet pacing into parse_pkt.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D38577

show more ...

ccr: Retire ccr_softc member in struct adapter.

Prior to Conrad's changes to replace session integer IDs with a
pointer to the driver-specific state in commit 1b0909d51a8aa, the
driver had to find t

ccr: Retire ccr_softc member in struct adapter.

Prior to Conrad's changes to replace session integer IDs with a
pointer to the driver-specific state in commit 1b0909d51a8aa, the
driver had to find the softc pointer from the adapter before it could
locate the ccr_session structure for a completed request. Since
Conrad's changes, the ccr_session pointer can now be obtained directly
from the crp. Add a backpoint from ccr_session back to ccr_softc and
use this in place of the ccr_softc member in cxgbe's struct adapter.

Sponsored by: Chelsio Communications

show more ...

cxgbe: Various whitespace fixes.

Mostly trailing whitespace and spaces before tabs.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37350

cxgbe: Use secq(9) to manage the timestamp generations.

This is mostly cosmetic, but it also doesn't leave a gap of time where
no structures are valid. Instead, we permit the ISR to continue to
use

cxgbe: Use secq(9) to manage the timestamp generations.

This is mostly cosmetic, but it also doesn't leave a gap of time where
no structures are valid. Instead, we permit the ISR to continue to
use the previous structure if the write to update cal_current isn't
yet visible.

Reviewed by: gallatin
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36669

show more ...

cxgbe: Compute timestamps via sbintime_t.

This uses fixed-point math already used elsewhere in the kernel for
sub-second time values. To avoid overflows this does require updating
the calibration o

cxgbe: Compute timestamps via sbintime_t.

This uses fixed-point math already used elsewhere in the kernel for
sub-second time values. To avoid overflows this does require updating
the calibration once a second rather than once every 30 seconds. Note
that the cxgbe driver already queries multiple registers once a second
for the statistics timers. This version also uses fewer instructions
with no branches (for the math portion) in the per-packet fast path.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36663

show more ...

Enable M_TSTMP in Chelsio cxgbe driver by creating a mechanism that can sync the time.

Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mb

Enable M_TSTMP in Chelsio cxgbe driver by creating a mechanism that can sync the time.

Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mbuf flags. This is because the timestamp
was just the free running 60bit clock. This change fixes that so that
we keep a synchronization by periodically (every 30 seconds after startup)
getting the timestamp and the current nanosecond time. We always keep
several sets around and the current one we always keep the current pair
and the previous pair of timestamps. This allows us to setup a ratio
between the two so we can correctly translate the time. Note that
we use special care to split the timestamp into seconds (per the clock tick)
and nanoseconds otherwise 64bit math would overflow.

Reviewed by: np
Sponsored by: Netflix Inc
Differential Revision: https://reviews.freebsd.org/D36315

show more ...

cxgbe(4): Add another setting to the knob that controls congestion.

hw.cxgbe.cong_drop=2 will generate backpressure *and* drop frames for
queues that are congested.

MFC after: 2 weeks
Sponsored by:

cxgbe(4): Add another setting to the knob that controls congestion.

hw.cxgbe.cong_drop=2 will generate backpressure *and* drop frames for
queues that are congested.

MFC after: 2 weeks
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Specify the ingress queue's type when creating it.

The firmware takes the type into account when setting up the PCIe
channel for the queue.

MFC after: 1 week
Sponsored by: Chelsio Communi

cxgbe(4): Specify the ingress queue's type when creating it.

The firmware takes the type into account when setting up the PCIe
channel for the queue.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe: don't support MJUMPAGESIZE != MCLBYTES

This could only happen on systems with PAGE_SIZE < 4K and FreeBSD
doesn't support such systems.

Reviewed by: np, imp, jhb
Sponsored by: DARPA, AFRL
Dif

cxgbe: don't support MJUMPAGESIZE != MCLBYTES

This could only happen on systems with PAGE_SIZE < 4K and FreeBSD
doesn't support such systems.

Reviewed by: np, imp, jhb
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D36321

show more ...