cxgbe(4): Stop work request queues in a reliable manner.

Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.

MFC after: 1 week
Sp

cxgbe(4): Stop work request queues in a reliable manner.

Clear the EQ_HW_ALLOCATED flag with the wrq lock held and discard all
work requests, pending or new, when it's not set.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe: use order_base_2

Use order_base_2 in place of an expression.

Reviewed by: alc, markj
Differential Revision: https://reviews.freebsd.org/D45536

cxgbev(4): Pay attention to the VLAN configuration for the VF.

Make sure that the transmit traffic is tagged correctly or else the
firmware will refuse to transmit and will report an ACL violation.

cxgbev(4): Pay attention to the VLAN configuration for the VF.

Make sure that the transmit traffic is tagged correctly or else the
firmware will refuse to transmit and will report an ACL violation.

On receive the hardware will make sure that tagged traffic is delivered
to the appropriate VM. The driver only asserts that the VLAN id that
was extracted from the wire traffic matches the VF's configuration.

All this works when associating a specific VLAN id with a VF. The
'trunk' setting likely needs more work.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

powerof2: replace loops with fls or ilog2

In several places, a loop tests for powers of two, or iterates through
powers of two. In those places, replace the loop with an invocation
of fls or ilog2

powerof2: replace loops with fls or ilog2

In several places, a loop tests for powers of two, or iterates through
powers of two. In those places, replace the loop with an invocation
of fls or ilog2 without changing the meaning of the code.

Reviewed by: alc, markj, kib, np, erj, avg (previous version)
Differential Revision: https://reviews.freebsd.org/D45494

show more ...

cxgbe(4): Allocate a taskqueue per port instead of per channel.

All the channels are not used on all boards and there's no point
allocating taskqueues that will never be used.

MFC after: 1 week
Spo

cxgbe(4): Allocate a taskqueue per port instead of per channel.

All the channels are not used on all boards and there's no point
allocating taskqueues that will never be used.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe: Support TCP_USE_DDP on offloaded TOE connections

When this socket option is enabled, relatively large contiguous
buffers are allocated and used to receive data from the remote
connection. Wh

cxgbe: Support TCP_USE_DDP on offloaded TOE connections

When this socket option is enabled, relatively large contiguous
buffers are allocated and used to receive data from the remote
connection. When data is received a wrapper M_EXT mbuf is queued to
the socket's receive buffer. This reduces the length of the linked
list of received mbufs and allows consumers to consume receive data in
larger chunks.

To minimize reprogramming the page pods in the adapter, receive
buffers for a given connection are recycled. When a buffer has been
fully consumed by the receiver and freed, the buffer is placed on a
per-connection free buffers list.

The size of the receive buffers defaults to 256k and can be set via
the hw.cxgbe.toe.ddp_rcvbuf_len sysctl. The
hw.cxgbe.toe.ddp_rcvbuf_cache sysctl (defaults to 4) determines the
maximum number of free buffers cached per connection. Note that this
limit does not apply to "in-flight" receive buffers that are
associated with mbufs in the socket's receive buffer.

Co-authored-by: Navdeep Parhar <np@FreeBSD.org>
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44001

show more ...

cxgbe: Add counters for POSIX async I/O requests handled by the driver

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43668

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

cxgbe: Move ethofld_transmit under #ifdef INET || INET.

This fixes a -Wunused-function warning when building LINT-NOIP.

Reported by: rpokala

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

cxgbe(4): Remove dead code.

Fixes: e7e084442227 cxgbe(4): Replace T4_PKT_TIMESTAMP with something slightly less hackish.
MFC after: 1 week
Sponsored by: Chelsio Communications

Mechanically convert cxgb(4) and cxgbe(4) to IfAPI

Reviewed by: np
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38597

cxgbe: Queue NIC TLS mbufs internally.

Set internal mbuf properties and queue NIC TLS mbuf chains to the
connection's TX queue directly.

Reviewed by: np
Sponsored by: Chelsio Communications
Differe

cxgbe: Queue NIC TLS mbufs internally.

Set internal mbuf properties and queue NIC TLS mbuf chains to the
connection's TX queue directly.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D38579

show more ...

cxgbe: Move helper functions for mbuf metadata to adapter.h.

Previously private to t4_sge.c, this allows other parts of the driver
(such as NIC TLS) to use these helpers directly.

Reviewed by: np
S

cxgbe: Move helper functions for mbuf metadata to adapter.h.

Previously private to t4_sge.c, this allows other parts of the driver
(such as NIC TLS) to use these helpers directly.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D38578

show more ...

cxgbe: Allow parse_pkt to internally queue a packet.

If parse_pkt returns EINPROGRESS, return from cxgbe_transmit
without queueing the packet in a txq. Use this to move the call
to ethofld_transmit

cxgbe: Allow parse_pkt to internally queue a packet.

If parse_pkt returns EINPROGRESS, return from cxgbe_transmit
without queueing the packet in a txq. Use this to move the call
to ethofld_transmit for packet pacing into parse_pkt.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D38577

show more ...

pfil: add pfil_mem_{in,out}() and retire pfil_run_hooks()

The 0b70e3e78b0 changed the original design of a single entry point
into pfil(9) chains providing separate functions for the filtering
point

pfil: add pfil_mem_{in,out}() and retire pfil_run_hooks()

The 0b70e3e78b0 changed the original design of a single entry point
into pfil(9) chains providing separate functions for the filtering
points that always provide mbufs and know the direction of a flow.
The motivation was to reduce branching. The logical continuation
would be to do the same for the filtering points that always provide
a memory pointer and retire the single entry point.

o Hooks now provide two functions: one for mbufs and optional for
memory pointers.
o pfil_hook_args() has a new member and pfil_add_hook() has a
requirement to zero out uninitialized data. Bump PFIL_VERSION.
o As it was before, a hook function for a memory pointer may realloc
into an mbuf. Such mbuf would be returned via a pointer that must
be provided in argument.
o The only hook that supports memory pointers is ipfw:default-link.
It is rewritten to provide two functions.
o All remaining uses of pfil_run_hooks() are converted to
pfil_mem_in().
o Transparent union of pfil_packet_t and tricks to fix pointer
alignment are retired. Internal pfil_realloc() reduces down to
m_devget() and thus is retired, too.

Reviewed by: mjg, ocochard
Differential revision: https://reviews.freebsd.org/D37977

show more ...

cxgbe: Various whitespace fixes.

Mostly trailing whitespace and spaces before tabs.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37350

cxgbe: Use secq(9) to manage the timestamp generations.

This is mostly cosmetic, but it also doesn't leave a gap of time where
no structures are valid. Instead, we permit the ISR to continue to
use

cxgbe: Use secq(9) to manage the timestamp generations.

This is mostly cosmetic, but it also doesn't leave a gap of time where
no structures are valid. Instead, we permit the ISR to continue to
use the previous structure if the write to update cal_current isn't
yet visible.

Reviewed by: gallatin
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36669

show more ...

cxgbe: Compute timestamps via sbintime_t.

This uses fixed-point math already used elsewhere in the kernel for
sub-second time values. To avoid overflows this does require updating
the calibration o

cxgbe: Compute timestamps via sbintime_t.

This uses fixed-point math already used elsewhere in the kernel for
sub-second time values. To avoid overflows this does require updating
the calibration once a second rather than once every 30 seconds. Note
that the cxgbe driver already queries multiple registers once a second
for the statistics timers. This version also uses fewer instructions
with no branches (for the math portion) in the per-packet fast path.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36663

show more ...

cxgbe(4): Fix potential integer overflow in t4_tstmp_to_ns.

Coverity flagged this in its latest run but it is not a problem in
practice as the card's core clock would have to be > 4.2GHz for any
ove

cxgbe(4): Fix potential integer overflow in t4_tstmp_to_ns.

Coverity flagged this in its latest run but it is not a problem in
practice as the card's core clock would have to be > 4.2GHz for any
overflow to occur.

CID 1498303: Integer handling issues (OVERFLOW_BEFORE_WIDEN)
Potentially overflowing expression "sc->params.vpd.cclk * 1000U" with type "unsigned int" (32 bits, unsigned) is evaluated using 32-bit arithmetic, and then used in a context that expects an expression of type "uint64_t" (64 bits, unsigned).

Reported by: Coverity Scan (CID 1498303)
Sponsored by: Chelsio Communications

show more ...

Enable M_TSTMP in Chelsio cxgbe driver by creating a mechanism that can sync the time.

Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mb

Enable M_TSTMP in Chelsio cxgbe driver by creating a mechanism that can sync the time.

Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mbuf flags. This is because the timestamp
was just the free running 60bit clock. This change fixes that so that
we keep a synchronization by periodically (every 30 seconds after startup)
getting the timestamp and the current nanosecond time. We always keep
several sets around and the current one we always keep the current pair
and the previous pair of timestamps. This allows us to setup a ratio
between the two so we can correctly translate the time. Note that
we use special care to split the timestamp into seconds (per the clock tick)
and nanoseconds otherwise 64bit math would overflow.

Reviewed by: np
Sponsored by: Netflix Inc
Differential Revision: https://reviews.freebsd.org/D36315

show more ...

cxgbe(4): Add knob to control congestion behavior for TOE queues.

hw.cxgbe.ofld_cong_drop works just like hw.cxgbe.cong_drop but for TOE
rx queues.

MFC after: 2 weeks
Sponsored by: Chelsio Communic

cxgbe(4): Add knob to control congestion behavior for TOE queues.

hw.cxgbe.ofld_cong_drop works just like hw.cxgbe.cong_drop but for TOE
rx queues.

MFC after: 2 weeks
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Add another setting to the knob that controls congestion.

hw.cxgbe.cong_drop=2 will generate backpressure *and* drop frames for
queues that are congested.

MFC after: 2 weeks
Sponsored by:

cxgbe(4): Add another setting to the knob that controls congestion.

hw.cxgbe.cong_drop=2 will generate backpressure *and* drop frames for
queues that are congested.

MFC after: 2 weeks
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Specify the ingress queue's type when creating it.

The firmware takes the type into account when setting up the PCIe
channel for the queue.

MFC after: 1 week
Sponsored by: Chelsio Communi

cxgbe(4): Specify the ingress queue's type when creating it.

The firmware takes the type into account when setting up the PCIe
channel for the queue.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe: don't support MJUMPAGESIZE != MCLBYTES

This could only happen on systems with PAGE_SIZE < 4K and FreeBSD
doesn't support such systems.

Reviewed by: np, imp, jhb
Sponsored by: DARPA, AFRL
Dif

cxgbe: don't support MJUMPAGESIZE != MCLBYTES

This could only happen on systems with PAGE_SIZE < 4K and FreeBSD
doesn't support such systems.

Reviewed by: np, imp, jhb
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D36321

show more ...