ipsec_offload: offload inner checksums calculations for UDP/TCP/TSO

and allow the interface driver to declare such support.

Sponsored by: NVIDIA networking
Differential revision: https://reviews.fr

ipsec_offload: offload inner checksums calculations for UDP/TCP/TSO

and allow the interface driver to declare such support.

Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44221

show more ...

ipsec_offload: kernel infrastructure

Inline IPSEC offload moves almost whole IPSEC processing from the
CPU/MCU and possibly crypto accelerator, to the network card.

The transmitted packet content i

ipsec_offload: kernel infrastructure

Inline IPSEC offload moves almost whole IPSEC processing from the
CPU/MCU and possibly crypto accelerator, to the network card.

The transmitted packet content is not touched by CPU during TX
operations, kernel only does the required policy and security
association lookups to find out that given flow is offloaded, and then
packet is transmitted as plain text to the card. For driver convenience,
a metadata is attached to the packet identifying SA which must process
the packet. Card does encryption of the payload, padding, calculates
authentication, and does the reformat according to the policy.

Similarly, on receive, card does the decapsulation, decryption, and
authentification. Kernel receives the identifier of SA that was
used to process the packet, together with the plain-text packet.

Overall, payload octets are only read or written by card DMA engine,
removing a lot of memory subsystem overhead, and saving CPU time because
IPSEC algos calculations are avoided.

If driver declares support for inline IPSEC offload (with the
IFCAP2_IPSEC_OFFLOAD capability set and registering method table struct
if_ipsec_accel_methods), kernel offers the SPD and SAD to driver.
Driver decides which policies and SAs can be offloaded based on
hardware capacity, and acks/nacks each SA for given interface to
kernel. Kernel needs to keep this information to make a decision to
skip software processing on TX, and to assume processing already done
on RX. This shadow SPD/SAD database of offloads is rooted from
policies (struct secpolicy accel_ifps, struct ifp_handle_sp) and SAs
(struct secasvar accel_ipfs, struct ifp_handle_sav).

Some extensions to the PF_KEY socket allow to limit interfaces for
which given SP/SA could be offloaded (proposed for offload). Also,
additional statistics extensions allow to observe allocation/octet/use
counters for specific SA.

Since SPs and SAs are typically instantiated in non-sleepable context,
while offloading them into card is expected to require costly async
manipulations of the card state, calls to the driver for offload and
termination are executed in the threaded taskqueue. It also solves
the issue of allocating resources needed for the offload database.
Neither ipf_handle_sp nor ipf_handle_sav do not add reference to the
owning SP/SA, the offload must be terminated before last reference is
dropped. ipsec_accel only adds transient references to ensure safe
pointer ownership by taskqueue.

Maintaining the SA counters for hardware-accelerated packets is the
duty of the driver. The helper ipsec_accel_drv_sa_lifetime_update()
is provided to hide accel infrastructure from drivers which would use
expected callout to query hardware periodically for updates.

Reviewed by: rscheff (transport, stack integration), np
Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44219

show more ...

ipsec_output(): add mtu argument

Similarly, mtu is needed to decide inline IPSEC offloiad for the driver.

Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44224

ipsec_output(): add outcoming ifp argument

The information about the interface is needed to coordinate inline
offloading of IPSEC processing with corresponding driver.

Sponsored by: NVIDIA networki

ipsec_output(): add outcoming ifp argument

The information about the interface is needed to coordinate inline
offloading of IPSEC processing with corresponding driver.

Sponsored by: NVIDIA networking
Differential revision: https://reviews.freebsd.org/D44223

show more ...

Add UDP encapsulation of ESP in IPv6

This patch provides UDP encapsulation of ESP packets over IPv6.
Ports the IPv4 code to IPv6 and adds support for IPv6 in udpencap.c
As required by the RFC and un

Add UDP encapsulation of ESP in IPv6

This patch provides UDP encapsulation of ESP packets over IPv6.
Ports the IPv4 code to IPv6 and adds support for IPv6 in udpencap.c
As required by the RFC and unlike in IPv4 encapsulation,
UDP checksums are calculated.

Co-authored-by: Aurelien Cazuc <aurelien.cazuc.external@stormshield.eu>
Sponsored-by: Stormshield
Sponsored-by: Wiktel
Sponsored-by: Klara, Inc.

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

tcp: embed inpcb into tcpcb

For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several struct

tcp: embed inpcb into tcpcb

For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several structures, that previously were allocated
separately.

The most import one is the inpcb itself. With embedding we can provide
strong guarantee that with a valid TCP inpcb the tcpcb is always valid
and vice versa. Also we reduce number of allocs/frees per connection.
The embedded inpcb is placed in the beginning of the struct tcpcb,
since in_pcballoc() requires that. However, later we may want to move
it around for cache line efficiency, and this can be done with a little
effort. The new intotcpcb() macro is ready for such move.

The congestion algorithm data, the TCP timers and osd(9) data are
also embedded into tcpcb, and temprorary struct tcpcb_mem goes away.
There was no extra allocation here, but we went through extra pointer
every time we accessed this data.

One interesting side effect is that now TCP data is allocated from
SMR-protected zone. Potentially this allows the TCP stacks or other
TCP related modules to utilize that for their own synchronization.

Large part of the change was done with sed script:

s/tp->ccv->/tp->t_ccv./g
s/tp->ccv/\&tp->t_ccv/g
s/tp->cc_algo/tp->t_cc/g
s/tp->t_timers->tt_/tp->tt_/g
s/CCV\(ccv, osd\)/\&CCV(ccv, t_osd)/g

Dependency side effect is that code that needs to know struct tcpcb
should also know struct inpcb, that added several <netinet/in_pcb.h>.

Differential revision: https://reviews.freebsd.org/D37127

show more ...

netipsec: move specific ipsecmethods declarations to ipsec_support.h

where struct ipsec_methods is defined. Not a functional change.
Allows further modification of method prototypes without breakin

netipsec: move specific ipsecmethods declarations to ipsec_support.h

where struct ipsec_methods is defined. Not a functional change.
Allows further modification of method prototypes without breaking
compilation of other ipsec compilation units.

Differential revision: https://reviews.freebsd.org/D36730

show more ...

ipsec_encap: setdf is only used for INET.

netinet: Remove unneeded mb_unmapped_to_ext() calls

in_cksum_skip() now handles unmapped mbufs on platforms where they're
permitted.

Reviewed by: glebius, jhb
MFC after: 1 week
Sponsored by: The Fr

netinet: Remove unneeded mb_unmapped_to_ext() calls

in_cksum_skip() now handles unmapped mbufs on platforms where they're
permitted.

Reviewed by: glebius, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33097

show more ...

sctp: Remove now-unneeded mb_unmapped_to_ext() calls

sctp_delayed_checksum() now handles unmapped mbufs, thanks to m_apply().

No functional change intended.

Reviewed by: tuexen
MFC after: 2 weeks

sctp: Remove now-unneeded mb_unmapped_to_ext() calls

sctp_delayed_checksum() now handles unmapped mbufs, thanks to m_apply().

No functional change intended.

Reviewed by: tuexen
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32942

show more ...

ipsec: fix typo part2

ipsec: fix typo in comment

ipsec: Add support for PMTUD for IPv6 tunnels

Discard and send ICMPv6 Packet Too Big to sender when we try to encapsulate
and forward a packet which total length exceeds the PMTU.
Logic is based on

ipsec: Add support for PMTUD for IPv6 tunnels

Discard and send ICMPv6 Packet Too Big to sender when we try to encapsulate
and forward a packet which total length exceeds the PMTU.
Logic is based on the IPv4 implementation.
Common code was moved to a separate function.

Differential revision: https://reviews.freebsd.org/D31771
Obtained from: Semihalf
Sponsored by: Stormshield

show more ...

ipsec: If no PMTU in hostcache assume it's equal to link's MTU

If we fail to find to PMTU in hostcache, we assume it's equal
to link's MTU.

This patch prevents packets larger then link's MTU to be

ipsec: If no PMTU in hostcache assume it's equal to link's MTU

If we fail to find to PMTU in hostcache, we assume it's equal
to link's MTU.

This patch prevents packets larger then link's MTU to be dropped
silently if there is no PMTU in hostcache.

Differential revision: https://reviews.freebsd.org/D31770
Obtained from: Semihalf
Sponsored by: Stormshield

show more ...

ipsec: Add PMTUD support for IPsec IPv4 over IPv6 tunnel

Add support for checking PMTU for IPv4 packets encapsulated in IPv6 tunnels.

Differential revision: https://reviews.freebsd.org/D31769
Spons

ipsec: Add PMTUD support for IPsec IPv4 over IPv6 tunnel

Add support for checking PMTU for IPv4 packets encapsulated in IPv6 tunnels.

Differential revision: https://reviews.freebsd.org/D31769
Sponsored by: Stormshield
Obtained from: Semihalf

show more ...

ipsec: Return error code if no matching SA was found

If we matched SP to a packet, but no associated SA was found
ipsec4_allocsa will return NULL while setting error=0.
This resulted in use after fr

ipsec: Return error code if no matching SA was found

If we matched SP to a packet, but no associated SA was found
ipsec4_allocsa will return NULL while setting error=0.
This resulted in use after free and potential kernel panic.
Return EINPROGRESS if the case described above instead.

Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D30994

show more ...

ipsec: Check PMTU before sending a frame.

If an encapsulated frame is going to have DF bit set check its desitnitions'
PMTU and if it won't fit drop it and:

Generate ICMP 3/4 message if the packet

ipsec: Check PMTU before sending a frame.

If an encapsulated frame is going to have DF bit set check its desitnitions'
PMTU and if it won't fit drop it and:

Generate ICMP 3/4 message if the packet was to be forwarded.
Return EMSGSIZE error otherwise.

Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D30993

show more ...

Convert unmapped mbufs before computing checksums in IPsec.

This is similar to the logic used in ip_output() to convert mbufs
prior to computing checksums. Unmapped mbufs can be sent when using
sen

Convert unmapped mbufs before computing checksums in IPsec.

This is similar to the logic used in ip_output() to convert mbufs
prior to computing checksums. Unmapped mbufs can be sent when using
sendfile() over IPsec or using KTLS over IPsec.

Reported by: Sony Arpita Das @ Chelsio QA
Reviewed by: np
Sponsored by: Chelsio
Differential Revision: https://reviews.freebsd.org/D28187

show more ...

net: clean up empty lines in .c and .h files

Enter and exit the network epoch for async IPsec callbacks.

When an IPsec packet has been encrypted or decrypted, the next step in
the packet's traversal through the network stack is invoked from a

Enter and exit the network epoch for async IPsec callbacks.

When an IPsec packet has been encrypted or decrypted, the next step in
the packet's traversal through the network stack is invoked from a
crypto worker thread, not from the original calling thread. These
threads need to enter the network epoch before passing packets down to
IP output routines or up to transport protocols.

Reviewed by: ae
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D25444

show more ...

Add the SCTP_SUPPORT kernel option.

This is in preparation for enabling a loadable SCTP stack. Analogous to
IPSEC/IPSEC_SUPPORT, the SCTP_SUPPORT kernel option must be configured
in order to suppor

Add the SCTP_SUPPORT kernel option.

This is in preparation for enabling a loadable SCTP stack. Analogous to
IPSEC/IPSEC_SUPPORT, the SCTP_SUPPORT kernel option must be configured
in order to support a loadable SCTP implementation.

Discussed with: tuexen
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

show more ...

sys: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
pro

sys: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

show more ...

MFH @ r323558.