inpcb: Close some SO_REUSEPORT_LB races, part 2

Suppose a thread is adds a socket to an existing TCP lbgroup that is
actively accepting connections. It has to do the following operations:
1. set SO

inpcb: Close some SO_REUSEPORT_LB races, part 2

Suppose a thread is adds a socket to an existing TCP lbgroup that is
actively accepting connections. It has to do the following operations:
1. set SO_REUSEPORT_LB on the socket
2. bind() the socket to the shared address/port
3. call listen()

Step 2 makes the inpcb visible to incoming connection requests.
However, at this point the inpcb cannot accept new connections. If
in_pcblookup() matches it, the remote end will see ECONNREFUSED even
when other listening sockets are present in the lbgroup. This means
that dynamically adding inpcbs to an lbgroup (e.g., by starting up new
workers) can trigger spurious connection failures for no good reason.
(A similar problem exists when removing inpcbs from an lbgroup, but that
is harder to fix and is not addressed by this patch; see the review for
a bit more commentary.)

Fix this by augmenting each lbgroup with a linked list of inpcbs that
are pending a listen() call. When adding an inpcb to an lbgroup, keep
the inpcb on this list if listen() hasn't been called, so it is not yet
visible to the lookup path. Then, add a new in_pcblisten() routine which
makes the inpcb visible within the lbgroup now that it's safe to let it
handle new connections.

Add a regression test which verifies that we don't get spurious
connection errors while adding sockets to an LB group.

Reviewed by: glebius
MFC after: 1 month
Sponsored by: Klara, Inc.
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D48544

show more ...

tests: remove reference to renamed test

Fixes: ae5c3dfd3e75bb287984947359d4f958aea505ec

fibs_test: Run in a jail

These tests assume exclusive use of one or more FIBs and so can't be run
in parallel. Running them in a VNET jail sidesteps this problem.

MFC after: 1 week
Sponsored by: K

fibs_test: Run in a jail

These tests assume exclusive use of one or more FIBs and so can't be run
in parallel. Running them in a VNET jail sidesteps this problem.

MFC after: 1 week
Sponsored by: Klara, Inc.
Sponsored by: Stormshield

show more ...

netinet tests: Make some tests more reliable when run in parallel

Many of the modified tests add epair interfaces to the host, though most
of the tests themselves run in a VNET jail. scapy in parti

netinet tests: Make some tests more reliable when run in parallel

Many of the modified tests add epair interfaces to the host, though most
of the tests themselves run in a VNET jail. scapy in particular becomes
unhappy when the list of interfaces changes as it is running, so, to
improve reliability of parallel test runs, isolate tests which add
epairs to the host.

Also serialize arp tests, as they examine the dmesg as part of the test.

The list of modified tests is not exhaustive, it was determined by
running the test suite with parallelization enabled and looking at
failures. It may be easier to just automatically enable VNET jailing of
all netinet tests, but let's be more particular for now.

Reviewed by: kp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D46606

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

tests/netinet: add UDP socket I/O tests

Start a file that would collect tests for I/O functionality of a UDP
socket, targeted on how a socket interacts with userland rather than with
wire side of th

tests/netinet: add UDP socket I/O tests

Start a file that would collect tests for I/O functionality of a UDP
socket, targeted on how a socket interacts with userland rather than with
wire side of the protocol.

First version tests that MSG_TRUNC and MSG_PEEK are working correctly.

show more ...

tests/netinet: add a demo of TCP implied connect

The TCP implied connect is an artifact left after T/TCP. To my surprise
it still works, hence the existence of this test. Please read this email
fi

tests/netinet: add a demo of TCP implied connect

The TCP implied connect is an artifact left after T/TCP. To my surprise
it still works, hence the existence of this test. Please read this email
first:

https://lists.freebsd.org/pipermail/freebsd-net/2010-August/026311.html

An interesting fact that this test takes 220 - 240 milliseconds to
execute on my Threadripper PRO. Flipping the '#if 0' to '#if 1' in the
test, thus bringing it back to normal connect(2), would speed the test up
a hundred times and I guess all this time is fork+exec of the test.

show more ...

netinet tests: test that we send IGMP join/leave packets as expected

Sponsored by: Rubicon Communications, LLC ("Netgate")

netinet tests: Add error handling tests for UDP with v4-mapped sockets

This provides a regression test for commit abca3ae7734f.

Add it to the existing v4-mapped address test file, and rename
accord

netinet tests: Add error handling tests for UDP with v4-mapped sockets

This provides a regression test for commit abca3ae7734f.

Add it to the existing v4-mapped address test file, and rename
accordingly.

Reviewed by: tuexen, karels, rrs
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39216

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

netinet tests: test carp source MAC address

Ensure that (multicast) CARP packets are sent with the expected source
MAC address.

Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Net

netinet tests: test carp source MAC address

Ensure that (multicast) CARP packets are sent with the expected source
MAC address.

Reviewed by: melifaro
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D39454

show more ...

tests/netinet: mark more tests that require python

Split netinet shell tests into one per line.

This makes diffs when adding or removing tests easier to read.

While here, sort the list of tests.

Reviewed by: kevans, melifaro, asomers, markj, emast

Split netinet shell tests into one per line.

This makes diffs when adding or removing tests easier to read.

While here, sort the list of tests.

Reviewed by: kevans, melifaro, asomers, markj, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36908

show more ...

netinet tests: Add test for IPv6 mapped-v4 bind problem

Test fix in 637f317c6d9c, verifying that when ports run out, we get
an EADDRNOTAVAIL error from bind() rather than an EADDRINUSE error
from co

netinet tests: Add test for IPv6 mapped-v4 bind problem

Test fix in 637f317c6d9c, verifying that when ports run out, we get
an EADDRNOTAVAIL error from bind() rather than an EADDRINUSE error
from connect(). Use small port range to exhaust ports and see which
error happens.

Reviewed by: tuexen, glebius, melifaro
Differential Revision: https://reviews.freebsd.org/D36056
MFC after: 3 days (with 637f317c6d9c)

show more ...

Add tests for TCP_MD5 getsockopt

Add tests for TCP_MD5 getsockopt for ipv6 and ipv4. These will only run
when the kernel module is loaded, but will be in place if further
regressions are found.

Rev

Add tests for TCP_MD5 getsockopt

Add tests for TCP_MD5 getsockopt for ipv6 and ipv4. These will only run
when the kernel module is loaded, but will be in place if further
regressions are found.

Reviewed by: rscheff
MFC after: 3 days
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35532

show more ...

test/libalias: Tests for instantiation

In order to modify libalias for performance, the existing
functionality must not change. Enforce this.

Reviewed by: kp
MFC after: 1 week
Differential Revisio

test/libalias: Tests for instantiation

In order to modify libalias for performance, the existing
functionality must not change. Enforce this.

Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30307

show more ...

base: remove if_wg(4) and associated utilities, manpage

After length decisions, we've decided that the if_wg(4) driver and
related work is not yet ready to live in the tree. This driver has
larger

base: remove if_wg(4) and associated utilities, manpage

After length decisions, we've decided that the if_wg(4) driver and
related work is not yet ready to live in the tree. This driver has
larger security implications than many, and thus will be held to
more scrutiny than other drivers.

Please also see the related message sent to the freebsd-hackers@
and freebsd-arch@ lists by Kyle Evans <kevans@FreeBSD.org> on
2021/03/16, with the subject line "Removing WireGuard Support From Base"
for additional context.

show more ...

if_wg: import latest fixup work from the wireguard-freebsd project

This is the culmination of about a week of work from three developers to
fix a number of functional and security issues. This patc

if_wg: import latest fixup work from the wireguard-freebsd project

This is the culmination of about a week of work from three developers to
fix a number of functional and security issues. This patch consists of
work done by the following folks:

- Jason A. Donenfeld <Jason@zx2c4.com>
- Matt Dunwoodie <ncon@noconroy.net>
- Kyle Evans <kevans@FreeBSD.org>

Notable changes include:
- Packets are now correctly staged for processing once the handshake has
completed, resulting in less packet loss in the interim.
- Various race conditions have been resolved, particularly w.r.t. socket
and packet lifetime (panics)
- Various tests have been added to assure correct functionality and
tooling conformance
- Many security issues have been addressed
- if_wg now maintains jail-friendly semantics: sockets are created in
the interface's home vnet so that it can act as the sole network
connection for a jail
- if_wg no longer fails to remove peer allowed-ips of 0.0.0.0/0
- if_wg now exports via ioctl a format that is future proof and
complete. It is additionally supported by the upstream
wireguard-tools (which we plan to merge in to base soon)
- if_wg now conforms to the WireGuard protocol and is more closely
aligned with security auditing guidelines

Note that the driver has been rebased away from using iflib. iflib
poses a number of challenges for a cloned device trying to operate in a
vnet that are non-trivial to solve and adds complexity to the
implementation for little gain.

The crypto implementation that was previously added to the tree was a
super complex integration of what previously appeared in an old out of
tree Linux module, which has been reduced to crypto.c containing simple
boring reference implementations. This is part of a near-to-mid term
goal to work with FreeBSD kernel crypto folks and take advantage of or
improve accelerated crypto already offered elsewhere.

There's additional test suite effort underway out-of-tree taking
advantage of the aforementioned jail-friendly semantics to test a number
of real-world topologies, based on netns.sh.

Also note that this is still a work in progress; work going further will
be much smaller in nature.

MFC after: 1 month (maybe)

show more ...

Add arp/ndp tests in addition to rtsock ll tests.

Add a regression test for the port-selection behavior fixed in r367680.

Reviewed by: markj, olivier, tuexen
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27173

Do a sweep and remove most WARNS=6 settings

Repeating the default WARNS here makes it slightly more difficult to
experiment with default WARNS changes, e.g. if we did something absolutely
bananas an

Do a sweep and remove most WARNS=6 settings

Repeating the default WARNS here makes it slightly more difficult to
experiment with default WARNS changes, e.g. if we did something absolutely
bananas and introduced a WARNS=7 and wanted to try lifting the default to
that.

Drop most of them; there is one in the blake2 kernel module, but I suspect
it should be dropped -- the default WARNS in the rest of the build doesn't
currently apply to kernel modules, and I haven't put too much thought into
whether it makes sense to make it so.

show more ...

Add basic test for net.fibs dynamic growth.

Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D26382

Make sys.netinet.output and sys.netinet6.output6 tests depend on python;
they need to be able to run net_receiver.py.

MFC after: 2 weeks
Sponsored by: DARPA

Add basic routing LPM tests.

Differential Revision: https://reviews.freebsd.org/D24684

carp tests: Basic functionality test

Set up three vnet jails, bridged together. Run carp between two of them.
Attempt to provoke locking / epoch issues.

Reviewed by: mav (previous version), melifar

carp tests: Basic functionality test

Set up three vnet jails, bridged together. Run carp between two of them.
Attempt to provoke locking / epoch issues.

Reviewed by: mav (previous version), melifaro, asomers
Differential Revision: https://reviews.freebsd.org/D24303

show more ...