rpcsec_tls/client: API refactoring between kernel and rpc.tlsclntd(8)

Now that the conversion of rpcsec_tls/client + rpc.tlsclntd(8) to the
netlink(4) socket as RPC transport started using kernel so

rpcsec_tls/client: API refactoring between kernel and rpc.tlsclntd(8)

Now that the conversion of rpcsec_tls/client + rpc.tlsclntd(8) to the
netlink(4) socket as RPC transport started using kernel socket pointer as
a reliable cookie, we can shave off quite a lot of complexity. We will
utilize the same kernel-generated cookie in all RPCs. And the need for
the daemon generated cookie in the form of timestamp+sequence vanishes.

In the clnt_vc.c we no longer need to store the userland cookie, but we
still need to observe the TLS life cycle of the client. We observe
RPCTLS_INHANDSHAKE state, that lives for a short time when the socket had
already been fetched by the daemon with the syscall, but the RPC call is
still waiting for the reply from daemon.

This time bump the RPC version.

Reviewed by: rmacklem
Differential Revision: https://reviews.freebsd.org/D48564

show more ...

krpc: assert that we don't support kernel RPC over unix(4)

Reviewed by: rmacklem
Differential Revision: https://reviews.freebsd.org/D48563

krpc/clnt_vc: clear vnet context before kthread_exit()

Fixes: b2ff4cb1931c2e1509a5741f6743322699ad1e00

krpc/clnt_vc: set vnet(9) context in clnt_vc kthread

The per-client kthread to offload TLS stuff was added ab0c29af0512d. Let
it run in the vnet(9) that matches associated socket.

krpc/clnt_vc: in clnt_vc_destroy() use more lapidary logic

on whether to close the socket or leave it.

krpc/clnt_vc: remove always false check

We just initialized ct_closeit to false a few lines above.

krpc: Ref cnt the client structures for TLS upcalls

A crash occurred during testing, where the client structures had
already been free'd when the upcall thread tried to lock them.

This patch acquir

krpc: Ref cnt the client structures for TLS upcalls

A crash occurred during testing, where the client structures had
already been free'd when the upcall thread tried to lock them.

This patch acquires a reference count on both of the structures
and these are released when the upcall is done, so that the
structures cannot be free'd prematurely. This happened because
the testing is done over a very slow vpn.

Found during a IETF bakeathon testing event this week.

MFC after: 5 days

show more ...

rpc: use new macros to lock socket buffers

Fixes: d80a97def9a1db6f07f5d2e68f7ad62b27918947

sockets: garbage collect SS_ISCONFIRMING

Fixes: 8df32b19dee92b5eaa4b488ae78dca6accfcb38e

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

clnt_vc.c: Replace msleep() with pause() to avoid assert panic

An msleep() in clnt_vc.c used a global "fake_wchan" wchan argument
along with the mutex in a CLIENT structure. As such, it was
possibl

clnt_vc.c: Replace msleep() with pause() to avoid assert panic

An msleep() in clnt_vc.c used a global "fake_wchan" wchan argument
along with the mutex in a CLIENT structure. As such, it was
possible to use different mutexes for the same wchan and
cause a panic assert. Since this is in a rarely executed code
path, the assert panic was only recently observed.

Since "fake_wchan" never gets a wakeup, this msleep() can
be replaced with a pause() to avoid the panic assert,
which is what this patch does.

Reviewed by: kib, markj
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D36977

show more ...

krpc: Fix NFS-over-TLS for KTLS1.3

When NFS-over-TLS uses KTLS1.3, the client can receive
post-handshake handshake records. These records can be
safely thown away, but are not handled correctly via

krpc: Fix NFS-over-TLS for KTLS1.3

When NFS-over-TLS uses KTLS1.3, the client can receive
post-handshake handshake records. These records can be
safely thown away, but are not handled correctly via the
rpctls_ct_handlerecord() upcall to the daemon.

Commit 373511338d95 changed soreceive_generic() so that it
will only return ENXIO for Alert records when MSG_TLSAPPDATA
is specified. As such, the post-handshake handshake
records will be returned to the krpc.

This patch modifies the krpc so that it will throw
these records away, which seems sufficient to make
NFS-over-TLS work with KTLS1.3. This change has
no effect on the use of KTLS1.2, since it does not
generate post-handshake handshake records.

MFC after: 2 weeks

show more ...

sockets: use socket buffer mutexes in struct socket directly

Since c67f3b8b78e the sockbuf mutexes belong to the containing socket,
and socket buffers just point to it. In 74a68313b50 macros that a

sockets: use socket buffer mutexes in struct socket directly

Since c67f3b8b78e the sockbuf mutexes belong to the containing socket,
and socket buffers just point to it. In 74a68313b50 macros that access
this mutex directly were added. Go over the core socket code and
eliminate code that reaches the mutex by dereferencing the sockbuf
compatibility pointer.

This change requires a KPI change, as some functions were given the
sockbuf pointer only without any hint if it is a receive or send buffer.

This change doesn't cover the whole kernel, many protocols still use
compatibility pointers internally. However, it allows operation of a
protocol that doesn't use them.

Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D35152

show more ...

clnt_vc_destroy: eliminiate write only variable stat

Sponsored by: Netflix

Don't require the socket lock for sorele().

Previously, sorele() always required the socket lock and dropped the
lock if the released reference was not the last reference. Many
callers locked the s

Don't require the socket lock for sorele().

Previously, sorele() always required the socket lock and dropped the
lock if the released reference was not the last reference. Many
callers locked the socket lock just before calling sorele() resulting
in a wasted lock/unlock when not dropping the last reference.

Move the previous implementation of sorele() into a new
sorele_locked() function and use it instead of sorele() for various
places in uipc_socket.c that called sorele() while already holding the
socket lock.

The sorele() macro now uses refcount_release_if_not_last() try to drop
the socket reference without locking the socket. If that shortcut
fails, it locks the socket and calls sorele_locked().

Reviewed by: kib, markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32741

show more ...

rpc: Make function tables const

No functional change intended.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

Add TLS support to the kernel RPC.

An internet draft titled "Towards Remote Procedure Call Encryption By Default"
describes how TLS is to be used for Sun RPC, with NFS as an intended use case.
This

Add TLS support to the kernel RPC.

An internet draft titled "Towards Remote Procedure Call Encryption By Default"
describes how TLS is to be used for Sun RPC, with NFS as an intended use case.
This patch adds client and server support for this to the kernel RPC,
using KERN_TLS and upcalls to daemons for the handshake, peer reset and
other non-application data record cases.

The upcalls to the daemons use three fields to uniquely identify the
TCP connection. They are the time.tv_sec, time.tv_usec of the connection
establshment, plus a 64bit sequence number. The time fields avoid problems
with re-use of the sequence number after a daemon restart.
For the server side, once a Null RPC with AUTH_TLS is received, kernel
reception on the socket is blocked and an upcall to the rpctlssd(8) daemon
is done to perform the TLS handshake. Upon completion, the completion
status of the handshake is stored in xp_tls as flag bits and the reply to
the Null RPC is sent.
For the client, if CLSET_TLS has been set, a new TCP connection will
send the Null RPC with AUTH_TLS to initiate the handshake. The client
kernel RPC code will then block kernel I/O on the socket and do an upcall
to the rpctlscd(8) daemon to perform the handshake.
If the upcall is successful, ct_rcvstate will be maintained to indicate
if/when an upcall is being done.

If non-application data records are received, the code does an upcall to
the appropriate daemon, which will do a SSL_read() of 0 length to handle
the record(s).

When the socket is being shut down, upcalls are done to the daemons, so
that they can perform SSL_shutdown() calls to perform the "peer reset".

The rpctlssd(8) and rpctlscd(8) daemons require a patched version of the
openssl library and, as such, will not be committed to head at this time.

Although the changes done by this patch are fairly numerous, there should
be no semantics change to the kernel RPC at this time.
A future commit to the NFS code will optionally enable use of TLS for NFS.

show more ...

Fix up a comment added by r362455.

Modify the way the client side krpc does soreceive() for TCP.

Without this patch, clnt_vc_soupcall() first does a soreceive() for
4 bytes (the Sun RPC over TCP record mark) and then soreceive(s) for

Modify the way the client side krpc does soreceive() for TCP.

Without this patch, clnt_vc_soupcall() first does a soreceive() for
4 bytes (the Sun RPC over TCP record mark) and then soreceive(s) for
the RPC message.
This first soreceive() almost always results in an mbuf allocation,
since having the 4byte record mark in a separate mbuf in the socket
rcv queue is unlikely.
This is somewhat inefficient and rather odd. It also will not work
for the ktls rx, since the latter returns a TLS record for each
soreceive().

This patch replaces the above with code similar to what the server side
of the krpc does for TCP, where it does a soreceive() for as much data
as possible and then parses RPC messages out of the received data.
A new field of the TCP socket structure called ct_raw is the list of
received mbufs that the RPC message(s) are parsed from.
I think this results in cleaner code and is needed for support of
nfs-over-tls.
It also fixes the code for the case where a server sends an RPC message
in multiple RPC message fragments. Although this is allowed by RFC5531,
no extant NFS server does this. However, it is probably good to fix this
in case some future NFS server does do this.

show more ...

Merge ^/head r325999 through r326131.

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

show more ...

Merge ^/head r317808 through r317970.

Fix the client side krpc from doing TCP reconnects for ERESTART from sosend().

When sosend() replies ERESTART in the client side krpc, it indicates that
the RPC message hasn't yet been sent and that

Fix the client side krpc from doing TCP reconnects for ERESTART from sosend().

When sosend() replies ERESTART in the client side krpc, it indicates that
the RPC message hasn't yet been sent and that the send queue is full or
locked while a signal is posted for the process.
Without this patch, this would result in a RPC_CANTSEND reply from
clnt_vc_call(), which would cause clnt_reconnect_call() to create a new
TCP transport connection. For most NFS servers, this wasn't a serious problem,
although it did imply retries of outstanding RPCs, which could possibly
have missed the DRC.
For an NFSv4.1 mount to AmazonEFS, this caused a serious problem, since
AmazonEFS often didn't retain the NFSv4.1 session and would reply with
NFS4ERR_BAD_SESSION. This implies to the client a crash/reboot which
requires open/lock state recovery.

Three options were considered to fix this:
- Return the ERESTART all the way up to the system call boundary and then
have the system call redone. This is fraught with risk, due to convoluted
code paths, asynchronous I/O RPCs etc. cperciva@ worked on this, but it
is still a work in prgress and may not be feasible.
- Set SB_NOINTR for the socket buffer. This fixes the problem, but makes
the sosend() completely non interruptible, which kib@ considered
inappropriate. It also would break forced dismount when a thread
was blocked in sosend().
- Modify the retry loop in clnt_vc_call(), so that it loops for this case
for up to 15sec. Testing showed that the sosend() usually succeeded by
the 2nd retry. The extreme case observed was 111 loop iterations, or
about 100msec of delay.
This third alternative is what is implemented in this patch, since the
change is:
- localized
- straightforward
- forced dismount is not broken by it.

This patch has been tested by cperciva@ extensively against AmazonEFS.

Reported by: cperciva
Tested by: cperciva
MFC after: 2 weeks

show more ...