sdt: Initialize probes in two passes

Suppose a kernel module A defines an SDT provider and probes, and kernel
linker file B, dependant on A, contains tracepoints for those probes.
When sdt.ko is loa

sdt: Initialize probes in two passes

Suppose a kernel module A defines an SDT provider and probes, and kernel
linker file B, dependant on A, contains tracepoints for those probes.
When sdt.ko is loaded, it iterates over all loaded KLDs to initialize
probe structures and register them with dtrace. In particular it uses
linker_file_foreach(), which is not sorted; in the above scenario, B may
be visited before A. Thus, it's possible for sdt_kld_load_probes() to
try to add tracepoints to an uninitialized SDT probe.

An example of the above arises when pfsync, pf, and sdt are loaded in
that exact order after commit 4bb3b36577645.

Fix this by initializing probe structures in the first pass over loaded
KLDs. Then, the second pass can safely add tracepoints to any probe
structure.

Note that the scenario where B and A are loaded after sdt.ko is already
handled properly, as there, the kld_load eventhandler is responsible for
registering probes with dtrace, and that eventhandler fires for
dependencies before it does for the dependent KLD. This presumes,
however, that there are no cycles in the dependency graph.

Reported by: jenkins
MFC after: 2 weeks

show more ...

sdt: Tear down probes in kernel modules during kldunload

Previously only providers in kernel modules were removed leaving
dangling pointers to tracepoints, etc. in unloaded kernel modules.

PR: 281

sdt: Tear down probes in kernel modules during kldunload

Previously only providers in kernel modules were removed leaving
dangling pointers to tracepoints, etc. in unloaded kernel modules.

PR: 281825
Reported by: Sony Arpita Das <sonyarpitad@chelsio.com>
Reviewed by: markj
Fixes: ddf0ed09bd8f sdt: Implement SDT probes using hot-patching
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46890

show more ...

sdt: Fix aframe handling after commit ddf0ed09bd8f

DTrace probes have an "aframes" attribute, used when unwinding the stack
from dtrace_probe(). It counts the number of leading frames to skip
when

sdt: Fix aframe handling after commit ddf0ed09bd8f

DTrace probes have an "aframes" attribute, used when unwinding the stack
from dtrace_probe(). It counts the number of leading frames to skip
when returning a stack trace, thus is used to hide internal functions.
Commit ddf0ed09bd8f set the aframes value for SDT probes to 0, which was
correct for an earlier iteration of the patch, but now doesn't take
sdt_probe()/sdt_probe6() into account.

Fix the aframes definition for SDT probes. Also try to improve
lockstat(1) output by adding an additional aframe for lockstat probes,
which otherwise show internal mtx(9), rwlock(9), etc. functions as the
probe "caller". This is not quite correct as the number of frames to
skip may differ depending on the lock type and kernel configuration (see
e.g., the MUTEX_NOINLINE kernel option), but this is not a new problem.

Reported by: mjg
Fixes: ddf0ed09bd8f ("sdt: Implement SDT probes using hot-patching")

show more ...

sdt: Support fetching the probe sixth argument with MI machinery

SDT calls dtrace_probe() directly, and this can be used to pass up to
five probe arguments directly. To pass the sixth argument (SDT

sdt: Support fetching the probe sixth argument with MI machinery

SDT calls dtrace_probe() directly, and this can be used to pass up to
five probe arguments directly. To pass the sixth argument (SDT
currently doesn't support more than this), we use a hack: just add
additional parameters to the call and cast dtrace_probe accordingly.
This happens to work on amd64, but doesn't work in general.

Modify SDT to call dtrace_probe() after storing arguments beyond the
first five in thread-local storage. Implement sdt_getargval() to fetch
extra argument values this way. An alternative would be to use invop
handlers instead and make sdt_probe_func point to a breakpoint
instruction, so that one can extract arguments using the breakpoint
exception trapframe, but this makes the providers more expensive when
enabled and doesn't seem justified. This approach works well unless we
want to add more than one or two more parameters to SDT probes, which
seems unlikely at present.

In particular, this fixes fetching the last argument of most ip and tcp
probes on arm64.

Reported by: rwatson
Reviewed by: Domagoj Stolfa
MFC after: 1 month
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D45648

show more ...

sdt: Implement SDT probes using hot-patching

The idea here is to avoid a memory access and conditional branch per
probe site. Instead, the probe is represented by an "unreachable"
unconditional fun

sdt: Implement SDT probes using hot-patching

The idea here is to avoid a memory access and conditional branch per
probe site. Instead, the probe is represented by an "unreachable"
unconditional function call. asm goto is used to store the address of
the probe site (represented by a no-op sled) and the address of the
function call into a tracepoint record. Each SDT probe carries a list
of tracepoints.

When the probe is enabled, the no-op sled corresponding to each
tracepoint is overwritten with a jmp to the corresponding label. The
implementation uses smp_rendezvous() to park all other CPUs while the
instruction is being overwritten, as this can't be done atomically in
general. The compiler moves argument marshalling code and the
sdt_probe() function call out-of-line, i.e., to the end of the function.

Per gallatin@ in D43504, this approach has less overhead when probes are
disabled. To make the implementation a bit simpler, I removed support
for probes with 7 arguments; nothing makes use of this except a
regression test case. It could be re-added later if need be.

The approach taken in this patch enables some more improvements:
1. We can now automatically fill out the "function" field of SDT probe
names. The SDT macros let the programmer specify the function and
module names, but this is really a bug and shouldn't have been
allowed. The intent was to be able to have the same probe in
multiple functions and to let the user restrict which probes actually
get enabled by specifying a function name or glob.
2. We can avoid branching on SDT_PROBES_ENABLED() by adding the ability
to include blocks of code in the out-of-line path. For example:

if (SDT_PROBES_ENABLED()) {
int reason = CLD_EXITED;

if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
SDT_PROBE1(proc, , , exit, reason);
}

could be written

SDT_PROBE1_EXT(proc, , , exit, reason,
int reason;

reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
);

In the future I would like to use this mechanism more generally, e.g.,
to remove branches and marshalling code used by hwpmc, and generally to
make it easier to add new tracepoint consumers without having to add
more conditional branches to hot code paths.

Reviewed by: Domagoj Stolfa, avg
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D44483

show more ...

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

sdt: Create providers and probes in separate passes when loading sdt.ko

The sdt module's load handler iterates over SDT linker sets for the
kernel and all loaded modules to create probes and provide

sdt: Create providers and probes in separate passes when loading sdt.ko

The sdt module's load handler iterates over SDT linker sets for the
kernel and all loaded modules to create probes and providers defined by
SDT(9). Probes in one module may belong to a provider in a different
module, but when a probe is created we assume that the provider is
already defined. To maintain this invariant, modify the load handler to
perform two separate passes over loaded modules: one to define providers
and the other to define probes.

The problem manifests when loading linux.ko, which depends on
linux_common.ko, which defines providers used by probes defined in
linux.ko.

Reported by: gallatin
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would a

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

show more ...

lockstat: track lockstat just like sdt probes

In particular flip the frequently tested var to bool.

Merge ^/head r323559 through r325504.

MFhead@r324884

sdt: make all sdt probe sites test one variable

This saves on cache misses at the expense of a slight grow of .text.

Note this is a bandaid for lack of hotpatching.

Discussed with: markj

MFhead@r324075

Use C99 initializers for DTrace provider methods.

This makes the definitions easier to read and more cscope-friendly.

MFC after: 1 week

MFH

Sponsored by: The FreeBSD Foundation

Initialize SDT probes during SI_SUB_DTRACE_PROVIDER.

This is consistent with all other DTrace providers and ensures that
SDT probes are available for boot-time tracing.

MFC after: 2 weeks

Merge from head

Sponsored by: Gandi.net

Merge from head

Sync up with head up to r289211.

Merge ^/head r288197 through r288456.

Merge from head

sdt: static-ize couple of variables

MFC after: 11 days

sdt module does not seem to actually use any symbol from opensolaris module

MFC after: 11 days