Import the Hardware Trace (HWT) framework.

The HWT framework provides infrastructure for hardware-assisted tracing. It
collects detailed information about software execution and records it as
"event

Import the Hardware Trace (HWT) framework.

The HWT framework provides infrastructure for hardware-assisted tracing. It
collects detailed information about software execution and records it as
"events" in highly compressed format into DRAM. The events cover information
about control flow changes of a program, whether branches taken or not,
exceptions taken, timing information, cycles elapsed and more. This allows
to reconstruct entire program flow of a given application.

This comes with separate machine-dependent tracing backends for trace
collection, trace decoder libraries and an instrumentation tool.

Reviewed by: kib (sys/kern bits)
Sponsored by: UKRI
Differential Revision: https://reviews.freebsd.org/D40466

show more ...

ktrace: trace and decode thr_new() thr_param

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

kern_thr.c: fix typo

Fixes: c25ddffc51a1e2c0e03fa
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

thr_new(2): check param.flags for reserved bits

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

thr_new: pass params.flags to cpu_set_user_tls()

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

kern_thr.c: normalize includes

Remove extra sys/param.h, provided by sys/systm.h.
Order the rest alphabetically.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differenti

kern_thr.c: normalize includes

Remove extra sys/param.h, provided by sys/systm.h.
Order the rest alphabetically.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44867

show more ...

kern_thr.c/kern_sig.c: remove sys/cdefs.h

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44867

thread: Ignore errors when copying out during thr_exit()

It does not seem reasonable to return to userspace after calling
umtx_thread_exit().

This is in preparation for annotating copyin() and rela

thread: Ignore errors when copying out during thr_exit()

It does not seem reasonable to return to userspace after calling
umtx_thread_exit().

This is in preparation for annotating copyin() and related functions
with __result_use_check.

Reviewed by: olce, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43143

show more ...

thread: Add a return value to cpu_set_upcall()

Some implementations copy data to userspace, an operation which can in
principle fail. In preparation for adding a __result_use_check
annotation to co

thread: Add a return value to cpu_set_upcall()

Some implementations copy data to userspace, an operation which can in
principle fail. In preparation for adding a __result_use_check
annotation to copyin() and related functions, let implementations of
cpu_set_upcall() return an error, and check for errors when copying data
to user memory.

Reviewed by: kib, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43100

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

thread_create(): call cpu_copy_thread() after td_pflags is zeroed

By calling the function too early we might still have the td_pflags
value cached from the previous struct thread use. cpu_copy_threa

thread_create(): call cpu_copy_thread() after td_pflags is zeroed

By calling the function too early we might still have the td_pflags
value cached from the previous struct thread use. cpu_copy_thread()
depends on correct value for TDP_KTHREAD at least on x86.

Reported, bisected, and tested by: pho
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D36069

show more ...

AST: rework

Make most AST handlers dynamically registered. This allows to have
subsystem-specific handler source located in the subsystem files,
instead of making subr_trap.c aware of it. For inst

AST: rework

Make most AST handlers dynamically registered. This allows to have
subsystem-specific handler source located in the subsystem files,
instead of making subr_trap.c aware of it. For instance, signal
delivery code on return to userspace is now moved to kern_sig.c.

Also, it allows to have some handlers designated as the cleanup (kclear)
type, which are called both at AST and on thread/process exit. For
instance, ast(), exit1(), and NFS server no longer need to be aware
about UFS softdep processing.

The dynamic registration also allows third-party modules to register AST
handlers if needed. There is one caveat with loadable modules: the
code does not make any effort to ensure that the module is not unloaded
before all threads processed through AST handler in it. In fact, this
is already present behavior for hwpmc.ko and ufs.ko. I do not think it
is worth the efforts and the runtime overhead to try to fix it.

Reviewed by: markj
Tested by: emaste (arm64), pho
Discussed with: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D35888

show more ...

umtx: Split umtx.h on two counterparts.

To prevent umtx.h polluting by future changes split it on two headers:
umtx.h - ABI header for userspace;
umtxvar.h - the kernel staff.

While here fix umtx_k

umtx: Split umtx.h on two counterparts.

To prevent umtx.h polluting by future changes split it on two headers:
umtx.h - ABI header for userspace;
umtxvar.h - the kernel staff.

While here fix umtx_key_match style.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D31248
MFC after: 2 weeks

show more ...

Provide ABI modules hooks for process exec/exit and thread exit.

Exec and exit are same as corresponding eventhandler hooks.

Thread exit hook is called somewhat earlier, while thread is still
owned

Provide ABI modules hooks for process exec/exit and thread exit.

Exec and exit are same as corresponding eventhandler hooks.

Thread exit hook is called somewhat earlier, while thread is still
owned by the process and enough context is available. Note that the
process lock is owned when the hook is called.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D27309

show more ...

thread: rework tidhash vs proc lock interaction

Apart from minor clean up this gets rid of proc unlock/lock cycle on thread
exit to work around LOR against tidhash lock.

threads: introduce a limit for total number

The intent is to replace the current id allocation method and a known upper
bound will be useful.

Reviewed by: kib (previous version), markj (previous ve

threads: introduce a limit for total number

The intent is to replace the current id allocation method and a known upper
bound will be useful.

Reviewed by: kib (previous version), markj (previous version)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D27100

show more ...

Merge ^/head r358269 through r358399.

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

Merge ^/head r357408 through r357661.

Fix a recursion on the thread lock by acquiring it after call rtp_to_pri().

Reported by: swills
Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D23495

schedlock 1/4

Eliminate recursion from most thread_lock consumers. Return from
sched_add() without the thread_lock held. This eliminates unnecessary
atomics and lock word loads as well as reducing

schedlock 1/4

Eliminate recursion from most thread_lock consumers. Return from
sched_add() without the thread_lock held. This eliminates unnecessary
atomics and lock word loads as well as reducing the hold time for
scheduler locks. This will eventually allow for lockless remote adds.

Discussed with: kib
Reviewed by: jhb
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D22626

show more ...

Don't acquire evclass_lock with a spinlock held

When the "pc" audit class is enabled and auditd is running, witness will
panic during thread exit because au_event_class tries to lock an rwlock
while

Don't acquire evclass_lock with a spinlock held

When the "pc" audit class is enabled and auditd is running, witness will
panic during thread exit because au_event_class tries to lock an rwlock
while holding a spinlock acquired upstack by thread_exit.

To fix this, move AUDIT_SYSCALL_EXIT futher upstack, before the spinlock is
acquired. Of thread_exit's 16 callers, it's only necessary to call
AUDIT_SYSCALL_EXIT from two, exit1 (for exiting processes) and kern_thr_exit
(for exiting threads). The other callers are all kernel threads, which
needen't call AUDIT_SYSCALL_EXIT because since they can't make syscalls
there will be nothing to audit. And exit1 already does call
AUDIT_SYSCALL_EXIT, making the second call in thread_exit redundant for that
case.

PR: 228444
Reported by: aniketp
Reviewed by: aniketp, kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16210

show more ...

hwpmc: log name->pid, name->tid mappings

By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid

hwpmc: log name->pid, name->tid mappings

By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid was which when the sample was taken.

% pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log

% pmc filter -x -T idle pmc.log pmc-noidle.log

show more ...

hwpmc: Implement per-thread counters for PMC sampling

This implements per-thread counters for PMC sampling. The thread
descriptors are stored in a list attached to the process descriptor.
These thre

hwpmc: Implement per-thread counters for PMC sampling

This implements per-thread counters for PMC sampling. The thread
descriptors are stored in a list attached to the process descriptor.
These thread descriptors can store any per-thread information necessary
for current or future features. For the moment, they just store the counters
for sampling.

The thread descriptors are created when the process descriptor is created.
Additionally, thread descriptors are created or freed when threads
are started or stopped. Because the thread exit function is called in a
critical section, we can't directly free the thread descriptors. Hence,
they are freed to a cache, which is also used as a source of allocations
when needed for new threads.

Approved by: sbruno
Obtained from: jtl
Sponsored by: Juniper Networks, Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15335

show more ...