rwmlock/rwlock/sx: Print the pointer of destroyed locks in panic messages

Suggested by: markj
Reviewed by: kib, markj
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D49

rwmlock/rwlock/sx: Print the pointer of destroyed locks in panic messages

Suggested by: markj
Reviewed by: kib, markj
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D49332

show more ...

lockmgr/rmlock/rwlock/sx: Make various assertions more robust

Print pointers to locks instead of their names to avoid a nested panic
if the lock object is corrupted.

Reviewed by: markj
Sponsored by

lockmgr/rmlock/rwlock/sx: Make various assertions more robust

Print pointers to locks instead of their names to avoid a nested panic
if the lock object is corrupted.

Reviewed by: markj
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D49331

show more ...

fix build with LOCK_PROFILING but without KDTRACE_HOOKS

Reviewed by: kib, markj
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D47822

sleepqueue: Remove kernel stack swapping support, part 10

- Remove kick_proc0().
- Make the return type of sleepq_broadcast(), sleepq_signal(), etc.,
void.
- Fix up callers.

Tested by: pho
Review

sleepqueue: Remove kernel stack swapping support, part 10

- Remove kick_proc0().
- Make the return type of sleepq_broadcast(), sleepq_signal(), etc.,
void.
- Fix up callers.

Tested by: pho
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D46128

show more ...

kern: move __always_inline to canonical position

Ahead of including inline in __always_inline, move __always_inline to
where inline goes.

Reviewed by: kib, olce
Sponsored by: Dell EMC Isilon
Differ

kern: move __always_inline to canonical position

Ahead of including inline in __always_inline, move __always_inline to
where inline goes.

Reviewed by: kib, olce
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D45708

show more ...

SCHEDULER_STOPPED(): Rely on a global variable

A commit from 2012 (5d7380f8e34f0083, r228424) introduced
'td_stopsched', on the ground that a global variable would cause all
CPUs to have a copy of i

SCHEDULER_STOPPED(): Rely on a global variable

A commit from 2012 (5d7380f8e34f0083, r228424) introduced
'td_stopsched', on the ground that a global variable would cause all
CPUs to have a copy of it in their cache, and consequently of all other
variables sharing the same cache line.

This is really a problem only if that cache line sees relatively
frequent modifications. This was unlikely to be the case back then
because nearby variables are almost never modified as well. In any
case, today we have a new tool at our disposal to ensure that this
variable goes into a read-mostly section containing frequently-accessed
variables ('__read_frequently'). Most of the cache lines covering this
section are likely to always be in every CPU cache. This makes the
second reason stated in the commit message (ensuring the field is in the
same cache line as some lock-related fields, since these are accessed in
close proximity) moot, as well as the second order effect of requiring
an additional line to be present in the cache (the one containing the
new 'scheduler_stopped' boolean, see below).

From a pure logical point of view, whether the scheduler is stopped is
a global state and is certainly not a per-thread quality.

Consequently, remove 'td_stopsched', which immediately frees a byte in
'struct thread'. Currently, the latter's size (and layout) stays
unchanged, but some of the later re-orderings will probably benefit from
this removal. Available bytes at the original position for
'td_stopsched' have been made explicit with the addition of the
'_td_pad0' member.

Store the global state in the new 'scheduler_stopped' boolean, which is
annotated with '__read_frequently'.

Replace uses of SCHEDULER_STOPPED_TD() with SCHEDULER_STOPPER() and
remove the former as it is now unnecessary.

Reviewed by: markj, kib
Approved by: markj (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43572

show more ...

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sx: fixup copy pasto in previous

Spotted by: glebius
Sponsored by: Rubicon Communications, LLC ("Netgate")

sx: unset td_wantedlock around going to sleep

Otherwise it can crash in sleepq_wait_sig -> sleepq_catch_signals ->
sig_ast_checksusp -> thread_suspend_check due to a mutex acquire.

Reported by: pho

sx: unset td_wantedlock around going to sleep

Otherwise it can crash in sleepq_wait_sig -> sleepq_catch_signals ->
sig_ast_checksusp -> thread_suspend_check due to a mutex acquire.

Reported by: pho
Sponsored by: Rubicon Communications, LLC ("Netgate")

show more ...

thread: add td_wantedlock

This enables obtaining lock information threads are actively waiting for
while sampling. Without the change one would only see a bunch of calls
to lock_delay(), where the s

thread: add td_wantedlock

This enables obtaining lock information threads are actively waiting for
while sampling. Without the change one would only see a bunch of calls
to lock_delay(), where the stacktrace often does not reveal what the
lock might be.

Note this is not the same as lock profiling, which only produces data
for cases which wait for locks.

struct thread already has a td_lockname field, but I did not use it
because it has different semantics -- denotes when the thread is off
cpu. At the same time it could not be converted to hold a lock_object
pointer because non-curthread access would no longer be guaranteed to be
safe -- by the time it reads the pointer the lock might have been taken,
released and the object containing it freed.

Sample usage with dtrace:
rm /tmp/out.kern_stacks ; dtrace -x stackframes=100 -n 'profile-997 { @[curthread->td_wantedlock != NULL ? stringof(curthread->td_wantedlock->lo_name) : stringof("\n"), stack()] = count(); }' -o /tmp/out.kern_stacks

This also facilitates addition of lock information to traces produced by
hwpmc.

Note: spinlocks are not supported at the moment.

Sponsored by: Rubicon Communications, LLC ("Netgate")

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

sx: whack set-but-not-used warn in _sx_slock_hard

Sponsored by: Rubicon Communications, LLC ("Netgate")

lockprof: pass lock type as an argument instead of reading the spin flag

Minor style cleanup

We prefer 'while (0)' to 'while(0)' according to grep and stlye(9)'s
space after keyword rule. Remove a few stragglers of the latter.
Many of these usages were inconsistent withi

Minor style cleanup

We prefer 'while (0)' to 'while(0)' according to grep and stlye(9)'s
space after keyword rule. Remove a few stragglers of the latter.
Many of these usages were inconsistent within the file.

MFC After: 3 days
Sponsored by: Netflix

show more ...

locks: push lock_delay_arg_init calls down

Minor cleanup to skip doing them when recursing on locks and so that
they can act on found lock value if need be.

sx: drop spurious volatile keyword

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

locks: fix a long standing bug for primitives with kdtrace but without spinning

In such a case the second argument to lock_delay_arg_init was NULL which was
immediately causing a null pointer deref.

locks: fix a long standing bug for primitives with kdtrace but without spinning

In such a case the second argument to lock_delay_arg_init was NULL which was
immediately causing a null pointer deref.

Since the sructure is only used for spin count, provide a dedicate routine
initializing it.

Reported by: andrew

show more ...

Merge ^/head r358269 through r358399.

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

locks: add default delay struct

Use it for all primitives. This makes everything fit in 8 bytes.

locks: convert delay times to u_short

int is just a waste of space for this purpose.

sleep(9), sleepqueue(9): const'ify wchan pointers

_sleep(9), wakeup(9), sleepqueue(9), et al do not dereference or modify the
channel pointers provided in any way; they are merely used as intptrs in

sleep(9), sleepqueue(9): const'ify wchan pointers

_sleep(9), wakeup(9), sleepqueue(9), et al do not dereference or modify the
channel pointers provided in any way; they are merely used as intptrs into a
dictionary structure to match waiters with wakers. Correctly annotate this
such that _sleep() and wakeup() may be used on const pointers without
invoking ugly patterns like __DECONST(). Plumb const through all of the
underlying sleepqueue bits.

No functional change.

Reviewed by: rlibby
Discussed with: kib, markj
Differential Revision: https://reviews.freebsd.org/D22914

show more ...

sx: check for SX_LOCK_SHARED | SX_LOCK_WRITE_SPINNER when exclusive-locking

First, this removes a spurious difference compared to rw locks.
More importantly though this avoids a trip through sleepq

sx: check for SX_LOCK_SHARED | SX_LOCK_WRITE_SPINNER when exclusive-locking

First, this removes a spurious difference compared to rw locks.
More importantly though this avoids a trip through sleepq code if the lock
happens to be caught in this state.

show more ...