sleepqueue: Remove kernel stack swapping support, part 10

- Remove kick_proc0().
- Make the return type of sleepq_broadcast(), sleepq_signal(), etc.,
void.
- Fix up callers.

Tested by: pho
Review

sleepqueue: Remove kernel stack swapping support, part 10

- Remove kick_proc0().
- Make the return type of sleepq_broadcast(), sleepq_signal(), etc.,
void.
- Fix up callers.

Tested by: pho
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D46128

show more ...

ktrace: Fix the build when options KTRACE is not configured

MFC after: 1 week
Reported by: John Nielsen <lists@jnielsen.net>

SCHEDULER_STOPPED(): Rely on a global variable

A commit from 2012 (5d7380f8e34f0083, r228424) introduced
'td_stopsched', on the ground that a global variable would cause all
CPUs to have a copy of i

SCHEDULER_STOPPED(): Rely on a global variable

A commit from 2012 (5d7380f8e34f0083, r228424) introduced
'td_stopsched', on the ground that a global variable would cause all
CPUs to have a copy of it in their cache, and consequently of all other
variables sharing the same cache line.

This is really a problem only if that cache line sees relatively
frequent modifications. This was unlikely to be the case back then
because nearby variables are almost never modified as well. In any
case, today we have a new tool at our disposal to ensure that this
variable goes into a read-mostly section containing frequently-accessed
variables ('__read_frequently'). Most of the cache lines covering this
section are likely to always be in every CPU cache. This makes the
second reason stated in the commit message (ensuring the field is in the
same cache line as some lock-related fields, since these are accessed in
close proximity) moot, as well as the second order effect of requiring
an additional line to be present in the cache (the one containing the
new 'scheduler_stopped' boolean, see below).

From a pure logical point of view, whether the scheduler is stopped is
a global state and is certainly not a per-thread quality.

Consequently, remove 'td_stopsched', which immediately frees a byte in
'struct thread'. Currently, the latter's size (and layout) stays
unchanged, but some of the later re-orderings will probably benefit from
this removal. Available bytes at the original position for
'td_stopsched' have been made explicit with the addition of the
'_td_pad0' member.

Store the global state in the new 'scheduler_stopped' boolean, which is
annotated with '__read_frequently'.

Replace uses of SCHEDULER_STOPPED_TD() with SCHEDULER_STOPPER() and
remove the former as it is now unnecessary.

Reviewed by: markj, kib
Approved by: markj (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43572

show more ...

condvar: Fix a user-after-free in _cv_wait() when ktrace is enabled

When a thread wakes up after sleeping on a CV, it must not dereference
the CV structure, as it may already have been freed. At le

condvar: Fix a user-after-free in _cv_wait() when ktrace is enabled

When a thread wakes up after sleeping on a CV, it must not dereference
the CV structure, as it may already have been freed. At least ZFS
relies on this invariant, see commit
c636f94bd2ff15be5b904939872b4bce31456c18 for example.

Thus, when logging context-switch events, copy the wmesg into a stack
buffer while it is still safe to do so, and log that after waking up.

While here, move the initial ktrcsw() call later, after assertions and
the SCHEDULER_STOPPED_TD() condition are checked.

Reported by: syzkaller
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D43450

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

Use sleepq_signal(SLEEPQ_DROP) in cv_signal().

Same as wakeup_one()/wakeup_any() commit before it reduces the lock
hold time and so contention.

MFC after: 1 week

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...

Merge ^/head r313644 through r313895.

Introduce SCHEDULER_STOPPED_TD for use when the thread pointer was already read

Sprinkle in few places.

Merge ^/head r305431 through r305622.

cv: do a lockless check for no waiters in cv_signal and cv_broadcastpri

In case of some consumers like zfs there are no waiters vast majority of
the time

Reviewed by: jhb
MFC after: 1 week

sys/kern: spelling fixes in comments.

No functional change.

MFH

Sponsored by: The FreeBSD Foundation

Rework handling of thread sleeps before timers are working.

Previously, calls to *sleep() and cv_*wait*() immediately returned during
early boot. Instead, permit threads that request a sleep withou

Rework handling of thread sleeps before timers are working.

Previously, calls to *sleep() and cv_*wait*() immediately returned during
early boot. Instead, permit threads that request a sleep without a
timeout to sleep as wakeup() works during early boot. Sleeps with
timeouts are harder to emulate without working timers, so just punt and
panic explicitly if any thread tries to use those before timers are
working. Any threads that depend on timeouts should either wait until
SI_SUB_KICK_SCHEDULER to start or they should use DELAY() until timers
are available.

Until APs are started earlier this should be a no-op as other kthreads
shouldn't get a chance to start running until after timers are working
regardless of when they were created.

Reviewed by: kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D5724

show more ...

Merge ^/head r296007 through r296368.

MFH

Sponsored by: The FreeBSD Foundation

Use SCHEDULER_STOPPED() in cv_*wait*() instead of checking panicstr.

Reviewed by: kib
MFC after: 1 month
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D5516

MFH @r294567

MFH

Sponsored by: The FreeBSD Foundation

Merge ^/head r293430 through r293685.

Prevent cv_waiters wraparound.

r282971 attempted to fix this problem by decrementing cv_waiters after
waking up from sleeping on a condition variable, but this can result in
a use-after-free if the

Prevent cv_waiters wraparound.

r282971 attempted to fix this problem by decrementing cv_waiters after
waking up from sleeping on a condition variable, but this can result in
a use-after-free if the CV is freed before all woken threads have had a
chance to run. Instead, avoid incrementing cv_waiters past INT_MAX, and
have cv_signal() explicitly check for sleeping threads once cv_waiters has
reached this bound.

Reviewed by: jhb
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D4822

show more ...

Revert r282971. It depends on condvar consumers not destroying condvars
until all threads sleeping on a condvar have resumed execution after being
awakened. However, there are cases where that guar

Revert r282971. It depends on condvar consumers not destroying condvars
until all threads sleeping on a condvar have resumed execution after being
awakened. However, there are cases where that guarantee is very hard to
provide.

show more ...

Previously, cv_waiters was only updated by cv_signal or cv_wait. If a
thread awakened due to a time out, then cv_waiters was not decremented.
If INT_MAX threads timed out on a cv without an interveni

Previously, cv_waiters was only updated by cv_signal or cv_wait. If a
thread awakened due to a time out, then cv_waiters was not decremented.
If INT_MAX threads timed out on a cv without an intervening cv_broadcast,
then cv_waiters could overflow. To fix this, have each sleeping thread
decrement cv_waiters when it resumes.

Note that previously cv_waiters was protected by the sleepq chain lock.
However, that lock is not held when threads resume from sleep. In
addition, the interlock is also not always reacquired after resuming
(cv_wait_unlock), nor is it always held by callers of cv_signal() or
cv_broadcast(). Instead, use atomic ops to update cv_waiters. Since
the sleepq chain lock is still held on every increment, it should
still be safe to compare cv_waiters against zero while holding the
lock in the wakeup routines as the only way the race should be lost
would result in extra calls to sleepq_signal() or sleepq_broadcast().

Differential Revision: https://reviews.freebsd.org/D2427
Reviewed by: benno
Reported by: benno (wrap of cv_waiters in the field)
MFC after: 2 weeks

show more ...

Merge from HEAD