kthread: Set *tdptr earlier in kproc_kthread_add()

See commit ae77041e0714 ("kthread: Set *newtdp earlier in
kthread_add1()") for details. That commit was incomplete since
g_init()'s first call to

kthread: Set *tdptr earlier in kproc_kthread_add()

See commit ae77041e0714 ("kthread: Set *newtdp earlier in
kthread_add1()") for details. That commit was incomplete since
g_init()'s first call to kproc_kthread_add() will cause
kproc_kthread_add() to take the `*procptr == NULL` branch, which avoids
kthread_create().

To ensure that the thread pointer is initialized before the thread
starts running, we have to start the kernel process with RFSTOPPED.
We could perhaps go further and use RFSTOPPED only when tdptr != NULL,
but it's probably better to have consistent behaviour.

Reviewed by: olce, kib
Reported by: syzbot+e91e798f3c088215ace6@syzkaller.appspotmail.com
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D44927

show more ...

kthread: Set *newtdp earlier in kthread_add1()

syzbot reported a single boot-time crash in g_event_procbody(), a page
fault when dereferencing g_event_td. g_event_td is initialized by the
kproc_kth

kthread: Set *newtdp earlier in kthread_add1()

syzbot reported a single boot-time crash in g_event_procbody(), a page
fault when dereferencing g_event_td. g_event_td is initialized by the
kproc_kthread_add() call which creates the GEOM event thread:

kproc_kthread_add(g_event_procbody, NULL, &g_proc, &g_event_td,
RFHIGHPID, 0, "geom", "g_event");

I believe that the caller of kproc_kthread_add() was preempted after
adding the new thread to the scheduler, and before setting *newtdp,
which is equal to g_event_td. Thus, since the first action of the GEOM
event thread is to lock itself, it ended up dereferencing a NULL
pointer.

Fix the problem simply by initializing *newtdp earlier. I see no harm
in that, and it matches kproc_create1(). The scheduler provides
sufficient synchronization to ensure that the store is visible to the
new thread, wherever it happens to run.

Reported by: syzbot+5397f4d39219b85a9409@syzkaller.appspotmail.com
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D42986

show more ...

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

kthread_add(): do not allow to attach the thread to a dead or dying process

Reported by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

kern_kthread: fork1() does not handle locked Giant

fork1() does not behave if called under Giant. For instance, it might
need to call thread_suspend_check() which explicitly verifies that Giant
is

kern_kthread: fork1() does not handle locked Giant

fork1() does not behave if called under Giant. For instance, it might
need to call thread_suspend_check() which explicitly verifies that Giant
is not locked. On the other hand, the kthread KPI is often called from
SYSINIT() which is still Giant-locked.

Correct this by dropping Giant in kthread_add() and kproc_create().

Reported by: pho
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D41694

show more ...

kern_kthread.c: some style

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D41694

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

Update comments referencing create_thread()

The equivalent function is now named thread_create(). Mention
kthread_add() where it is also relevant.

Reviewed by: kib, markj
MFC after: 3 days
Sponsore

Update comments referencing create_thread()

The equivalent function is now named thread_create(). Mention
kthread_add() where it is also relevant.

Reviewed by: kib, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38180

show more ...

umtx: Split umtx.h on two counterparts.

To prevent umtx.h polluting by future changes split it on two headers:
umtx.h - ABI header for userspace;
umtxvar.h - the kernel staff.

While here fix umtx_k

umtx: Split umtx.h on two counterparts.

To prevent umtx.h polluting by future changes split it on two headers:
umtx.h - ABI header for userspace;
umtxvar.h - the kernel staff.

While here fix umtx_key_match style.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D31248
MFC after: 2 weeks

show more ...

Set TDP_KTHREAD before calling cpu_fork() and cpu_copy_thread().

This permits these routines to use special logic for initializing MD
kthread state.

For the kproc case, this required moving the log

Set TDP_KTHREAD before calling cpu_fork() and cpu_copy_thread().

This permits these routines to use special logic for initializing MD
kthread state.

For the kproc case, this required moving the logic to set these flags
from kproc_create() into do_fork().

Reviewed by: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29207

show more ...

Provide ABI modules hooks for process exec/exit and thread exit.

Exec and exit are same as corresponding eventhandler hooks.

Thread exit hook is called somewhat earlier, while thread is still
owned

Provide ABI modules hooks for process exec/exit and thread exit.

Exec and exit are same as corresponding eventhandler hooks.

Thread exit hook is called somewhat earlier, while thread is still
owned by the process and enough context is available. Note that the
process lock is owned when the hook is called.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D27309

show more ...

thread: rework tidhash vs proc lock interaction

Apart from minor clean up this gets rid of proc unlock/lock cycle on thread
exit to work around LOR against tidhash lock.

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

Do a lockless check in kthread_suspend_check

Otherwise an idle system running lockstat sleep 10 reports contention on
process lock comming from bufdaemon.

While here fix a style nit.

Merge ^/head r357270 through r357349.

Remove duplicated empty lines from kern/*.c

No functional changes.

schedlock 1/4

Eliminate recursion from most thread_lock consumers. Return from
sched_add() without the thread_lock held. This eliminates unnecessary
atomics and lock word loads as well as reducing

schedlock 1/4

Eliminate recursion from most thread_lock consumers. Return from
sched_add() without the thread_lock held. This eliminates unnecessary
atomics and lock word loads as well as reducing the hold time for
scheduler locks. This will eventually allow for lockless remote adds.

Discussed with: kib
Reviewed by: jhb
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D22626

show more ...

Merge ^/head r340427 through r340868.

proc: always store parent pid in p_oppid

Doing so removes the dependency on proctree lock from sysctl process list
export which further reduces contention during poudriere -j 128 runs.

Reviewed by:

proc: always store parent pid in p_oppid

Doing so removes the dependency on proctree lock from sysctl process list
export which further reduces contention during poudriere -j 128 runs.

Reviewed by: kib (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17825

show more ...

hwpmc: log name->pid, name->tid mappings

By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid

hwpmc: log name->pid, name->tid mappings

By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid was which when the sample was taken.

% pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log

% pmc filter -x -T idle pmc.log pmc-noidle.log

show more ...

Implement several enhancements to NUMA policies.

Add a new "interleave" allocation policy which stripes pages across
domains with a stride or width keeping contiguity within a multi-page
region.

Mo

Implement several enhancements to NUMA policies.

Add a new "interleave" allocation policy which stripes pages across
domains with a stride or width keeping contiguity within a multi-page
region.

Move the kernel to the dedicated numbered cpuset #2 making it possible
to assign kernel threads and memory policy separately from user. This
also eliminates the need for the complicated interrupt binding code.

Add a sysctl API for viewing and manipulating domainsets. Refactor some
of the cpuset_t manipulation code using the generic bitset type so that
it can be used for both. This probably belongs in a dedicated subr file.

Attempt to improve the include situation.

Reviewed by: kib
Discussed with: jhb (cpuset parts)
Tested by: pho (before review feedback)
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14839

show more ...

Merge ^/head r327341 through r327623.

Instrument thread creations for the the benefit of the TSLOG framework.
This assists in tracking time spent while the boot is being "held" waiting
for something to happen.

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...