sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

kcov: use __func__ in KASSERT instead of old function name

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

kcov: Disable address and memory sanitizers in get_kinfo()

get_kinfo() is only called from the coverage sanitizer callbacks, which
are similarly uninstrumented.

Sponsored by: The FreeBSD Foundation

kcov: Integrate with KMSAN

- kern_kcov.c needs to be compiled with -fsanitize=kernel-memory when
KMSAN is configured since it calls into various other subsystems.
- Disable address and memory sani

kcov: Integrate with KMSAN

- kern_kcov.c needs to be compiled with -fsanitize=kernel-memory when
KMSAN is configured since it calls into various other subsystems.
- Disable address and memory sanitizers in kcov(4)'s coverage sanitizer
callbacks, as they do not provide useful checking. Moreover, with
KMSAN we may otherwise get false positives since the caller (coverage
sanitizer runtime) is not instrumented.
- Disable KASAN and KMSAN interceptors in subr_coverage.c, as they do
not provide any benefit but do introduce overhead when fuzzing.

Sponsored by: The FreeBSD Foundation

show more ...

Generalize bus_space(9) and atomic(9) sanitizer interceptors

Make it easy to define interceptors for new sanitizer runtimes, rather
than assuming KCSAN. Lay a bit of groundwork for KASAN and KMSAN.

Generalize bus_space(9) and atomic(9) sanitizer interceptors

Make it easy to define interceptors for new sanitizer runtimes, rather
than assuming KCSAN. Lay a bit of groundwork for KASAN and KMSAN.

When a sanitizer is compiled in, atomic(9) and bus_space(9) definitions
in atomic_san.h are used by default instead of the inline
implementations in the platform's atomic.h. These definitions are
implemented in the sanitizer runtime, which includes
machine/{atomic,bus}.h with SAN_RUNTIME defined to pull in the actual
implementations.

No functional change intended.

MFC after: 1 month
Sponsored by: The FreeBSD Foundation

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

Mark COVERAGE and KCOV as part of KCSAN

While not strictly true this stops them from trying to use the KCSAN atomic
hooks and allows these to be compiled into the same kernel.

Sponsored by: Innovat

Mark COVERAGE and KCOV as part of KCSAN

While not strictly true this stops them from trying to use the KCSAN atomic
hooks and allows these to be compiled into the same kernel.

Sponsored by: Innovate UK

show more ...

Merge ^/head r358269 through r358399.

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

Merge ^/head r357270 through r357349.

Remove duplicated empty lines from kern/*.c

No functional changes.

Merge ^/head r356931 through r357118.

Consistently use busy and vm_page_valid() rather than touching page bits
directly. This improves API compliance, asserts, etc.

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd

Consistently use busy and vm_page_valid() rather than touching page bits
directly. This improves API compliance, asserts, etc.

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D23283

show more ...

Merge ^/head r352105 through r352307.

Change synchonization rules for vm_page reference counting.

There are several mechanisms by which a vm_page reference is held,
preventing the page from being freed back to the page allocator. In
pa

Change synchonization rules for vm_page reference counting.

There are several mechanisms by which a vm_page reference is held,
preventing the page from being freed back to the page allocator. In
particular, holding the page's object lock is sufficient to prevent the
page from being freed; holding the busy lock or a wiring is sufficent as
well. These references are protected by the page lock, which must
therefore be acquired for many per-page operations. This results in
false sharing since the page locks are external to the vm_page
structures themselves and each lock protects multiple structures.

Transition to using an atomically updated per-page reference counter.
The object's reference is counted using a flag bit in the counter. A
second flag bit is used to atomically block new references via
pmap_extract_and_hold() while removing managed mappings of a page.
Thus, the reference count of a page is guaranteed not to increase if the
page is unbusied, unmapped, and the object's write lock is held. As
a consequence of this, the page lock no longer protects a page's
identity; operations which move pages between objects are now
synchronized solely by the objects' locks.

The vm_page_wire() and vm_page_unwire() KPIs are changed. The former
requires that either the object lock or the busy lock is held. The
latter no longer has a return value and may free the page if it releases
the last reference to that page. vm_page_unwire_noq() behaves the same
as before; the caller is responsible for checking its return value and
freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is
introduced for use in pmap_extract_and_hold(). It fails if the page is
concurrently being unmapped, typically triggering a fallback to the
fault handler. vm_page_wire() no longer requires the page lock and
vm_page_unwire() now internally acquires the page lock when releasing
the last wiring of a page (since the page lock still protects a page's
queue state). In particular, synchronization details are no longer
leaked into the caller.

The change excises the page lock from several frequently executed code
paths. In particular, vm_object_terminate() no longer bounces between
page locks as it releases an object's pages, and direct I/O and
sendfile(SF_NOCACHE) completions no longer require the page lock. In
these latter cases we now get linear scalability in the common scenario
where different threads are operating on different files.

__FreeBSD_version is bumped. The DRM ports have been updated to
accomodate the KPI changes.

Reviewed by: jeff (earlier version)
Tested by: gallatin (earlier version), pho
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20486

show more ...

MFHead @348740

Sponsored by: The FreeBSD Foundation

kcov depends on eventhandler.h.

MFC after: 3 days

Merge ^/head r344513 through r344548.

MFhead@r344527

Check the index hasn't changed after writing the cmp entry.

If an interrupt fires while writing the cmp entry we may have a partial
entry. Work around this by using atomic_cmpset to set the new inde

Check the index hasn't changed after writing the cmp entry.

If an interrupt fires while writing the cmp entry we may have a partial
entry. Work around this by using atomic_cmpset to set the new index. If it
fails we need to set the previous index value and try again as the entry
may be in an inconsistent state.

This fixes messages similar to the following from syzkaller:
bad comp 224 type 2163727253

Reviewed by: tuexen
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D19287

show more ...

Merge ^/head r344178 through r344512.

Allow the kcov buffer to be mmaped multiple times.

After r344391 this restriction is no longer needed.

Sponsored by: DARPA, AFRL