buf: Add a runningbufclaim() helper

No functional change intended.

Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D47696

vfs: Fix runningspace tuning after maxphys was bumped

The previous maximum value for the upper watermark was based on the old
value of MAXPHYS. Raise it to allow more parallel I/O on large systems.

vfs: Fix runningspace tuning after maxphys was bumped

The previous maximum value for the upper watermark was based on the old
value of MAXPHYS. Raise it to allow more parallel I/O on large systems.

This is still a rather flawed mechanism since it's applied without
regard to the number of filesystems or block devices between which this
mechanism sits, but we might as well bump the limits at this point, as
they haven't been revised in quite a long time.

Reviewed by: imp, kib
MFC after: 2 weeks
Fixes: cd8537910406 ("Make MAXPHYS tunable. Bump MAXPHYS to 1M.")
Differential Revision: https://reviews.freebsd.org/D47398

show more ...

bufwrite(): adjust the comment

The statement about 'do not deadlock there' is false, since this write
might need other writes to finish, which cannot be started due to
runningbufspace.

PR: 282449
R

bufwrite(): adjust the comment

The statement about 'do not deadlock there' is false, since this write
might need other writes to finish, which cannot be started due to
runningbufspace.

PR: 282449
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 3 days

show more ...

bufwrite(): style

Use bool for vp_md. Compactify the calculation.
Explicitly check for non-zero when testing flags.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 3 days

getblk: track "non-sterile" bufobj to avoid bo lock on miss if sterile

This is a scheme to avoid taking the bufobj lock and doing a second
lookup in the case where in getblk we do an unlocked lookup

getblk: track "non-sterile" bufobj to avoid bo lock on miss if sterile

This is a scheme to avoid taking the bufobj lock and doing a second
lookup in the case where in getblk we do an unlocked lookup and find no
buf. Was there really no buf, or were we in the middle of a reassignbuf
race? By tracking any use of reassignbuf with a flag, we can know if
there can't have been a race because there has been no reassignbuf.
Because this scheme is spoiled on the first use of reassignbuf, it is
mostly only beneficial for cases where a certain vnode is never expected
to use dirty bufs at all.

Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D45571

show more ...

getblk: reduce time under bufobj lock

Use the new pctrie combined insert/lookup facility to reduce work and
time under the bufobj interlock when associating a buf with a vnode.

We now do one lookup

getblk: reduce time under bufobj lock

Use the new pctrie combined insert/lookup facility to reduce work and
time under the bufobj interlock when associating a buf with a vnode.

We now do one lookup in the dirty tree and one combined lookup/insert in
the clean tree instead of one lookup in dirty, two in clean, and then an
insert in clean. We also avoid touching the possibly unrelated buf at
the tail of the queue.

Also correct an issue where the actual order of the tail queue depended
on the insertion order due to sign issues.

Reviewed by: kib (previous version), dougm, markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D45395

show more ...

db_show_buffer: minor cleanup

Do some light cleanup to make the output format more consistent for
readability.

Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.

db_show_buffer: minor cleanup

Do some light cleanup to make the output format more consistent for
readability.

Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D45442

show more ...

getblk: fail faster with GB_LOCK_NOWAIT

If we asked not to wait on a lock, and then we failed to get a buf lock
because we would have had to wait, then just return the error. This
avoids taking the

getblk: fail faster with GB_LOCK_NOWAIT

If we asked not to wait on a lock, and then we failed to get a buf lock
because we would have had to wait, then just return the error. This
avoids taking the bufobj lock and a second trip to lockmgr.

Reviewed by: mckusick, kib, markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D45245

show more ...

getblkx(9): be more tolerant but also strict with the buffer size checks

It is possible that on-disk filesystem format causes allocation of
buffers of size larger than maxbcachebuf. Currently, getb

getblkx(9): be more tolerant but also strict with the buffer size checks

It is possible that on-disk filesystem format causes allocation of
buffers of size larger than maxbcachebuf. Currently, getblkx() and
indirectly bufkva_alloc() panic in that situation.

It is more useful to return an error instead, allowing the system to
continue running.

PR: 277414
Reported by: Robert Morris <rtm@lcs.mit.edu>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation

show more ...

Add DEBUG_POISON_POINTER

If you have a pointer which you know points to stale data, you can
fill it with junk so that dereference later will trap

Reviewed by: kib
Sponsored by: Rubicon Communicatio

Add DEBUG_POISON_POINTER

If you have a pointer which you know points to stale data, you can
fill it with junk so that dereference later will trap

Reviewed by: kib
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D40946

show more ...

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

buf: Add sysctl flag CTLFLAG_TUN to loader tunable

The sysctl variable 'vfs.unmapped_buf_allowed' is actually a loader
tunable. Add sysctl flag CTLFLAG_TUN to it so that `sysctl -T` will
report it c

buf: Add sysctl flag CTLFLAG_TUN to loader tunable

The sysctl variable 'vfs.unmapped_buf_allowed' is actually a loader
tunable. Add sysctl flag CTLFLAG_TUN to it so that `sysctl -T` will
report it correctly.

No functional change intended.

Reviewed by: kib, imp
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D42113

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

memdesc: Retire MEMDESC_BIO.

Instead, change memdesc_bio to examine the bio and return a memdesc of
a more generic type describing the data buffer.

Reviewed by: imp
Sponsored by: Chelsio Communicat

memdesc: Retire MEMDESC_BIO.

Instead, change memdesc_bio to examine the bio and return a memdesc of
a more generic type describing the data buffer.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D41029

show more ...

vfs_bio: initialize pctries

bufobj_init depends on fields bo_dirty.bv_root and bo_clean.bv_root
being zeroed on entry and pctrie_init zeroing whatever is passed to
them, and so does not call pctrie_

vfs_bio: initialize pctries

bufobj_init depends on fields bo_dirty.bv_root and bo_clean.bv_root
being zeroed on entry and pctrie_init zeroing whatever is passed to
them, and so does not call pctrie_init for either of them. That fails
if pctrie_init ever changes to do something other that zeroing data,
so add explicit calls to them.

Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D40978

show more ...

tslog: Annotate parts of SYSINIT cpu

Booting an amd64 kernel on Firecracker with 1 CPU and 128 MB of RAM,
SYSINIT cpu takes roughly 2770 us:
* 2280 us in vm_ksubmap_init
* 535 us in kmem_malloc

tslog: Annotate parts of SYSINIT cpu

Booting an amd64 kernel on Firecracker with 1 CPU and 128 MB of RAM,
SYSINIT cpu takes roughly 2770 us:
* 2280 us in vm_ksubmap_init
* 535 us in kmem_malloc
* 450 us in pmap_zero_page
* 1720 us in pmap_growkernel
* 1620 us in pmap_zero_page
* 80 us in bufinit
* 480 us in cpu_setregs
* 430 us in cpu_setregs calling load_cr0

Much of this is hypervisor overhead: load_cr0 is slow because it traps
to the hypervisor, and 99% of the time in pmap_zero_page is spent when
we first touch the page, presumably due to the host Linux kernel
faulting in backing pages one by one.

Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D40327

show more ...

buf: Make the number of pbufs slightly more dynamic

Various subsystems pre-allocate a set of pbufs, allocated to implement
I/O operations. pbuf allocations are transient, unlike most buf
allocation

buf: Make the number of pbufs slightly more dynamic

Various subsystems pre-allocate a set of pbufs, allocated to implement
I/O operations. pbuf allocations are transient, unlike most buf
allocations.

Most subsystems preallocate nswbuf or nswbuf/2 pbufs each. The
preallocation ensures that pbuf allocation will succeed in low memory
conditions, which might help avoid deadlocks. Currently we initialize
nswbuf = min(nbuf / 4, 256).

nbuf/4 > 256 on anything but the smallest systems. For example,
nswbuf is 256 in a VM with 128MB of memory. In this configuration, a
firecracker VM with one CPU preallocates over 900 pbufs. This consumes
2MB of RAM and adds several milliseconds to the kernel's (very small)
boot time.

Scale nswbuf by ncpu in the common case. I think this makes more sense
than scaling by the amount of RAM, since pbuf allocations are transient
and aren't used for caching. With the change, we get nswbuf=256 with 8
CPUs. With fewer than 8 CPUs we'll preallocate fewer pbufs than before,
and with more we'll preallocate more.

Event: BSDCan 2023
Reported by: cperciva
Reviewed by: glebius, kib
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D40216

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

buf: Dynamically allocate per-CPU buffer queues

To reduce static bloat. No functional change intended.

PR: 269572
Reviewed by: mjg, kib, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Founda

buf: Dynamically allocate per-CPU buffer queues

To reduce static bloat. No functional change intended.

PR: 269572
Reviewed by: mjg, kib, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39808

show more ...

buf: Make buf_daemon_shutdown() a no-op after a panic

As in commit 9d7cc536e261a7, there is no need to do anything in this
context.

MFC after: 1 week

buf: Make bufspace_daemon_shutdown() a no-op after a panic

This function doesn't need to do anything in that context, and calling
wakeup() can lead to recursive panics.

Discussed with: mhorne
MFC a

buf: Make bufspace_daemon_shutdown() a no-op after a panic

This function doesn't need to do anything in that context, and calling
wakeup() can lead to recursive panics.

Discussed with: mhorne
MFC after: 1 week

show more ...

allocbuf(): convert direct panic() calls to KASSERT()s

Also do minor style adjustments.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revisio

allocbuf(): convert direct panic() calls to KASSERT()s

Also do minor style adjustments.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38549

show more ...

mi_switch(): clean up switch types and their usage

Overall, this is a non-functional change, except for kernels built with
SCHED_STATS. However, the switch types are useful for communicating the
int

mi_switch(): clean up switch types and their usage

Overall, this is a non-functional change, except for kernels built with
SCHED_STATS. However, the switch types are useful for communicating the
intent of the caller.

1. Ensure that every caller provides a type. In most cases, we upgrade
the basic yield to sched_relinquish() aka SWT_RELINQUISH.
2. The case of sched_bind() is distinct, so add a new switch type SWT_BIND.
3. Remove the two unused types, SWT_PREEMPT and SWT_SLEEPQTIMO.
4. Remove SWT_NONE altogether and assert that callers always provide
a type flag.
5. Reference the mi_switch(9) man page in the comments, as these flags
will be documented there.

Reviewed by: kib, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38184

show more ...

vfs: whack mips remnant

This reverts commit 8ffa01a06199df4d14b56a9261dc2a8b3b156a2f.

Adjust function definition in vfs_bio.c to avoid clang 15 warnings

With clang 15, the following -Werror warning is produced:

sys/kern/vfs_bio.c:3430:11: error: a function declaration without a

Adjust function definition in vfs_bio.c to avoid clang 15 warnings

With clang 15, the following -Werror warning is produced:

sys/kern/vfs_bio.c:3430:11: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
buf_daemon()
^
void

This is because buf_daemon() is declared with a (void) argument list,
but defined with an empty argument list. Make the definition match the
declaration.

MFC after: 3 days

show more ...