pbuf_ctor(): Stop using LK_NOWAIT, use LK_NOWITNESS

The LK_NOWAIT was added to suppress a witness warning, but LK_NOWITNESS
is more what we mean. This makes pbuf_ctor() more consistent with
buf_all

pbuf_ctor(): Stop using LK_NOWAIT, use LK_NOWITNESS

The LK_NOWAIT was added to suppress a witness warning, but LK_NOWITNESS
is more what we mean. This makes pbuf_ctor() more consistent with
buf_alloc(), although, unlike buf_alloc(), for pbuf there should not be
any danger of a wild locker relying on the type stability of the buf to
attempt a lock. That is, this is essentially cosmetic.

Relevant history:
- 531f8cfea06b Use dedicated lock name for pbufs
- 5875b94c7493 buf_alloc(): lock the buffer with LK_NOWAIT
- c9e023541aef pbuf_ctor(): lock the buffer with LK_NOWAIT
- 1fb00c8f1060 buf_alloc(): Stop using LK_NOWAIT, use LK_NOWITNESS

Reviewed by: rew, kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D45360

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

vm: fix a number of functions to match the expected prototypes

Noticed while attempting to make boolean_t unsigned: some vm-related
function declarations and defintions were using boolean_t where th

vm: fix a number of functions to match the expected prototypes

Noticed while attempting to make boolean_t unsigned: some vm-related
function declarations and defintions were using boolean_t where they
should have used int, and vice versa.

MFC after: 1 week
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D39753

show more ...

vm_pager_allocate(): override resulting object type

For dynamically allocated pager type, which inherits the parent's alloc
method, type of the returned object is set to the parent's type
otherwise.

vm_pager_allocate(): override resulting object type

For dynamically allocated pager type, which inherits the parent's alloc
method, type of the returned object is set to the parent's type
otherwise.

Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37097

show more ...

vm_pager: add method to veto page allocation

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37097

vm_pager: add methods for page insertion and removal notifications

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.fre

vm_pager: add methods for page insertion and removal notifications

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37097

show more ...

sys/vm: Add TSLOG to some functions

The functions pbuf_init, kva_alloc, and keg_alloc_slab are significant
contributors to the kernel boot time when FreeBSD boots inside the
Firecracker VMM. Instru

sys/vm: Add TSLOG to some functions

The functions pbuf_init, kva_alloc, and keg_alloc_slab are significant
contributors to the kernel boot time when FreeBSD boots inside the
Firecracker VMM. Instrument them so they show up on flamecharts.

show more ...

vm_pager: Remove the default pager

It's unused now. Keep the OBJ_DEFAULT identifier, but make it an alias
of OBJT_SWAP for the benefit of out-of-tree code.

Reviewed by: alc, imp, kib
Tested by: ph

vm_pager: Remove the default pager

It's unused now. Keep the OBJ_DEFAULT identifier, but make it an alias
of OBJT_SWAP for the benefit of out-of-tree code.

Reviewed by: alc, imp, kib
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35790

show more ...

pbuf_ctor(): lock the buffer with LK_NOWAIT

This LOR happens when reading from a file backed MD device:

lock order reversal:
1st 0xfffffe00431eaac0 pbufwait (pbufwait, lockmgr) @ /cobra/src/sys/vm

pbuf_ctor(): lock the buffer with LK_NOWAIT

This LOR happens when reading from a file backed MD device:

lock order reversal:
1st 0xfffffe00431eaac0 pbufwait (pbufwait, lockmgr) @ /cobra/src/sys/vm/vm_pager.c:471
2nd 0xfffff80003f17930 ufs (ufs, lockmgr) @ /cobra/src/sys/dev/md/md.c:977
lock order pbufwait -> ufs attempted at:
#0 0xffffffff80c78ead at witness_checkorder+0xbdd
#1 0xffffffff80bd6a52 at lockmgr_lock_flags+0x182
#2 0xffffffff80f52d5c at ffs_lock+0x6c
#3 0xffffffff80d0f3f4 at _vn_lock+0x54
#4 0xffffffff80708629 at mdstart_vnode+0x499
#5 0xffffffff807060ec at md_kthread+0x20c
#6 0xffffffff80bbfcd0 at fork_exit+0x80
#7 0xffffffff810b809e at fork_trampoline+0xe

This LOR was previously blessed by witness before commit 531f8cfea06b
("Use dedicated lock name for pbufs").

Instead of blessing ufs and pbufwait, use LK_NOWAIT to prevent recording
the lock order. LK_NOWAIT will be a nop here as the lock is dropped in
pbuf_dtor(). The takes the same approach as 5875b94c7493 ("buf_alloc():
lock the buffer with LK_NOWAIT").

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D34183

show more ...

Use dedicated lock name for pbufs

Also remove a pointer to array variable, use array address directly.

Reviewed by: markj, mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1

Use dedicated lock name for pbufs

Also remove a pointer to array variable, use array address directly.

Reviewed by: markj, mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34072

show more ...

vm_pager: Optimize an assertion

Obtained from: jeff (object_concurrency patches)
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32946

amd64: Populate the KMSAN shadow maps and integrate with the VM

- During boot, allocate PDP pages for the shadow maps. The region above
KERNBASE is currently not shadowed.
- Create a dummy shadow

amd64: Populate the KMSAN shadow maps and integrate with the VM

- During boot, allocate PDP pages for the shadow maps. The region above
KERNBASE is currently not shadowed.
- Create a dummy shadow for the vm page array. For now, this array is
not protected by the shadow map to help reduce kernel memory usage.
- Grow shadows when growing the kernel map.
- Increase the default kernel stack size when KMSAN is enabled. As with
KASAN, sanitizer instrumentation appears to create stack frames large
enough that the default value is not sufficient.
- Disable UMA's use of the direct map when KMSAN is configured. KMSAN
cannot validate the direct map.
- Disable unmapped I/O when KMSAN configured.
- Lower the limit on paging buffers when KMSAN is configured. Each
buffer has a static MAXPHYS-sized allocation of KVA, which in turn
eats 2*MAXPHYS of space in the shadow map.

Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31295

show more ...

tmpfs: dynamically register tmpfs pager

Remove OBJT_SWAP_TMPFS. Move tmpfs-specific swap pager bits into
tmpfs_subr.c.

There is no longer any code to directly support tmpfs in sys/vm, most
tmpfs kn

tmpfs: dynamically register tmpfs pager

Remove OBJT_SWAP_TMPFS. Move tmpfs-specific swap pager bits into
tmpfs_subr.c.

There is no longer any code to directly support tmpfs in sys/vm, most
tmpfs knowledge is shared by non-anon swap object type implementation.
The tmpfs-specific methods are provided by registered tmpfs pager, which
inherits from the swap pager.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30168

show more ...

vm: Add KPI to dynamically register pagers

Pager is allowed to inherit part of its implementation from the existing
pager, which is done by copying non-NULL virtual method slots.

Reviewed by: markj

vm: Add KPI to dynamically register pagers

Pager is allowed to inherit part of its implementation from the existing
pager, which is done by copying non-NULL virtual method slots.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30168

show more ...

vm_object_kvme_type(): reimplement by embedding kvme_type into pagerops

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://review

vm_object_kvme_type(): reimplement by embedding kvme_type into pagerops

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30168

show more ...

Constify vm_pager-related virtual tables.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30070

Add OBJT_SWAP_TMPFS pager

This is OBJT_SWAP pager, specialized for tmpfs. Right now, both swap pager
and generic vm code have to explicitly handle swap objects which are tmpfs
vnode v_object, in th

Add OBJT_SWAP_TMPFS pager

This is OBJT_SWAP pager, specialized for tmpfs. Right now, both swap pager
and generic vm code have to explicitly handle swap objects which are tmpfs
vnode v_object, in the special ways. Replace (almost) all such places with
proper methods.

Since VM still needs a notion of the 'swap object', regardless of its
use, add yet another type-classification flag OBJ_SWAP. Set it in
vm_object_allocate() where other type-class flags are set.

This change almost completely eliminates the knowledge of tmpfs from VM,
and opens a way to make OBJT_SWAP_TMPFS loadable from tmpfs.ko.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30070

show more ...

pagertab: use designated initializers

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30070

Add pgo_getvp method

This eliminates the staircase of conditions in vm_map_entry_set_vnode_text().

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Different

Add pgo_getvp method

This eliminates the staircase of conditions in vm_map_entry_set_vnode_text().

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30070

show more ...

Add pgo_mightbedirty method

Used to implement vm_object_mightbedirty()

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://review

Add pgo_mightbedirty method

Used to implement vm_object_mightbedirty()

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30070

show more ...

vm_pager: add pgo_set_writeable_dirty method

specialized for swap and vnode pagers, and used to implement
vm_object_set_writeable_dirty().

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBS

vm_pager: add pgo_set_writeable_dirty method

specialized for swap and vnode pagers, and used to implement
vm_object_set_writeable_dirty().

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30070

show more ...

Make MAXPHYS tunable. Bump MAXPHYS to 1M.

Replace MAXPHYS by runtime variable maxphys. It is initialized from
MAXPHYS by default, but can be also adjusted with the tunable kern.maxphys.

Make b_pag

Make MAXPHYS tunable. Bump MAXPHYS to 1M.

Replace MAXPHYS by runtime variable maxphys. It is initialized from
MAXPHYS by default, but can be also adjusted with the tunable kern.maxphys.

Make b_pages[] array in struct buf flexible. Size b_pages[] for buffer
cache buffers exactly to atop(maxbcachebuf) (currently it is sized to
atop(MAXPHYS)), and b_pages[] for pbufs is sized to atop(maxphys) + 1.
The +1 for pbufs allow several pbuf consumers, among them vmapbuf(),
to use unaligned buffers still sized to maxphys, esp. when such
buffers come from userspace (*). Overall, we save significant amount
of otherwise wasted memory in b_pages[] for buffer cache buffers,
while bumping MAXPHYS to desired high value.

Eliminate all direct uses of the MAXPHYS constant in kernel and driver
sources, except a place which initialize maxphys. Some random (and
arguably weird) uses of MAXPHYS, e.g. in linuxolator, are converted
straight. Some drivers, which use MAXPHYS to size embeded structures,
get private MAXPHYS-like constant; their convertion is out of scope
for this work.

Changes to cam/, dev/ahci, dev/ata, dev/mpr, dev/mpt, dev/mvs,
dev/siis, where either submitted by, or based on changes by mav.

Suggested by: mav (*)
Reviewed by: imp, mav, imp, mckusick, scottl (intermediate versions)
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D27225

show more ...

Merge ^/head r358239 through r358262.

sys/vm: quiet -Wwrite-strings

Discussed with: kib
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D23796