vm_pageout: Scan inactive dirty pages less aggressively

Consider a database workload where the bulk of RAM is used for a
fixed-size file-backed cache. Any leftover pages are used for
filesystem cac

vm_pageout: Scan inactive dirty pages less aggressively

Consider a database workload where the bulk of RAM is used for a
fixed-size file-backed cache. Any leftover pages are used for
filesystem caching or anonymous memory. In particular, there is little
memory pressure and the inactive queue is scanned rarely.

Once in a while, the free page count dips a bit below the setpoint,
triggering an inactive queue scan. Since almost all of the memory there
is used by the database cache, the scan encounters only referenced
and/or dirty pages, moving them to the active and laundry queues. In
particular, it ends up completely depleting the inactive queue, even for
a small, non-urgent free page shortage.

This scan might process many gigabytes worth of pages in one go,
triggering VM object lock contention (on the DB cache file's VM object)
and consuming CPU, which can cause application latency spikes.

Observing this behaviour, my observation is that we should abort
scanning once we've encountered many dirty pages without meeting the
shortage. In general we've tried to make the page daemon control loops
avoid large bursts of work, and if a scan fails to turn up clean pages,
there's not much use in moving everything to laundry queue at once. In
particular, pacing this work ensures that the page daemon isn't
frequently acquiring and releasing the VM object lock over long periods,
especially when multiple page daemon threads are in use.

Modify the inactive scan to abort early if we encounter enough dirty
pages without meeting the shortage. If the shortage hasn't been met,
this will trigger shortfall laundering, wherein the laundry thread will
clean as many pages as needed to meet the instantaneous shortfall.
Laundered pages will be placed near the head of the inactive queue, so
will be immediately visible to the page daemon during its next scan of
the inactive queue.

Reviewed by: alc, kib
MFC after: 1 month
Sponsored by: Modirum MDPay
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D48337

show more ...

tmpfs_vnops: move swap work to swap_pager

Two functions in tmpfs_vnops.c use an interface provided by
swap_pager.c. Move most of the implementation of those functions to
swap_pager.c so that they ca

tmpfs_vnops: move swap work to swap_pager

Two functions in tmpfs_vnops.c use an interface provided by
swap_pager.c. Move most of the implementation of those functions to
swap_pager.c so that they can be implemented more effectively, with
access to implementation details of the swap pager.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D47212

show more ...

swap_pager: move scan_all_shadowed, use iterators

Move vm_object_scan_all_shadowed from vm_object.c to swap_pager.c, and
rename it. In the moved function, use vm_page and swblk iterators to
advance

swap_pager: move scan_all_shadowed, use iterators

Move vm_object_scan_all_shadowed from vm_object.c to swap_pager.c, and
rename it. In the moved function, use vm_page and swblk iterators to
advance through the objects. Avoid checking a backing page for
busyness or validity more than once, or when it is beyond the upper
bound of the scan.

Reviewed by: kib, markj
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D47150

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

Make swap_pager_freespace() global

also make it return the count of the swap pages freed, which are not
simultaneously resident in the object.

Reviewed by: markj
Tested by: pho
Sponsored by: The Fr

Make swap_pager_freespace() global

also make it return the count of the swap pages freed, which are not
simultaneously resident in the object.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37097

show more ...

swapoff: add one more variant of the syscall

Requested and reviewed by: brooks
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33343

swapoff(2): add a SWAPOFF_FORCE flag

The flag requests skipping the heuristic which tries to avoid leaving
system with more allocated memory than available from RAM and remanining
swap.

Reviewed by

swapoff(2): add a SWAPOFF_FORCE flag

The flag requests skipping the heuristic which tries to avoid leaving
system with more allocated memory than available from RAM and remanining
swap.

Reviewed by: markj
Discussed with: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33165

show more ...

swapoff(2): replace special device name argument with a structure

For compatibility, add a placeholder pointer to the start of the
added struct swapoff_new_args, and use it to distinguish old vs. ne

swapoff(2): replace special device name argument with a structure

For compatibility, add a placeholder pointer to the start of the
added struct swapoff_new_args, and use it to distinguish old vs. new
style of syscall invocation.

Reviewed by: markj
Discussed with: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33165

show more ...

forward declare struct thread

sys/sysctl.h moved struct thread forward declaration under #ifdef
_KERNEL and so this header fails when included from userland. Add a
forward declaration here.

Fixes:

forward declare struct thread

sys/sysctl.h moved struct thread forward declaration under #ifdef
_KERNEL and so this header fails when included from userland. Add a
forward declaration here.

Fixes: 99eefc727eba
Sponsored by: Netflix

show more ...

Externalize nsw_cluster_max and initialize it early.

GEOM_ELI needs to know the value, cause it will soon have special
memory handling for IO operations associated with swap.

Move initialization to

Externalize nsw_cluster_max and initialize it early.

GEOM_ELI needs to know the value, cause it will soon have special
memory handling for IO operations associated with swap.

Move initialization to swap_pager_init(), which is executed at
SI_SUB_VM, unlike swap_pager_swap_init(), which would be executed
only when a swap is configured. GEOM_ELI might need the value at
SI_SUB_DRIVERS, when disks are tasted by GEOM.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D24400

show more ...

swap_pager: Handle large swap_pager_reserve() requests

This interface is used solely by md(4) when the MD_RESERVE flag is
specified, as in `mdconfig -a -t swap -s 1G -o reserve`. It
pre-allocates s

swap_pager: Handle large swap_pager_reserve() requests

This interface is used solely by md(4) when the MD_RESERVE flag is
specified, as in `mdconfig -a -t swap -s 1G -o reserve`. It
pre-allocates swap blocks for the entire object.

The number of blocks to be reserved is specified as a vm_size_t, but
swp_pager_getswapspace() can allocate at most INT_MAX blocks. vm_size_t
also seems like the incorrect type to use here it refers only to the
size of the VM object, not the size of a mapping. So:
- change the type of "size" in swap_pager_reserve() to vm_pindex_t, and
- clamp the requested number of blocks for a single
swp_pager_getswapspace() call to INT_MAX.

Reported by: syzkaller
Reviewed by: dougm, alc, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31875

show more ...

tmpfs: dynamically register tmpfs pager

Remove OBJT_SWAP_TMPFS. Move tmpfs-specific swap pager bits into
tmpfs_subr.c.

There is no longer any code to directly support tmpfs in sys/vm, most
tmpfs kn

tmpfs: dynamically register tmpfs pager

Remove OBJT_SWAP_TMPFS. Move tmpfs-specific swap pager bits into
tmpfs_subr.c.

There is no longer any code to directly support tmpfs in sys/vm, most
tmpfs knowledge is shared by non-anon swap object type implementation.
The tmpfs-specific methods are provided by registered tmpfs pager, which
inherits from the swap pager.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30168

show more ...

Add pgo_freespace method

Makes the code in vm_object collapse/page_remove cleaner

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: htt

Add pgo_freespace method

Makes the code in vm_object collapse/page_remove cleaner

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30070

show more ...

sysctl vm.objects: report backing object and swap use

For anonymous objects, provide a handle kvo_me naming the object,
and report the handle of the backing object. This allows userspace
to deconst

sysctl vm.objects: report backing object and swap use

For anonymous objects, provide a handle kvo_me naming the object,
and report the handle of the backing object. This allows userspace
to deconstruct the shadow chain. Right now the handle is the address
of the object in KVA, but this is not guaranteed.

For the same anonymous objects, report the swap space used for actually
swapped out pages, in kvo_swapped field. I do not believe that it is
useful to report full 64bit counter there, so only uint32_t value is
returned, clamped to the max.

For kinfo_vmentry, report anonymous object handle backing the entry,
so that the shadow chain for the specific mapping can be deconstructed.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29771

show more ...

Merge ^/head r358000 through r358048.

Remove swblk_t.

It was used only to store the bounds of each swap device. However,
since swblk_t is a signed 32-bit int and daddr_t is a signed 64-bit
int, swp_pager_isondev() may return an invalid

Remove swblk_t.

It was used only to store the bounds of each swap device. However,
since swblk_t is a signed 32-bit int and daddr_t is a signed 64-bit
int, swp_pager_isondev() may return an invalid result if swap devices
are repeatedly added and removed and sw_end for a device ends up
becoming a negative number.

Note that the removed comment about maximum swap size still applies.

Reviewed by: jeff, kib
Tested by: pho
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23666

show more ...

Merge ^/head r325999 through r326131.

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

show more ...

MFH @ r323558.

MFhead@r323343

Make the swap_pager_full variable static.

r290920 removed the use of the variable from vm/vm_pageout.c.

Submitted by: ota@j.email.ne.jp
PR: 221356
MFC after: 1 week

MFhead@r320180

Merge ^/head r319973 through 321382.

Merge ^/head r320994 through r321238.