swap_pager: clean up pctrie usage

Define wrapper functions for the pctrie operations specific to
swap_pager, to hide some verbose details. Separate the meta_transfer
and meta_free functions into se

swap_pager: clean up pctrie usage

Define wrapper functions for the pctrie operations specific to
swap_pager, to hide some verbose details. Separate the meta_transfer
and meta_free functions into separate functions.

Reviewed by: kib (previous version)
Tested by: pho (previous version)
Differential Revision: https://reviews.freebsd.org/D46315

show more ...

swap_pager_freespace: fix freed count

Function swp_pager_meta_transfer uses 'pindex' as the start address of
the swblk in calculating which page to lookup in order to count freed
pages. However, th

swap_pager_freespace: fix freed count

Function swp_pager_meta_transfer uses 'pindex' as the start address of
the swblk in calculating which page to lookup in order to count freed
pages. However, the lookup for a swblk at 'pindex' or greater may
produce one greater than 'pindex', given by sb->p, and that's the
value that should be used to compute a page adddress.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D46234

show more ...

swap_pager: speedup meta_transfer

Add a parameter to swp_pager_meta_build, for the benefit of
swp_pager_meta_transfer.

swp_pager_meta_transfer calls swp_pager_xfer_source, which may look up
the sam

swap_pager: speedup meta_transfer

Add a parameter to swp_pager_meta_build, for the benefit of
swp_pager_meta_transfer.

swp_pager_meta_transfer calls swp_pager_xfer_source, which may look up
the same trie entry twice - first, by calling sw_pager_meta_lookup,
and then as the first step in swp_pager_meta_build. A boolean
parameter to swp_pager_meta_build tells that function not to replace a
previously assigned swapblk with a new one, and setting it in this
call makes the first meta_lookup call unnecessary.

swp_pager_meta_transfer calls swp_pager_xfer_source, which may release
and reacquire the source object write lock, because the call to
swp_pager_meta_build may acquire and then release the destination
object write block. But it probably doesn't, so fiddling with the
source object write block was probably unnecessary. This boolean
parameter to swp_pager_meta_build tells it to return immediately if
memory allocation problems are about to require a lock
release/reacquisitiion, so that the caller can release/reacquire the
source object write lock only if truly necessary, around a second call
the swp_pager_meta_build with that boolean parameter not set. This
should make manipulation of the source object write lock rarer.

Reviewed by: alc, kib (previous version)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D45781

show more ...

swap_pager: cleanup swapoff_object

Function swap_pager_swapoff_object calls vm_pager_unswapped (via
swp_pager_force_dirty) for every page that must be unswapped. That
means that there's an unneeded

swap_pager: cleanup swapoff_object

Function swap_pager_swapoff_object calls vm_pager_unswapped (via
swp_pager_force_dirty) for every page that must be unswapped. That
means that there's an unneeded check for lock ownership (the caller
always owns it), a needless PCTRIE_LOOKUP (the caller has already
found it), a call to free one page of swap space only, and a check to
see if all blocks are empty, when the caller usually knows that the
check is useless.

Isolate the essential part, needed however swap_pager_unswapped is
invoked, into a smaller function swap_pager_unswapped_acct. From
swapoff_object, invoke swp_pager_update_freerange for each appropriate
page, so that there are potentially fewer calls to
swp_pager_freeswapspace. Consider freeing a set of blocks (a struct
swblk) only after having invalidated all those blocks.

Replace the doubly-nested loops with a single loop, and refetch and
rescan a swblk only when the object write lock has been released and
reacquired.

After getting a page from swap, dirty it immediately to address a race
condition observed by @kib.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D45668

show more ...

Revert "swap_pager: small improvement to find_least"

This reverts commit dd0e5c02ab13b9eb240d42a71a8f41a8b036bd33.

pctrie: avoid typecast

Have PCTRIE_RECLAIM_CALLBACK typecast one function pointer type to
another, to relieve the writer of the call back function from having
to cast its first argument from void* t

pctrie: avoid typecast

Have PCTRIE_RECLAIM_CALLBACK typecast one function pointer type to
another, to relieve the writer of the call back function from having
to cast its first argument from void* to member type.

Reviewed by: rlibby
Differential Revision: https://reviews.freebsd.org/D45586

show more ...

swap_pager: pctrie_reclaim_cb in meta_free_all

Replace the lookup-remove loop in swp_pager_meta_free_all with a call
to SWAP_PCTRIE_RECLAIM_CALLBACK, to eliminate repeated trie searches.

Reviewed b

swap_pager: pctrie_reclaim_cb in meta_free_all

Replace the lookup-remove loop in swp_pager_meta_free_all with a call
to SWAP_PCTRIE_RECLAIM_CALLBACK, to eliminate repeated trie searches.

Reviewed by: rlibby
Differential Revision: https://reviews.freebsd.org/D45583

show more ...

swap_pager: add new page range struct

Define a page_range struct to pair up the two values passed to
freerange functions. Have swp_pager_freeswapspace also take a
page_range argument rather than a p

swap_pager: add new page range struct

Define a page_range struct to pair up the two values passed to
freerange functions. Have swp_pager_freeswapspace also take a
page_range argument rather than a pair of arguments.

In swp_pager_meta_free_all, drop a needless test and use a new
helper function to do the cleanup for each swap block.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D45562

show more ...

swap_pager: small improvement to find_least

Drop an unneeded test, a branch and a needless computation to save a
few instructions.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.or

swap_pager: small improvement to find_least

Drop an unneeded test, a branch and a needless computation to save a
few instructions.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D45558

show more ...

swap-like pagers: assert that writemapping decrease does not pass zero

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews

swap-like pagers: assert that writemapping decrease does not pass zero

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D45119

show more ...

swap_pager: Unbusy readahead pages after an I/O error

The swap pager itself allocates readahead pages, so should take care to
unbusy them after a read error, just as it does in the non-error case.

swap_pager: Unbusy readahead pages after an I/O error

The swap pager itself allocates readahead pages, so should take care to
unbusy them after a read error, just as it does in the non-error case.

PR: 277538
Reviewed by: olce, dougm, alc, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D44646

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

swap_pager: Fix a race in swap_pager_swapoff_object()

When we disable swapping to a device, we scan the full VM object list
looking for objects with swap trie nodes that reference the device in
ques

swap_pager: Fix a race in swap_pager_swapoff_object()

When we disable swapping to a device, we scan the full VM object list
looking for objects with swap trie nodes that reference the device in
question. The pages corresponding to those nodes are paged in.

While paging in, we drop the VM object lock. Moreover, we do not hold a
reference for the object; swap_pager_swapoff_object() merely bumps the
paging-in-progress counter. vm_object_terminate() waits for this
counter to drain before proceeding and freeing pages.

However, swap_pager_swapoff_object() decrements the counter before
re-acquiring the VM object lock, which means that vm_object_terminate()
can race to acquire the lock and free the pages. Then,
swap_pager_swapoff_object() ends up unbusying a freed page. Fix the
problem by acquiring the lock before waking up sleepers.

PR: 273610
Reported by: Graham Perrin <grahamperrin@gmail.com>
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D42029

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

vm: fix a number of functions to match the expected prototypes

Noticed while attempting to make boolean_t unsigned: some vm-related
function declarations and defintions were using boolean_t where th

vm: fix a number of functions to match the expected prototypes

Noticed while attempting to make boolean_t unsigned: some vm-related
function declarations and defintions were using boolean_t where they
should have used int, and vice versa.

MFC after: 1 week
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D39753

show more ...

Provide consistent prototype for swp_pager_meta_free()

This should fix 32bit build breakage.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Make swap_pager_freespace() global

also make it return the count of the swap pages freed, which are not
simultaneously resident in the object.

Reviewed by: markj
Tested by: pho
Sponsored by: The Fr

Make swap_pager_freespace() global

also make it return the count of the swap pages freed, which are not
simultaneously resident in the object.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37097

show more ...

swap_pager_find_least(): assert that the function is called on the right object type

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: h

swap_pager_find_least(): assert that the function is called on the right object type

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37024

show more ...

swap_pager: style, wrap long lines

Reviewed by: brooks, imp (previous version)
Discussed with: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.fr

swap_pager: style, wrap long lines

Reviewed by: brooks, imp (previous version)
Discussed with: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D36540

show more ...

vm_overcommit: put into __read_mostly section

Suggested by: mjg
Reviewed by: brooks, imp (previous version)
Discussed with: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential

vm_overcommit: put into __read_mostly section

Suggested by: mjg
Reviewed by: brooks, imp (previous version)
Discussed with: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D36540

show more ...

vm: make vm.overcommit available externally

Reviewed by: brooks, imp (previous version)
Discussed with: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://r

vm: make vm.overcommit available externally

Reviewed by: brooks, imp (previous version)
Discussed with: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D36540

show more ...

swap_pager: Reduce the scope of the object lock in putpages

We don't need to hold the object lock while allocating swap space, so
don't.

Reviewed by: dougm, kib, markj
MFC after: 1 week
Differentia

swap_pager: Reduce the scope of the object lock in putpages

We don't need to hold the object lock while allocating swap space, so
don't.

Reviewed by: dougm, kib, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35839

show more ...

vm_object: Release object swap charge in the swap pager destructor

With the removal of OBJT_DEFAULT, we can simply handle this in
swap_pager_dealloc(). No functional change intended.

Suggested by:

vm_object: Release object swap charge in the swap pager destructor

With the removal of OBJT_DEFAULT, we can simply handle this in
swap_pager_dealloc(). No functional change intended.

Suggested by: alc
Reviewed by: alc, kib
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35787

show more ...

swap_pager: Removing handling for objects with OBJ_SWAP clear

With the removal of OBJT_DEFAULT, we can assume that pager operations
provide an object with OBJ_SWAP set. Also, we do not need to conv

swap_pager: Removing handling for objects with OBJ_SWAP clear

With the removal of OBJT_DEFAULT, we can assume that pager operations
provide an object with OBJ_SWAP set. Also, we do not need to convert
objects from type OBJT_DEFAULT. Thus, remove checks for OBJ_SWAP and
remove code which modifies the object type. In some places, replace the
check for OBJ_SWAP with a check for whether any swap blocks are
assigned.

Reviewed by: alc, kib
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35786

show more ...

vm: Use __diagused for variables only used in KASSERT().