The "lookup_is_valid" field is used as a "bool". Make it one.

Convert vm_fault_hold()'s Boolean variables that are only used
internally to "bool". Add a comment describing why the one
remaining "b

The "lookup_is_valid" field is used as a "bool". Make it one.

Convert vm_fault_hold()'s Boolean variables that are only used
internally to "bool". Add a comment describing why the one
remaining "boolean_t" was not converted.

Reviewed by: kib
MFC after: 8 days

show more ...

With one exception, "hardfault" is used like a "bool". Change that
exception and make it a "bool".

Reviewed by: kib
MFC after: 7 days

Add one more use of unlock_vp().

Discussed with: kib
X-MFC With: r308094

Add unlock_vp() helper.
Trim space.

Discussed with: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Merge ^/head r307383 through r307735.

If vm_fault_hold(9) finds that fs.m is wired, do not free it after a
pager error, leave the page to the wire owner. E.g. the page might be
a part of the invalidated buffer.

Reported and tested by:

If vm_fault_hold(9) finds that fs.m is wired, do not free it after a
pager error, leave the page to the wire owner. E.g. the page might be
a part of the invalidated buffer.

Reported and tested by: pho
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D8197

show more ...

Merge ^/head r306906 through r307382.

Plug a potential vnode lock leak in vm_fault_hold().

Reviewed by: alc, kib
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8242

Add a comment describing the 'fast path' that was introduced in r270011.

Reviewed by: kib
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Break up vm_fault()'s implementation of the read-ahead and delete-behind
optimizations into two distinct pieces. The first piece consists of the
code that should only be performed once per page faul

Break up vm_fault()'s implementation of the read-ahead and delete-behind
optimizations into two distinct pieces. The first piece consists of the
code that should only be performed once per page fault and requires the map
to be locked. The second piece consists of the code that should be
performed each time a pager is called on an object in the shadow chain.
(This second piece expects the map to be unlocked.)

Previously, the entire implementation could be executed multiple times.
Moreover, the second and subsequent executions would occur with the map
unlocked. Usually, the ensuing unsynchronized accesses to the map were
harmless because the map was not changing. Nonetheless, it was possible for
a use-after-free error to occur, where vm_fault() wrote to a freed map
entry. This change corrects that problem.

Reported by: avg
Reviewed by: kib
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

show more ...

Change the type of the map entry's next_read field from a vm_pindex_t to a
vm_offset_t. (This field is used to detect sequential access to the virtual
address range represented by the map entry.) T

Change the type of the map entry's next_read field from a vm_pindex_t to a
vm_offset_t. (This field is used to detect sequential access to the virtual
address range represented by the map entry.) There are three reasons to
make this change. First, a vm_offset_t is smaller on 32-bit architectures.
Consequently, a struct vm_map_entry is now smaller on 32-bit architectures.
Second, a vm_offset_t can be written atomically, whereas it may not be
possible to write a vm_pindex_t atomically on a 32-bit architecture. Third,
using a vm_pindex_t makes the next_read field dependent on which object in
the shadow chain is being read from.

Replace an "XXX" comment.

Reviewed by: kib
Approved by: re (gjb)
Sponsored by: EMC / Isilon Storage Division

show more ...

Change type of the 'dead' variable to boolean.

Requested by: alc
MFC after: 1 week
Approved by: re (gjb)

If the vm_fault() handler raced with the vm_object_collapse()
sleepable scan, iteration over the shadow chain looking for a page
could find an OBJ_DEAD object. Such state of the mapping is only
tran

If the vm_fault() handler raced with the vm_object_collapse()
sleepable scan, iteration over the shadow chain looking for a page
could find an OBJ_DEAD object. Such state of the mapping is only
transient, the dead object will be terminated and removed from the
chain shortly. We must not return KERN_PROTECTION_FAILURE unless the
object type is changed to OBJT_DEAD in the chain, indicating that
paging on this address is really impossible. Returning
KERN_PROTECTION_FAILURE prematurely causes spurious SIGSEGV delivered
to processes, or kernel accesses to UVA spuriously failing with
EFAULT.

If the object with OBJ_DEAD flag is found, only return
KERN_PROTECTION_FAILURE when object type is already OBJT_DEAD.
Otherwise, sleep a tick and retry the fault handling.

Ideally, we would wait until the OBJ_DEAD flag is resolved, e.g. by
waiting until the paging on this object is finished. But to do so, we
need to reference the dead object, while vm_object_collapse() insists
on owning the final reference on the collapsed object. This could be
fixed by e.g. changing the assert to shared reference release between
vm_fault() and vm_object_collapse(), but it seems to be too much
complications for rare boundary condition.

PR: 204426
Tested by: pho
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
X-Differential revision: https://reviews.freebsd.org/D6085
MFC after: 2 weeks
Approved by: re (gjb)

show more ...

Use vm_page_replace_checked() instead of vm_page_rename() for implementing
optimized copy-on-write faults. This has two advantages: (1) one less radix
tree operation is performed and (2) vm_page_rep

Use vm_page_replace_checked() instead of vm_page_rename() for implementing
optimized copy-on-write faults. This has two advantages: (1) one less radix
tree operation is performed and (2) vm_page_replace_checked() cannot fail,
making the code simpler.

Submitted by: Ryan Libby
Reviewed by: kib
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D4478

show more ...

Correct an error in a comment: One of the conditions for page allocation
is actually the opposite of that stated in the comment.

Remove an unnecessary assignment. Use an assertion to document the f

Correct an error in a comment: One of the conditions for page allocation
is actually the opposite of that stated in the comment.

Remove an unnecessary assignment. Use an assertion to document the fact
that no assignment is needed.

Rewrite another comment to clarify that the page is not completely valid.

Reviewed by: kib

show more ...

When descending a shadow chain of objects, it makes no sense to update
the current offset (spelled: "fs.pindex") until it is known whether a
backing object exists. In fact, if not for the fact that

When descending a shadow chain of objects, it makes no sense to update
the current offset (spelled: "fs.pindex") until it is known whether a
backing object exists. In fact, if not for the fact that the backing
object offset is zero when there is no backing object, this update would
produce a broken offset.

Reviewed by: kib

show more ...

Clean up the handling of errors from vm_pager_get_pages(). Mostly, this
cleanup consists of fixes to comments. However, there is one change to
code: Remove special-case handling of errors involving

Clean up the handling of errors from vm_pager_get_pages(). Mostly, this
cleanup consists of fixes to comments. However, there is one change to
code: Remove special-case handling of errors involving the kernel map.
We do not perform I/O on the kernel map, so there is no need for this
special case.

Reviewed by: kib (an earlier version)

show more ...

MFH

Sponsored by: The FreeBSD Foundation

Add four new RCTL resources - readbps, readiops, writebps and writeiops,
for limiting disk (actually filesystem) IO.

Note that in some cases these limits are not quite precise. It's ok,
as long as i

Add four new RCTL resources - readbps, readiops, writebps and writeiops,
for limiting disk (actually filesystem) IO.

Note that in some cases these limits are not quite precise. It's ok,
as long as it's within some reasonable bounds.

Testing - and review of the code, in particular the VFS and VM parts - is
very welcome.

MFC after: 1 month
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5080

show more ...

MFH r289384-r293170

Sponsored by: The FreeBSD Foundation

MFH @r292599

This includes the pluggable TCP framework and other chnages to the
netstack to track for VNET stability.

Security: The FreeBSD Foundation

A change to KPI of vm_pager_get_pages() and underlying VOP_GETPAGES().

o With new KPI consumers can request contiguous ranges of pages, and
unlike before, all pages will be kept busied on return,

A change to KPI of vm_pager_get_pages() and underlying VOP_GETPAGES().

o With new KPI consumers can request contiguous ranges of pages, and
unlike before, all pages will be kept busied on return, like it was
done before with the 'reqpage' only. Now the reqpage goes away. With
new interface it is easier to implement code protected from race
conditions.

Such arrayed requests for now should be preceeded by a call to
vm_pager_haspage() to make sure that request is possible. This
could be improved later, making vm_pager_haspage() obsolete.

Strenghtening the promises on the business of the array of pages
allows us to remove such hacks as swp_pager_free_nrpage() and
vm_pager_free_nonreq().

o New KPI accepts two integer pointers that may optionally point at
values for read ahead and read behind, that a pager may do, if it
can. These pages are completely owned by pager, and not controlled
by the caller.

This shifts the UFS-specific readahead logic from vm_fault.c, which
should be file system agnostic, into vnode_pager.c. It also removes
one VOP_BMAP() request per hard fault.

Discussed with: kib, alc, jeff, scottl
Sponsored by: Nginx, Inc.
Sponsored by: Netflix

show more ...

vm_fault_hold: handle vm_page_rename failure

On vm_page_rename failure, fix a missing object unlock and a double free of
a page.

First remove the old page, then rename into other page into first_ob

vm_fault_hold: handle vm_page_rename failure

On vm_page_rename failure, fix a missing object unlock and a double free of
a page.

First remove the old page, then rename into other page into first_object,
then free the old page. This avoids the problem on rename failure. This is
a little ugly but seems to be the most straightforward solution.

Tested with:
$ sysctl debug.fail_point.uma_zalloc_arg="1%return"
$ kyua test -k /usr/tests/sys/Kyuafile

Submitted by: Ryan Libby <rlibby@gmail.com>
Reviewed by: kib
Seen by: alc
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D4326

show more ...

Merge from head

Sponsored by: Gandi.net

Finish merging from head, messed up in previous attempt