Merge r278538 through r280226.

Merge from head

Convert PTE eviction lock from an RW lock to a RM lock. It is held for
writing approximately never (< 0.00000001% under heavy VM load, and it can
go for months without ever being acquired in normal o

Convert PTE eviction lock from an RW lock to a RM lock. It is held for
writing approximately never (< 0.00000001% under heavy VM load, and it can
go for months without ever being acquired in normal operation). This
provides a 10% (2-minute) improvement in wall clock time for make -j32
buildworld on a 4-core 32-thread POWER8.

show more ...

Merge from head

Merge ^/head r279893 through r279984.

Deallocate any leftover page table entries in the LPAR at boot. This
prevents contamination from a previous kernel (e.g. after shutdown -r).

Merge from head

Merge r278538 through r279514.

Merge ^/head r279163 through r279308.

New pmap implementation for 64-bit PowerPC processors. The main focus of
this change is to improve concurrency:
- Drop global state stored in the shadow overflow page table (and all other
global st

New pmap implementation for 64-bit PowerPC processors. The main focus of
this change is to improve concurrency:
- Drop global state stored in the shadow overflow page table (and all other
global state)
- Remove all global locks
- Use per-PTE lock bits to allow parallel page insertion
- Reconstruct state when requested for evicted PTEs instead of buffering
it during overflow

This drops total wall time for make buildworld on a 32-thread POWER8 system
by a factor of two and system time by a factor of three, providing performance
20% better than similarly clocked Core i7 Xeons per-core. Performance on
smaller SMP systems, where PMAP lock contention was not as much of an issue,
is nearly unchanged.

Tested on: POWER8, POWER5+, G5 UP, G5 SMP (64-bit and 32-bit kernels)
Merged from: user/nwhitehorn/ppc64-pmap-rework
Looked over by: jhibbits, andreast
MFC after: 3 months
Relnotes: yes
Sponsored by: FreeBSD Foundation

show more ...

Merge from HEAD

MFH: r278202,r278205-r278590

Sponsored by: The FreeBSD Foundation

Merge ^/head r278351 through r278498.

Mark invalid page table entries correctly for PMAP as well as for the
hypervisor. This prevents an infinite loop where processes with evicted
pages would page fault forever when PMAP decided the evic

Mark invalid page table entries correctly for PMAP as well as for the
hypervisor. This prevents an infinite loop where processes with evicted
pages would page fault forever when PMAP decided the evicted pages on
which the process was faulting was actually present and did not need to
be restored.

Found while building LLVM with make -j32.

Sponsored by: FreeBSD Foundation

show more ...

Fix typo in PTE insertion overflow handling: use the page we're actually
returning, not the one we just looked at.

Add some error checking on the supplied page size list. This makes sure
that we (a) get the correct large page size to provide to pmap and (b)
we can alert the user if running under incorrectly-confi

Add some error checking on the supplied page size list. This makes sure
that we (a) get the correct large page size to provide to pmap and (b)
we can alert the user if running under incorrectly-configured PowerKVM
on POWER7 and POWER8 systems.

MFC after: 1 week

show more ...

Merge head

IFC @264767

Merge head up to r262222 (last merge was incomplete).

MFC @ r259205 in preparation for some SVM updates. (for real this time)

Limit the large page size to 16MB for now.

MFC after: 1 week

Merge head r232040 through r258006.

Ouch, do the shift properly. An uin64_t needs the right postfix when shifting.

MFC @ r256071

This is just prior to the bhyve_npt_pmap import so will allow
just the change to be merged for easier debug.

Merge head r233826 through r256722.