sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

locks: fix two potential overflows in the lock delay code

With large numbers of CPUs, the calculation of the maximum lock delay
could overflow, leading to an unexpectedly low delay. In fact, the
max

locks: fix two potential overflows in the lock delay code

With large numbers of CPUs, the calculation of the maximum lock delay
could overflow, leading to an unexpectedly low delay. In fact, the
maximum delay would calculate to 0 on systems with between 128 and
255 cores (inclusive). Also, when calculating the new delay in
lock_delay(), the delay would overflow if the old delay was >= 32,768.

This commit fixes these two overflows. It also updates the maximum
delay from 32,678 to SHRT_MAX.

Reviewed by: gallatin, jhb, mjg
Fixes: 6b8dd26e7c5f ("locks: convert delay times to u_short")
MFC after: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D39372

show more ...

locks: whack a failed experiment in form of restrict_starvation

This was never enabled and only pollutes the code. The issue will
be addressed later in a different manner.

Sponsored by: Rubicon Com

locks: whack a failed experiment in form of restrict_starvation

This was never enabled and only pollutes the code. The issue will
be addressed later in a different manner.

Sponsored by: Rubicon Communications, LLC ("Netgate")

show more ...

lockprof: move panic check after inspecting the state

lockprof: pass lock type as an argument instead of reading the spin flag

lockprof: add contested-only profiling

This allows tracking all wait times with much smaller runtime impact.

For example when doing -j 104 buildkernel on tmpfs:

no profiling: 2921.70s user 282.72s

lockprof: add contested-only profiling

This allows tracking all wait times with much smaller runtime impact.

For example when doing -j 104 buildkernel on tmpfs:

no profiling: 2921.70s user 282.72s system 6598% cpu 48.562 total
all acquires: 2926.87s user 350.53s system 6656% cpu 49.237 total
contested only: 2919.64s user 290.31s system 6583% cpu 48.756 total

show more ...

lockprof: retire lock_prof_skipcount

The implementation uses a global variable for *ALL* calls, defeating the
point of sampling in the first place. Remove it as it clearly remains
unused.

lock_delay(9): improve interaction with restrict_starvation

After e7a5b3bd058, the la->delay value was adjusted after
being set by the starvation_limit code block, which is wrong.

Reported By: avg

lock_delay(9): improve interaction with restrict_starvation

After e7a5b3bd058, the la->delay value was adjusted after
being set by the starvation_limit code block, which is wrong.

Reported By: avg
Reviewed By: avg
Fixes: e7a5b3bd058
Sponsored By: NetApp, Inc.
Sponsored By: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D29513

show more ...

Modify lock_delay() to increase the delay time after spinning

Modify lock_delay() to increase the delay time after spinning,
not before. Previously we would spin at least twice instead of once.
In

Modify lock_delay() to increase the delay time after spinning

Modify lock_delay() to increase the delay time after spinning,
not before. Previously we would spin at least twice instead of once.
In NetApp's benchmarks this fixes a performance regression compared
to FreeBSD 10, which called cpu_spinwait() directly.

Reviewed By: mjg
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27331

show more ...

lockmgr: shrink struct lock by 8 bytes on LP64

Currently the struct has a 4 byte padding stemming from 3 ints.

1. prio comfortably fits in short, unfortunately there is no dedicated
type for it

lockmgr: shrink struct lock by 8 bytes on LP64

Currently the struct has a 4 byte padding stemming from 3 ints.

1. prio comfortably fits in short, unfortunately there is no dedicated
type for it and plumbing it throughout the codebase is not worth it
right now, instead an assert is added which covers also flags for
safety
2. lk_exslpfail can in principle exceed u_short, but the count is
already not considered reliable and it only ever gets modified
straight to 0. In other words it can be incrementing with an upper
bound of USHRT_MAX

With these in place struct lock shrinks from 48 to 40 bytes.

Reviewed by: kib (previous version)
Differential Revision: https://reviews.freebsd.org/D28680

show more ...

kern: clean up empty lines in .c and .h files

Merge ^/head r358269 through r358399.

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

locks: add default delay struct

Use it for all primitives. This makes everything fit in 8 bytes.

locks: convert delay times to u_short

int is just a waste of space for this purpose.

lockprof: use IPI-injecetd fences to fix hangs on stat dump and reset

The previously used quiesce_all_cpus walks all CPUs and waits until curthread
can run on them. Even on contemporary machines thi

lockprof: use IPI-injecetd fences to fix hangs on stat dump and reset

The previously used quiesce_all_cpus walks all CPUs and waits until curthread
can run on them. Even on contemporary machines this becomes a significant
problem under load when it can literally take minutes for the operation to
complete. With the patch the stall is normally less than 1 second.

Reviewed by: kib, jeff (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21740

show more ...

Merge ^/head r352587 through r352763.

lockprof: move per-cpu data to dpcpu

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21747

Merge ^/head r352537 through r352586.

lockprof: use CPUFOREACH and drop always false lp_cpu NULL checks

Sponsored by: The FreeBSD Foundation

MFHead@r345275

MFhead@r344996

Drop "All rights reserved" from my copyright statements.

Reviewed by: rgrimes
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D19485

locks: tweak backoff a little bit

Previous limits were chosen when locking primitives had spurious lock
accesses.

Flipping the starting point to 1 (or rather 2 as the first call shifts it)
provides

locks: tweak backoff a little bit

Previous limits were chosen when locking primitives had spurious lock
accesses.

Flipping the starting point to 1 (or rather 2 as the first call shifts it)
provides a modest win when mild contention is seen while not hurting worse
cases. Tested on a bunch of one, two and four socket old and new systems
(Westmere, Skylake, Threadreaper and others) by doing concurrent page faults,
buildkernel/buildworld and other stuff (although not all systems got all the
tests).

Another thing is the upper limit. It is semi-arbitrarily chosen as it was
getting out of hand for slightly less small systems (e.g. a 128-thread one).

Note that backoff is fundamentally a speculative bandaid and this change just
makes it fit a little bit better. It remains completely oblivious to the
hardware topology or the contention pattern. This is being experimented with.

show more ...