x86: Allow sharing of perfomance counter interrupts

This patch refactors the Performance Counter interrupt setup code to
allow sharing the interrupt line between multiple drivers.
More specifically,

x86: Allow sharing of perfomance counter interrupts

This patch refactors the Performance Counter interrupt setup code to
allow sharing the interrupt line between multiple drivers.
More specifically, Performance Counter interrupts are used by both
hwpmc(4) and hwt(4)'s upcoming Intel Processor Trace backend.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D46420

show more ...

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

hwpmc: tidy pcd_finalize methods

Make them a little bit more consistent with one another in terms of what
is done:
- Add/reduce assertions to what is most useful: a loop to ensure
pcpu_fini free

hwpmc: tidy pcd_finalize methods

Make them a little bit more consistent with one another in terms of what
is done:
- Add/reduce assertions to what is most useful: a loop to ensure
pcpu_fini freed everything
- Add PMCDBG trace entries

The exception is the dmc620/cmn600 classes, which behave a little
differently, so leave them untouched.

Reviewed by: jkoshy
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41270

show more ...

pmc: better distinguish pmu-events allocation path

Background:

The pm_ev field of struct pmc_op_pmcallocate and struct pmc
traditionally contains the index of the chosen event, corresponding to
the

pmc: better distinguish pmu-events allocation path

Background:

The pm_ev field of struct pmc_op_pmcallocate and struct pmc
traditionally contains the index of the chosen event, corresponding to
the __PMC_EVENTS array in pmc_events.h. This is a static list of events,
maintained by FreeBSD.

In the usual case, libpmc translates the user supplied event name
(string) into the pm_ev index, which is passed as an argument to the
allocation syscall. On the kernel side, the allocation method for the
relevant hwpmc class translates the given index into the event code that
will be written to an event selection register.

In 2018, a new source of performance event definitions was introduced:
the pmu-events json files, which are maintained by the Linux kernel. The
result was better coverage for newer Intel processors with a reduced
maintenance burden for libpmc/hwpmc. Intel and AMD CPUs were
unconditionally switched to allocate events from pmu-events instead of
the traditional scheme (959826ca1bb0a, 81eb4dcf9e0d).

Under the pmu-events scheme, the pm_ev field contains an index
corresponding to the selected event from the pmu-events table, something
which the kernel has no knowledge of. The configuration for the
performance counting registers is instead passed via class-dependent
fields (struct pmc_md_op_pmcallocate).

In 2021 I changed the allocation logic so that it would attempt to
pull from the pmu-events table first, and fall-back to the traditional
method (dfb4fb41166bc3). Later, pmu-events support for arm64 and power8
CPUs was added (28dd6730a5d6 and b48a2770d48b).

The problem that remains is that the pm_ev field is overloaded, without
a definitive way to determine whether the event allocation came from the
pmu-events table or FreeBSD's statically-defined PMC events. This
resulted in a recent fix, 21f7397a61f7.

Change:

To disambiguate these two supported but separate use-cases, add a new
flag, PMC_F_EV_PMU, to be set as part of the allocation, indicating that
the event index came from pmu-events.

This is useful in two ways:
1. On the kernel side, we can validate the syscall arguments better.
Some classes support only the traditional event scheme (e.g.
hwpmc_armv7), while others support only the pmu-events method (e.g.
hwpmc_core for Intel). We can now check for this. The hwpmc_arm64
class supports both methods, so the new flag supersedes the existing
MD flag, PM_MD_EVENT_RAW.

2. The flag will be tracked in struct pmc for the duration of its
lifetime, meaning it is communicated back to userspace. This allows
libpmc to perform the reverse index-to-event-name translation
without speculating about the meaning of the index value.

Adding the flag is a backwards-incompatible ABI change. We recently
bumped the major version of the hwpmc module, so this breakage is
acceptable.

Reviewed by: jkoshy
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40753

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

hwpmc: pass pmc pointer to more class methods

In many cases this avoids an extra lookup, since the callers always have
pm at hand. We can also eliminate several assertions, mostly for pm !=
NULL. Th

hwpmc: pass pmc pointer to more class methods

In many cases this avoids an extra lookup, since the callers always have
pm at hand. We can also eliminate several assertions, mostly for pm !=
NULL. The class methods are an internal interface, and the callers
already handle such a scenario. No functional change intended.

Reviewed by: jkoshy
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39915

show more ...

hwpmc: remove pmd_pcpu_{init,fini} callbacks

These are unused on all platforms.

Reviewed by: jkoshy, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://re

hwpmc: remove pmd_pcpu_{init,fini} callbacks

These are unused on all platforms.

Reviewed by: jkoshy, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39912

show more ...

hwpmc: don't use deprecated copystr(9)

It is just wrapper around strlcpy(), but results in more complicated
code. Clean this up to use strlcpy() or snprintf() as appropriate.

Reviewed by: jkoshy
MF

hwpmc: don't use deprecated copystr(9)

It is just wrapper around strlcpy(), but results in more complicated
code. Clean this up to use strlcpy() or snprintf() as appropriate.

Reviewed by: jkoshy
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39910

show more ...

hwpmc: Fix a typo

MFC after: 1 week

hwpmc: Avoid touching MSR_DEBUGCTLMSR inside VMs.

At least KVM in Proxmox seems not happy about it. Just to be safe
block it for all VMs, since it should just improve profile accuracy.

MFC after:

hwpmc: Avoid touching MSR_DEBUGCTLMSR inside VMs.

At least KVM in Proxmox seems not happy about it. Just to be safe
block it for all VMs, since it should just improve profile accuracy.

MFC after: 1 week

show more ...

hwpmc: Reenable PME before reenabling counters.

Doing otherwise may lead to lost interrupts, that in combination
with hardware PMCs freezing may leave them frozen forever. It may
also slightly impr

hwpmc: Reenable PME before reenabling counters.

Doing otherwise may lead to lost interrupts, that in combination
with hardware PMCs freezing may leave them frozen forever. It may
also slightly improve profiling accuracy.

MFC after: 2 weeks

show more ...

hwpmc: Add IDs for few more Intel Atom CPUs.

MFC after: 1 month

hwpmc: Skip GLOBAL_CTRL updates on stop_pmc().

After we wipe PMC configuration, including its local enable bit(s),
we don't really care about its global enable bit. Global enable bits
now may only

hwpmc: Skip GLOBAL_CTRL updates on stop_pmc().

After we wipe PMC configuration, including its local enable bit(s),
we don't really care about its global enable bit. Global enable bits
now may only be cleared by interrupt handler in case of error (sample
buffer overflow). Being set is actually a reset default for them.

This saves one WRMSR per process-scope PMC per context switch, that
is clearly visible in profiles.

MFC after: 1 month

show more ...

hwpmc: Bump Intel's IA32_PERFEVTSELx width to 64 bits.

Haswell added there bits 32/33 for TSX, and AlderLake added bit 34
for Adaptive PEBS Record.

MFC after: 1 month

hwpmc: Add basic Intel Alderlake CPUs support.

The PMC subsystem is not designed for non-uniform CPU capabilities
(P/E-cores are different), but at least several working architectural
events like cp

hwpmc: Add basic Intel Alderlake CPUs support.

The PMC subsystem is not designed for non-uniform CPU capabilities
(P/E-cores are different), but at least several working architectural
events like cpu_clk_unhalted.thread_p should be better than nothing.

MFC after: 1 month

show more ...

hwpmc: Update Intel's programmable counters restrictions.

Primarily remove unneeded restrictions from later CPUs.

MFC after: 1 month

hwpmc: Correct selection of Intel fixed counters.

Intel json's use event=0 to specify fixed counter number via umask.
Alternatively fixed counters have equivalent programmable event/umask.

MFC afte

hwpmc: Correct selection of Intel fixed counters.

Intel json's use event=0 to specify fixed counter number via umask.
Alternatively fixed counters have equivalent programmable event/umask.

MFC after: 1 month

show more ...

hwpmc: Use hardware PMCs freezing on PMI on Intel v2+.

Since version 2 Intel CPUs can freeze PMCs when intering PMI to reduce
PMI effects on collected statistics. Since version 4 hardware supports

hwpmc: Use hardware PMCs freezing on PMI on Intel v2+.

Since version 2 Intel CPUs can freeze PMCs when intering PMI to reduce
PMI effects on collected statistics. Since version 4 hardware supports
"streamlined" mechanism, not requiring IA_GLOBAL_CTRL MSR access.

MFC after: 1 month

show more ...

hwpmc: drop 0x before %p printf format string

%p already includes the 0x.

Sponsored by: The FreeBSD Foundation

hwpmc: Add IDs for Intel Comet/Ice/Tiger/Rocketlake CPUs.

MFC after: 1 month

hwpmc: don't validate capabilities in allocation method

These checks were inconsistently applied across the various hwpmc
classes. The condition is already checked by the generic code in
hwpmc_mod.c

hwpmc: don't validate capabilities in allocation method

These checks were inconsistently applied across the various hwpmc
classes. The condition is already checked by the generic code in
hwpmc_mod.c, so remove them.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31388

show more ...

hwpmc: consistently validate PMC class in allocation method

It is always a good idea. In one case, attempting to allocate N+1 PMCs
from a class with N hardware counters would incorrectly attempt to

hwpmc: consistently validate PMC class in allocation method

It is always a good idea. In one case, attempting to allocate N+1 PMCs
from a class with N hardware counters would incorrectly attempt to
allocate from the next class in the list. Without this validation, this
can lead to all kinds of strange behaviour.

Since powerpc_allocate_pmc() is used by both the mpc7xxx and ppc970
classes, add a new global to track which is active (it will never be
both).

Reviewed by: luporl, ray
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31387

show more ...

x86_msr_op: extend the KPI to allow MSR read and single-CPU operations

Reivewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D3

x86_msr_op: extend the KPI to allow MSR read and single-CPU operations

Reivewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31386

show more ...

hwpmc: mostly clean up cc --analyze

Sponsored by: Rubicon Communications, LLC ("Netgate")