dtrace/arm64: Fix enumeration of FBT return probes

On arm64, the FBT provider treats tail calls as return probes. Ignoring
the question of whether this is really correct, the implementation is
wron

dtrace/arm64: Fix enumeration of FBT return probes

On arm64, the FBT provider treats tail calls as return probes. Ignoring
the question of whether this is really correct, the implementation is
wrong: instr is a pointer to uint32_t, so the removed multiplication by
the instruction size is wrong. As a result, FBT would create return
probes for intra-function branches.

MFC after: 2 weeks
Sponsored by: Innovate UK

show more ...

arm64: Use void pointers for arguments to arm64_get_writable_addr

No functional change, but this reduces diffs with CheriBSD downstream.

Reviewed by: andrew
Sponsored by: University of Cambridge, G

arm64: Use void pointers for arguments to arm64_get_writable_addr

No functional change, but this reduces diffs with CheriBSD downstream.

Reviewed by: andrew
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D44344

show more ...

arm64: Switch the address argument to cpu_*cache* to a pointer

No functional change, but this reduces diffs with CheriBSD downstream.

Reviewed by: andrew
Sponsored by: University of Cambridge, Goog

arm64: Switch the address argument to cpu_*cache* to a pointer

No functional change, but this reduces diffs with CheriBSD downstream.

Reviewed by: andrew
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D44342

show more ...

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

dtrace: fix fbt regression for aarch64

fbt computes incorrect instruction position for AArch64 kernel module symbol.

The issue is with the for loop, it does an extra increment of instr pointer
afte

dtrace: fix fbt regression for aarch64

fbt computes incorrect instruction position for AArch64 kernel module symbol.

The issue is with the for loop, it does an extra increment of instr pointer
after the required instruction is found. Hence, a wrong instruction is
targeted for patching.

Signed-off-by: Igor Ostapenko <pm@igoro.pro>

Fixes: 980746e5cb26 ("fbt: simplify arm64 function-prologue parsing")
Reviewed by: markj
Pull Request: https://github.com/freebsd/freebsd-src/pull/855
MFC after: 1 week

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

fbt: simplify arm64 function-prologue parsing

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40364

dtrace: deduplicate arm64 breakpoint definition

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40363

fbt: get rid of redundant defines

No functional change intended.

Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39882

Teach DTrace about BTI on arm64

The Branch Target Identification (BTI) Armv8-A extension adds new
instructions that can be placed where we may indirrectly branch to,
e.g. at the start of a function

Teach DTrace about BTI on arm64

The Branch Target Identification (BTI) Armv8-A extension adds new
instructions that can be placed where we may indirrectly branch to,
e.g. at the start of a function called via a function pointer. We can't
emulate these in DTrace as the kernel will have raised a different
exception before the DTrace handler has run.

Skip over the BTI instruction if it's used as the first instruction in
a function.

Sponsored by: The FreeBSD Foundation

show more ...

Fix dtrace fbt return probes on arm64

As with arm and riscv fix return fbt probes on arm64. arg0 should be
the offset within the function of the return instruction and arg1
should be the return valu

Fix dtrace fbt return probes on arm64

As with arm and riscv fix return fbt probes on arm64. arg0 should be
the offset within the function of the return instruction and arg1
should be the return value.

Reviewed by: kp, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33440

show more ...

Allow ddb and dtrace use the DMAP region on arm64

When writing to memory on arm64 we may be trying to be accessing a
read-only page. In this case try to access via the DMAP region to
get a writable

Allow ddb and dtrace use the DMAP region on arm64

When writing to memory on arm64 we may be trying to be accessing a
read-only page. In this case try to access via the DMAP region to
get a writable location.

While here simplify writing data in DDB and stop trashing the size as
it is passed into the cache handling functions.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32053

show more ...

Tune DTrace 'aframes' for the FBT and profile providers on arm64.
In both cases, too few frames were trimmed, leading to exception handling
or DTrace internals being exposed in stack traces exposed b

Tune DTrace 'aframes' for the FBT and profile providers on arm64.
In both cases, too few frames were trimmed, leading to exception handling
or DTrace internals being exposed in stack traces exposed by D's stack()
primitive.

MFC after: 3 days
Reviewed by: emaste, andrew

show more ...

Handle functions that use a nop in the arm64 fbt

To trace leaf asm functions we can insert a single nop instruction as
the first instruction in a function and trigger off this.

Reviewed by: gnn
Spo

Handle functions that use a nop in the arm64 fbt

To trace leaf asm functions we can insert a single nop instruction as
the first instruction in a function and trigger off this.

Reviewed by: gnn
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D28132

show more ...

Handle using a sub instruction in the arm64 fbt

Some stack frames are too large for a store pair instruction we already
detect in the arm64 fbt code. Add support for handling subtracting the
stack p

Handle using a sub instruction in the arm64 fbt

Some stack frames are too large for a store pair instruction we already
detect in the arm64 fbt code. Add support for handling subtracting the
stack pointer directly.

Sponsored by: Innovate UK

show more ...

Only allow a store through sp in the arm64 fbt

When searching for an instruction to patch out in the arm64 function
boundary trace we search for a store pair with a write back. This
instruction is c

Only allow a store through sp in the arm64 fbt

When searching for an instruction to patch out in the arm64 function
boundary trace we search for a store pair with a write back. This
instruction is commonly used to store two registers to the stack
and update the stack pointer to hold space for more.

This works in many cases, however not all functions use this, e.g.
when the stack frame is too large. In these cases we may find another
instruction of the same type that doesn't store through the stack
pointer. Filter these instructions out and assume if we see one we
are past the function prologue.

Reported by: rwatson
Sponsored by: Innovate UK

show more ...

Changes that improve DTrace FBT reliability on freebsd/arm64:

- Implement a dtrace_getnanouptime(), matching the existing
dtrace_getnanotime(), to avoid DTrace calling out to a potentially
instr

Changes that improve DTrace FBT reliability on freebsd/arm64:

- Implement a dtrace_getnanouptime(), matching the existing
dtrace_getnanotime(), to avoid DTrace calling out to a potentially
instrumentable function.

(These should probably both be under KDTRACE_HOOKS. Also, it's not clear
to me that they are correct implementations for the DTrace thread time
functions they are used in .. fixes for another commit.)

- Don't allow FBT to instrument functions involved in EL1 exception handling
that are involved in FBT trap processing: handle_el1h_sync() and
do_el1h_sync().

- Don't allow FBT to instrument DDB and KDB functions, as that makes it
rather harder to debug FBT problems.

Prior to these changes, use of FBT on FreeBSD/arm64 rapidly led to kernel
panics due to recursion in DTrace.

Reliable FBT on FreeBSD/arm64 is reliant on another change from @andrew to
have the aarch64 instrumentor more carefully check that instructions it
replaces are against the stack pointer, which can otherwise lead to memory
corruption. That change remains under review.

MFC after: 2 weeks
Reviewed by: andrew, kp, markj (earlier version), jrtc27 (earlier version)
Differential revision: https://reviews.freebsd.org/D27766

show more ...

Merge ^/head r338298 through r338391.

Allow multiple FBT probes to share a tracepoint.

With GNU ifuncs, multiple FBT probes may correspond to the same
instruction. fbt_invop() assumed that this could not happen and
would return after t

Allow multiple FBT probes to share a tracepoint.

With GNU ifuncs, multiple FBT probes may correspond to the same
instruction. fbt_invop() assumed that this could not happen and
would return after the first probe found in the global FBT hash
table, which might not be the one that's enabled. Fix the problem
on x86 by linking probes that share a tracepoint and having each
linked probe fire when the tracepoint is hit.

PR: 230846
Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16921

show more ...

Make the second argument of dtrace_invop() a trapframe pointer.

Currently this argument is a pointer into the stack which is used by FBT
to fetch the first five probe arguments. On all non-x86 archi

Make the second argument of dtrace_invop() a trapframe pointer.

Currently this argument is a pointer into the stack which is used by FBT
to fetch the first five probe arguments. On all non-x86 architectures it's
simply the trapframe address, so this change has no functional impact. On
amd64 it's a pointer into the trapframe such that stack[1 .. 5] gives the
first five argument registers, which are deliberately grouped together in
the amd64 trapframe definition.

A trapframe argument simplifies the invop handlers on !x86 and makes the
x86 FBT invop handler easier to understand. Moreover, it allows for invop
handlers that may want to modify the register set of the interrupted thread.

show more ...

Merge from head

Sponsored by: Gandi.net

Finish merging from head, messed up in previous attempt

Merge from head

Catch up with head, primarily for the 1.14.4.0 firmware.

Merge ^/head r285924 through r286421.