dtrace/arm64: Fix dtrace_gethrtime()

This routine returns a monotonic count of the number of nanoseconds elapsed
since the previous call. On arm64 it uses the generic system timer. The
implementat

dtrace/arm64: Fix dtrace_gethrtime()

This routine returns a monotonic count of the number of nanoseconds elapsed
since the previous call. On arm64 it uses the generic system timer. The
implementation multiplies the counter value by 10**9 then divides by the counter
frequency, but this multiplication can overflow. This can result in trace
records with non-monotonic timestamps, which breaks libdtrace's temporal
ordering algorithm.

An easy fix is to reverse the order of operations, since the counter frequency
will in general be smaller than 10**9. (In fact, it's mandated to be 1Ghz in
ARMv9, which makes life simple.) However, this can give a fair bit of error.
Adopt the calculation used on amd64, with tweaks to handle frequencies as low as
1MHz: the ARM generic timer documentation suggests that ARMv8 timers are
typically in the 1MHz-50MHz range, which is true on arm64 systems that I have
access to.

MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D49244

show more ...

dtrace/arm64: Simplify variable declarations in the invop handler

Remove some unused variables and reduce the scope of some others.

No functional change intended.

MFC after: 1 week
Sponsored by: I

dtrace/arm64: Simplify variable declarations in the invop handler

Remove some unused variables and reduce the scope of some others.

No functional change intended.

MFC after: 1 week
Sponsored by: Innovate UK

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

dtrace/arm64: Store the fault address when suppressing a page fault

Reviewed by: andrew
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40539

dtrace: Add a "regs" variable

This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired. It does
not work for SDT providers b

dtrace: Add a "regs" variable

This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired. It does
not work for SDT providers because their probes are implemented as plain
function calls and so don't save registers. It's not clear what
semantics "regs" should have for them anyway.

This is akin to "uregs", which nominally provides access to the
userspace registers. In fact, DIF already had a DIF_VAR_REGS variable
defined, it was simply unimplemented.

Usage example: print the contents of %rdi upon each call to
amd64_syscall():

fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}

Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.
Currently there are no similar definitions for non-x86 platforms.

Reviewed by: christos
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36799

show more ...

Fix dtrace fbt return probes on arm64

As with arm and riscv fix return fbt probes on arm64. arg0 should be
the offset within the function of the return instruction and arg1
should be the return valu

Fix dtrace fbt return probes on arm64

As with arm and riscv fix return fbt probes on arm64. arg0 should be
the offset within the function of the return instruction and arg1
should be the return value.

Reviewed by: kp, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33440

show more ...

Handle functions that use a nop in the arm64 fbt

To trace leaf asm functions we can insert a single nop instruction as
the first instruction in a function and trigger off this.

Reviewed by: gnn
Spo

Handle functions that use a nop in the arm64 fbt

To trace leaf asm functions we can insert a single nop instruction as
the first instruction in a function and trigger off this.

Reviewed by: gnn
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D28132

show more ...

Reimplement the arm64 dtrace_gethrtime(), which provides the
high-resolution nanosecond timestamp used for the DTrace 'timestamp'
built-in variable. The new implementation uses the EL0 cycle
counter

Reimplement the arm64 dtrace_gethrtime(), which provides the
high-resolution nanosecond timestamp used for the DTrace 'timestamp'
built-in variable. The new implementation uses the EL0 cycle
counter and frequency registers in ARMv8-A. This replaces a
previous implementation that relied on an instrumentation-safe
implementation of getnanotime(), which provided only timer
resolution.

MFC after: 3 days
Reviewed by: andrew, bsdimp (older version)
Useful comments appreciated: jrtc27, emaste

show more ...

Handle using a sub instruction in the arm64 fbt

Some stack frames are too large for a store pair instruction we already
detect in the arm64 fbt code. Add support for handling subtracting the
stack p

Handle using a sub instruction in the arm64 fbt

Some stack frames are too large for a store pair instruction we already
detect in the arm64 fbt code. Add support for handling subtracting the
stack pointer directly.

Sponsored by: Innovate UK

show more ...

Changes that improve DTrace FBT reliability on freebsd/arm64:

- Implement a dtrace_getnanouptime(), matching the existing
dtrace_getnanotime(), to avoid DTrace calling out to a potentially
instr

Changes that improve DTrace FBT reliability on freebsd/arm64:

- Implement a dtrace_getnanouptime(), matching the existing
dtrace_getnanotime(), to avoid DTrace calling out to a potentially
instrumentable function.

(These should probably both be under KDTRACE_HOOKS. Also, it's not clear
to me that they are correct implementations for the DTrace thread time
functions they are used in .. fixes for another commit.)

- Don't allow FBT to instrument functions involved in EL1 exception handling
that are involved in FBT trap processing: handle_el1h_sync() and
do_el1h_sync().

- Don't allow FBT to instrument DDB and KDB functions, as that makes it
rather harder to debug FBT problems.

Prior to these changes, use of FBT on FreeBSD/arm64 rapidly led to kernel
panics due to recursion in DTrace.

Reliable FBT on FreeBSD/arm64 is reliant on another change from @andrew to
have the aarch64 instrumentor more carefully check that instructions it
replaces are against the stack pointer, which can otherwise lead to memory
corruption. That change remains under review.

MFC after: 2 weeks
Reviewed by: andrew, kp, markj (earlier version), jrtc27 (earlier version)
Differential revision: https://reviews.freebsd.org/D27766

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

Don't overflow the trap frame when accessing lr or xzr.

When emulating a load pair or store pair in dtrace on arm64 we need to
copy the data between the stack and trap frame. When the registers are

Don't overflow the trap frame when accessing lr or xzr.

When emulating a load pair or store pair in dtrace on arm64 we need to
copy the data between the stack and trap frame. When the registers are
either the link register or the zero register we will access memory
past the end of the trap frame as these are encoded as registers 30 and
31 respectively while the array they access only has 30 entries.

Fix this by creating 2 helper functions to perform the operation with
special cases for these registers.

Sponsored by: Innovate UK

show more ...

Corrected misspelled versions of rendezvous.

The MFC will include a compat definition of smp_no_rendevous_barrier()
that calls smp_no_rendezvous_barrier().

Reviewed by: gnn, kib
MFC after: 1 week
D

Corrected misspelled versions of rendezvous.

The MFC will include a compat definition of smp_no_rendevous_barrier()
that calls smp_no_rendezvous_barrier().

Reviewed by: gnn, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D10313

show more ...

Merge ^/head r308227 through r308490.

Fix improper use of "its".

Sponsored by: Dell EMC Isilon

Try to unbreak the build after r300611 by including the header
defining VM_MIN_KERNEL_ADDRESS.

Sponsored by: DARPA/AFRL

Mark all memory before the kernel as toxic to DTrace.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Make the second argument of dtrace_invop() a trapframe pointer.

Currently this argument is a pointer into the stack which is used by FBT
to fetch the first five probe arguments. On all non-x86 archi

Make the second argument of dtrace_invop() a trapframe pointer.

Currently this argument is a pointer into the stack which is used by FBT
to fetch the first five probe arguments. On all non-x86 architectures it's
simply the trapframe address, so this change has no functional impact. On
amd64 it's a pointer into the trapframe such that stack[1 .. 5] gives the
first five argument registers, which are deliberately grouped together in
the amd64 trapframe definition.

A trapframe argument simplifies the invop handlers on !x86 and makes the
x86 FBT invop handler easier to understand. Moreover, it allows for invop
handlers that may want to modify the register set of the interrupted thread.

show more ...

MFH

Sponsored by: The FreeBSD Foundation

Remove unused variables dtrace_in_probe and dtrace_in_probe_addr.

Merge from head

Catch up with head, primarily for the 1.14.4.0 firmware.

Merge ^/head r284737 through r285152.

First cut of DTrace for AArch64.

Reviewed by: andrew, emaste
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2738