kinst: do not look for a function epilogue

kinst currently only traces functions that start and end with the usual
function prologue and epilogue respectively. Ignoring functions that do
not have an

kinst: do not look for a function epilogue

kinst currently only traces functions that start and end with the usual
function prologue and epilogue respectively. Ignoring functions that do
not have an epilogue however, makes the filtering too strict, as this
means that we can not trace functions that never return (e.g
vnlru_proc()). This patch relaxes the filtering and only checks whether
the function pushes the frame pointer.

Reviewed by: markj
Approved by: markj
Differential Revision: https://reviews.freebsd.org/D41876

show more ...

kinst: check for 'push %rbp' anywhere in the function

Currently kinst checks if only the first instruction is 'push %rbp',
essentially excluding functions that do push RBP, but not in the first
inst

kinst: check for 'push %rbp' anywhere in the function

Currently kinst checks if only the first instruction is 'push %rbp',
essentially excluding functions that do push RBP, but not in the first
instruction. This patch modifies kinst to check for 'push %rbp', as
well, as a following 'pop %rbp', anywhere in the function. This behavior
also matches that of FBT.

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40283

show more ...

kinst: update LICENSE headers

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40875

kinst: rename t_kinst to t_kinst_tramp

The forthcoming RISC-V and ARM64 ports of kinst introduce a new field
named "t_kinst_curprobe", so "t_kinst" (which points to a trampoline)
becomes a misleadin

kinst: rename t_kinst to t_kinst_tramp

The forthcoming RISC-V and ARM64 ports of kinst introduce a new field
named "t_kinst_curprobe", so "t_kinst" (which points to a trampoline)
becomes a misleading name.

No functional change intended.

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40507

show more ...

kinst: use bool where appropriate

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40412

kinst: fix memcpy() tracing crash

Tracing memcpy() would crash the kernel, because we'd also trace the
memcpy() calls from kinst_invop(). To fix this, introduce kinst_memcpy()
whose arguments are 'v

kinst: fix memcpy() tracing crash

Tracing memcpy() would crash the kernel, because we'd also trace the
memcpy() calls from kinst_invop(). To fix this, introduce kinst_memcpy()
whose arguments are 'volatile', so that we avoid having the compiler
replace it with a regular memcpy().

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40284

show more ...

kinst: add kinst_excluded()

Exclude functions that are not safe-to-trace.

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
ifferential Revision: https://reviews.f

kinst: add kinst_excluded()

Exclude functions that are not safe-to-trace.

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
ifferential Revision: https://reviews.freebsd.org/D39229

show more ...

kinst: use dtrace_dis_get_byte() instead of own copy

No functional change intended.

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https:

kinst: use dtrace_dis_get_byte() instead of own copy

No functional change intended.

Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39872

show more ...

kinst: do not use DPCPU_ID_GET if we already have the trampoline

No functional change intended.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebs

kinst: do not use DPCPU_ID_GET if we already have the trampoline

No functional change intended.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39870

show more ...

kinst: Add per-CPU interrupt trampolines

In the common case, kinst emulates a traced instruction by copying it to
a trampoline, where it is followed by a jump back to the original code,
and pointing

kinst: Add per-CPU interrupt trampolines

In the common case, kinst emulates a traced instruction by copying it to
a trampoline, where it is followed by a jump back to the original code,
and pointing the interrupted thread's %rip at the trampoline. In
particular, the trampoline is executed with the same CPU context as the
original instruction, so if interrupts are enabled at the point where
the probe fires, they will be enabled when the trampoline is
subsequently executed.

It can happen that an interrupt is raised while a thread is executing a
kinst trampoline. In that case, it is possible that the interrupt
handler will trigger a kinst probe, so we must ensure that the thread
does not recurse and overwrite its trampoline before it is finished
executing the original contents, otherwise an attempt to trace code
called from interrupt handlers can crash the kernel.

To that end, add a per-CPU trampoline, used when the probe fired with
interrupts disabled. Note that this is not quite complete since it does
not handle the possibility of kinst probes firing while executing an NMI
handler.

Also ensure that we do not trace instructions which set IF, since in
that case it is not clear which trampoline (the per-thread trampoline or
the per-CPU trampoline) we should use, and since such instructions are
rare.

Reported and tested by: Domagoj Stolfa
Reviewed by: christos
Fixes: f0bc4ed144fc ("kinst: Initial revision")
Differential Revision: https://reviews.freebsd.org/D37619

show more ...

dtrace: Fix up %rip for invop probes on x86

When a breakpoint exception is raised, the saved value of %rip points to
the instruction following the breakpoint. However, when fetching the
value of %r

dtrace: Fix up %rip for invop probes on x86

When a breakpoint exception is raised, the saved value of %rip points to
the instruction following the breakpoint. However, when fetching the
value of %rip using regs[], it's more natural to provide the address of
the breakpoint itself, so modify the kinst and fbt providers accordingly.

Reported by: khng
Reviewed by: christos, khng
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D37218

show more ...

kinst: Initial revision

This is a new DTrace provider which allows arbitrary kernel instructions
to be traced. Currently it is implemented only for amd64.

kinst probes are created on demand by lib

kinst: Initial revision

This is a new DTrace provider which allows arbitrary kernel instructions
to be traced. Currently it is implemented only for amd64.

kinst probes are created on demand by libdtrace, and there is a probe
for each kernel instruction. Probes are named
kinst:<module>:<function>:<offset>, where "offset" is the offset of the
target instruction relative to the beginning of the function. Omitting
"offset" causes all instructions in the function to be traced.

kinst works similarly to FBT in that it places a breakpoint on the
target instruction and hooks into the kernel breakpoint handler.
Because kinst has to be able to trace arbitrary instructions, it does
not emulate most of them in software but rather causes the traced thread
to execute a copy of the instruction before returning to the original
code.

The provider is quite low-level and as-is will be useful mostly only to
kernel developers. However, it provides a great deal of visibility into
kernel code execution and could be used as a building block for
higher-level tooling which can in some sense translate between C sources
and generated machine code. In particular, the "regs" variable recently
added to D allows the CPU's register file to be accessed from kinst
probes.

kinst is experimental and should not be used on production systems for
now.

In collaboration with: markj
Sponsored by: Google, Inc. (GSoC 2022)
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D36851

show more ...