tracepoints: Update static_call before tp_funcs when adding a tracepoint

Because of the significant overhead that retpolines pose on indirect
calls, the tracepoint code was updated to use the new "s

tracepoints: Update static_call before tp_funcs when adding a tracepoint

Because of the significant overhead that retpolines pose on indirect
calls, the tracepoint code was updated to use the new "static_calls" that
can modify the running code to directly call a function instead of using
an indirect caller, and this function can be changed at runtime.

In the tracepoint code that calls all the registered callbacks that are
attached to a tracepoint, the following is done:

it_func_ptr = rcu_dereference_raw((&__tracepoint_##name)->funcs);
if (it_func_ptr) {
__data = (it_func_ptr)->data;
static_call(tp_func_##name)(__data, args);
}

If there's just a single callback, the static_call is updated to just call
that callback directly. Once another handler is added, then the static
caller is updated to call the iterator, that simply loops over all the
funcs in the array and calls each of the callbacks like the old method
using indirect calling.

The issue was discovered with a race between updating the funcs array and
updating the static_call. The funcs array was updated first and then the
static_call was updated. This is not an issue as long as the first element
in the old array is the same as the first element in the new array. But
that assumption is incorrect, because callbacks also have a priority
field, and if there's a callback added that has a higher priority than the
callback on the old array, then it will become the first callback in the
new array. This means that it is possible to call the old callback with
the new callback data element, which can cause a kernel panic.

static_call = callback1()
funcs[] = {callback1,data1};
callback2 has higher priority than callback1

CPU 1 CPU 2
----- -----

new_funcs = {callback2,data2},
{callback1,data1}

rcu_assign_pointer(tp->funcs, new_funcs);

/*
* Now tp->funcs has the new array
* but the static_call still calls callback1
*/

it_func_ptr = tp->funcs [ new_funcs ]
data = it_func_ptr->data [ data2 ]
static_call(callback1, data);

/* Now callback1 is called with
* callback2's data */

[ KERNEL PANIC ]

update_static_call(iterator);

To prevent this from happening, always switch the static_call to the
iterator before assigning the tp->funcs to the new array. The iterator will
always properly match the callback with its data.

To trigger this bug:

In one terminal:

while :; do hackbench 50; done

In another terminal

echo 1 > /sys/kernel/tracing/events/sched/sched_waking/enable
while :; do
echo 1 > /sys/kernel/tracing/set_event_pid;
sleep 0.5
echo 0 > /sys/kernel/tracing/set_event_pid;
sleep 0.5
done

And it doesn't take long to crash. This is because the set_event_pid adds
a callback to the sched_waking tracepoint with a high priority, which will
be called before the sched_waking trace event callback is called.

Note, the removal to a single callback updates the array first, before
changing the static_call to single callback, which is the proper order as
the first element in the array is the same as what the static_call is
being changed to.

Link: https://lore.kernel.org/io-uring/4ebea8f0-58c9-e571-fd30-0ce4f6f09c70@samba.org/

Cc: stable@vger.kernel.org
Fixes: d25e37d89dd2f ("tracepoint: Optimize using static_call()")
Reported-by: Stefan Metzmacher <metze@samba.org>
tested-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

show more ...

Merge tag 'trace-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:

- Added option for per CPU threads to the hwlat tracer

- Ha

Merge tag 'trace-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:

- Added option for per CPU threads to the hwlat tracer

- Have hwlat tracer handle hotplug CPUs

- New tracer: osnoise, that detects latency caused by interrupts,
softirqs and scheduling of other tasks.

- Added timerlat tracer that creates a thread and measures in detail
what sources of latency it has for wake ups.

- Removed the "success" field of the sched_wakeup trace event. This has
been hardcoded as "1" since 2015, no tooling should be looking at it
now. If one exists, we can revert this commit, fix that tool and try
to remove it again in the future.

- tgid mapping fixed to handle more than PID_MAX_DEFAULT pids/tgids.

- New boot command line option "tp_printk_stop", as tp_printk causes
trace events to write to console. When user space starts, this can
easily live lock the system. Having a boot option to stop just after
boot up is useful to prevent that from happening.

- Have ftrace_dump_on_oops boot command line option take numbers that
match the numbers shown in /proc/sys/kernel/ftrace_dump_on_oops.

- Bootconfig clean ups, fixes and enhancements.

- New ktest script that tests bootconfig options.

- Add tracepoint_probe_register_may_exist() to register a tracepoint
without triggering a WARN*() if it already exists. BPF has a path
from user space that can do this. All other paths are considered a
bug.

- Small clean ups and fixes

* tag 'trace-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (49 commits)
tracing: Resize tgid_map to pid_max, not PID_MAX_DEFAULT
tracing: Simplify & fix saved_tgids logic
treewide: Add missing semicolons to __assign_str uses
tracing: Change variable type as bool for clean-up
trace/timerlat: Fix indentation on timerlat_main()
trace/osnoise: Make 'noise' variable s64 in run_osnoise()
tracepoint: Add tracepoint_probe_register_may_exist() for BPF tracing
tracing: Fix spelling in osnoise tracer "interferences" -> "interference"
Documentation: Fix a typo on trace/osnoise-tracer
trace/osnoise: Fix return value on osnoise_init_hotplug_support
trace/osnoise: Make interval u64 on osnoise_main
trace/osnoise: Fix 'no previous prototype' warnings
tracing: Have osnoise_main() add a quiescent state for task rcu
seq_buf: Make trace_seq_putmem_hex() support data longer than 8
seq_buf: Fix overflow in seq_buf_putmem_hex()
trace/osnoise: Support hotplug operations
trace/hwlat: Support hotplug operations
trace/hwlat: Protect kdata->kthread with get/put_online_cpus
trace: Add timerlat tracer
trace: Add osnoise tracer
...

show more ...

tracepoint: Add tracepoint_probe_register_may_exist() for BPF tracing

All internal use cases for tracepoint_probe_register() is set to not ever
be called with the same function and data. If it is, i

tracepoint: Add tracepoint_probe_register_may_exist() for BPF tracing

All internal use cases for tracepoint_probe_register() is set to not ever
be called with the same function and data. If it is, it is considered a
bug, as that means the accounting of handling tracepoints is corrupted.
If the function and data for a tracepoint is already registered when
tracepoint_probe_register() is called, it will call WARN_ON_ONCE() and
return with EEXISTS.

The BPF system call can end up calling tracepoint_probe_register() with
the same data, which now means that this can trigger the warning because
of a user space process. As WARN_ON_ONCE() should not be called because
user space called a system call with bad data, there needs to be a way to
register a tracepoint without triggering a warning.

Enter tracepoint_probe_register_may_exist(), which can be called, but will
not cause a WARN_ON() if the probe already exists. It will still error out
with EEXIST, which will then be sent to the user space that performed the
BPF system call.

This keeps the previous testing for issues with other users of the
tracepoint code, while letting BPF call it with duplicated data and not
warn about it.

Link: https://lore.kernel.org/lkml/20210626135845.4080-1-penguin-kernel@I-love.SAKURA.ne.jp/
Link: https://syzkaller.appspot.com/bug?id=41f4318cf01762389f4d1c1c459da4f542fe5153

Cc: stable@vger.kernel.org
Fixes: c4f6699dfcb85 ("bpf: introduce BPF_RAW_TRACEPOINT")
Reported-by: syzbot <syzbot+721aa903751db87aa244@syzkaller.appspotmail.com>
Reported-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Tested-by: syzbot+721aa903751db87aa244@syzkaller.appspotmail.com
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

show more ...

Merge branch 'next' into for-linus

Prepare input updates for 5.13 merge window.

Merge tag 'v5.12-rc7' into ecryptfs/next

Required to pick up idmapped mount changes which changed some function
parameters.

Merge tag 'v5.12-rc4' into next

Sync up with the mainline to bring in newest APIs.

Merge drm/drm-next into drm-misc-next

Noralf needs some patches in 5.12-rc3, and we've been delaying the 5.12
merge due to the swap issue so it looks like a good time.

Signed-off-by: Maxime Ripard

Merge drm/drm-next into drm-misc-next

Noralf needs some patches in 5.12-rc3, and we've been delaying the 5.12
merge due to the swap issue so it looks like a good time.

Signed-off-by: Maxime Ripard <maxime@cerno.tech>

show more ...

Merge tag 'irqchip-fixes-5.12-1' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into irq/urgent

Pull irqchip fixes from Marc Zyngier:

- More compatible strings for the Ingenic

Merge tag 'irqchip-fixes-5.12-1' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into irq/urgent

Pull irqchip fixes from Marc Zyngier:

- More compatible strings for the Ingenic irqchip (introducing the
JZ4760B SoC)
- Select GENERIC_IRQ_MULTI_HANDLER on the ARM ep93xx platform
- Drop all GENERIC_IRQ_MULTI_HANDLER selections from the irqchip
Kconfig, now relying on the architecture to get it right
- Drop the debugfs_file field from struct irq_domain, now that
debugfs can track things on its own

show more ...

Merge drm/drm-next into drm-intel-next

Sync up with upstream.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>

Merge branch 'fixes-rc2' into fixes

Merge tag 'tags/sound-sdw-kconfig-fixes' into for-linus

ALSA/ASoC/SOF/SoundWire: fix Kconfig issues

In January, Intel kbuild bot and Arnd Bergmann reported multiple
issues with randconfig. This pat

Merge tag 'tags/sound-sdw-kconfig-fixes' into for-linus

ALSA/ASoC/SOF/SoundWire: fix Kconfig issues

In January, Intel kbuild bot and Arnd Bergmann reported multiple
issues with randconfig. This patchset builds on Arnd's suggestions to

a) expose ACPI and PCI devices in separate modules, while sof-acpi-dev
and sof-pci-dev become helpers. This will result in minor changes
required for developers/testers, i.e. modprobe snd-sof-pci will no
longer result in a probe. The SOF CI was already updated to deal with
this module dependency change and introduction of new modules.

b) Fix SOF/SoundWire/DSP_config dependencies by moving the code
required to detect SoundWire presence in ACPI tables to sound/hda.

Link: https://lore.kernel.org/r/20210302003125.1178419-1-pierre-louis.bossart@linux.intel.com

show more ...

Merge remote-tracking branch 'net/master'

Merge branch 'next' into for-linus

Prepare input updates for 5.12 merge window.

Merge tag 'v5.11' into next

Merge with mainline to get latest APIs and device tree bindings.

Merge tag 'trace-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:

- Update to the way irqs and preemption is tracked via the tr

Merge tag 'trace-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:

- Update to the way irqs and preemption is tracked via the trace event
PC field

- Fix handling of unregistering event failing due to allocate memory.
This is only triggered by failure injection, as it is pretty much
guaranteed to have less than a page allocation succeed.

- Do not show the useless "filter" or "enable" files for the "ftrace"
trace system, as they have no effect on doing anything.

- Add a warning if kprobes are registered more than once.

- Synthetic events now have their fields parsed by semicolons. Old
formats without semicolons will still work, but new features will
require them.

- New option to allow trace events to show %p without hashing in trace
file. The trace file can only be read by root, and reading the raw
event buffer did not have any pointers hashed, so this does not
expose anything new.

- New directory in tools called tools/tracing, where a new tool that
reads sequential latency reports from the ftrace latency tracers.

- Other minor fixes and cleanups.

* tag 'trace-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (33 commits)
kprobes: Fix to delay the kprobes jump optimization
tracing/tools: Add the latency-collector to tools directory
tracing: Make hash-ptr option default
tracing: Add ptr-hash option to show the hashed pointer value
tracing: Update the stage 3 of trace event macro comment
tracing: Show real address for trace event arguments
selftests/ftrace: Add '!event' synthetic event syntax check
selftests/ftrace: Update synthetic event syntax errors
tracing: Add a backward-compatibility check for synthetic event creation
tracing: Update synth command errors
tracing: Rework synthetic event command parsing
tracing/dynevent: Delegate parsing to create function
kprobes: Warn if the kprobe is reregistered
ftrace: Remove unused ftrace_force_update()
tracepoints: Code clean up
tracepoints: Do not punish non static call users
tracepoints: Remove unnecessary "data_args" macro parameter
tracing: Do not create "enable" or "filter" files for ftrace event subsystem
kernel: trace: preemptirq_delay_test: add cpu affinity
tracepoint: Do not fail unregistering a probe due to memory failure
...

show more ...

tracepoints: Code clean up

Restructure the code a bit to make it simpler, fix some formatting problems
and add READ_ONCE/WRITE_ONCE to make sure there's no compiler load/store
tearing to the variabl

tracepoints: Code clean up

Restructure the code a bit to make it simpler, fix some formatting problems
and add READ_ONCE/WRITE_ONCE to make sure there's no compiler load/store
tearing to the variables that can be accessed across CPUs.

Started with Mathieu Desnoyers's patch:

Link: https://lore.kernel.org/lkml/20210203175741.20665-1-mathieu.desnoyers@efficios.com/

And will keep his signature, but I will take the responsibility of this
being correct, and keep the authorship.

Link: https://lkml.kernel.org/r/20210204143004.61126582@gandalf.local.home

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

show more ...

tracepoint: Do not fail unregistering a probe due to memory failure

The list of tracepoint callbacks is managed by an array that is protected
by RCU. To update this array, a new array is allocated,

tracepoint: Do not fail unregistering a probe due to memory failure

The list of tracepoint callbacks is managed by an array that is protected
by RCU. To update this array, a new array is allocated, the updates are
copied over to the new array, and then the list of functions for the
tracepoint is switched over to the new array. After a completion of an RCU
grace period, the old array is freed.

This process happens for both adding a callback as well as removing one.
But on removing a callback, if the new array fails to be allocated, the
callback is not removed, and may be used after it is freed by the clients
of the tracepoint.

There's really no reason to fail if the allocation for a new array fails
when removing a function. Instead, the function can simply be replaced by a
stub function that could be cleaned up on the next modification of the
array. That is, instead of calling the function registered to the
tracepoint, it would call a stub function in its place.

Link: https://lore.kernel.org/r/20201115055256.65625-1-mmullins@mmlx.us
Link: https://lore.kernel.org/r/20201116175107.02db396d@gandalf.local.home
Link: https://lore.kernel.org/r/20201117211836.54acaef2@oasis.local.home
Link: https://lkml.kernel.org/r/20201118093405.7a6d2290@gandalf.local.home

[ Note, this version does use undefined compiler behavior (assuming that
a stub function with no parameters or return, can be called by a location
that thinks it has parameters but still no return value. Static calls
do the same thing, so this trick is not without precedent.

There's another solution that uses RCU tricks and is more complex, but
can be an alternative if this solution becomes an issue.

Link: https://lore.kernel.org/lkml/20210127170721.58bce7cc@gandalf.local.home/
]

Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: KP Singh <kpsingh@chromium.org>
Cc: netdev <netdev@vger.kernel.org>
Cc: bpf <bpf@vger.kernel.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Florian Weimer <fw@deneb.enyo.de>
Fixes: 97e1c18e8d17b ("tracing: Kernel Tracepoints")
Reported-by: syzbot+83aa762ef23b6f0d1991@syzkaller.appspotmail.com
Reported-by: syzbot+d29e58bb557324e55e5e@syzkaller.appspotmail.com
Reported-by: Matt Mullins <mmullins@mmlx.us>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Tested-by: Matt Mullins <mmullins@mmlx.us>

show more ...

Merge tag 'amba-make-remove-return-void' of https://git.pengutronix.de/git/ukl/linux into devel-stable

Tag for adaptions to struct amba_driver::remove changing prototype

Merge branch 'cpuidle-fix' into fixes

Merge drm/drm-next into drm-intel-gt-next

Backmerging to get a common base for merging topic branches between
drm-intel-next and drm-intel-gt-next.

Signed-off-by: Joonas Lahtinen <joonas.lahtinen@l

Merge drm/drm-next into drm-intel-gt-next

Backmerging to get a common base for merging topic branches between
drm-intel-next and drm-intel-gt-next.

Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

show more ...

Merge drm/drm-next into drm-intel-next

sync-up to not fall too much behind.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Merge branch 'for-linus' into for-next

Back-merge of 5.11-devel branch for syncing the result changes.

Signed-off-by: Takashi Iwai <tiwai@suse.de>

Merge tag 'asoc-fix-v5.11-rc2' of https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v5.11

A collection of mostly driver specific fixes, plus a maintainers

Merge tag 'asoc-fix-v5.11-rc2' of https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v5.11

A collection of mostly driver specific fixes, plus a maintainership
update for TI and a fix for DAPM driver removal paths.

show more ...

Merge drm/drm-next into drm-misc-next

Staying in sync to drm-next, and to be able to pull ttm fixes.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>

Merge tag 'v5.11-rc1' into spi-5.11

Linux 5.11-rc1