Merge tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Pull bpf updates from Daniel Borkmann:
"Major changes:

- Redesign the verifier error reporting: failures n

Merge tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Pull bpf updates from Daniel Borkmann:
"Major changes:

- Redesign the verifier error reporting: failures now carry source
and instruction annotations along with the causal event history
that led to them, making program rejections far easier to debug and
repair (Kumar Kartikeya Dwivedi)

- Add arena argument support to kfuncs and struct_ops through the new
__arena and __arena__nullable suffixes (Tejun Heo, Puranjay Mohan,
Kumar Kartikeya Dwivedi, Ihor Solodrai)

- Signed BPF program loader rework to accommodate both BPF and
security community needs where the kernel runs the signature
verification at BPF_PROG_LOAD time before the LSM admission hook
(Daniel Borkmann)

- Add a set of ksock kfuncs which let BPF LSM and syscall programs
create, connect and send on UDP sockets in order to emit telemetry
data (Mahe Tardy)

- Unify helper and kfunc call argument verification and classify
kfunc arguments purely from BTF into a generated bpf_func_proto
which is computed once at add-call time (Amery Hung)

Other features and fixes:

- Enable EXECMEM_ROX_CACHE for BPF allocations on x86 (Mike Rapoport)

- Add bidirectional VLAN support to bpf_fib_lookup() through the new
BPF_FIB_LOOKUP_VLAN and BPF_FIB_LOOKUP_VLAN_INPUT flags (Avinash
Duduskar)

- Infer zext_dst from static register liveness analysis to fix 32-bit
zero-extension semantics, and remove the artificial limitations on
pointer types eligible for spilling (Eduard Zingerman)

- Inline the numeric open-coded iterator kfuncs so that bpf_for()
loops no longer pay a kfunc call on every iteration (Puranjay
Mohan)

- Add an arena-based bitmap data structure to libarena along with
serial and parallel selftests (Emil Tsalapatis)

- Teach resolve_btfids to discover kfuncs from the kernel's BTF ID
sets and to emit kfunc BTF decl tags, reducing the kernel build's
dependency on pahole features (Ihor Solodrai)

- Add BPF_F_ADJ_ROOM_DECAP_* flags to bpf_skb_adjust_room() so that
tunnel decapsulation can update the GSO and encapsulation state of
the skb (Nick Hudson)

- Fix the ring buffer pending_pos walk and the available-data
accounting on 32-bit position wrap (Israel Téllez García)

- Add memory usage accounting for arena maps and fix an mmap_lock
deadlock on arena lock failure (Jiayuan Chen)

- Add tracing_multi link info support to the kernel UAPI and bpftool,
and refactor the stack map code to run with preemption disabled
(Jiri Olsa)

- Support BPF_F_EGRESS in bpf_redirect_peer() to emit the skb in the
egress direction of the target's peer device (Jordan Rife)

- Add a KF_SPINLOCK_SAFE kfunc flag so that providers, in particular
modules, can declare kfuncs safe to call under bpf_spin_lock
instead of relying on the verifier's hard-coded allowlist (Kaitao
Cheng)

- Introduce global percpu data for BPF programs with libbpf probing
and bpftool skeleton support, and stop exposing uninitialized
kernel heap memory when copying per-CPU map values (Leon Hwang)

- Add s390 JIT support for load-acquire and store-release
instructions (Maxim Khmelevskii)

- Fix a CFI mismatch in the task work callback and an arm64 KASAN
false positive after bpf_throw() (Mykyta Yatsenko)

- Reject writes through untrusted BTF pointers and bound the
rdonly/rdwr_buf_size kfunc arguments (Nicholas Dudar)

- Invalidate RCU pointers only after the final spin unlock and
account for preempt and IRQ disabled regions as overlapping RCU
protection (Ning Ding)

- Support mixing bpf2bpf calls and tail calls on RV64, add signed
operations and 32-bit atomics to the RV32 JIT, and add timed
may_goto support (Pu Lehui, Kuan-Wei Chiu, Feng Jiang)

- Fix a use-after-free on mm_struct in bpf_find_vma() for foreign
tasks and an mmap_lock leak in the irq_work path (Sanghyun Park)

- Populate mmap-able BPF array map memory lazily which makes mmap()
O(1) instead of proportional to the map size (Song Liu)

- Introduce a jit_required flag and reject programs with inlined
helpers when no JIT is available, where the interpreter would
otherwise jump into an invalid address (Tiezhu Yang)

- Fix the x86 JIT per-CPU address resolution into an extended
register where the REX prefix dropped the high destination register
bit (Vineet Gupta)

- Reject MEM_ALLOC BTF accesses past object bounds, arena frees below
the arena base, and mixed arena and ordinary atomic paths (Yiyang
Chen)

- Fix the trampoline handling of 128-bit arguments and of return
values larger than 8 bytes (Yonghong Song)

- Ensure that any fault prone load is rewritten with exception table
handling, and fix the arena load-acquire and atomic fetch handling
in the x86, arm64, riscv and s390 JITs (Daniel Borkmann)

- Many more fixes and cleanups across the verifier, arena,
trampolines, sockmap, cgroup, ring buffer, x86/arm64/riscv/s390
JITs, libbpf, bpftool, resolve_btfids and selftests"

* tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (373 commits)
selftests/bpf: Add tests for a store on a fault prone qdisc pointer
selftests/bpf: Add tests for fault prone loads out of RCU pointers
selftests/bpf: Add tests for pointer type merge at a shared load
selftests/bpf: Remove duplicate copies of the arena spinlock qnodes
selftests/bpf: Retry stat generation in cgroup_iter_memcg
selftests/bpf: Test pseudo-function policy diagnostics
bpf: Distinguish function references in policy diagnostics
bpf: Preserve source attribution without source text
selftests/bpf: Test kfunc argument diagnostics
bpf: Correct kfunc argument diagnostics
bpf: Use canonical stack argument names in diagnostics
bpf: Preserve R0 lineage across helper calls
selftests/bpf: Exercise negative optlen in cgroup getsockopt hook
bpf: Reject negative optlen in cgroup getsockopt hook
selftests/bpf: tc_tunnel - validate decap GSO and encapsulation state
bpf: Clear decap state on skb_adjust_room shrink path
bpf: Allow new DECAP flags and add guard rails
bpf: Add BPF_F_ADJ_ROOM_DECAP_* flags for tunnel decapsulation
bpf: Refactor masks for ADJ_ROOM flags and encap validation
bpf: Name the enum for BPF_FUNC_skb_adjust_room flags
...

show more ...

selftests/bpf: Improve readability in iter test for percpu data

The original 'offsetof()' + offset is equal to the new 'offsetof()'. Use
the new 'offsetof()' instead.

Rename two variables btw:

* o

selftests/bpf: Improve readability in iter test for percpu data

The original 'offsetof()' + offset is equal to the new 'offsetof()'. Use
the new 'offsetof()' instead.

Rename two variables btw:

* offsetof_num -> num_off
* percpu_data_sum -> sum

Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260814173206.93082-6-leon.hwang@linux.dev

show more ...

Merge branch 'bpf-introduce-global-percpu-data'

Leon Hwang says:

====================
bpf: Introduce global percpu data

This patch set introduces global percpu data, similar to commit
6316f78306c1

Merge branch 'bpf-introduce-global-percpu-data'

Leon Hwang says:

====================
bpf: Introduce global percpu data

This patch set introduces global percpu data, similar to commit
6316f78306c1 ("Merge branch 'support-global-data'"), to reduce restrictions
in C for BPF programs.

With this enhancement, it becomes possible to define and use global percpu
variables, like the DEFINE_PER_CPU() macro in the kernel
include/linux/percpu-defs.h.

The section name for global peurcpu data is ".percpu". Even though, a one-byte
percpu variable (e.g., char run SEC(".percpu") = 0;) can trigger a crash
with Clang 17 [1], users are expected to use such small variables as global
percpu data with newer Clang versions, which don't have the issue.

The idea stems from the bpfsnoop [2], which itself was inspired by
retsnoop [3]. During testing of bpfsnoop on the v6.6 kernel, two LBR
(Last Branch Record) entries were observed related to the
bpf_get_smp_processor_id() helper.

Since commit 1ae6921009e5 ("bpf: inline bpf_get_smp_processor_id() helper"),
the bpf_get_smp_processor_id() helper has been inlined on x86_64, reducing
the overhead and consequently minimizing these two LBR records.

However, the introduction of global percpu data offers a more robust
solution. By leveraging the percpu_array map and percpu instruction,
global percpu data can be implemented intrinsically.

This feature also facilitates sharing percpu information between tail
callers and callees or between freplace callers and callees through a
shared global percpu variable. Previously, this was achieved using a
1-entry percpu_array map, which this patch set aims to improve upon.

Links:
[1] https://lore.kernel.org/bpf/fd1b3f58-c27f-403d-ad99-644b7d06ecb3@linux.dev/
[2] https://github.com/bpfsnoop/bpfsnoop
[3] https://github.com/anakryiko/retsnoop

Changes:
v11 -> v12:
* Improve feature check in bpf_object__create_maps() in libbpf.
* Add percpu_array map support in bpf_map__set_value_size() in libbpf.
* Exercise bpf_map__set_value_size() in selftest.
* Drop dead warning in bpf_object__populate_internal_map() in libbpf.
(Sashiko)
* v11: https://lore.kernel.org/bpf/20260806163125.11172-1-leon.hwang@linux.dev/

v10 -> v11:
* Drop env->prog->jit_requested check when inlining insns for global
percpu data.
* Do not autocreate percpu_array map when kernel does not have global
percpu data support in libbpf.
* Check map->btf_value_type_id in bpftool's is_skel_data().
* Exercise bpf_map__lookup_elem() in selftest.
* Collect Reviewed-by tags from Emil, thanks.
* Drop all duplicate blank lines in kernel/bpf/*.c. (Emil)
* Factor out check_map_mem_read() helper. (Emil)
* Check bpf_jit_supports_percpu_insn() first in
percpu_array_map_direct_value_addr/meta(). (Emil)
* Add comment for 'map->libbpf_type == LIBBPF_MAP_PERCPU' in libbpf's
map_is_mmapable(). (Emil)
* Init update_flags as a const var in libbpf's
bpf_object__populate_internal_map(). (Emil)
* Keep is_mmapable_map() beyond is_skel_data() in bpftool. (Emil)
* Add 'run' and 'cpu_id' in selftest. (Emil)
* Drop subskel test. Verify the generated subskel manually. (Emil)
* Add comment to the raw insns in selftest. (Emil)
* v10: https://lore.kernel.org/bpf/20260715153254.92010-1-leon.hwang@linux.dev/

v9 -> v10:
* Rebase latest bpf-next tree to resolve code conflict in verifier in
patch #1.
* v9: https://lore.kernel.org/bpf/20260713154024.30851-1-leon.hwang@linux.dev/

v8 -> v9:
* Use real name for percpu data maps in libbpf in patch #4.
* Add long map name test in patch #6.
* Move parse_cpu_mask_file() to test_percpu_data_on_cpus() in test in
patch #6.
* Validate map type in get_map_ident() for percpu data maps in patch #5.
* Update code comment in verifier in patch #2. (per Andrii)
* Pass 'type' to internal_map_name in libbpf in patch #4. (per Andrii)
* Factor out the helper is_skel_data() in bpftool in patch #5.
(per Quentin and Andrii)
* v8: https://lore.kernel.org/bpf/20260629152406.52582-1-leon.hwang@linux.dev/

v7 -> v8:
* Send patch #1 and #2 separately that fix interpreter fallback issues.
(Andrii)
* Use 'array->elem_size' to avoid 'range' local variable in
percpu_array_map_direct_value_meta(). (Andrii)
* Keep original map name for percpu data's map in libbpf. (Andrii)
* Factor out helper bpf_map_is_skel_data() in bpftool. (Andrii)
* Update commit message of direct access read-only percpu_array map.
(Andrii)
* Add test to verify that it is disallowed to directly write data of
read-only percpu_array map. (Andrii)
* Drop unused 'num_cpus' in test. (bot+bpf-ci)
* Factor out helper test_percpu_data_on_cpus() in test. (bot+bpf-ci)
* v7: https://lore.kernel.org/bpf/20260622143557.22955-1-leon.hwang@linux.dev/

v6 -> v7:
* Use tgt_endian() in bpf_gen__map_update_elem() in patch #6. (Sashiko)
* Use sizeof(args) in verifier_snprintf test in patch #10. (Sashiko)
* Drop xlated test of v6. (Alexei)
* v6: https://lore.kernel.org/bpf/20260615152646.27639-1-leon.hwang@linux.dev/

v5 -> v6:
* Prevent running user addr_space_cast and addr_percpu insns in
interpreter. (Sashiko)
* Cast __percpu pointer to u64 with (__force unsigned long). (lkp)
* Exclude BPF_MAP_TYPE_PERCPU_ARRAY in check_mem_access() before calling
bpf_map_direct_read(), and add a test to verify it.
(Sashiko, bot+bpf-ci)
* Skip percpu data variables for subskeleton in bpftool. (Sashiko)
* Protect skel->percpu using mprotect(..., PROT_READ) in light skeleton.
(Sashiko, bot+bpf-ci)
* Drop roundup() in tests. (Sashiko)
* Call test_global_percpu_data_verifier_log() without
test__start_subtest(). (Sashiko)
* Cast insn->imm to __u64 with (__u32) in xlated test. (Sashiko)
* Check cnt using the new idx in xlated test. (Sashiko)
* v5: https://lore.kernel.org/bpf/20260608145113.65857-1-leon.hwang@linux.dev/

v4 -> v5:
* Add prog->jit_requested check to prevent running percpu data in
interpreter in patch #1.
* Factor out verifier log tests using its own patch.
* Address comments from Alexei:
* Move map_type check from check_mem_access() to bpf_map_direct_read()
in patch #2.
* Move BPF_MAP_TYPE_INSN_ARRAY map_type check from const_reg_xfer() to
bpf_map_direct_read() in patch #2.
* Add a test to verify that the off of xlated ldimm64 insn matches the
off encoded in the ELF ldimm64 insn.
* Drop patch #5 of v4.
* Address reviews from Sashiko:
* Update commit message of patch #6 to indicate that maps.percpu->mmaped
has been marked as read-only in libbpf.
* Lookup elem on specified CPU using BPF_F_CPU in tests.
* Drop unnecessary err == -EOPNOTSUPP in test.
* Locate target field using its offset in the iter test.
* v4: https://lore.kernel.org/bpf/20260414132421.63409-1-leon.hwang@linux.dev/

v3 -> v4:
* Drop duplicate blank lines in verifier.
* Add percpu data feature probe in libbpf.
* Update percpu_array map using BPF_F_ALL_CPUS flag for lskel, if no cpu flag
is set.
* Add two tests to verify verifier log.
* Add a test to verify mov64_percpu_reg instruction.
* Add a test to verify bpf_iter for percpu data map.
* Update percpu_array map using BPF_F_ALL_CPUS flag in libbpf
(per Alexei and Andrii).
* Address comments from Andrii:
* Use .percpu as section identifier.
* Use bpf_jit_supports_percpu_insn() instead of CONFIG_SMP.
* Drop bpf_map__is_internal_percpu() API.
* Drop unnecessary __aligned(8) in libbpf, verified by selftest.
* Make mmap data read-only after loading prog.
v3: https://lore.kernel.org/bpf/20250526162146.24429-1-leon.hwang@linux.dev/

v2 -> v3:
* Use ".data..percpu" as PERCPU_DATA_SEC.
* Address comment from Alexei:
* Add u8, array of ints and struct { .. } vars to selftest.
v2: https://lore.kernel.org/bpf/20250213161931.46399-1-leon.hwang@linux.dev/

v1 -> v2:
* Address comments from Andrii:
* Use LIBBPF_MAP_PERCPU and SEC_PERCPU.
* Reuse mmaped of libbpf's struct bpf_map for .percpu map data.
* Set .percpu struct pointer to NULL after loading skeleton.
* Make sure value size of .percpu map is __aligned(8).
* Use raw_tp and opts.cpu to test global percpu variables on all CPUs.
* Address comments from Alexei:
* Test non-zero offset of global percpu variable.
* Test case about BPF_PSEUDO_MAP_IDX_VALUE.
v1: https://lore.kernel.org/bpf/20250127162158.84906-1-leon.hwang@linux.dev/

rfc -> v1:
* Address comments from Andrii:
* Keep one image of global percpu variable for all CPUs.
* Reject non-ARRAY map in bpf_map_direct_read(), check_reg_const_str(),
and check_bpf_snprintf_call() in verifier.
* Split out libbpf changes from kernel-side changes.
* Use ".percpu" as PERCPU_DATA_SEC.
* Use enum libbpf_map_type to distinguish BSS, DATA, RODATA and
PERCPU_DATA.
* Avoid using errno for checking err from libbpf_num_possible_cpus().
* Use "map '%s': " prefix for error message.
rfc: https://lore.kernel.org/bpf/20250113152437.67196-1-leon.hwang@linux.dev/
====================

Link: https://patch.msgid.link/20260813152324.97937-1-leon.hwang@linux.dev
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

show more ...

selftests/bpf: Verify bpf_iter for global percpu data

Add a test to verify that it is OK to iter the percpu_array map used for
global percpu data.

Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
S

selftests/bpf: Verify bpf_iter for global percpu data

Add a test to verify that it is OK to iter the percpu_array map used for
global percpu data.

Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260813152324.97937-11-leon.hwang@linux.dev

show more ...

selftests/bpf: Test verifier log for global percpu data

Add two tests to verify the verifier log
"R%d points to percpu_array map which cannot be used as const string\n".

Signed-off-by: Leon Hwang <

selftests/bpf: Test verifier log for global percpu data

Add two tests to verify the verifier log
"R%d points to percpu_array map which cannot be used as const string\n".

Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260813152324.97937-10-leon.hwang@linux.dev

show more ...

selftests/bpf: Add tests to verify global percpu data

If the arch, like s390x, does not support percpu insn, these cases won't
test global percpu data by checking FEAT_PERCPU_DATA support.

The foll

selftests/bpf: Add tests to verify global percpu data

If the arch, like s390x, does not support percpu insn, these cases won't
test global percpu data by checking FEAT_PERCPU_DATA support.

The following APIs have been tested for global percpu data:

1. bpf_map__set_initial_value()
2. bpf_map__initial_value()
3. bpf_map__set_value_size()
4. generated percpu struct pointer pointing to internal map's mmaped data
5. bpf_map__lookup_elem() for global percpu data map
6. bpf_map_lookup_elem_flags() for global percpu data map

At the same time, the case is also tested with 'bpftool gen skeleton -L'.

Assisted-by: Codex:gpt-5.5-xhigh
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260813152324.97937-8-leon.hwang@linux.dev

show more ...