Changes in diagnostics.c

5a8cd539ac19f7a68e68e1d25ef9ca2ff55b8500 - Merge tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Thu, 20 Aug 2026 16:36:20 +0200

Merge tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-nextPull bpf updates from Daniel Borkmann: "Major changes: - Redesign the verifier error reporting: failures now carry source and instruction annotations along with the causal event history that led to them, making program rejections far easier to debug and repair (Kumar Kartikeya Dwivedi) - Add arena argument support to kfuncs and struct_ops through the new __arena and __arena__nullable suffixes (Tejun Heo, Puranjay Mohan, Kumar Kartikeya Dwivedi, Ihor Solodrai) - Signed BPF program loader rework to accommodate both BPF and security community needs where the kernel runs the signature verification at BPF_PROG_LOAD time before the LSM admission hook (Daniel Borkmann) - Add a set of ksock kfuncs which let BPF LSM and syscall programs create, connect and send on UDP sockets in order to emit telemetry data (Mahe Tardy) - Unify helper and kfunc call argument verification and classify kfunc arguments purely from BTF into a generated bpf_func_proto which is computed once at add-call time (Amery Hung) Other features and fixes: - Enable EXECMEM_ROX_CACHE for BPF allocations on x86 (Mike Rapoport) - Add bidirectional VLAN support to bpf_fib_lookup() through the new BPF_FIB_LOOKUP_VLAN and BPF_FIB_LOOKUP_VLAN_INPUT flags (Avinash Duduskar) - Infer zext_dst from static register liveness analysis to fix 32-bit zero-extension semantics, and remove the artificial limitations on pointer types eligible for spilling (Eduard Zingerman) - Inline the numeric open-coded iterator kfuncs so that bpf_for() loops no longer pay a kfunc call on every iteration (Puranjay Mohan) - Add an arena-based bitmap data structure to libarena along with serial and parallel selftests (Emil Tsalapatis) - Teach resolve_btfids to discover kfuncs from the kernel's BTF ID sets and to emit kfunc BTF decl tags, reducing the kernel build's dependency on pahole features (Ihor Solodrai) - Add BPF_F_ADJ_ROOM_DECAP_* flags to bpf_skb_adjust_room() so that tunnel decapsulation can update the GSO and encapsulation state of the skb (Nick Hudson) - Fix the ring buffer pending_pos walk and the available-data accounting on 32-bit position wrap (Israel Téllez García) - Add memory usage accounting for arena maps and fix an mmap_lock deadlock on arena lock failure (Jiayuan Chen) - Add tracing_multi link info support to the kernel UAPI and bpftool, and refactor the stack map code to run with preemption disabled (Jiri Olsa) - Support BPF_F_EGRESS in bpf_redirect_peer() to emit the skb in the egress direction of the target's peer device (Jordan Rife) - Add a KF_SPINLOCK_SAFE kfunc flag so that providers, in particular modules, can declare kfuncs safe to call under bpf_spin_lock instead of relying on the verifier's hard-coded allowlist (Kaitao Cheng) - Introduce global percpu data for BPF programs with libbpf probing and bpftool skeleton support, and stop exposing uninitialized kernel heap memory when copying per-CPU map values (Leon Hwang) - Add s390 JIT support for load-acquire and store-release instructions (Maxim Khmelevskii) - Fix a CFI mismatch in the task work callback and an arm64 KASAN false positive after bpf_throw() (Mykyta Yatsenko) - Reject writes through untrusted BTF pointers and bound the rdonly/rdwr_buf_size kfunc arguments (Nicholas Dudar) - Invalidate RCU pointers only after the final spin unlock and account for preempt and IRQ disabled regions as overlapping RCU protection (Ning Ding) - Support mixing bpf2bpf calls and tail calls on RV64, add signed operations and 32-bit atomics to the RV32 JIT, and add timed may_goto support (Pu Lehui, Kuan-Wei Chiu, Feng Jiang) - Fix a use-after-free on mm_struct in bpf_find_vma() for foreign tasks and an mmap_lock leak in the irq_work path (Sanghyun Park) - Populate mmap-able BPF array map memory lazily which makes mmap() O(1) instead of proportional to the map size (Song Liu) - Introduce a jit_required flag and reject programs with inlined helpers when no JIT is available, where the interpreter would otherwise jump into an invalid address (Tiezhu Yang) - Fix the x86 JIT per-CPU address resolution into an extended register where the REX prefix dropped the high destination register bit (Vineet Gupta) - Reject MEM_ALLOC BTF accesses past object bounds, arena frees below the arena base, and mixed arena and ordinary atomic paths (Yiyang Chen) - Fix the trampoline handling of 128-bit arguments and of return values larger than 8 bytes (Yonghong Song) - Ensure that any fault prone load is rewritten with exception table handling, and fix the arena load-acquire and atomic fetch handling in the x86, arm64, riscv and s390 JITs (Daniel Borkmann) - Many more fixes and cleanups across the verifier, arena, trampolines, sockmap, cgroup, ring buffer, x86/arm64/riscv/s390 JITs, libbpf, bpftool, resolve_btfids and selftests"* tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (373 commits) selftests/bpf: Add tests for a store on a fault prone qdisc pointer selftests/bpf: Add tests for fault prone loads out of RCU pointers selftests/bpf: Add tests for pointer type merge at a shared load selftests/bpf: Remove duplicate copies of the arena spinlock qnodes selftests/bpf: Retry stat generation in cgroup_iter_memcg selftests/bpf: Test pseudo-function policy diagnostics bpf: Distinguish function references in policy diagnostics bpf: Preserve source attribution without source text selftests/bpf: Test kfunc argument diagnostics bpf: Correct kfunc argument diagnostics bpf: Use canonical stack argument names in diagnostics bpf: Preserve R0 lineage across helper calls selftests/bpf: Exercise negative optlen in cgroup getsockopt hook bpf: Reject negative optlen in cgroup getsockopt hook selftests/bpf: tc_tunnel - validate decap GSO and encapsulation state bpf: Clear decap state on skb_adjust_room shrink path bpf: Allow new DECAP flags and add guard rails bpf: Add BPF_F_ADJ_ROOM_DECAP_* flags for tunnel decapsulation bpf: Refactor masks for ADJ_ROOM flags and encap validation bpf: Name the enum for BPF_FUNC_skb_adjust_room flags ... List of files: /linux/kernel/bpf/diagnostics.c

6bd520a6e3b66010e6e87ef77a1756c6b6ce30b1 - bpf: Preserve source attribution without source text

Sun, 16 Aug 2026 03:57:39 +0200

bpf: Preserve source attribution without source textGCC emits BTF line records with a file name and line number, but leaves thesource line string empty. bpf_diag_source() currently treats that empty stringas if the complete line record were unavailable, so diagnostics fall back toan instruction number and discard the function, file, and line attribution.Print the available source location before deciding whether source context canbe rendered. When source text is absent, omit only the source context and retainthe diagnostic annotation and instruction context.Fixes: b9c5d822f677 ("bpf: Add source and instruction diagnostic context")Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://lore.kernel.org/bpf/20260816015746.2632990-12-memxor@gmail.com List of files: /linux/kernel/bpf/diagnostics.c

09a0c2d678643aa8362ed83ea21b3a21566b318a - bpf: Use canonical stack argument names in diagnostics

Sun, 16 Aug 2026 03:57:33 +0200

bpf: Use canonical stack argument names in diagnosticsThe main diagnostic identifies the first outgoing stack slot as stackargument 1 and the sixth function argument. The causal history insteadlabels the same value as stack arg6, making it look like a different slot.Render causal-history targets in the verifier's canonical stack-argumentlocation form. The first outgoing slot is now shown as *(R11-8), matchingreg_arg_name(), while the main diagnostic retains its fuller slot andordinal description.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://lore.kernel.org/bpf/eb1be5327d136b7e5bd6d68e76fef6de20c40790.camel@gmail.comLink: https://lore.kernel.org/bpf/20260816015746.2632990-6-memxor@gmail.com List of files: /linux/kernel/bpf/diagnostics.c

ce7c9f6c599b640c1be288fd1af3289d1406d559 - Merge branch 'redesign-verification-errors'

Sat, 15 Aug 2026 20:11:17 +0200

Merge branch 'redesign-verification-errors'Kumar Kartikeya Dwivedi says:====================Redesign Verification ErrorsTL;DR: This set reworks verifier error messages to include source andinstruction annotations, together with more causal context, makingfailures easier to understand and more actionable when debugging andrepairing BPF programs.Changelog:----------v4 -> v5v4: https://lore.kernel.org/bpf/20260812233326.3575958-1-memxor@gmail.com * Defer Verifier Limit reports and the dependent call-chain allocation guards to follow-up work, reducing the series from 16 to 14 patches. (Eduard) * Make kfunc-name disassembly read-only before module-kfunc metadata is resolved, retain instruction context without usable source metadata, consolidate its fallback, and restrict source discovery to the containing subprogram. (Eduard, Sashiko) * Retain the newest diagnostic history in a bounded 64 MiB rotating buffer, use absolute logical positions across verifier path switches, report evicted shared history, and grow storage geometrically. (Eduard) * Complete active-path history for BPF_LD_IMM64 and atomic fetches, call clobbers and returns, outgoing stack arguments, legacy packet loads, and RCU pointer transitions. (Eduard, Sashiko) * Preserve causal lineage across equal snapshots, nullable pointer-cast branches, and repeated same-depth function invocations using unique diagnostic frame identities. Bound each rendered causal path to the oldest and newest 32 matching events with an omission summary. (Eduard) * Harden diagnostics for malformed release-kfunc signatures, fixed-size argument ranges, and dynptr, iterator, memory-size, and required-RCU failures by reporting the actual offending type or invariant. (Eduard, Sashiko) * Remove unrelated formatting and cross-patch churn, dead or single-use helpers and filter paths, and align helper placement, includes, and commit descriptions with the patches that first need them. (Eduard)v3 -> v4v3: https://lore.kernel.org/bpf/20260713153910.2556007-1-memxor@gmail.com * Introduce helpers with their first callers and add printf annotations. (Eduard, Sashiko) * Remove "report" from diagnostic function names. (Sashiko) * Reuse bpf_linfo_source and seq_buf, simplify internal names, and use shared formatting storage. (Eduard) * Use compact common event fields and record branches at successor entry. (Eduard) * Bound event storage at 1 MiB, use kvrealloc(), and drop events non-fatally. (Eduard, Sashiko) * Restore diagnostic history only for activated queued states, preserving the active failure trace during cleanup. (Eduard, Sashiko) * Record register changes through begin/end and scrub helpers, deriving targets and origins without caller-saved snapshots. (Eduard) * Store lineage marks on events and rewind shared formatting storage after rendering each event. (Eduard) * Record iterator return values before snapshotting alternate paths. (Sashiko) * Use the current verifier instruction for global-subprogram dynptr errors. (Sashiko) * Use the supplied call name for nullable global-subprogram arguments. (Sashiko) * Describe global calls under locks as a verifier restriction rather than a sleepability failure. (Sashiko) * Keep diagnostic strings unsplit and put long call openings on their own line. (Eduard) * Keep kfunc metadata zeroed before early fetch and allowability failures. (Sashiko) * Drop the Verifier Internal Error report patch. (Eduard) * Distinguish never-initialized registers from invalidated registers. (local review) * Preserve the legacy different-lock verifier message. (local review) * Preserve nullable type qualifiers and stable mismatch formatting. (local review) * Mark truncated call chains with an ellipsis. (local review)v2 -> v3v2: https://lore.kernel.org/bpf/20260619205934.1312876-1-memxor@gmail.com * Address various comments from Eduard and Sashiko. * Move instruction context from a separate gutter into a new section following source context, since surrounding source lines and BPF instructions do not map one-to-one. * Fix active-path branch reconstruction when switching to queued states, and expand register histories to follow value lineage across spills, fills, stack reads, helper/kfunc clobbers, and dynptr invalidation. * Misc improvements and refinements.v1 -> v2v1: https://lore.kernel.org/bpf/20260605063412.974640-1-memxor@gmail.com * Reworked diagnostic history from per-verifier-state log to active path log with positions saved and reset when verifier search backtracks. (Eduard) * Moved reusable diagnostic formatting storage into struct bpf_diag under struct bpf_verifier_env, and removed large per-report scratch buffers from verifier stack frames. (Eduard) * Added stack-slot events so diagnostics follow ordinary stack spill/fill value flow and invalidations in register-scoped histories. (Eduard) * Reused existing source and BTF formatting helpers for diagnostics, including bpf_get_linfo_file_line() and btf_type_snprintf_show_name(). (Eduard) * Fixed diagnostic edge cases around signed offset text, BPF_MAX_VAR_OFF reporting, negative-offset clamping, poisoned stack reads, and borrowed-reference invalidations. (Eduard) * Fixed various miscellaneous diagnostic bugs. (Sashiko) * Misc improvements and refinements.---Motivation~~~~~~~~~~The verifier log is the primary interface through which the verifiercommunicates to the user its verdict on whether a program was acceptedor rejected.To aid the debugging of rejection decisions, the verifier also reportsthe symbolic state of the program at each instruction, across every exploredpath of the BPF program. Such detailed information is critical tointrospect the correctness of verification decisions, and provideinsight into why a given program may have failed to load in the kernel.A constant pain point in the BPF ecosystem throughout the years hasbeen the difficulty of debugging verification errors. The human-readable errormessages produced in response to a failure in satisfying safety-relatedconstraints are often terse, context-dependent, or insufficient forunderstanding why a given error may have happened. Users must fall backto the verbose instruction-by-instruction breakdown of how the symbolicstate evolved to surface the root cause. For programs with a huge logvolume due to high verification complexity, such logs quickly becomeinscrutable.All of this has made life difficult for users lacking an understandingof how the verifier works, and the various heuristics and idiosyncrasiesused by it. In some cases, even seasoned BPF experts spend significanttime reverse engineering why a program may have failed, and have toreach into the verifier's source code to form a complete picture of theverification process.Such a steep learning curve and cognitive burden also hurts the speed ofBPF development, as the verifier sits right in the middle of the user'siteration loop while they make use of BPF to solve any given problem.Expertise in debugging verifier errors does not scale in terms of teamsdeploying these programs in production across a diverse set of kernels.Overall, this leads to a poorer developer experience, causes visibleuser dissatisfaction, and remains a drag on wider BPF adoption. Withsome of the more recent developments where users increasingly leverageAI tooling [0] to author their code, this bottleneck becomes even morecritical to address, since it throttles the much faster iteration loopof AI agents. [0]: https://lwn.net/Articles/1075067Approach~~~~~~~~This series starts moving selected failures from terse terminal messagestoward diagnostics that carry the relevant context for a verificationfailure. The existing verbose log remains the low-level trace. For selectedfailures, the new report is emitted after this trace and answers theimmediate debugging questions: - what verifier rule failed, - why the current state does not satisfy it, - where the failing instruction maps to source, - which earlier branch or state event made this path fail, - what kind of source change would satisfy the verifier.The series adds a text-only diagnostics framework under kernel/bpf anduses it to augment selected verifier errors. Existing verbose(env, ...)messages are kept, so current selftest expectations and existing logconsumers continue to see the legacy text. The new report has a uniformouter shape: Verification failed: : Reason: exact reason for the verification failure, with details At: source and instruction annotation Causal path: compressed branch and verifier-state events relevant for debugging Suggestion: speculation on potential fixes to repair the programThe outer shape is shared, but report construction is category-specific.The categories are intentionally broad and reviewable. This revisioncovers representative cases in Register Type Safety, Memory Safety,Resource Lifetime Safety, Call Type Safety, Execution Context Safety,Program Structure and Policy.It does not attempt to convert every verbose(env, ...) site for now.Additional verbose-only errors can be moved into the same frameworkincrementally.The following excerpts are copied from this current run on this branch: ./test_progs -j1 \ -a cpumask/test_populate_invalid_destination,\ cpumask/test_alloc_no_release,\ verifier_helper_value_access/via_variable_no_max_check_1,\ verifier_sock/invalidate_pkt_pointers_from_global_func \ -vvThey show the old terminal error and the exact new diagnostic report,including the source and instruction annotations.Call Type Safety, cpumask/test_populate_invalid_destination: Legacy: R1 type=scalar expected=fp Diagnostic: Verification failed: Call Type Safety: Invalid call argument Reason: The first argument (R1) to bpf_cpumask_populate does not satisfy the verifier contract: the kfunc expects 24 bytes of memory for (struct bpf_cpumask), but it is an integer scalar and not verifier-known memory. At: test_populate_invalid_destination @ cpumask_failure.c:234:8 Source context: 232 | ... 233 | ... >>> 234 | ret = bpf_cpumask_populate(invalid, &bits, sizeof(bits)); | ^-- error: invalid first argument (R1) for bpf_cpumask_populate 235 | if (!ret) 236 | err = 2; Instruction context: 2 | (b7) r1 = 1193046 3 | (b7) r3 = 8 >>> 4 | (85) call bpf_cpumask_populate#62860 5 | (56) if w0 != 0x0 goto pc+4 6 | (18) r1 = 0xffffc9000028e000 Causal path: test_populate_invalid_destination @ cpumask_failure.c:234:8 Source context: 232 | ... 233 | ... >>> 234 | ret = bpf_cpumask_populate(invalid, &bits, sizeof(bits)); | ^-- update: R1 changed from context pointer at offset 0 to integer scalar value | 1193046 235 | if (!ret) 236 | err = 2; Instruction context: 0 | (bf) r2 = r10 1 | (07) r2 += -8 >>> 2 | (b7) r1 = 1193046 3 | (b7) r3 = 8 4 | (85) call bpf_cpumask_populate#62860 Suggestion: Pass stack, map, context, or other verifier-known memory of the expected type and size, not an integer cast to a pointer.Register Type Safety, verifier_sock/invalidate_pkt_pointers_from_global_func: Legacy: R7 invalid mem access 'scalar' Diagnostic: Verification failed: Register Type Safety: Invalid dereference Reason: R7 is an integer scalar here, not a pointer to memory. At: invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1067:5 Source context: 1065 | ... 1066 | skb_pull_data1(sk, 0); >>> 1067 | *p = 42; /* this is unsafe */ | ^-- error: invalid dereference of R7 (an integer scalar) 1068 | ... 1069 | } Instruction context: 8 | (85) call pc+4 9 | (b4) w1 = 42 >>> 10 | (63) *(u32 *)(r7 +0) = r1 11 | (bc) w0 = w6 12 | (95) exit Causal path: invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1062:29 Source context: 1060 | int invalidate_pkt_pointers_from_global_func(struct __sk_buff *sk) 1061 | ... >>> 1062 | int *p = (void *)(long)sk->data; | ^-- update: R7 changed from uninitialized value to pkt at offset 0 1063 | ... 1064 | if ((void *)(p + 1) > (void *)(long)sk->data_end) Instruction context: 0 | (b4) w6 = 2 1 | (61) r2 = *(u32 *)(r1 +80) >>> 2 | (61) r7 = *(u32 *)(r1 +76) 3 | (bf) r3 = r7 4 | (07) r3 += 4 invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1064:22 Source context: 1062 | int *p = (void *)(long)sk->data; 1063 | ... >>> 1064 | if ((void *)(p + 1) > (void *)(long)sk->data_end) | ^-- branch: took the false branch of this conditional, goto not followed 1065 | ... 1066 | skb_pull_data1(sk, 0); Instruction context: 3 | (bf) r3 = r7 4 | (07) r3 += 4 >>> 5 | (2d) if r3 > r2 goto pc+5 6 | (b4) w6 = 0 7 | (b4) w2 = 0 invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1066:2 Source context: 1064 | if ((void *)(p + 1) > (void *)(long)sk->data_end) 1065 | ... >>> 1066 | skb_pull_data1(sk, 0); | ^-- invalidated: R7: packet data may have moved; previous value was pkt at | offset 0 1067 | *p = 42; /* this is unsafe */ 1068 | ... Instruction context: 6 | (b4) w6 = 0 7 | (b4) w2 = 0 >>> 8 | (85) call pc+4 9 | (b4) w1 = 42 10 | (63) *(u32 *)(r7 +0) = r1 Suggestion: Preserve a pointer-valued register where needed, or reload and revalidate the pointer after scalar arithmetic, helper calls, or other operations that can invalidate it.Memory Safety, verifier_helper_value_access/via_variable_no_max_check_1: Legacy: R1 unbounded memory access, make sure to bounds check any such access Diagnostic: Verification failed: Memory Safety: Access outside bounds Reason: The verifier cannot prove offset + access_size <= object_size. Here, the maximal bound for a memory access is 4294967295 and exceeds maximum allowed offset of 536870912. R1 is map_value; offset is variable: known bits 0x0, unknown mask 0xffffffff; signed range [0, 4294967295], unsigned range [0, 4294967295]; access_size is 1; object_size is 48. At: via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2 Source context: 625 | ... 626 | ... >>> 627 | asm volatile (" \ | ^-- error: access may be outside object bounds 628 | ... 629 | ... Instruction context: 11 | (b7) r2 = 1 12 | (b7) r3 = 0 >>> 13 | (85) call bpf_probe_read_kernel#113 14 | (95) exit Causal path: via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2 Source context: 625 | ... 626 | ... >>> 627 | asm volatile (" \ | ^-- update: R0 changed from uninitialized value to nullable map value from | map_hash_48b at offset 0 628 | ... 629 | ... Instruction context: 4 | (18) r1 = 0xffff88810a3ea000 >>> 6 | (85) call bpf_map_lookup_elem#1 7 | (15) if r0 == 0x0 goto pc+6 8 | (bf) r1 = r0 via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2 Source context: 625 | ... 626 | ... >>> 627 | asm volatile (" \ | ^-- branch: took the false branch of this conditional, goto not followed 628 | ... 629 | ... Instruction context: 6 | (85) call bpf_map_lookup_elem#1 >>> 7 | (15) if r0 == 0x0 goto pc+6 8 | (bf) r1 = r0 9 | (61) r3 = *(u32 *)(r0 +0) via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2 Source context: 625 | ... 626 | ... >>> 627 | asm volatile (" \ | ^-- update: R1 changed from uninitialized value to map value from map_hash_48b | at offset 0 628 | ... 629 | ... Instruction context: 6 | (85) call bpf_map_lookup_elem#1 7 | (15) if r0 == 0x0 goto pc+6 >>> 8 | (bf) r1 = r0 9 | (61) r3 = *(u32 *)(r0 +0) 10 | (0f) r1 += r3 via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2 Source context: 625 | ... 626 | ... >>> 627 | asm volatile (" \ | ^-- update: R1 changed from map value from map_hash_48b at offset 0 to map value | from map_hash_48b with variable offset: known bits 0x0, unknown mask | 0xffffffff, signed range [0, 4294967295], unsigned range [0, 4294967295] 628 | ... 629 | ... Instruction context: 8 | (bf) r1 = r0 9 | (61) r3 = *(u32 *)(r0 +0) >>> 10 | (0f) r1 += r3 11 | (b7) r2 = 1 12 | (b7) r3 = 0 Suggestion: Add or adjust a bounds check that proves offset + access_size stays within the object.Resource Lifetime Safety, cpumask/test_alloc_no_release: Legacy: Unreleased reference id=2 alloc_insn=0 BPF_EXIT instruction in main prog would lead to reference leak Diagnostic: Verification failed: Resource Lifetime Safety: Unreleased resource Reason: Owned resource (id=2) was acquired at instruction 0 and still needs to be released before this exit path. At: test_alloc_no_release @ cpumask_failure.c:36:5 Source context: 34 | ... 35 | ... >>> 36 | int BPF_PROG(test_alloc_no_release, struct task_struct *task, u64 clone_flags) | ^-- error: owned resource (id=2) still needs release 37 | ... 38 | ... Instruction context: 19 | (7b) *(u64 *)(r10 -8) = r6 20 | (b4) w0 = 0 >>> 21 | (95) exit Causal path: test_alloc_no_release @ cpumask_common.h:78:12 Source context: 76 | ... 77 | ... >>> 78 | cpumask = bpf_cpumask_create(); | ^-- acquired: owned resource (id=2) 79 | if (!cpumask) { 80 | err = 1; Instruction context: >>> 0 | (85) call bpf_cpumask_create#62851 1 | (bf) r6 = r0 2 | (55) if r6 != 0x0 goto pc+5 test_alloc_no_release @ cpumask_common.h:79:6 Source context: 77 | ... 78 | cpumask = bpf_cpumask_create(); >>> 79 | if (!cpumask) { | ^-- branch: took the true branch of this conditional, goto followed 80 | err = 1; 81 | ... Instruction context: 0 | (85) call bpf_cpumask_create#62851 1 | (bf) r6 = r0 >>> 2 | (55) if r6 != 0x0 goto pc+5 3 | (18) r1 = 0xffffc90000252000 test_alloc_no_release @ cpumask_common.h:84:6 Source context: 82 | ... 83 | ... >>> 84 | if (!bpf_cpumask_empty(cast(cpumask))) { | ^-- branch: took the true branch of this conditional, goto followed 85 | err = 2; 86 | bpf_cpumask_release(cpumask); Instruction context: 9 | (85) call bpf_cpumask_empty#62852 10 | (54) w0 &= 1 >>> 11 | (56) if w0 != 0x0 goto pc+7 12 | (18) r1 = 0xffffc90000252000 Suggestion: Release or transfer ownership of the acquired resource on every path before the program exits.Patch layout: - Patches 1-2 add the initial renderer, source-line lookup, and separate source and instruction context blocks. Reusable report sections arrive with their first category-specific consumers. - Patches 3-7 add bounded, growable environment-owned diagnostic history. It grows to 64 MiB and then retains the newest events in a rotating buffer. The history follows the active verifier path and is pruned when backtracking; it records branch outcomes, material register changes, reference lifetime events, and execution-context events so reports can explain the path and causal state transitions that led to the failure. - Patches 8-14 add the first category-specific reports. These patches hook selected verifier failure sites and choose the evidence that is useful for that error class.Evaluation~~~~~~~~~~The evaluation below is retained from v4 while v5 changes are in progress.It includes two Verifier Limit cases removed from v5 and must be refreshedbefore posting.To quantitatively assess diagnostic quality beyond subjective humanfeedback, we use AI models (called over APIs) and veristat metrics tocompare results.Models are used as a way to measure repair utility of the extradiagnostics over a fixed test set. Each prompt contains only a sanitizedsource snippet and either the legacy verifier log or the new diagnosticlog. To avoid leaking the answer through the test itself, comments,annotations, and other source hints that describe the intended failurewere removed. The model is not given internet access, repository access,test execution, verifier access, or the expected fix. The expectedcauses and intended repairs are kept outside the prompt. Under thoseconstraints, correctness, exact repair rate, output size, reasoningtokens, cost, and wall time provide a proxy for whether the additionalverifier context makes the failure easier to understand and turn into asource-level fix.Verifier cost is assessed by forcing the collection of diagnosticsinformation during normal verification. By default, this information iscollected and processed only when verbose logs are enabled, but forcingit even without a verbose log helps us measure the CPU time and memorycost of the extra data.Both evaluations are covered in the sections below.Repair Quality--------------Repair quality is measured by asking API-only models to propose sourcefixes from a sanitized source snippet and verifier log. The criterion isscore >= 3 on a 0-4 local grading scale, where 3 means a likely fix withincomplete detail and 4 means an actionable source-level fix. Score 4 isreported separately as the exact repair rate. The reported model setcontains 596 completed API responses: 298 diagnostic and 298 legacy.Main results (details available in Appendix): Metric Diagnostic Legacy Delta ---------------------------------- ----------- ----------- -------- Answers 298 298 Success rate 97.0% 97.3% -0.3 pp Exact repair rate 82.2% 72.1% +10.1 pp Mean score 3.79 3.69 +0.10 Solver cost $8.93 $10.37 -13.8% Mean output tokens per answer 1662 1975 -15.8% Mean reasoning tokens per answer 951 1080 -11.9% Mean wall time per answer 37.3s 44.1s -15.4%Diagnostic prompts carry more input context. The resulting answers arestill shorter and cheaper. In this run, diagnostics do not materiallychange the coarse success rate, but they increase exact repairs by 10.1percentage points while reducing cost, output tokens, reasoning tokens,and wall time.Verifier cost-------------Verifier cost is measured with veristat over the BPF selftest programsselected by tools/testing/selftests/bpf/veristat.cfg, with fiverepetitions per configuration. With diagnostics gated by log level, walltime and verifier duration stay close to baseline. Forcing diagnosticson for every verifier run adds modest overhead on this workload.memory.peak is measured with cgroup v2 memory accounting for eachprogram load. The table reports the mean wall time, the mean summedverifier duration, and the mean of the per-repetition maximummemory.peak values. Configuration Wall time mean Verifier duration memory.peak ---------------------------- -------------- ----------------- ----------- bpf-next baseline 25.78s 9.86s 142 MiB diagnostics, gated 26.64s 10.16s 144 MiB diagnostics, forced on 28.01s 11.00s 148 MiBTODO~~~~Known follow-up work: - Convert more verbose-only verifier errors into category-specific reports. - Integrate loop-convergence failure summarization from Eduard. - Report candidate kfuncs/helpers for releasing owned resources. - Explore association of source variables with verifier registers where debug info permits it. - Refine suggestions per category and, where useful, link diagnostics to maintained documentation. - Bring verifier warnings into the same reporting framework.Appendix: AI repair details~~~~~~~~~~~~~~~~~~~~~~~~~~~The 20 verifier-failing selftest cases are: Case Diff Category Selftest selector ------- ------ -------------------------- --------------------------------------------- case-001 easy Call Type Safety cpumask/test_populate_invalid_destination case-002 easy Resource Lifetime Safety cpumask/test_alloc_no_release case-003 easy Register Type Safety verifier_spill_fill/check_corrupted_spill_fill case-004 easy Register Type Safety test_global_funcs/global_func12 case-005 easy Execution Context Safety preempt_lock/preempt_sleepable_helper case-006 easy Policy verifier_helper_restricted/in_bpf_prog_type_kprobe_1 case-007 medium Memory Safety dynptr/dynptr_slice_var_len1 case-008 medium Call Type Safety dynptr/test_dynptr_skb_small_buff case-009 medium Call Type Safety task_kfunc/task_kfunc_acquire_untrusted case-010 medium Register Type Safety test_global_funcs/global_func6 case-011 medium Resource Lifetime Safety dynptr/ringbuf_missing_release2 case-012 medium Execution Context Safety irq/irq_sleepable_helper_global_subprog case-013 medium Verifier Limit test_global_funcs/global_func1 case-014 hard Memory Safety verifier_helper_value_access/via_variable_no_max_check_1 case-015 hard Register Type Safety verifier_sock/invalidate_pkt_pointers_from_global_func case-016 hard Resource Lifetime Safety verifier_ref_tracking/check_free_in_one_subbranch case-017 hard Resource Lifetime Safety irq/irq_restore_ooo case-018 hard Resource Lifetime Safety res_spin_lock_failure/res_spin_lock_ooo_unlock case-019 hard Program Structure verifier_loops1/bounded_recursion case-020 hard Verifier Limit verifier_liveness_exp/liveness_exponential_complexityThe grading scale is: - 4: identifies the verifier cause and gives an actionable source-level fix. - 3: gives a likely fix, but with incomplete explanation or detail. - 2: identifies part of the issue, but not enough to fix confidently. - 1: gives only a broad verifier-area answer, or a wrong/insufficient fix. - 0: does not identify the intended verifier failure.Detailed effort metrics for the model set: Metric Variant Mean Median P99 ----------------------- ---------- -------- -------- -------- Cost per answer diagnostic $0.030 $0.019 $0.203 Cost per answer legacy $0.035 $0.018 $0.223 Input tokens diagnostic 1391 1220 4048 Input tokens legacy 1052 805 3655 Output tokens diagnostic 1662 954 8680 Output tokens legacy 1975 1034 9912 Reasoning tokens diagnostic 951 208 8108 Reasoning tokens legacy 1080 228 6322 Wall time diagnostic 37.3s 18.3s 222.7s Wall time legacy 44.1s 19.8s 255.5sPer-model results for diagnostic prompts: Model profile Ans Succ Exact Mean Cost OutK ReasK Wall ----------------------------------------- --- ----- ----- ---- ------- ---- ----- ----- anthropic-haiku-4.5-default 20 90.0 80.0 3.70 $0.087 11.4 0.0 5.0s anthropic-opus-4.8-high 20 100.0 90.0 3.90 $0.819 25.5 0.0 15.5s anthropic-opus-4.8-medium 20 95.0 90.0 3.85 $0.870 27.5 0.0 12.7s anthropic-sonnet-4.6-high 20 95.0 80.0 3.75 $0.824 48.9 0.0 21.6s anthropic-sonnet-4.6-medium 20 100.0 65.0 3.65 $0.278 12.4 0.0 6.6s openai-gpt-5.3-codex-high 20 100.0 80.0 3.80 $0.601 39.8 33.9 25.0s openai-gpt-5.3-codex-medium 20 95.0 85.0 3.80 $0.287 17.5 11.4 13.5s openai-gpt-5.5-high 20 100.0 90.0 3.90 $2.356 74.4 65.2 56.8s openai-gpt-5.5-low 20 100.0 90.0 3.90 $0.686 18.7 8.5 21.3s openai-gpt-5.5-medium 19 100.0 84.2 3.84 $1.353 41.1 31.8 37.4s openai-gpt-5.5-none 20 95.0 90.0 3.85 $0.457 11.1 0.0 10.4s openrouter-deepseek-r1-0528 20 100.0 75.0 3.75 $0.145 61.5 53.8 98.3s openrouter-deepseek-v3.2 19 100.0 78.9 3.79 $0.028 64.2 58.1 87.3s openrouter-glm-5.1-high 20 95.0 80.0 3.75 $0.113 28.8 20.7 19.3s openrouter-qwen3-coder 20 90.0 75.0 3.65 $0.028 12.4 0.0 7.1sPer-model results for legacy prompts: Model profile Ans Succ Exact Mean Cost OutK ReasK Wall ----------------------------------------- --- ----- ----- ---- ------- ---- ----- ----- anthropic-haiku-4.5-default 20 90.0 45.0 3.35 $0.081 11.6 0.0 5.0s anthropic-opus-4.8-high 20 90.0 70.0 3.60 $1.192 42.2 0.0 17.5s anthropic-opus-4.8-medium 20 95.0 85.0 3.80 $1.001 34.5 0.0 13.4s anthropic-sonnet-4.6-high 20 100.0 75.0 3.75 $1.181 74.1 0.0 24.4s anthropic-sonnet-4.6-medium 20 95.0 65.0 3.60 $0.420 23.4 0.0 12.3s openai-gpt-5.3-codex-high 20 100.0 85.0 3.85 $0.562 37.8 31.6 27.1s openai-gpt-5.3-codex-medium 20 100.0 75.0 3.75 $0.318 20.3 13.7 13.6s openai-gpt-5.5-high 19 100.0 78.9 3.79 $2.613 84.0 75.4 98.1s openai-gpt-5.5-low 20 100.0 75.0 3.75 $0.664 19.0 9.7 21.7s openai-gpt-5.5-medium 20 100.0 75.0 3.75 $1.602 50.2 41.0 56.1s openai-gpt-5.5-none 20 95.0 85.0 3.80 $0.416 10.7 0.0 10.9s openrouter-deepseek-r1-0528 20 95.0 70.0 3.65 $0.149 64.6 57.5 92.5s openrouter-deepseek-v3.2 20 100.0 60.0 3.60 $0.030 74.3 67.8 98.3s openrouter-glm-5.1-high 19 100.0 63.2 3.63 $0.115 32.1 24.9 30.4s openrouter-qwen3-coder 20 100.0 75.0 3.75 $0.022 9.5 0.0 5.4s====================Link: https://patch.msgid.link/20260815064612.378577-1-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

ac545b00ca56368b8acb498107ad4f1d91a5245d - bpf: Report Policy helper and kfunc errors

Sat, 15 Aug 2026 08:46:09 +0200

bpf: Report Policy helper and kfunc errorsAugment selected helper and kfunc allowability failures with Policy reports.These reports explain which requested operation is forbidden and why, withoutadding path history for non-path-dependent policy checks.Cover unprivileged bpf2bpf and kfunc use, helper program-type restrictions,GPL-only helpers, helper-specific allow callbacks, kfunc allowability, anddestructive kfunc capability checks.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://patch.msgid.link/20260815064612.378577-15-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

a8f4278353947d019c990a77d574dfd4d1dc9b46 - bpf: Report Program Structure CFG errors

Sat, 15 Aug 2026 08:46:08 +0200

bpf: Report Program Structure CFG errorsAugment selected whole-program and subprogram CFG validation failures withProgram Structure reports. These errors are structural rather thanpath-dependent, so the reports focus on source and instruction contextinstead of causal history.Cover direct and indirect jumps outside the program or current subprogram,unprivileged backedges, missing and out-of-range jump tables, targets in thesecond half of an ldimm64, unreachable instructions, subprogram fallthrough,and recursive bpf2bpf call graph edges.Format long jump-range reasons directly in diagnostics.c, and keep thefallthrough suggestion aligned with the verifier check by suggesting exit orexplicit jumps.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://patch.msgid.link/20260815064612.378577-14-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

99a6a288a82bf00ba0b01e72bf85164e848d1d18 - bpf: Report Execution Context Safety errors

Sat, 15 Aug 2026 08:46:07 +0200

bpf: Report Execution Context Safety errorsAugment selected sleepability and critical-section failures with ExecutionContext Safety reports. Keep the existing verifier messages and add sourcecontext, path history, and suggestions tied to the active context.Use the context history recorded earlier to anchor causal paths to lock, IRQ,RCU, and preempt regions instead of unrelated register updates.Cover global calls while holding a lock, sleepable global function calls,sleepable helpers, sleepable kfunc calls from disallowed contexts, operationsthat exit while a context is still active, and unmatched context exits.Signed-off-by: Kumar Kartikeya Dwivedi Link: https://patch.msgid.link/20260815064612.378577-13-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

66e2727395dd994e26ace31d331013acc9ce8f2e - bpf: Report Call Type Safety argument errors

Sat, 15 Aug 2026 08:46:06 +0200

bpf: Report Call Type Safety argument errorsAugment selected helper and kfunc argument-contract failures with Call TypeSafety reports. Keep the existing terse verifier messages and add reason,source context, causal register or stack-argument history, and targetedsuggestions.Cover helper register-type mismatch, helper and kfunc non-NULL pointerrequirements, release-helper ownership requirements, scalar and constant kfuncarguments, trusted and RCU pointer contracts, kfunc memory arguments,memory/length pairs, refcounted kptrs, constant strings, and IRQ flag stackarguments.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://patch.msgid.link/20260815064612.378577-12-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

5d5764627555f6beda39bc03af56428dec0c4582 - bpf: Report Resource Lifetime reference leaks

Sat, 15 Aug 2026 08:46:05 +0200

bpf: Report Resource Lifetime reference leaksAugment selected Resource Lifetime Safety failures with structured diagnosticswhile preserving the existing verifier messages.Report unreleased references from check_reference_leak() usingreference-scoped diagnostic history, and add state reports for dynptr,iterator, lock, and IRQ-flag lifetime misuse.IRQ restore mismatch and out-of-order diagnostics use IRQ context-scopedhistory when an IRQ-disabled region is active, so retained save/restore contextis still visible after per-state history removal.Signed-off-by: Kumar Kartikeya Dwivedi Link: https://patch.msgid.link/20260815064612.378577-11-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

2bdc90f5319451d825466375b4c0d7fc1e52c501 - bpf: Report Memory Safety bounds errors

Sat, 15 Aug 2026 08:46:04 +0200

bpf: Report Memory Safety bounds errorsAugment selected memory-range verifier failures with Memory Safety reportswhile preserving the existing terse verifier messages for compatibility.Cover stack spill corruption, uninitialized stack reads, variable stack helperaccesses, and check_mem_region_access() range-proof failures. The bounds reportspells out the required offset + access_size <= object_size proof with concretevalues and uses scoped diagnostic history for causal context.Signed-off-by: Kumar Kartikeya Dwivedi Link: https://patch.msgid.link/20260815064612.378577-10-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

d63284e62b3185cafe40d5fab17245ee60b6cffe - bpf: Report Register Type Safety errors

Sat, 15 Aug 2026 08:46:03 +0200

bpf: Report Register Type Safety errorsAugment selected register-state verifier failures with Register Type Safetyreports. The existing verbose verifier messages remain in place; the newreports add reason, source context, causal path, and suggestions.Cover invalid pointer dereferences, unreadable registers, missing outgoingstack arguments for bpf2bpf and kfunc calls, and rejected pointer arithmetic.Use scoped diagnostic history so reports start from the latest relevant valuechange and then show later branch outcomes.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://patch.msgid.link/20260815064612.378577-9-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

956a66e5c33fb53003ca2bc043a90ff0b671b3a5 - bpf: Track verifier context diagnostic events

Sat, 15 Aug 2026 08:46:02 +0200

bpf: Track verifier context diagnostic eventsRecord verifier context transitions in the diagnostic history so later reportscan anchor causal paths to the critical section that made an operation invalid.This covers lock, IRQ, RCU, and preempt regions without adding any newverifier error reports. Category-specific commits decide where those recordedevents should be rendered.Use context depth when selecting scoped history so nested regions anchor at theouter active region, and fall back to the earliest retained event when thematching entry was pruned.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://patch.msgid.link/20260815064612.378577-8-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

9ecd70304e28985af297726cd961a33a6fec5f67 - bpf: Track verifier reference diagnostic events

Sat, 15 Aug 2026 08:46:01 +0200

bpf: Track verifier reference diagnostic eventsAdd reference acquire and release events to diagnostic history so ResourceLifetime Safety reports can show the lifetime of a specific reference id alongthe path.Record acquisitions after the verifier assigns the reference id. Recordreleases only after release_reference_nomark() succeeds, including thekptr_xchg RCU conversion path and owning-to-non-owning conversion path thatconsume an owning reference.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://patch.msgid.link/20260815064612.378577-7-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

af4ea6e20fff383cdc2f01b9a372b4c7a0abf5ff - bpf: Track verifier register diagnostic events

Sat, 15 Aug 2026 08:46:00 +0200

bpf: Track verifier register diagnostic eventsRecord material register and outgoing stack argument changes so diagnostics canexplain how a value reached its current type, bounds, or unreadable state.Store old and new register types, scalar ranges, tnum value and mask, map andBTF type identity, and basic operand metadata in the environment-owneddiagnostic event stream.Record invalidations when packet data moves, references are released, orborrowed references leave their protected region. Register-scoped historystarts at the latest matching modification and then shows later branchoutcomes.Also record fixed stack spills and overwrites, and tag register fills fromstack so register-scoped history can follow value flow through spilled stackslots.The type_is_map_ptr() helper previously lived as a static function inkernel/bpf/log.c since commit 0c95c9fdb696 ("bpf: emit map name in registerstate if applicable and available"). Move it verbatim toinclude/linux/bpf_verifier.h as a static inline, next to the other typeclassifiers, so diagnostics.c can reuse it without duplicating the case list.Signed-off-by: Kumar Kartikeya Dwivedi Link: https://patch.msgid.link/20260815064612.378577-6-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

daf8248701b621d8df7ea134793dbb750d4887c0 - bpf: Add verifier diagnostic event log

Sat, 15 Aug 2026 08:45:58 +0200

bpf: Add verifier diagnostic event logAdd an environment-owned diagnostic history for verifier reports. Eventpayloads keep the user-facing branch history shape, while storage livesin bpf_verifier_env and follows the active verifier path.Grow the event array geometrically up to a 64 MiB limit. Once storagereaches the limit, or an allocation fails, overwrite the oldest event sodiagnostics retain the newest useful suffix without adding per-eventmetadata.Represent saved positions as absolute logical sequence numbers. A restoretruncates to a retained position. If its prefix has already been evicted,clear the abandoned suffix and preserve the missing-history position. Thiskeeps marks stable across rotation without increasing their size.Add the branch event renderer and branch recording.Signed-off-by: Kumar Kartikeya Dwivedi Link: https://patch.msgid.link/20260815064612.378577-4-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

b9c5d822f677e065971481c4bafe5d84b5451082 - bpf: Add source and instruction diagnostic context

Sat, 15 Aug 2026 08:45:57 +0200

bpf: Add source and instruction diagnostic contextTeach verifier diagnostics to annotate an instruction with BTF sourceline information and nearby BPF instructions. The renderer keeps sourcetext in a fixed-width lane and prints instructions in a stable right-handgutter.Wrap annotation text under the source line so long error labels remainreadable while the source and instruction lanes keep their fixed layout.Keeping source and instruction context in one commit preserves the visuallayout contract that later diagnostic reports rely on.Signed-off-by: Kumar Kartikeya Dwivedi Link: https://patch.msgid.link/20260815064612.378577-3-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c

5ad746166341e3c07250ee09518d7e4ab5cfb966 - bpf: Add verifier diagnostics report helpers

Sat, 15 Aug 2026 08:45:56 +0200

bpf: Add verifier diagnostics report helpersAdd the initial diagnostics renderer for verifier reports and wire it intothe BPF build. The helper emits the common failure header through theverifier log.Later patches add prose wrapping, reusable report sections, and source andinstruction context for category-specific diagnostics.Gate the helpers on normal verifier log output from the start, soBPF_LOG_STATS-only loads do not collect or render diagnostics.Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Eduard Zingerman Link: https://patch.msgid.link/20260815064612.378577-2-memxor@gmail.comSigned-off-by: Eduard Zingerman List of files: /linux/kernel/bpf/diagnostics.c