Lines Matching +full:fault +full:- +full:log +full:- +full:enable
2 Fault injection capabilities infrastructure
5 See also drivers/md/md-faulty.c and "every_nth" module option for scsi_debug.
8 Available fault injection capabilities
9 --------------------------------------
11 - failslab
15 - fail_page_alloc
19 - fail_usercopy
23 - fail_futex
25 injects futex deadlock and uaddr fault errors.
27 - fail_sunrpc
31 - fail_make_request
34 /sys/block/<device>/make-it-fail or
35 /sys/block/<device>/<partition>/make-it-fail. (submit_bio_noacct())
37 - fail_mmc_request
42 - fail_function
48 - fail_skb_realloc
56 When the fault is injected and the reallocation is triggered, cached pointers
61 By creating these controlled fault scenarios, the system can catch instances
70 The effectiveness of this fault detection is enhanced when KASAN is
71 enabled, as it helps identify invalid memory references and use-after-free
74 - NVMe fault injection
81 - Null test block driver fault injection
90 Configure fault-injection capabilities behavior
91 -----------------------------------------------
96 fault-inject-debugfs kernel module provides some debugfs entries for runtime
97 configuration of fault-injection capabilities.
99 - /sys/kernel/debug/fail*/probability:
105 Note that one-failure-per-hundred is a very high error rate
109 - /sys/kernel/debug/fail*/interval:
114 Note that if you enable this, by setting interval>1, you will
117 - /sys/kernel/debug/fail*/times:
119 specifies how many times failures may happen at most. A value of -1
122 - /sys/kernel/debug/fail*/space:
128 - /sys/kernel/debug/fail*/verbose
134 log line per failure; '2' will print a call trace too -- useful
135 to debug the problems revealed by fault injection.
137 - /sys/kernel/debug/fail*/task-filter:
143 /proc/<pid>/make-it-fail==1.
145 - /sys/kernel/debug/fail*/require-start,
146 /sys/kernel/debug/fail*/require-end,
147 /sys/kernel/debug/fail*/reject-start,
148 /sys/kernel/debug/fail*/reject-end:
157 - /sys/kernel/debug/fail*/stacktrace-depth:
160 for a caller within [require-start,require-end) OR
161 [reject-start,reject-end).
163 - /sys/kernel/debug/fail_page_alloc/ignore-gfp-highmem:
170 - /sys/kernel/debug/failslab/cache-filter
178 - /sys/kernel/debug/failslab/ignore-gfp-wait:
179 - /sys/kernel/debug/fail_page_alloc/ignore-gfp-wait:
186 - /sys/kernel/debug/fail_page_alloc/min-order:
191 - /sys/kernel/debug/fail_futex/ignore-private:
198 - /sys/kernel/debug/fail_sunrpc/ignore-client-disconnect:
205 - /sys/kernel/debug/fail_sunrpc/ignore-server-disconnect:
212 - /sys/kernel/debug/fail_sunrpc/ignore-cache-wait:
219 - /sys/kernel/debug/fail_function/inject:
221 Format: { 'function-name' | '!function-name' | '' }
228 - /sys/kernel/debug/fail_function/injectable:
233 - NULL: retval must be 0.
234 - ERRNO: retval must be -1 to -MAX_ERRNO (-4096).
235 - ERR_NULL: retval must be 0 or -1 to -MAX_ERRNO (-4096).
237 - /sys/kernel/debug/fail_function/<function-name>/retval:
243 $ printf %#x -12 > retval
245 - /sys/kernel/debug/fail_skb_realloc/devname:
275 - /proc/<pid>/fail-nth,
276 /proc/self/task/<tid>/fail-nth:
278 Write to this file of integer N makes N-th call in the task fail.
280 that the fault setup with a previous write to this file was injected.
281 A positive integer N indicates that the fault wasn't yet injected.
284 like probability, interval, times, etc. But per-capability settings
285 (e.g. fail_futex/ignore-private) take precedence over it.
292 --------------------------
300 Since the function-level error injection forcibly changes the code path
305 - The function returns an error code if it fails, and the callers must check
308 - The function does not execute any code which can change any state before
331 There are 4 types of errors defined in include/asm-generic/error-injection.h
338 This function will return an `-errno` error code if it fails. e.g. return
339 -EINVAL if the input is wrong. This will include the functions which will
340 return an address which encodes `-errno` by ERR_PTR() macro.
343 This function will return an `-errno` or `NULL` if it fails. If the caller
348 This function will return `true` (non-zero positive value) if it fails.
355 How to add new fault injection capability
356 -----------------------------------------
358 - #include <linux/fault-inject.h>
360 - define the fault attributes
364 Please see the definition of struct fault_attr in fault-inject.h
367 - provide a way to configure fault attributes
369 - boot option
371 If you need to enable the fault injection capability from boot time, you can
376 - debugfs entries
383 - module parameters
385 If the scope of the fault injection capability is limited to a
387 configure the fault attributes.
389 - add a hook to insert failures
396 --------------------
398 - Inject slab allocation failures into module init/exit code::
403 echo Y > /sys/kernel/debug/$FAILTYPE/task-filter
406 echo -1 > /sys/kernel/debug/$FAILTYPE/times
409 echo Y > /sys/kernel/debug/$FAILTYPE/ignore-gfp-wait
413 bash -c "echo 1 > /proc/self/make-it-fail && exec $*"
416 if [ $# -eq 0 ]
428 faulty_system modprobe -r $m
431 ------------------------------------------------------------------------------
433 - Inject page allocation failures only for a specific module::
440 if [ -z $module ]
448 if [ ! -d /sys/module/$module/sections ]
454 cat /sys/module/$module/sections/.text > /sys/kernel/debug/$FAILTYPE/require-start
455 cat /sys/module/$module/sections/.data > /sys/kernel/debug/$FAILTYPE/require-end
457 echo N > /sys/kernel/debug/$FAILTYPE/task-filter
460 echo -1 > /sys/kernel/debug/$FAILTYPE/times
463 echo Y > /sys/kernel/debug/$FAILTYPE/ignore-gfp-wait
464 echo Y > /sys/kernel/debug/$FAILTYPE/ignore-gfp-highmem
465 echo 10 > /sys/kernel/debug/$FAILTYPE/stacktrace-depth
472 ------------------------------------------------------------------------------
474 - Inject open_ctree error while btrfs mount::
478 rm -f testfile.img
480 DEVICE=$(losetup --show -f testfile.img)
481 mkfs.btrfs -f $DEVICE
482 mkdir -p tmpmnt
487 printf %#x -12 > /sys/kernel/debug/$FAILTYPE/$FAILFUNC/retval
488 echo N > /sys/kernel/debug/$FAILTYPE/task-filter
491 echo -1 > /sys/kernel/debug/$FAILTYPE/times
495 mount -t btrfs $DEVICE tmpmnt
496 if [ $? -ne 0 ]
507 losetup -d $DEVICE
510 ------------------------------------------------------------------------------
512 - Inject only skbuff allocation failures ::
517 echo 1 > /sys/kernel/debug/failslab/cache-filter
518 # Turn on fault injection
524 ----------------------------------------------------
526 tools/testing/fault-injection/failcmd.sh. Please run a command
527 "./tools/testing/fault-injection/failcmd.sh --help" for more information and
532 Run a command "make -C tools/testing/selftests/ run_tests" with injecting slab
535 # ./tools/testing/fault-injection/failcmd.sh \
536 -- make -C tools/testing/selftests/ run_tests
541 # ./tools/testing/fault-injection/failcmd.sh --times=100 \
542 -- make -C tools/testing/selftests/ run_tests
548 ./tools/testing/fault-injection/failcmd.sh --times=100 \
549 -- make -C tools/testing/selftests/ run_tests
551 Systematic faults using fail-nth
552 ---------------------------------
554 The following code systematically faults 0-th, 1-st, 2-nd and so on
573 system("echo N > /sys/kernel/debug/failslab/ignore-gfp-wait");
574 sprintf(buf, "/proc/self/task/%ld/fail-nth", syscall(SYS_gettid));
586 printf("%d-th fault %c: res=%d/%d\n", i, atoi(buf) ? 'N' : 'Y',
596 1-th fault Y: res=-1/23
597 2-th fault Y: res=-1/23
598 3-th fault Y: res=-1/12
599 4-th fault Y: res=-1/12
600 5-th fault Y: res=-1/23
601 6-th fault Y: res=-1/23
602 7-th fault Y: res=-1/23
603 8-th fault Y: res=-1/12
604 9-th fault Y: res=-1/12
605 10-th fault Y: res=-1/12
606 11-th fault Y: res=-1/12
607 12-th fault Y: res=-1/12
608 13-th fault Y: res=-1/12
609 14-th fault Y: res=-1/12
610 15-th fault Y: res=-1/12
611 16-th fault N: res=0/12