1.\" 2.\" Copyright (c) 2009 Isilon Inc http://www.isilon.com/ 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice(s), this list of conditions and the following disclaimer as 9.\" the first lines of this file unmodified other than the possible 10.\" addition of one or more copyright notices. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice(s), this list of conditions and the following disclaimer in the 13.\" documentation and/or other materials provided with the distribution. 14.\" 15.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY 16.\" EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18.\" DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY 19.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 20.\" (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 21.\" SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 22.\" CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH 25.\" DAMAGE. 26.\" 27.\" $FreeBSD$ 28.\" 29.Dd May 10, 2009 30.Dt FAIL 9 31.Os 32.Sh NAME 33.Nm KFAIL_POINT_CODE , 34.Nm KFAIL_POINT_RETURN , 35.Nm KFAIL_POINT_RETURN_VOID , 36.Nm KFAIL_POINT_ERROR , 37.Nm KFAIL_POINT_GOTO , 38.Nm fail_point , 39.Nm DEBUG_FP 40.Nd fail points 41.Sh SYNOPSIS 42.In sys/fail.h 43.Fn KFAIL_POINT_CODE "parent" "name" "code" 44.Fn KFAIL_POINT_RETURN "parent" "name" 45.Fn KFAIL_POINT_RETURN_VOID "parent" "name" 46.Fn KFAIL_POINT_ERROR "parent" "name" "error_var" 47.Fn KFAIL_POINT_GOTO "parent" "name" "error_var" "label" 48.Sh DESCRIPTION 49Fail points are used to add code points where errors may be injected 50in a user controlled fashion. 51Fail points provide a convenient wrapper around user-provided error 52injection code, providing a 53.Xr sysctl 9 54MIB, and a parser for that MIB that describes how the error 55injection code should fire. 56.Pp 57The base fail point macro is 58.Fn KFAIL_POINT_CODE 59where 60.Fa parent 61is a sysctl tree (frequently 62.Sy DEBUG_FP 63for kernel fail points, but various subsystems may wish to provide 64their own fail point trees), and 65.Fa name 66is the name of the MIB in that tree, and 67.Fa code 68is the error injection code. 69The 70.Fa code 71argument does not require braces, but it is considered good style to 72use braces for any multi-line code arguments. 73Inside the 74.Fa code 75argument, the evaluation of 76.Sy RETURN_VALUE 77is derived from the 78.Fn return 79value set in the sysctl MIB. 80See 81.Sx SYSCTL VARIABLES 82below. 83.Pp 84The remaining 85.Fn KFAIL_POINT_* 86macros are wrappers around common error injection paths: 87.Bl -inset 88.It Fn KFAIL_POINT_RETURN parent name 89is the equivalent of 90.Sy KFAIL_POINT_CODE(..., return RETURN_VALUE) 91.It Fn KFAIL_POINT_RETURN_VOID parent name 92is the equivalent of 93.Sy KFAIL_POINT_CODE(..., return) 94.It Fn KFAIL_POINT_ERROR parent name error_var 95is the equivalent of 96.Sy KFAIL_POINT_CODE(..., error_var = RETURN_VALUE) 97.It Fn KFAIL_POINT_GOTO parent name error_var label 98is the equivalent of 99.Sy KFAIL_POINT_CODE(..., { error_var = RETURN_VALUE; goto label;}) 100.El 101.Sh SYSCTL VARIABLES 102The 103.Fn KFAIL_POINT_* 104macros add sysctl MIBs where specified. 105Many base kernel MIBs can be found in the 106.Sy debug.fail_point 107tree (referenced in code by 108.Sy DEBUG_FP ) . 109.Pp 110The sysctl variable may be set using the following grammar: 111.Bd -literal 112 <fail_point> :: 113 <term> ( "->" <term> )* 114 115 <term> :: 116 ( (<float> "%") | (<integer> "*" ) )* 117 <type> 118 [ "(" <integer> ")" ] 119 120 <float> :: 121 <integer> [ "." <integer> ] | 122 "." <integer> 123 124 <type> :: 125 "off" | "return" | "sleep" | "panic" | "break" | "print" 126.Ed 127.Pp 128The <type> argument specifies which action to take: 129.Bl -tag -width ".Dv return" 130.It Sy off 131Take no action (does not trigger fail point code) 132.It Sy return 133Trigger fail point code with specified argument 134.It Sy sleep 135Sleep the specified number of milliseconds 136.It Sy panic 137Panic 138.It Sy break 139Break into the debugger, or trap if there is no debugger support 140.It Sy print 141Print that the fail point executed 142.El 143.Pp 144The <float>% and <integer>* modifiers prior to <type> control when 145<type> is executed. 146The <float>% form (e.g. "1.2%") can be used to specify a 147probability that <type> will execute. 148The <integer>* form (e.g. "5*") can be used to specify the number of 149times <type> should be executed before this <term> is disabled. 150Only the last probability and the last count are used if multiple 151are specified, i.e. "1.2%2%" is the same as "2%". 152When both a probability and a count are specified, the probability 153is evaluated before the count, i.e. "2%5*" means "2% of the time, 154but only 5 times total". 155.Pp 156The operator -> can be used to express cascading terms. 157If you specify <term1>-><term2>, it means that if <term1> does not 158.Ql execute , 159<term2> is evaluated. 160For the purpose of this operator, the return() and print() operators 161are the only types that cascade. 162A return() term only cascades if the code executes, and a print() 163term only cascades when passed a non-zero argument. 164.Sh EXAMPLES 165.Bl -tag 166.It Sy sysctl debug.fail_point.foobar="2.1%return(5)" 16721/1000ths of the time, execute 168.Fa code 169with RETURN_VALUE set to 5. 170.It Sy sysctl debug.fail_point.foobar="2%return(5)->5%return(22)" 1712/100ths of the time, execute 172.Fa code 173with RETURN_VALUE set to 5. 174If that does not happen, 5% of the time execute 175.Fa code 176with RETURN_VALUE set to 22. 177.It Sy sysctl debug.fail_point.foobar="5*return(5)->0.1%return(22)" 178For 5 times, return 5. 179After that, 1/1000th of the time, return 22. 180.It Sy sysctl debug.fail_point.foobar="0.1%5*return(5)" 181Return 5 for 1 in 1000 executions, but only 5 times total. 182.It Sy sysctl debug.fail_point.foobar="1%*sleep(50)" 1831/100th of the time, sleep 50ms. 184.El 185.Sh AUTHORS 186.An -nosplit 187This manual page was written by 188.An Zach Loafman Aq zml@FreeBSD.org . 189.Sh CAVEATS 190It is easy to shoot yourself in the foot by setting fail points too 191aggressively or setting too many in combination. 192For example, forcing 193.Fn malloc 194to fail consistently is potentially harmful to uptime. 195.Pp 196The 197.Fn sleep 198sysctl setting may not be appropriate in all situations. 199Currently, 200.Fn fail_point_eval 201does not verify whether the context is appropriate for calling 202.Fn msleep . 203