xref: /freebsd/sys/contrib/openzfs/man/man8/zinject.8 (revision 61145dc2b94f12f6a47344fb9aac702321880e43)
1.\" SPDX-License-Identifier: CDDL-1.0
2.\"
3.\" CDDL HEADER START
4.\"
5.\" The contents of this file are subject to the terms of the
6.\" Common Development and Distribution License (the "License").
7.\" You may not use this file except in compliance with the License.
8.\"
9.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
10.\" or https://opensource.org/licenses/CDDL-1.0.
11.\" See the License for the specific language governing permissions
12.\" and limitations under the License.
13.\"
14.\" When distributing Covered Code, include this CDDL HEADER in each
15.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
16.\" If applicable, add the following below this CDDL HEADER, with the
17.\" fields enclosed by brackets "[]" replaced with your own identifying
18.\" information: Portions Copyright [yyyy] [name of copyright owner]
19.\"
20.\" CDDL HEADER END
21.\"
22.\" Copyright 2013 Darik Horn <dajhorn@vanadac.com>. All rights reserved.
23.\" Copyright (c) 2024, 2025, Klara, Inc.
24.\"
25.\" lint-ok: WARNING: sections out of conventional order: Sh SYNOPSIS
26.\"
27.Dd January 14, 2025
28.Dt ZINJECT 8
29.Os
30.
31.Sh NAME
32.Nm zinject
33.Nd ZFS Fault Injector
34.Sh DESCRIPTION
35.Nm
36creates artificial problems in a ZFS pool by simulating data corruption
37or device failures.
38This program is dangerous.
39.
40.Sh SYNOPSIS
41.Bl -tag -width Ds
42.It Xo
43.Nm zinject
44.Xc
45List injection records.
46.
47.It Xo
48.Nm zinject
49.Fl b Ar objset : Ns Ar object : Ns Ar level : Ns Ar start : Ns Ar end
50.Op Fl f Ar frequency
51.Fl amu
52.Op pool
53.Xc
54Force an error into the pool at a bookmark.
55.
56.It Xo
57.Nm zinject
58.Fl c Ar id Ns | Ns Sy all
59.Xc
60Cancel injection records.
61.
62.It Xo
63.Nm zinject
64.Fl d Ar vdev
65.Fl A Sy degrade Ns | Ns Sy fault
66.Ar pool
67.Xc
68Force a vdev into the DEGRADED or FAULTED state.
69.
70.It Xo
71.Nm zinject
72.Fl d Ar vdev
73.Fl D Ar latency : Ns Ar lanes
74.Op Fl T Ar read|write
75.Ar pool
76.Xc
77Add an artificial delay to I/O requests on a particular
78device, such that the requests take a minimum of
79.Ar latency
80milliseconds to complete.
81Each delay has an associated number of
82.Ar lanes
83which defines the number of concurrent
84I/O requests that can be processed.
85.Pp
86For example, with a single lane delay of 10 ms
87.No (\& Ns Fl D Ar 10 : Ns Ar 1 ) ,
88the device will only be able to service a single I/O request
89at a time with each request taking 10 ms to complete.
90So, if only a single request is submitted every 10 ms, the
91average latency will be 10 ms; but if more than one request
92is submitted every 10 ms, the average latency will be more
93than 10 ms.
94.Pp
95Similarly, if a delay of 10 ms is specified to have two
96lanes
97.No (\& Ns Fl D Ar 10 : Ns Ar 2 ) ,
98then the device will be able to service
99two requests at a time, each with a minimum latency of 10 ms.
100So, if two requests are submitted every 10 ms, then
101the average latency will be 10 ms; but if more than two
102requests are submitted every 10 ms, the average latency
103will be more than 10 ms.
104.Pp
105Also note, these delays are additive.
106So two invocations of
107.Fl D Ar 10 : Ns Ar 1
108are roughly equivalent to a single invocation of
109.Fl D Ar 10 : Ns Ar 2 .
110This also means, that one can specify multiple
111lanes with differing target latencies.
112For example, an invocation of
113.Fl D Ar 10 : Ns Ar 1
114followed by
115.Fl D Ar 25 : Ns Ar 2
116will create 3 lanes on the device: one lane with a latency
117of 10 ms and two lanes with a 25 ms latency.
118.
119.It Xo
120.Nm zinject
121.Fl d Ar vdev
122.Op Fl e Ar device_error
123.Op Fl L Ar label_error
124.Op Fl T Ar failure
125.Op Fl f Ar frequency
126.Op Fl F
127.Ar pool
128.Xc
129Force a vdev error.
130.
131.It Xo
132.Nm zinject
133.Fl i Ar seconds
134.Ar pool
135.Xc
136Add an artificial delay during the future import of a pool.
137This injector is automatically cleared after the import is finished.
138.
139.It Xo
140.Nm zinject
141.Fl I
142.Op Fl s Ar seconds Ns | Ns Fl g Ar txgs
143.Ar pool
144.Xc
145Simulate a hardware failure that fails to honor a cache flush.
146.
147.It Xo
148.Nm zinject
149.Fl p Ar function
150.Ar pool
151.Xc
152Panic inside the specified function.
153.
154.It Xo
155.Nm zinject
156.Fl t Sy data
157.Fl C Ar dvas
158.Op Fl e Ar device_error
159.Op Fl f Ar frequency
160.Op Fl l Ar level
161.Op Fl r Ar range
162.Op Fl amq
163.Ar path
164.Xc
165Force an error into the contents of a file.
166.
167.It Xo
168.Nm zinject
169.Fl t Sy dnode
170.Fl C Ar dvas
171.Op Fl e Ar device_error
172.Op Fl f Ar frequency
173.Op Fl l Ar level
174.Op Fl amq
175.Ar path
176.Xc
177Force an error into the metadnode for a file or directory.
178.
179.It Xo
180.Nm zinject
181.Fl t Ar mos_type
182.Fl C Ar dvas
183.Op Fl e Ar device_error
184.Op Fl f Ar frequency
185.Op Fl l Ar level
186.Op Fl r Ar range
187.Op Fl amqu
188.Ar pool
189.Xc
190Force an error into the MOS of a pool.
191.El
192.Sh OPTIONS
193.Bl -tag -width "-C dvas"
194.It Fl a
195Flush the ARC before injection.
196.It Fl b Ar objset : Ns Ar object : Ns Ar level : Ns Ar start : Ns Ar end
197Force an error into the pool at this bookmark tuple.
198Each number is in hexadecimal, and only one block can be specified.
199.It Fl C Ar dvas
200Inject the given error only into specific DVAs.
201The mask should be specified as a list of 0-indexed DVAs separated by commas
202.No (e.g . Ar 0,2 Ns No ).
203This option is not applicable to logical data errors such as
204.Sy decompress
205and
206.Sy decrypt .
207.It Fl d Ar vdev
208A vdev specified by path or GUID.
209.It Fl e Ar device_error
210Specify
211.Bl -tag -compact -width "decompress"
212.It Sy checksum
213for an ECKSUM error,
214.It Sy decompress
215for a data decompression error,
216.It Sy decrypt
217for a data decryption error,
218.It Sy corrupt
219to flip a bit in the data after a read,
220.It Sy dtl
221for an ECHILD error,
222.It Sy io
223for an EIO error where reopening the device will succeed,
224.It Sy nxio
225for an ENXIO error where reopening the device will fail, or
226.It Sy noop
227to drop the IO without executing it, and return success.
228.El
229.Pp
230For EIO and ENXIO, the "failed" reads or writes still occur.
231The probe simply sets the error value reported by the I/O pipeline
232so it appears the read or write failed.
233Decryption errors only currently work with file data.
234.It Fl f Ar frequency
235Only inject errors a fraction of the time.
236Expressed as a real number percentage between
237.Sy 0.0001
238and
239.Sy 100 .
240.It Fl F
241Fail faster.
242Do fewer checks.
243.It Fl f Ar txgs
244Run for this many transaction groups before reporting failure.
245.It Fl h
246Print the usage message.
247.It Fl l Ar level
248Inject an error at a particular block level.
249The default is
250.Sy 0 .
251.It Fl L Ar label_error
252Set the label error region to one of
253.Sy nvlist ,
254.Sy pad1 ,
255.Sy pad2 ,
256or
257.Sy uber .
258.It Fl m
259Automatically remount the underlying filesystem.
260.It Fl q
261Quiet mode.
262Only print the handler number added.
263.It Fl r Ar range
264Inject an error over a particular logical range of an object, which
265will be translated to the appropriate blkid range according to the
266object's properties.
267.It Fl s Ar seconds
268Run for this many seconds before reporting failure.
269.It Fl T Ar type
270Inject the error into I/O of this type.
271.Bl -tag -compact -width "read, write, flush, claim, free"
272.It Sy read , Sy write , Sy flush , Sy claim , Sy free
273Fundamental I/O types
274.It Sy all
275All fundamental I/O types
276.It Sy probe
277Device probe I/O
278.El
279.It Fl t Ar mos_type
280Set this to
281.Bl -tag -compact -width "spacemap"
282.It Sy mos
283for any data in the MOS,
284.It Sy mosdir
285for an object directory,
286.It Sy config
287for the pool configuration,
288.It Sy bpobj
289for the block pointer list,
290.It Sy spacemap
291for the space map,
292.It Sy metaslab
293for the metaslab, or
294.It Sy errlog
295for the persistent error log.
296.El
297.It Fl u
298Unload the pool after injection.
299.El
300.
301.Sh ENVIRONMENT VARIABLES
302.Bl -tag -width "ZF"
303.It Ev ZFS_HOSTID
304Run
305.Nm
306in debug mode.
307.El
308.
309.Sh SEE ALSO
310.Xr zfs 8 ,
311.Xr zpool 8
312