xref: /freebsd/sys/contrib/openzfs/man/man8/zfs-send.8 (revision c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5)
1.\"
2.\" CDDL HEADER START
3.\"
4.\" The contents of this file are subject to the terms of the
5.\" Common Development and Distribution License (the "License").
6.\" You may not use this file except in compliance with the License.
7.\"
8.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9.\" or http://www.opensolaris.org/os/licensing.
10.\" See the License for the specific language governing permissions
11.\" and limitations under the License.
12.\"
13.\" When distributing Covered Code, include this CDDL HEADER in each
14.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15.\" If applicable, add the following below this CDDL HEADER, with the
16.\" fields enclosed by brackets "[]" replaced with your own identifying
17.\" information: Portions Copyright [yyyy] [name of copyright owner]
18.\"
19.\" CDDL HEADER END
20.\"
21.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved.
22.\" Copyright 2011 Joshua M. Clulow <josh@sysmgr.org>
23.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved.
24.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
25.\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
26.\" Copyright (c) 2014 by Adam Stevko. All rights reserved.
27.\" Copyright (c) 2014 Integros [integros.com]
28.\" Copyright 2019 Richard Laager. All rights reserved.
29.\" Copyright 2018 Nexenta Systems, Inc.
30.\" Copyright 2019 Joyent, Inc.
31.\"
32.Dd April 15, 2021
33.Dt ZFS-SEND 8
34.Os
35.
36.Sh NAME
37.Nm zfs-send
38.Nd generate backup stream of ZFS dataset
39.Sh SYNOPSIS
40.Nm zfs
41.Cm send
42.Op Fl DLPRbcehnpsvw
43.Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot
44.Ar snapshot
45.Nm zfs
46.Cm send
47.Op Fl DLPcensvw
48.Op Fl i Ar snapshot Ns | Ns Ar bookmark
49.Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
50.Nm zfs
51.Cm send
52.Fl -redact Ar redaction_bookmark
53.Op Fl DLPcenpv
54.Op Fl i Ar snapshot Ns | Ns Ar bookmark
55.Ar snapshot
56.Nm zfs
57.Cm send
58.Op Fl Penv
59.Fl t
60.Ar receive_resume_token
61.Nm zfs
62.Cm send
63.Op Fl Pnv
64.Fl S Ar filesystem
65.Nm zfs
66.Cm redact
67.Ar snapshot redaction_bookmark
68.Ar redaction_snapshot Ns …
69.
70.Sh DESCRIPTION
71.Bl -tag -width ""
72.It Xo
73.Nm zfs
74.Cm send
75.Op Fl DLPRbcehnpvw
76.Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot
77.Ar snapshot
78.Xc
79Creates a stream representation of the second
80.Ar snapshot ,
81which is written to standard output.
82The output can be redirected to a file or to a different system
83.Po for example, using
84.Xr ssh 1
85.Pc .
86By default, a full stream is generated.
87.Bl -tag -width "-D"
88.It Fl D , -dedup
89Deduplicated send is no longer supported.
90This flag is accepted for backwards compatibility, but a regular,
91non-deduplicated stream will be generated.
92.It Fl I Ar snapshot
93Generate a stream package that sends all intermediary snapshots from the first
94snapshot to the second snapshot.
95For example,
96.Fl I Em @a Em fs@d
97is similar to
98.Fl i Em @a Em fs@b Ns \&; Fl i Em @b Em fs@c Ns \&; Fl i Em @c Em fs@d .
99The incremental source may be specified as with the
100.Fl i
101option.
102.It Fl L , -large-block
103Generate a stream which may contain blocks larger than 128KB.
104This flag has no effect if the
105.Sy large_blocks
106pool feature is disabled, or if the
107.Sy recordsize
108property of this filesystem has never been set above 128KB.
109The receiving system must have the
110.Sy large_blocks
111pool feature enabled as well.
112See
113.Xr zpool-features 7
114for details on ZFS feature flags and the
115.Sy large_blocks
116feature.
117.It Fl P , -parsable
118Print machine-parsable verbose information about the stream package generated.
119.It Fl R , -replicate
120Generate a replication stream package, which will replicate the specified
121file system, and all descendent file systems, up to the named snapshot.
122When received, all properties, snapshots, descendent file systems, and clones
123are preserved.
124.Pp
125If the
126.Fl i
127or
128.Fl I
129flags are used in conjunction with the
130.Fl R
131flag, an incremental replication stream is generated.
132The current values of properties, and current snapshot and file system names are
133set when the stream is received.
134If the
135.Fl F
136flag is specified when this stream is received, snapshots and file systems that
137do not exist on the sending side are destroyed.
138If the
139.Fl R
140flag is used to send encrypted datasets, then
141.Fl w
142must also be specified.
143.It Fl e , -embed
144Generate a more compact stream by using
145.Sy WRITE_EMBEDDED
146records for blocks which are stored more compactly on disk by the
147.Sy embedded_data
148pool feature.
149This flag has no effect if the
150.Sy embedded_data
151feature is disabled.
152The receiving system must have the
153.Sy embedded_data
154feature enabled.
155If the
156.Sy lz4_compress
157feature is active on the sending system, then the receiving system must have
158that feature enabled as well.
159Datasets that are sent with this flag may not be
160received as an encrypted dataset, since encrypted datasets cannot use the
161.Sy embedded_data
162feature.
163See
164.Xr zpool-features 7
165for details on ZFS feature flags and the
166.Sy embedded_data
167feature.
168.It Fl b , -backup
169Sends only received property values whether or not they are overridden by local
170settings, but only if the dataset has ever been received.
171Use this option when you want
172.Nm zfs Cm receive
173to restore received properties backed up on the sent dataset and to avoid
174sending local settings that may have nothing to do with the source dataset,
175but only with how the data is backed up.
176.It Fl c , -compressed
177Generate a more compact stream by using compressed WRITE records for blocks
178which are compressed on disk and in memory
179.Po see the
180.Sy compression
181property for details
182.Pc .
183If the
184.Sy lz4_compress
185feature is active on the sending system, then the receiving system must have
186that feature enabled as well.
187If the
188.Sy large_blocks
189feature is enabled on the sending system but the
190.Fl L
191option is not supplied in conjunction with
192.Fl c ,
193then the data will be decompressed before sending so it can be split into
194smaller block sizes.
195Streams sent with
196.Fl c
197will not have their data recompressed on the receiver side using
198.Fl o Sy compress Ns = Ar value .
199The data will stay compressed as it was from the sender.
200The new compression property will be set for future data.
201.It Fl w , -raw
202For encrypted datasets, send data exactly as it exists on disk.
203This allows backups to be taken even if encryption keys are not currently loaded.
204The backup may then be received on an untrusted machine since that machine will
205not have the encryption keys to read the protected data or alter it without
206being detected.
207Upon being received, the dataset will have the same encryption
208keys as it did on the send side, although the
209.Sy keylocation
210property will be defaulted to
211.Sy prompt
212if not otherwise provided.
213For unencrypted datasets, this flag will be equivalent to
214.Fl Lec .
215Note that if you do not use this flag for sending encrypted datasets, data will
216be sent unencrypted and may be re-encrypted with a different encryption key on
217the receiving system, which will disable the ability to do a raw send to that
218system for incrementals.
219.It Fl h , -holds
220Generate a stream package that includes any snapshot holds (created with the
221.Nm zfs Cm hold
222command), and indicating to
223.Nm zfs Cm receive
224that the holds be applied to the dataset on the receiving system.
225.It Fl i Ar snapshot
226Generate an incremental stream from the first
227.Ar snapshot
228.Pq the incremental source
229to the second
230.Ar snapshot
231.Pq the incremental target .
232The incremental source can be specified as the last component of the snapshot
233name
234.Po the
235.Sy @
236character and following
237.Pc
238and it is assumed to be from the same file system as the incremental target.
239.Pp
240If the destination is a clone, the source may be the origin snapshot, which must
241be fully specified
242.Po for example,
243.Em pool/fs@origin ,
244not just
245.Em @origin
246.Pc .
247.It Fl n , -dryrun
248Do a dry-run
249.Pq Qq No-op
250send.
251Do not generate any actual send data.
252This is useful in conjunction with the
253.Fl v
254or
255.Fl P
256flags to determine what data will be sent.
257In this case, the verbose output will be written to standard output
258.Po contrast with a non-dry-run, where the stream is written to standard output
259and the verbose output goes to standard error
260.Pc .
261.It Fl p , -props
262Include the dataset's properties in the stream.
263This flag is implicit when
264.Fl R
265is specified.
266The receiving system must also support this feature.
267Sends of encrypted datasets must use
268.Fl w
269when using this flag.
270.It Fl s , -skip-missing
271Allows sending a replication stream even when there are snapshots missing in the
272hierarchy.
273When a snapshot is missing, instead of throwing an error and aborting the send,
274a warning is printed to the standard error stream and the dataset to which it belongs
275and its descendents are skipped.
276This flag can only be used in conjunction with
277.Fl R .
278.It Fl v , -verbose
279Print verbose information about the stream package generated.
280This information includes a per-second report of how much data has been sent.
281.Pp
282The format of the stream is committed.
283You will be able to receive your streams on future versions of ZFS.
284.El
285.It Xo
286.Nm zfs
287.Cm send
288.Op Fl DLPcenvw
289.Op Fl i Ar snapshot Ns | Ns Ar bookmark
290.Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
291.Xc
292Generate a send stream, which may be of a filesystem, and may be incremental
293from a bookmark.
294If the destination is a filesystem or volume, the pool must be read-only, or the
295filesystem must not be mounted.
296When the stream generated from a filesystem or volume is received, the default
297snapshot name will be
298.Qq --head-- .
299.Bl -tag -width "-D"
300.It Fl D , -dedup
301Deduplicated send is no longer supported.
302This flag is accepted for backwards compatibility, but a regular,
303non-deduplicated stream will be generated.
304.It Fl L , -large-block
305Generate a stream which may contain blocks larger than 128KB.
306This flag has no effect if the
307.Sy large_blocks
308pool feature is disabled, or if the
309.Sy recordsize
310property of this filesystem has never been set above 128KB.
311The receiving system must have the
312.Sy large_blocks
313pool feature enabled as well.
314See
315.Xr zpool-features 7
316for details on ZFS feature flags and the
317.Sy large_blocks
318feature.
319.It Fl P , -parsable
320Print machine-parsable verbose information about the stream package generated.
321.It Fl c , -compressed
322Generate a more compact stream by using compressed WRITE records for blocks
323which are compressed on disk and in memory
324.Po see the
325.Sy compression
326property for details
327.Pc .
328If the
329.Sy lz4_compress
330feature is active on the sending system, then the receiving system must have
331that feature enabled as well.
332If the
333.Sy large_blocks
334feature is enabled on the sending system but the
335.Fl L
336option is not supplied in conjunction with
337.Fl c ,
338then the data will be decompressed before sending so it can be split into
339smaller block sizes.
340.It Fl w , -raw
341For encrypted datasets, send data exactly as it exists on disk.
342This allows backups to be taken even if encryption keys are not currently loaded.
343The backup may then be received on an untrusted machine since that machine will
344not have the encryption keys to read the protected data or alter it without
345being detected.
346Upon being received, the dataset will have the same encryption
347keys as it did on the send side, although the
348.Sy keylocation
349property will be defaulted to
350.Sy prompt
351if not otherwise provided.
352For unencrypted datasets, this flag will be equivalent to
353.Fl Lec .
354Note that if you do not use this flag for sending encrypted datasets, data will
355be sent unencrypted and may be re-encrypted with a different encryption key on
356the receiving system, which will disable the ability to do a raw send to that
357system for incrementals.
358.It Fl e , -embed
359Generate a more compact stream by using
360.Sy WRITE_EMBEDDED
361records for blocks which are stored more compactly on disk by the
362.Sy embedded_data
363pool feature.
364This flag has no effect if the
365.Sy embedded_data
366feature is disabled.
367The receiving system must have the
368.Sy embedded_data
369feature enabled.
370If the
371.Sy lz4_compress
372feature is active on the sending system, then the receiving system must have
373that feature enabled as well.
374Datasets that are sent with this flag may not be received as an encrypted dataset,
375since encrypted datasets cannot use the
376.Sy embedded_data
377feature.
378See
379.Xr zpool-features 7
380for details on ZFS feature flags and the
381.Sy embedded_data
382feature.
383.It Fl i Ar snapshot Ns | Ns Ar bookmark
384Generate an incremental send stream.
385The incremental source must be an earlier snapshot in the destination's history.
386It will commonly be an earlier snapshot in the destination's file system, in
387which case it can be specified as the last component of the name
388.Po the
389.Sy #
390or
391.Sy @
392character and following
393.Pc .
394.Pp
395If the incremental target is a clone, the incremental source can be the origin
396snapshot, or an earlier snapshot in the origin's filesystem, or the origin's
397origin, etc.
398.It Fl n , -dryrun
399Do a dry-run
400.Pq Qq No-op
401send.
402Do not generate any actual send data.
403This is useful in conjunction with the
404.Fl v
405or
406.Fl P
407flags to determine what data will be sent.
408In this case, the verbose output will be written to standard output
409.Po contrast with a non-dry-run, where the stream is written to standard output
410and the verbose output goes to standard error
411.Pc .
412.It Fl v , -verbose
413Print verbose information about the stream package generated.
414This information includes a per-second report of how much data has been sent.
415.El
416.It Xo
417.Nm zfs
418.Cm send
419.Fl -redact Ar redaction_bookmark
420.Op Fl DLPcenpv
421.Op Fl i Ar snapshot Ns | Ns Ar bookmark
422.Ar snapshot
423.Xc
424Generate a redacted send stream.
425This send stream contains all blocks from the snapshot being sent that aren't
426included in the redaction list contained in the bookmark specified by the
427.Fl -redact
428(or
429.Fl d )
430flag.
431The resulting send stream is said to be redacted with respect to the snapshots
432the bookmark specified by the
433.Fl -redact No flag was created with.
434The bookmark must have been created by running
435.Nm zfs Cm redact
436on the snapshot being sent.
437.Pp
438This feature can be used to allow clones of a filesystem to be made available on
439a remote system, in the case where their parent need not (or needs to not) be
440usable.
441For example, if a filesystem contains sensitive data, and it has clones where
442that sensitive data has been secured or replaced with dummy data, redacted sends
443can be used to replicate the secured data without replicating the original
444sensitive data, while still sharing all possible blocks.
445A snapshot that has been redacted with respect to a set of snapshots will
446contain all blocks referenced by at least one snapshot in the set, but will
447contain none of the blocks referenced by none of the snapshots in the set.
448In other words, if all snapshots in the set have modified a given block in the
449parent, that block will not be sent; but if one or more snapshots have not
450modified a block in the parent, they will still reference the parent's block, so
451that block will be sent.
452Note that only user data will be redacted.
453.Pp
454When the redacted send stream is received, we will generate a redacted
455snapshot.
456Due to the nature of redaction, a redacted dataset can only be used in the
457following ways:
458.Bl -enum -width "a."
459.It
460To receive, as a clone, an incremental send from the original snapshot to one
461of the snapshots it was redacted with respect to.
462In this case, the stream will produce a valid dataset when received because all
463blocks that were redacted in the parent are guaranteed to be present in the
464child's send stream.
465This use case will produce a normal snapshot, which can be used just like other
466snapshots.
467.
468.It
469To receive an incremental send from the original snapshot to something
470redacted with respect to a subset of the set of snapshots the initial snapshot
471was redacted with respect to.
472In this case, each block that was redacted in the original is still redacted
473(redacting with respect to additional snapshots causes less data to be redacted
474(because the snapshots define what is permitted, and everything else is
475redacted)).
476This use case will produce a new redacted snapshot.
477.It
478To receive an incremental send from a redaction bookmark of the original
479snapshot that was created when redacting with respect to a subset of the set of
480snapshots the initial snapshot was created with respect to
481anything else.
482A send stream from such a redaction bookmark will contain all of the blocks
483necessary to fill in any redacted data, should it be needed, because the sending
484system is aware of what blocks were originally redacted.
485This will either produce a normal snapshot or a redacted one, depending on
486whether the new send stream is redacted.
487.It
488To receive an incremental send from a redacted version of the initial
489snapshot that is redacted with respect to a subject of the set of snapshots the
490initial snapshot was created with respect to.
491A send stream from a compatible redacted dataset will contain all of the blocks
492necessary to fill in any redacted data.
493This will either produce a normal snapshot or a redacted one, depending on
494whether the new send stream is redacted.
495.It
496To receive a full send as a clone of the redacted snapshot.
497Since the stream is a full send, it definitionally contains all the data needed
498to create a new dataset.
499This use case will either produce a normal snapshot or a redacted one, depending
500on whether the full send stream was redacted.
501.El
502.Pp
503These restrictions are detected and enforced by
504.Nm zfs Cm receive ;
505a redacted send stream will contain the list of snapshots that the stream is
506redacted with respect to.
507These are stored with the redacted snapshot, and are used to detect and
508correctly handle the cases above.
509Note that for technical reasons,
510raw sends and redacted sends cannot be combined at this time.
511.It Xo
512.Nm zfs
513.Cm send
514.Op Fl Penv
515.Fl t
516.Ar receive_resume_token
517.Xc
518Creates a send stream which resumes an interrupted receive.
519The
520.Ar receive_resume_token
521is the value of this property on the filesystem or volume that was being
522received into.
523See the documentation for
524.Nm zfs Cm receive Fl s
525for more details.
526.It Xo
527.Nm zfs
528.Cm send
529.Op Fl Pnv
530.Op Fl i Ar snapshot Ns | Ns Ar bookmark
531.Fl S
532.Ar filesystem
533.Xc
534Generate a send stream from a dataset that has been partially received.
535.Bl -tag -width "-L"
536.It Fl S , -saved
537This flag requires that the specified filesystem previously received a resumable
538send that did not finish and was interrupted.
539In such scenarios this flag
540enables the user to send this partially received state.
541Using this flag will always use the last fully received snapshot
542as the incremental source if it exists.
543.El
544.It Xo
545.Nm zfs
546.Cm redact
547.Ar snapshot redaction_bookmark
548.Ar redaction_snapshot Ns …
549.Xc
550Generate a new redaction bookmark.
551In addition to the typical bookmark information, a redaction bookmark contains
552the list of redacted blocks and the list of redaction snapshots specified.
553The redacted blocks are blocks in the snapshot which are not referenced by any
554of the redaction snapshots.
555These blocks are found by iterating over the metadata in each redaction snapshot
556to determine what has been changed since the target snapshot.
557Redaction is designed to support redacted zfs sends; see the entry for
558.Nm zfs Cm send
559for more information on the purpose of this operation.
560If a redact operation fails partway through (due to an error or a system
561failure), the redaction can be resumed by rerunning the same command.
562.El
563.Ss Redaction
564ZFS has support for a limited version of data subsetting, in the form of
565redaction.
566Using the
567.Nm zfs Cm redact
568command, a
569.Sy redaction bookmark
570can be created that stores a list of blocks containing sensitive information.
571When provided to
572.Nm zfs Cm send ,
573this causes a
574.Sy redacted send
575to occur.
576Redacted sends omit the blocks containing sensitive information,
577replacing them with REDACT records.
578When these send streams are received, a
579.Sy redacted dataset
580is created.
581A redacted dataset cannot be mounted by default, since it is incomplete.
582It can be used to receive other send streams.
583In this way datasets can be used for data backup and replication,
584with all the benefits that zfs send and receive have to offer,
585while protecting sensitive information from being
586stored on less-trusted machines or services.
587.Pp
588For the purposes of redaction, there are two steps to the process.
589A redact step, and a send/receive step.
590First, a redaction bookmark is created.
591This is done by providing the
592.Nm zfs Cm redact
593command with a parent snapshot, a bookmark to be created, and a number of
594redaction snapshots.
595These redaction snapshots must be descendants of the parent snapshot,
596and they should modify data that is considered sensitive in some way.
597Any blocks of data modified by all of the redaction snapshots will
598be listed in the redaction bookmark, because it represents the truly sensitive
599information.
600When it comes to the send step, the send process will not send
601the blocks listed in the redaction bookmark, instead replacing them with
602REDACT records.
603When received on the target system, this will create a
604redacted dataset, missing the data that corresponds to the blocks in the
605redaction bookmark on the sending system.
606The incremental send streams from
607the original parent to the redaction snapshots can then also be received on
608the target system, and this will produce a complete snapshot that can be used
609normally.
610Incrementals from one snapshot on the parent filesystem and another
611can also be done by sending from the redaction bookmark, rather than the
612snapshots themselves.
613.Pp
614In order to make the purpose of the feature more clear, an example is provided.
615Consider a zfs filesystem containing four files.
616These files represent information for an online shopping service.
617One file contains a list of usernames and passwords, another contains purchase histories,
618a third contains click tracking data, and a fourth contains user preferences.
619The owner of this data wants to make it available for their development teams to
620test against, and their market research teams to do analysis on.
621The development teams need information about user preferences and the click
622tracking data, while the market research teams need information about purchase
623histories and user preferences.
624Neither needs access to the usernames and passwords.
625However, because all of this data is stored in one ZFS filesystem,
626it must all be sent and received together.
627In addition, the owner of the data
628wants to take advantage of features like compression, checksumming, and
629snapshots, so they do want to continue to use ZFS to store and transmit their data.
630Redaction can help them do so.
631First, they would make two clones of a snapshot of the data on the source.
632In one clone, they create the setup they want their market research team to see;
633they delete the usernames and passwords file,
634and overwrite the click tracking data with dummy information.
635In another, they create the setup they want the development teams
636to see, by replacing the passwords with fake information and replacing the
637purchase histories with randomly generated ones.
638They would then create a redaction bookmark on the parent snapshot,
639using snapshots on the two clones as redaction snapshots.
640The parent can then be sent, redacted, to the target
641server where the research and development teams have access.
642Finally, incremental sends from the parent snapshot to each of the clones can be sent
643to and received on the target server; these snapshots are identical to the
644ones on the source, and are ready to be used, while the parent snapshot on the
645target contains none of the username and password data present on the source,
646because it was removed by the redacted send operation.
647.
648.Sh SEE ALSO
649.Xr zfs-bookmark 8 ,
650.Xr zfs-receive 8 ,
651.Xr zfs-redact 8 ,
652.Xr zfs-snapshot 8
653