1.\" 2.\" CDDL HEADER START 3.\" 4.\" The contents of this file are subject to the terms of the 5.\" Common Development and Distribution License (the "License"). 6.\" You may not use this file except in compliance with the License. 7.\" 8.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9.\" or http://www.opensolaris.org/os/licensing. 10.\" See the License for the specific language governing permissions 11.\" and limitations under the License. 12.\" 13.\" When distributing Covered Code, include this CDDL HEADER in each 14.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15.\" If applicable, add the following below this CDDL HEADER, with the 16.\" fields enclosed by brackets "[]" replaced with your own identifying 17.\" information: Portions Copyright [yyyy] [name of copyright owner] 18.\" 19.\" CDDL HEADER END 20.\" 21.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved. 22.\" Copyright 2011 Joshua M. Clulow <josh@sysmgr.org> 23.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved. 24.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved. 25.\" Copyright (c) 2014, Joyent, Inc. All rights reserved. 26.\" Copyright (c) 2014 by Adam Stevko. All rights reserved. 27.\" Copyright (c) 2014 Integros [integros.com] 28.\" Copyright 2019 Richard Laager. All rights reserved. 29.\" Copyright 2018 Nexenta Systems, Inc. 30.\" Copyright 2019 Joyent, Inc. 31.\" 32.Dd April 15, 2021 33.Dt ZFS-SEND 8 34.Os 35. 36.Sh NAME 37.Nm zfs-send 38.Nd generate backup stream of ZFS dataset 39.Sh SYNOPSIS 40.Nm zfs 41.Cm send 42.Op Fl DLPRbcehnpsvw 43.Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot 44.Ar snapshot 45.Nm zfs 46.Cm send 47.Op Fl DLPcensvw 48.Op Fl i Ar snapshot Ns | Ns Ar bookmark 49.Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot 50.Nm zfs 51.Cm send 52.Fl -redact Ar redaction_bookmark 53.Op Fl DLPcenpv 54.Op Fl i Ar snapshot Ns | Ns Ar bookmark 55.Ar snapshot 56.Nm zfs 57.Cm send 58.Op Fl Penv 59.Fl t 60.Ar receive_resume_token 61.Nm zfs 62.Cm send 63.Op Fl Pnv 64.Fl S Ar filesystem 65.Nm zfs 66.Cm redact 67.Ar snapshot redaction_bookmark 68.Ar redaction_snapshot Ns … 69. 70.Sh DESCRIPTION 71.Bl -tag -width "" 72.It Xo 73.Nm zfs 74.Cm send 75.Op Fl DLPRbcehnpvw 76.Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot 77.Ar snapshot 78.Xc 79Creates a stream representation of the second 80.Ar snapshot , 81which is written to standard output. 82The output can be redirected to a file or to a different system 83.Po for example, using 84.Xr ssh 1 85.Pc . 86By default, a full stream is generated. 87.Bl -tag -width "-D" 88.It Fl D , -dedup 89Deduplicated send is no longer supported. 90This flag is accepted for backwards compatibility, but a regular, 91non-deduplicated stream will be generated. 92.It Fl I Ar snapshot 93Generate a stream package that sends all intermediary snapshots from the first 94snapshot to the second snapshot. 95For example, 96.Fl I Em @a Em fs@d 97is similar to 98.Fl i Em @a Em fs@b Ns \&; Fl i Em @b Em fs@c Ns \&; Fl i Em @c Em fs@d . 99The incremental source may be specified as with the 100.Fl i 101option. 102.It Fl L , -large-block 103Generate a stream which may contain blocks larger than 128KB. 104This flag has no effect if the 105.Sy large_blocks 106pool feature is disabled, or if the 107.Sy recordsize 108property of this filesystem has never been set above 128KB. 109The receiving system must have the 110.Sy large_blocks 111pool feature enabled as well. 112See 113.Xr zpool-features 7 114for details on ZFS feature flags and the 115.Sy large_blocks 116feature. 117.It Fl P , -parsable 118Print machine-parsable verbose information about the stream package generated. 119.It Fl R , -replicate 120Generate a replication stream package, which will replicate the specified 121file system, and all descendent file systems, up to the named snapshot. 122When received, all properties, snapshots, descendent file systems, and clones 123are preserved. 124.Pp 125If the 126.Fl i 127or 128.Fl I 129flags are used in conjunction with the 130.Fl R 131flag, an incremental replication stream is generated. 132The current values of properties, and current snapshot and file system names are 133set when the stream is received. 134If the 135.Fl F 136flag is specified when this stream is received, snapshots and file systems that 137do not exist on the sending side are destroyed. 138If the 139.Fl R 140flag is used to send encrypted datasets, then 141.Fl w 142must also be specified. 143.It Fl e , -embed 144Generate a more compact stream by using 145.Sy WRITE_EMBEDDED 146records for blocks which are stored more compactly on disk by the 147.Sy embedded_data 148pool feature. 149This flag has no effect if the 150.Sy embedded_data 151feature is disabled. 152The receiving system must have the 153.Sy embedded_data 154feature enabled. 155If the 156.Sy lz4_compress 157feature is active on the sending system, then the receiving system must have 158that feature enabled as well. 159Datasets that are sent with this flag may not be 160received as an encrypted dataset, since encrypted datasets cannot use the 161.Sy embedded_data 162feature. 163See 164.Xr zpool-features 7 165for details on ZFS feature flags and the 166.Sy embedded_data 167feature. 168.It Fl b , -backup 169Sends only received property values whether or not they are overridden by local 170settings, but only if the dataset has ever been received. 171Use this option when you want 172.Nm zfs Cm receive 173to restore received properties backed up on the sent dataset and to avoid 174sending local settings that may have nothing to do with the source dataset, 175but only with how the data is backed up. 176.It Fl c , -compressed 177Generate a more compact stream by using compressed WRITE records for blocks 178which are compressed on disk and in memory 179.Po see the 180.Sy compression 181property for details 182.Pc . 183If the 184.Sy lz4_compress 185feature is active on the sending system, then the receiving system must have 186that feature enabled as well. 187If the 188.Sy large_blocks 189feature is enabled on the sending system but the 190.Fl L 191option is not supplied in conjunction with 192.Fl c , 193then the data will be decompressed before sending so it can be split into 194smaller block sizes. 195Streams sent with 196.Fl c 197will not have their data recompressed on the receiver side using 198.Fl o Sy compress Ns = Ar value . 199The data will stay compressed as it was from the sender. 200The new compression property will be set for future data. 201Note that uncompressed data from the sender will still attempt to 202compress on the receiver, unless you specify 203.Fl o Sy compress Ns = Em off . 204.It Fl w , -raw 205For encrypted datasets, send data exactly as it exists on disk. 206This allows backups to be taken even if encryption keys are not currently loaded. 207The backup may then be received on an untrusted machine since that machine will 208not have the encryption keys to read the protected data or alter it without 209being detected. 210Upon being received, the dataset will have the same encryption 211keys as it did on the send side, although the 212.Sy keylocation 213property will be defaulted to 214.Sy prompt 215if not otherwise provided. 216For unencrypted datasets, this flag will be equivalent to 217.Fl Lec . 218Note that if you do not use this flag for sending encrypted datasets, data will 219be sent unencrypted and may be re-encrypted with a different encryption key on 220the receiving system, which will disable the ability to do a raw send to that 221system for incrementals. 222.It Fl h , -holds 223Generate a stream package that includes any snapshot holds (created with the 224.Nm zfs Cm hold 225command), and indicating to 226.Nm zfs Cm receive 227that the holds be applied to the dataset on the receiving system. 228.It Fl i Ar snapshot 229Generate an incremental stream from the first 230.Ar snapshot 231.Pq the incremental source 232to the second 233.Ar snapshot 234.Pq the incremental target . 235The incremental source can be specified as the last component of the snapshot 236name 237.Po the 238.Sy @ 239character and following 240.Pc 241and it is assumed to be from the same file system as the incremental target. 242.Pp 243If the destination is a clone, the source may be the origin snapshot, which must 244be fully specified 245.Po for example, 246.Em pool/fs@origin , 247not just 248.Em @origin 249.Pc . 250.It Fl n , -dryrun 251Do a dry-run 252.Pq Qq No-op 253send. 254Do not generate any actual send data. 255This is useful in conjunction with the 256.Fl v 257or 258.Fl P 259flags to determine what data will be sent. 260In this case, the verbose output will be written to standard output 261.Po contrast with a non-dry-run, where the stream is written to standard output 262and the verbose output goes to standard error 263.Pc . 264.It Fl p , -props 265Include the dataset's properties in the stream. 266This flag is implicit when 267.Fl R 268is specified. 269The receiving system must also support this feature. 270Sends of encrypted datasets must use 271.Fl w 272when using this flag. 273.It Fl s , -skip-missing 274Allows sending a replication stream even when there are snapshots missing in the 275hierarchy. 276When a snapshot is missing, instead of throwing an error and aborting the send, 277a warning is printed to the standard error stream and the dataset to which it belongs 278and its descendents are skipped. 279This flag can only be used in conjunction with 280.Fl R . 281.It Fl v , -verbose 282Print verbose information about the stream package generated. 283This information includes a per-second report of how much data has been sent. 284.Pp 285The format of the stream is committed. 286You will be able to receive your streams on future versions of ZFS. 287.El 288.It Xo 289.Nm zfs 290.Cm send 291.Op Fl DLPcenvw 292.Op Fl i Ar snapshot Ns | Ns Ar bookmark 293.Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot 294.Xc 295Generate a send stream, which may be of a filesystem, and may be incremental 296from a bookmark. 297If the destination is a filesystem or volume, the pool must be read-only, or the 298filesystem must not be mounted. 299When the stream generated from a filesystem or volume is received, the default 300snapshot name will be 301.Qq --head-- . 302.Bl -tag -width "-D" 303.It Fl D , -dedup 304Deduplicated send is no longer supported. 305This flag is accepted for backwards compatibility, but a regular, 306non-deduplicated stream will be generated. 307.It Fl L , -large-block 308Generate a stream which may contain blocks larger than 128KB. 309This flag has no effect if the 310.Sy large_blocks 311pool feature is disabled, or if the 312.Sy recordsize 313property of this filesystem has never been set above 128KB. 314The receiving system must have the 315.Sy large_blocks 316pool feature enabled as well. 317See 318.Xr zpool-features 7 319for details on ZFS feature flags and the 320.Sy large_blocks 321feature. 322.It Fl P , -parsable 323Print machine-parsable verbose information about the stream package generated. 324.It Fl c , -compressed 325Generate a more compact stream by using compressed WRITE records for blocks 326which are compressed on disk and in memory 327.Po see the 328.Sy compression 329property for details 330.Pc . 331If the 332.Sy lz4_compress 333feature is active on the sending system, then the receiving system must have 334that feature enabled as well. 335If the 336.Sy large_blocks 337feature is enabled on the sending system but the 338.Fl L 339option is not supplied in conjunction with 340.Fl c , 341then the data will be decompressed before sending so it can be split into 342smaller block sizes. 343.It Fl w , -raw 344For encrypted datasets, send data exactly as it exists on disk. 345This allows backups to be taken even if encryption keys are not currently loaded. 346The backup may then be received on an untrusted machine since that machine will 347not have the encryption keys to read the protected data or alter it without 348being detected. 349Upon being received, the dataset will have the same encryption 350keys as it did on the send side, although the 351.Sy keylocation 352property will be defaulted to 353.Sy prompt 354if not otherwise provided. 355For unencrypted datasets, this flag will be equivalent to 356.Fl Lec . 357Note that if you do not use this flag for sending encrypted datasets, data will 358be sent unencrypted and may be re-encrypted with a different encryption key on 359the receiving system, which will disable the ability to do a raw send to that 360system for incrementals. 361.It Fl e , -embed 362Generate a more compact stream by using 363.Sy WRITE_EMBEDDED 364records for blocks which are stored more compactly on disk by the 365.Sy embedded_data 366pool feature. 367This flag has no effect if the 368.Sy embedded_data 369feature is disabled. 370The receiving system must have the 371.Sy embedded_data 372feature enabled. 373If the 374.Sy lz4_compress 375feature is active on the sending system, then the receiving system must have 376that feature enabled as well. 377Datasets that are sent with this flag may not be received as an encrypted dataset, 378since encrypted datasets cannot use the 379.Sy embedded_data 380feature. 381See 382.Xr zpool-features 7 383for details on ZFS feature flags and the 384.Sy embedded_data 385feature. 386.It Fl i Ar snapshot Ns | Ns Ar bookmark 387Generate an incremental send stream. 388The incremental source must be an earlier snapshot in the destination's history. 389It will commonly be an earlier snapshot in the destination's file system, in 390which case it can be specified as the last component of the name 391.Po the 392.Sy # 393or 394.Sy @ 395character and following 396.Pc . 397.Pp 398If the incremental target is a clone, the incremental source can be the origin 399snapshot, or an earlier snapshot in the origin's filesystem, or the origin's 400origin, etc. 401.It Fl n , -dryrun 402Do a dry-run 403.Pq Qq No-op 404send. 405Do not generate any actual send data. 406This is useful in conjunction with the 407.Fl v 408or 409.Fl P 410flags to determine what data will be sent. 411In this case, the verbose output will be written to standard output 412.Po contrast with a non-dry-run, where the stream is written to standard output 413and the verbose output goes to standard error 414.Pc . 415.It Fl v , -verbose 416Print verbose information about the stream package generated. 417This information includes a per-second report of how much data has been sent. 418.El 419.It Xo 420.Nm zfs 421.Cm send 422.Fl -redact Ar redaction_bookmark 423.Op Fl DLPcenpv 424.Op Fl i Ar snapshot Ns | Ns Ar bookmark 425.Ar snapshot 426.Xc 427Generate a redacted send stream. 428This send stream contains all blocks from the snapshot being sent that aren't 429included in the redaction list contained in the bookmark specified by the 430.Fl -redact 431(or 432.Fl d ) 433flag. 434The resulting send stream is said to be redacted with respect to the snapshots 435the bookmark specified by the 436.Fl -redact No flag was created with. 437The bookmark must have been created by running 438.Nm zfs Cm redact 439on the snapshot being sent. 440.Pp 441This feature can be used to allow clones of a filesystem to be made available on 442a remote system, in the case where their parent need not (or needs to not) be 443usable. 444For example, if a filesystem contains sensitive data, and it has clones where 445that sensitive data has been secured or replaced with dummy data, redacted sends 446can be used to replicate the secured data without replicating the original 447sensitive data, while still sharing all possible blocks. 448A snapshot that has been redacted with respect to a set of snapshots will 449contain all blocks referenced by at least one snapshot in the set, but will 450contain none of the blocks referenced by none of the snapshots in the set. 451In other words, if all snapshots in the set have modified a given block in the 452parent, that block will not be sent; but if one or more snapshots have not 453modified a block in the parent, they will still reference the parent's block, so 454that block will be sent. 455Note that only user data will be redacted. 456.Pp 457When the redacted send stream is received, we will generate a redacted 458snapshot. 459Due to the nature of redaction, a redacted dataset can only be used in the 460following ways: 461.Bl -enum -width "a." 462.It 463To receive, as a clone, an incremental send from the original snapshot to one 464of the snapshots it was redacted with respect to. 465In this case, the stream will produce a valid dataset when received because all 466blocks that were redacted in the parent are guaranteed to be present in the 467child's send stream. 468This use case will produce a normal snapshot, which can be used just like other 469snapshots. 470. 471.It 472To receive an incremental send from the original snapshot to something 473redacted with respect to a subset of the set of snapshots the initial snapshot 474was redacted with respect to. 475In this case, each block that was redacted in the original is still redacted 476(redacting with respect to additional snapshots causes less data to be redacted 477(because the snapshots define what is permitted, and everything else is 478redacted)). 479This use case will produce a new redacted snapshot. 480.It 481To receive an incremental send from a redaction bookmark of the original 482snapshot that was created when redacting with respect to a subset of the set of 483snapshots the initial snapshot was created with respect to 484anything else. 485A send stream from such a redaction bookmark will contain all of the blocks 486necessary to fill in any redacted data, should it be needed, because the sending 487system is aware of what blocks were originally redacted. 488This will either produce a normal snapshot or a redacted one, depending on 489whether the new send stream is redacted. 490.It 491To receive an incremental send from a redacted version of the initial 492snapshot that is redacted with respect to a subject of the set of snapshots the 493initial snapshot was created with respect to. 494A send stream from a compatible redacted dataset will contain all of the blocks 495necessary to fill in any redacted data. 496This will either produce a normal snapshot or a redacted one, depending on 497whether the new send stream is redacted. 498.It 499To receive a full send as a clone of the redacted snapshot. 500Since the stream is a full send, it definitionally contains all the data needed 501to create a new dataset. 502This use case will either produce a normal snapshot or a redacted one, depending 503on whether the full send stream was redacted. 504.El 505.Pp 506These restrictions are detected and enforced by 507.Nm zfs Cm receive ; 508a redacted send stream will contain the list of snapshots that the stream is 509redacted with respect to. 510These are stored with the redacted snapshot, and are used to detect and 511correctly handle the cases above. 512Note that for technical reasons, 513raw sends and redacted sends cannot be combined at this time. 514.It Xo 515.Nm zfs 516.Cm send 517.Op Fl Penv 518.Fl t 519.Ar receive_resume_token 520.Xc 521Creates a send stream which resumes an interrupted receive. 522The 523.Ar receive_resume_token 524is the value of this property on the filesystem or volume that was being 525received into. 526See the documentation for 527.Nm zfs Cm receive Fl s 528for more details. 529.It Xo 530.Nm zfs 531.Cm send 532.Op Fl Pnv 533.Op Fl i Ar snapshot Ns | Ns Ar bookmark 534.Fl S 535.Ar filesystem 536.Xc 537Generate a send stream from a dataset that has been partially received. 538.Bl -tag -width "-L" 539.It Fl S , -saved 540This flag requires that the specified filesystem previously received a resumable 541send that did not finish and was interrupted. 542In such scenarios this flag 543enables the user to send this partially received state. 544Using this flag will always use the last fully received snapshot 545as the incremental source if it exists. 546.El 547.It Xo 548.Nm zfs 549.Cm redact 550.Ar snapshot redaction_bookmark 551.Ar redaction_snapshot Ns … 552.Xc 553Generate a new redaction bookmark. 554In addition to the typical bookmark information, a redaction bookmark contains 555the list of redacted blocks and the list of redaction snapshots specified. 556The redacted blocks are blocks in the snapshot which are not referenced by any 557of the redaction snapshots. 558These blocks are found by iterating over the metadata in each redaction snapshot 559to determine what has been changed since the target snapshot. 560Redaction is designed to support redacted zfs sends; see the entry for 561.Nm zfs Cm send 562for more information on the purpose of this operation. 563If a redact operation fails partway through (due to an error or a system 564failure), the redaction can be resumed by rerunning the same command. 565.El 566.Ss Redaction 567ZFS has support for a limited version of data subsetting, in the form of 568redaction. 569Using the 570.Nm zfs Cm redact 571command, a 572.Sy redaction bookmark 573can be created that stores a list of blocks containing sensitive information. 574When provided to 575.Nm zfs Cm send , 576this causes a 577.Sy redacted send 578to occur. 579Redacted sends omit the blocks containing sensitive information, 580replacing them with REDACT records. 581When these send streams are received, a 582.Sy redacted dataset 583is created. 584A redacted dataset cannot be mounted by default, since it is incomplete. 585It can be used to receive other send streams. 586In this way datasets can be used for data backup and replication, 587with all the benefits that zfs send and receive have to offer, 588while protecting sensitive information from being 589stored on less-trusted machines or services. 590.Pp 591For the purposes of redaction, there are two steps to the process. 592A redact step, and a send/receive step. 593First, a redaction bookmark is created. 594This is done by providing the 595.Nm zfs Cm redact 596command with a parent snapshot, a bookmark to be created, and a number of 597redaction snapshots. 598These redaction snapshots must be descendants of the parent snapshot, 599and they should modify data that is considered sensitive in some way. 600Any blocks of data modified by all of the redaction snapshots will 601be listed in the redaction bookmark, because it represents the truly sensitive 602information. 603When it comes to the send step, the send process will not send 604the blocks listed in the redaction bookmark, instead replacing them with 605REDACT records. 606When received on the target system, this will create a 607redacted dataset, missing the data that corresponds to the blocks in the 608redaction bookmark on the sending system. 609The incremental send streams from 610the original parent to the redaction snapshots can then also be received on 611the target system, and this will produce a complete snapshot that can be used 612normally. 613Incrementals from one snapshot on the parent filesystem and another 614can also be done by sending from the redaction bookmark, rather than the 615snapshots themselves. 616.Pp 617In order to make the purpose of the feature more clear, an example is provided. 618Consider a zfs filesystem containing four files. 619These files represent information for an online shopping service. 620One file contains a list of usernames and passwords, another contains purchase histories, 621a third contains click tracking data, and a fourth contains user preferences. 622The owner of this data wants to make it available for their development teams to 623test against, and their market research teams to do analysis on. 624The development teams need information about user preferences and the click 625tracking data, while the market research teams need information about purchase 626histories and user preferences. 627Neither needs access to the usernames and passwords. 628However, because all of this data is stored in one ZFS filesystem, 629it must all be sent and received together. 630In addition, the owner of the data 631wants to take advantage of features like compression, checksumming, and 632snapshots, so they do want to continue to use ZFS to store and transmit their data. 633Redaction can help them do so. 634First, they would make two clones of a snapshot of the data on the source. 635In one clone, they create the setup they want their market research team to see; 636they delete the usernames and passwords file, 637and overwrite the click tracking data with dummy information. 638In another, they create the setup they want the development teams 639to see, by replacing the passwords with fake information and replacing the 640purchase histories with randomly generated ones. 641They would then create a redaction bookmark on the parent snapshot, 642using snapshots on the two clones as redaction snapshots. 643The parent can then be sent, redacted, to the target 644server where the research and development teams have access. 645Finally, incremental sends from the parent snapshot to each of the clones can be sent 646to and received on the target server; these snapshots are identical to the 647ones on the source, and are ready to be used, while the parent snapshot on the 648target contains none of the username and password data present on the source, 649because it was removed by the redacted send operation. 650. 651.Sh SEE ALSO 652.Xr zfs-bookmark 8 , 653.Xr zfs-receive 8 , 654.Xr zfs-redact 8 , 655.Xr zfs-snapshot 8 656