1# Welcome to libarchive! 2 3The libarchive project develops a portable, efficient C library that 4can read and write streaming archives in a variety of formats. It 5also includes implementations of the common `tar`, `cpio`, and `zcat` 6command-line tools that use the libarchive library. 7 8## Questions? Issues? 9 10* https://www.libarchive.org is the home for ongoing 11 libarchive development, including documentation, 12 and links to the libarchive mailing lists. 13* To report an issue, use the issue tracker at 14 https://github.com/libarchive/libarchive/issues 15* To submit an enhancement to libarchive, please 16 submit a pull request via GitHub: https://github.com/libarchive/libarchive/pulls 17 18## Contents of the Distribution 19 20This distribution bundle includes the following major components: 21 22* **libarchive**: a library for reading and writing streaming archives 23* **tar**: the 'bsdtar' program is a full-featured 'tar' implementation built on libarchive 24* **cpio**: the 'bsdcpio' program is a different interface to essentially the same functionality 25* **cat**: the 'bsdcat' program is a simple replacement tool for zcat, bzcat, xzcat, and such 26* **unzip**: the 'bsdunzip' program is a simple replacement tool for Info-ZIP's unzip 27* **examples**: Some small example programs that you may find useful. 28* **examples/minitar**: a compact sample demonstrating use of libarchive. 29* **contrib**: Various items sent to me by third parties; please contact the authors with any questions. 30 31The top-level directory contains the following information files: 32 33* **NEWS** - highlights of recent changes 34* **COPYING** - what you can do with this 35* **INSTALL** - installation instructions 36* **README** - this file 37* **CMakeLists.txt** - input for "cmake" build tool, see INSTALL 38* **configure** - configuration script, see INSTALL for details. If your copy of the source lacks a `configure` script, you can try to construct it by running the script in `build/autogen.sh` (or use `cmake`). 39 40The following files in the top-level directory are used by the 'configure' script: 41 42* `Makefile.am`, `aclocal.m4`, `configure.ac` - used to build this distribution, only needed by maintainers 43* `Makefile.in`, `config.h.in` - templates used by configure script 44 45## Documentation 46 47In addition to the informational articles and documentation 48in the online [libarchive Wiki](https://github.com/libarchive/libarchive/wiki), 49the distribution also includes a number of manual pages: 50 51 * bsdtar.1 explains the use of the bsdtar program 52 * bsdcpio.1 explains the use of the bsdcpio program 53 * bsdcat.1 explains the use of the bsdcat program 54 * libarchive.3 gives an overview of the library as a whole 55 * archive_read.3, archive_write.3, archive_write_disk.3, and 56 archive_read_disk.3 provide detailed calling sequences for the read 57 and write APIs 58 * archive_entry.3 details the "struct archive_entry" utility class 59 * archive_internals.3 provides some insight into libarchive's 60 internal structure and operation. 61 * libarchive-formats.5 documents the file formats supported by the library 62 * cpio.5, mtree.5, and tar.5 provide detailed information about these 63 popular archive formats, including hard-to-find details about 64 modern cpio and tar variants. 65 66The manual pages above are provided in the 'doc' directory in 67a number of different formats. 68 69You should also read the copious comments in `archive.h` and the 70source code for the sample programs for more details. Please let us 71know about any errors or omissions you find. 72 73## Supported Formats 74 75Currently, the library automatically detects and reads the following formats: 76 77 * Old V7 tar archives 78 * POSIX ustar 79 * GNU tar format (including GNU long filenames, long link names, and sparse files) 80 * Solaris 9 extended tar format (including ACLs) 81 * POSIX pax interchange format 82 * POSIX octet-oriented cpio 83 * SVR4 ASCII cpio 84 * Binary cpio (big-endian or little-endian) 85 * PWB binary cpio 86 * ISO9660 CD-ROM images (with optional Rockridge or Joliet extensions) 87 * ZIP archives (with uncompressed or "deflate" compressed entries, including support for encrypted Zip archives) 88 * ZIPX archives (with support for bzip2, ppmd8, lzma and xz compressed entries) 89 * GNU and BSD 'ar' archives 90 * 'mtree' format 91 * 7-Zip archives (including archives that use zstandard compression) 92 * Microsoft CAB format 93 * LHA and LZH archives 94 * RAR and RAR 5.0 archives (with some limitations due to RAR's proprietary status) 95 * XAR archives 96 97The library also detects and handles any of the following before evaluating the archive: 98 99 * uuencoded files 100 * files with RPM wrapper 101 * gzip compression 102 * bzip2 compression 103 * compress/LZW compression 104 * lzma, lzip, and xz compression 105 * lz4 compression 106 * lzop compression 107 * zstandard compression 108 109The library can create archives in any of the following formats: 110 111 * POSIX ustar 112 * POSIX pax interchange format 113 * "restricted" pax format, which will create ustar archives except for 114 entries that require pax extensions (for long filenames, ACLs, etc). 115 * Old GNU tar format 116 * Old V7 tar format 117 * POSIX octet-oriented cpio 118 * SVR4 "newc" cpio 119 * Binary cpio (little-endian) 120 * PWB binary cpio 121 * shar archives 122 * ZIP archives (with uncompressed or "deflate" compressed entries) 123 * GNU and BSD 'ar' archives 124 * 'mtree' format 125 * ISO9660 format 126 * 7-Zip archives 127 * XAR archives 128 129When creating archives, the result can be filtered with any of the following: 130 131 * uuencode 132 * gzip compression 133 * bzip2 compression 134 * compress/LZW compression 135 * lzma, lzip, and xz compression 136 * lz4 compression 137 * lzop compression 138 * zstandard compression 139 140## Notes about the Library Design 141 142The following notes address many of the most common 143questions we are asked about libarchive: 144 145* This is a heavily stream-oriented system. That means that 146 it is optimized to read or write the archive in a single 147 pass from beginning to end. For example, this allows 148 libarchive to process archives too large to store on disk 149 by processing them on-the-fly as they are read from or 150 written to a network or tape drive. This also makes 151 libarchive useful for tools that need to produce 152 archives on-the-fly (such as webservers that provide 153 archived contents of a users account). 154 155* In-place modification and random access to the contents 156 of an archive are not directly supported. For some formats, 157 this is not an issue: For example, tar.gz archives are not 158 designed for random access. In some other cases, libarchive 159 can re-open an archive and scan it from the beginning quickly 160 enough to provide the needed abilities even without true 161 random access. Of course, some applications do require true 162 random access; those applications should consider alternatives 163 to libarchive. 164 165* The library is designed to be extended with new compression and 166 archive formats. The only requirement is that the format be 167 readable or writable as a stream and that each archive entry be 168 independent. There are articles on the libarchive Wiki explaining 169 how to extend libarchive. 170 171* On read, compression and format are always detected automatically. 172 173* The same API is used for all formats; it should be very 174 easy for software using libarchive to transparently handle 175 any of libarchive's archiving formats. 176 177* Libarchive's automatic support for decompression can be used 178 without archiving by explicitly selecting the "raw" and "empty" 179 formats. 180 181* I've attempted to minimize static link pollution. If you don't 182 explicitly invoke a particular feature (such as support for a 183 particular compression or format), it won't get pulled in to 184 statically-linked programs. In particular, if you don't explicitly 185 enable a particular compression or decompression support, you won't 186 need to link against the corresponding compression or decompression 187 libraries. This also reduces the size of statically-linked 188 binaries in environments where that matters. 189 190* The library is generally _thread safe_ depending on the platform: 191 it does not define any global variables of its own. However, some 192 platforms do not provide fully thread-safe versions of key C library 193 functions. On those platforms, libarchive will use the non-thread-safe 194 functions. Patches to improve this are of great interest to us. 195 196* The function `archive_write_disk_header()` is _not_ thread safe on 197 POSIX machines and could lead to security issue resulting in world 198 writeable directories. Thus it must be mutexed by the calling code. 199 This is due to calling `umask(oldumask = umask(0))`, which sets the 200 umask for the whole process to 0 for a short time frame. 201 In case other thread calls the same function in parallel, it might 202 get interrupted by it and cause the executable to use umask=0 for the 203 remaining execution. 204 This will then lead to implicitely created directories to have 777 205 permissions without sticky bit. 206 207* In particular, libarchive's modules to read or write a directory 208 tree do use `chdir()` to optimize the directory traversals. This 209 can cause problems for programs that expect to do disk access from 210 multiple threads. Of course, those modules are completely 211 optional and you can use the rest of libarchive without them. 212 213* The library is _not_ thread aware, however. It does no locking 214 or thread management of any kind. If you create a libarchive 215 object and need to access it from multiple threads, you will 216 need to provide your own locking. 217 218* On read, the library accepts whatever blocks you hand it. 219 Your read callback is free to pass the library a byte at a time 220 or mmap the entire archive and give it to the library at once. 221 On write, the library always produces correctly-blocked output. 222 223* The object-style approach allows you to have multiple archive streams 224 open at once. bsdtar uses this in its "@archive" extension. 225 226* The archive itself is read/written using callback functions. 227 You can read an archive directly from an in-memory buffer or 228 write it to a socket, if you wish. There are some utility 229 functions to provide easy-to-use "open file," etc, capabilities. 230 231* The read/write APIs are designed to allow individual entries 232 to be read or written to any data source: You can create 233 a block of data in memory and add it to a tar archive without 234 first writing a temporary file. You can also read an entry from 235 an archive and write the data directly to a socket. If you want 236 to read/write entries to disk, there are convenience functions to 237 make this especially easy. 238 239* Note: The "pax interchange format" is a POSIX standard extended tar 240 format that should be used when the older _ustar_ format is not 241 appropriate. It has many advantages over other tar formats 242 (including the legacy GNU tar format) and is widely supported by 243 current tar implementations. 244 245