README
1
2XZ Embedded
3===========
4
5 XZ Embedded is a relatively small, limited implementation of the .xz
6 file format. Currently only decoding is implemented.
7
8 XZ Embedded was written for use in the Linux kernel, but the code can
9 be easily used in other environments too, including regular userspace
10 applications. See userspace/xzminidec.c for an example program.
11
12 This README contains information that is useful only when the copy
13 of XZ Embedded isn't part of the Linux kernel tree. You should also
14 read linux/Documentation/xz.txt even if you aren't using XZ Embedded
15 as part of Linux; information in that file is not repeated in this
16 README.
17
18Compiling the Linux kernel module
19
20 The xz_dec module depends on crc32 module, so make sure that you have
21 it enabled (CONFIG_CRC32).
22
23 Building the xz_dec and xz_dec_test modules without support for BCJ
24 filters:
25
26 cd linux/lib/xz
27 make -C /path/to/kernel/source \
28 KCPPFLAGS=-I"$(pwd)/../../include" M="$(pwd)" \
29 CONFIG_XZ_DEC=m CONFIG_XZ_DEC_TEST=m
30
31 Building the xz_dec and xz_dec_test modules with support for BCJ
32 filters:
33
34 cd linux/lib/xz
35 make -C /path/to/kernel/source \
36 KCPPFLAGS=-I"$(pwd)/../../include" M="$(pwd)" \
37 CONFIG_XZ_DEC=m CONFIG_XZ_DEC_TEST=m CONFIG_XZ_DEC_BCJ=y \
38 CONFIG_XZ_DEC_X86=y CONFIG_XZ_DEC_POWERPC=y \
39 CONFIG_XZ_DEC_IA64=y CONFIG_XZ_DEC_ARM=y \
40 CONFIG_XZ_DEC_ARMTHUMB=y CONFIG_XZ_DEC_SPARC=y
41
42 If you want only one or a few of the BCJ filters, omit the appropriate
43 variables. CONFIG_XZ_DEC_BCJ=y is always required to build the support
44 code shared between all BCJ filters.
45
46 Most people don't need the xz_dec_test module. You can skip building
47 it by omitting CONFIG_XZ_DEC_TEST=m from the make command line.
48
49Compiler requirements
50
51 XZ Embedded should compile as either GNU-C89 (used in the Linux
52 kernel) or with any C99 compiler. Getting the code to compile with
53 non-GNU C89 compiler or a C++ compiler should be quite easy as
54 long as there is a data type for unsigned 64-bit integer (or the
55 code is modified not to support large files, which needs some more
56 care than just using 32-bit integer instead of 64-bit).
57
58 If you use GCC, try to use a recent version. For example, on x86-32,
59 xz_dec_lzma2.c compiled with GCC 3.3.6 is 15-25 % slower than when
60 compiled with GCC 4.3.3.
61
62Embedding into userspace applications
63
64 To embed the XZ decoder, copy the following files into a single
65 directory in your source code tree:
66
67 linux/include/linux/xz.h
68 linux/lib/xz/xz_crc32.c
69 linux/lib/xz/xz_dec_lzma2.c
70 linux/lib/xz/xz_dec_stream.c
71 linux/lib/xz/xz_lzma2.h
72 linux/lib/xz/xz_private.h
73 linux/lib/xz/xz_stream.h
74 userspace/xz_config.h
75
76 Alternatively, xz.h may be placed into a different directory but then
77 that directory must be in the compiler include path when compiling
78 the .c files.
79
80 Your code should use only the functions declared in xz.h. The rest of
81 the .h files are meant only for internal use in XZ Embedded.
82
83 You may want to modify xz_config.h to be more suitable for your build
84 environment. Probably you should at least skim through it even if the
85 default file works as is.
86
87Supporting concatenated .xz files
88
89 Regular .xz files can be concatenated as is and the xz command line
90 tool will decompress all streams from a concatenated file (a few
91 other popular formats and tools support this too). This kind of .xz
92 files aren't as uncommon as one might think because pxz, an early
93 threaded XZ compressor, created this kind of .xz files.
94
95 The xz_dec_run() function will stop after decompressing one stream.
96 This is good when XZ data is stored inside some other file format.
97 However, if one is decompressing regular standalone .xz files, one
98 will want to decompress all streams in the file. This is easy with
99 xz_dec_catrun(). To include support for xz_dec_catrun(), you need
100 to #define XZ_DEC_CONCATENATED in xz_config.h or in compiler flags.
101
102Integrity check support
103
104 XZ Embedded always supports the integrity check types None and
105 CRC32. Support for CRC64 is optional. SHA-256 is currently not
106 supported in XZ Embedded although the .xz format does support it.
107 The xz tool from XZ Utils uses CRC64 by default, but CRC32 is usually
108 enough in embedded systems to keep the code size smaller.
109
110 If you want support for CRC64, you need to copy linux/lib/xz/xz_crc64.c
111 into your application, and #define XZ_USE_CRC64 in xz_config.h or in
112 compiler flags.
113
114 When using the internal CRC32 or CRC64, their lookup tables need to be
115 initialized with xz_crc32_init() and xz_crc64_init(), respectively.
116 See xz.h for details.
117
118 To use external CRC32 or CRC64 code instead of the code from
119 xz_crc32.c or xz_crc64.c, the following #defines may be used
120 in xz_config.h or in compiler flags:
121
122 #define XZ_INTERNAL_CRC32 0
123 #define XZ_INTERNAL_CRC64 0
124
125 Then it is up to you to provide compatible xz_crc32() or xz_crc64()
126 functions.
127
128 If the .xz file being decompressed uses an integrity check type that
129 isn't supported by XZ Embedded, it is treated as an error and the
130 file cannot be decompressed. For multi-call mode, this can be modified
131 by #defining XZ_DEC_ANY_CHECK. Then xz_dec_run() will return
132 XZ_UNSUPPORTED_CHECK when unsupported check type is detected. After
133 that decompression can be continued normally except that the
134 integrity check won't be verified. In single-call mode there's
135 no way to continue decoding, so XZ_DEC_ANY_CHECK is almost useless
136 in single-call mode.
137
138BCJ filter support
139
140 If you want support for one or more BCJ filters, you need to copy also
141 linux/lib/xz/xz_dec_bcj.c into your application, and use appropriate
142 #defines in xz_config.h or in compiler flags. You don't need these
143 #defines in the code that just uses XZ Embedded via xz.h, but having
144 them always #defined doesn't hurt either.
145
146 #define Instruction set BCJ filter endianness
147 XZ_DEC_X86 x86-32 or x86-64 Little endian only
148 XZ_DEC_POWERPC PowerPC Big endian only
149 XZ_DEC_IA64 Itanium (IA-64) Big or little endian
150 XZ_DEC_ARM ARM Little endian only
151 XZ_DEC_ARMTHUMB ARM-Thumb Little endian only
152 XZ_DEC_SPARC SPARC Big or little endian
153
154 While some architectures are (partially) bi-endian, the endianness
155 setting doesn't change the endianness of the instructions on all
156 architectures. That's why Itanium and SPARC filters work for both big
157 and little endian executables (Itanium has little endian instructions
158 and SPARC has big endian instructions).
159
160 There currently is no filter for little endian PowerPC or big endian
161 ARM or ARM-Thumb. Implementing filters for them can be considered if
162 there is a need for such filters in real-world applications.
163
164Notes about shared libraries
165
166 If you are including XZ Embedded into a shared library, you very
167 probably should rename the xz_* functions to prevent symbol
168 conflicts in case your library is linked against some other library
169 or application that also has XZ Embedded in it (which may even be
170 a different version of XZ Embedded). TODO: Provide an easy way
171 to do this.
172
173 Please don't create a shared library of XZ Embedded itself unless
174 it is fine to rebuild everything depending on that shared library
175 everytime you upgrade to a newer version of XZ Embedded. There are
176 no API or ABI stability guarantees between different versions of
177 XZ Embedded.
178
179