1.. SPDX-License-Identifier: 0BSD 2 3============================ 4XZ data compression in Linux 5============================ 6 7Introduction 8============ 9 10XZ is a general purpose data compression format with high compression 11ratio. The XZ decompressor in Linux is called XZ Embedded. It supports 12the LZMA2 filter and optionally also Branch/Call/Jump (BCJ) filters 13for executable code. CRC32 is supported for integrity checking. 14 15See the `XZ Embedded`_ home page for the latest version which includes 16a few optional extra features that aren't required in the Linux kernel 17and information about using the code outside the Linux kernel. 18 19For userspace, `XZ Utils`_ provide a zlib-like compression library 20and a gzip-like command line tool. 21 22.. _XZ Embedded: https://tukaani.org/xz/embedded.html 23.. _XZ Utils: https://tukaani.org/xz/ 24 25XZ related components in the kernel 26=================================== 27 28The xz_dec module provides XZ decompressor with single-call (buffer 29to buffer) and multi-call (stateful) APIs in include/linux/xz.h. 30 31For decompressing the kernel image, initramfs, and initrd, there 32is a wrapper function in lib/decompress_unxz.c. Its API is the 33same as in other decompress_*.c files, which is defined in 34include/linux/decompress/generic.h. 35 36For kernel makefiles, three commands are provided for use with 37``$(call if_changed)``. They require the xz tool from XZ Utils. 38 39- ``$(call if_changed,xzkern)`` is for compressing the kernel image. 40 It runs the script scripts/xz_wrap.sh which uses arch-optimized 41 options and a big LZMA2 dictionary. 42 43- ``$(call if_changed,xzkern_with_size)`` is like ``xzkern`` above but 44 this also appends a four-byte trailer containing the uncompressed size 45 of the file. The trailer is needed by the boot code on some archs. 46 47- Other things can be compressed with ``$(call if_needed,xzmisc)`` 48 which will use no BCJ filter and 1 MiB LZMA2 dictionary. 49 50Notes on compression options 51============================ 52 53Since the XZ Embedded supports only streams with CRC32 or no integrity 54check, make sure that you don't use some other integrity check type 55when encoding files that are supposed to be decoded by the kernel. 56With liblzma from XZ Utils, you need to use either ``LZMA_CHECK_CRC32`` 57or ``LZMA_CHECK_NONE`` when encoding. With the ``xz`` command line tool, 58use ``--check=crc32`` or ``--check=none`` to override the default 59``--check=crc64``. 60 61Using CRC32 is strongly recommended unless there is some other layer 62which will verify the integrity of the uncompressed data anyway. 63Double checking the integrity would probably be waste of CPU cycles. 64Note that the headers will always have a CRC32 which will be validated 65by the decoder; you can only change the integrity check type (or 66disable it) for the actual uncompressed data. 67 68In userspace, LZMA2 is typically used with dictionary sizes of several 69megabytes. The decoder needs to have the dictionary in RAM: 70 71- In multi-call mode the dictionary is allocated as part of the 72 decoder state. The reasonable maximum dictionary size for in-kernel 73 use will depend on the target hardware: a few megabytes is fine for 74 desktop systems while 64 KiB to 1 MiB might be more appropriate on 75 some embedded systems. 76 77- In single-call mode the output buffer is used as the dictionary 78 buffer. That is, the size of the dictionary doesn't affect the 79 decompressor memory usage at all. Only the base data structures 80 are allocated which take a little less than 30 KiB of memory. 81 For the best compression, the dictionary should be at least 82 as big as the uncompressed data. A notable example of single-call 83 mode is decompressing the kernel itself (except on PowerPC). 84 85The compression presets in XZ Utils may not be optimal when creating 86files for the kernel, so don't hesitate to use custom settings to, 87for example, set the dictionary size. Also, xz may produce a smaller 88file in single-threaded mode so setting that explicitly is recommended. 89Example:: 90 91 xz --threads=1 --check=crc32 --lzma2=dict=512KiB inputfile 92 93xz_dec API 94========== 95 96This is available with ``#include <linux/xz.h>``. 97 98.. kernel-doc:: include/linux/xz.h 99