1# ZSTD-On-ZFS Library Manual 2 3## Introduction 4 5This subtree contains the ZSTD library used in ZFS. It is heavily cut-down by 6dropping any unneeded files, and combined into a single file, but otherwise is 7intentionally unmodified. Please do not alter the file containing the zstd 8library, besides upgrading to a newer ZSTD release. 9 10Tree structure: 11 12* `zfs_zstd.c` is the actual `zzstd` kernel module. 13* `lib/` contains the the unmodified, [_"amalgamated"_](https://github.com/facebook/zstd/blob/dev/contrib/single_file_libs/README.md) 14 version of the `Zstandard` library, generated from our template file 15* `zstd-in.c` is our template file for generating the library 16* `include/`: This directory contains supplemental includes for platform 17 compatibility, which are not expected to be used by ZFS elsewhere in the 18 future. Thus we keep them private to ZSTD. 19 20## Updating ZSTD 21 22To update ZSTD the following steps need to be taken: 23 241. Grab the latest release of [ZSTD](https://github.com/facebook/zstd/releases). 252. Update `module/zstd/zstd-in.c` if required. (see 26 `zstd/contrib/single_file_libs/zstd-in.c` in the zstd repository) 273. Generate the "single-file-library" and put it to `module/zstd/lib/`. 284. Copy the following files to `module/zstd/lib/`: 29 - `zstd/lib/zstd.h` 30 - `zstd/lib/common/zstd_errors.h` 31 32This can be done using a few shell commands from inside the zfs repo: 33 34~~~sh 35cd PATH/TO/ZFS 36 37url="https://github.com/facebook/zstd" 38release="$(curl -s "${url}"/releases/latest | grep -oP '(?<=v)[\d\.]+')" 39zstd="/tmp/zstd-${release}/" 40 41wget -O /tmp/zstd.tar.gz \ 42 "${url}/releases/download/v${release}/zstd-${release}.tar.gz" 43tar -C /tmp -xzf /tmp/zstd.tar.gz 44 45cp ${zstd}/lib/zstd.h module/zstd/lib/ 46cp ${zstd}/lib/zstd_errors.h module/zstd/lib/ 47${zstd}/contrib/single_file_libs/combine.sh \ 48 -r ${zstd}/lib -o module/zstd/lib/zstd.c module/zstd/zstd-in.c 49~~~ 50 51Note: if the zstd library for zfs is updated to a newer version, 52the macro list in include/zstd_compat_wrapper.h usually needs to be updated. 53this can be done with some hand crafting of the output of the following 54script: nm zstd.o | awk '{print "#define "$3 " zfs_" $3}' > macrotable 55 56 57## Altering ZSTD and breaking changes 58 59If ZSTD made changes that break compatibility or you need to make breaking 60changes to the way we handle ZSTD, it is required to maintain backwards 61compatibility. 62 63We already save the ZSTD version number within the block header to be used 64to add future compatibility checks and/or fixes. However, currently it is 65not actually used in such a way. 66