1#------------------------------------------------------------------------------ 2# $File: compress,v 1.83 2022/08/16 11:16:39 christos Exp $ 3# compress: file(1) magic for pure-compression formats (no archives) 4# 5# compress, gzip, pack, compact, huf, squeeze, crunch, freeze, yabba, etc. 6# 7# Formats for various forms of compressed data 8# Formats for "compress" proper have been moved into "compress.c", 9# because it tries to uncompress it to figure out what's inside. 10 11# standard unix compress 120 string \037\235 compress'd data 13!:mime application/x-compress 14!:apple LZIVZIVU 15>2 byte&0x80 >0 block compressed 16>2 byte&0x1f x %d bits 17 18# gzip (GNU zip, not to be confused with Info-ZIP or PKWARE zip archiver) 19# URL: https://en.wikipedia.org/wiki/Gzip 20# Reference: https://tools.ietf.org/html/rfc1952 21# Update: Joerg Jenderek, Apr 2019 22# Edited by Chris Chittleborough <cchittleborough@yahoo.com.au>, March 2002 23# * Original filename is only at offset 10 if "extra field" absent 24# * Produce shorter output - notably, only report compression methods 25# other than 8 ("deflate", the only method defined in RFC 1952). 26# Note: find defs -iname '*.trid.xml' -exec grep -q '<Bytes>1F8B08' {} \; -ls 27# TODO: 28# FBR Blueberry FlashBack screen Record https://www.flashbackrecorder.com/ 29# KPR KOffice/Calligra KPresenter application/x-kpresenter 30# KPT KOffice/Calligra KPresenter template? application/x-kpresenter 31# SAV Diggles Saved Game File http://www.innonics.com 32# SAV FarCry (demo) saved game http://www.farcry-thegame.com 33# DAT ZOAGZIP game data format http://en.wikipedia.org/wiki/SD_Gundam_Capsule_Fighter 340 string \037\213 35# to display gzip compressed (strength=100=2*50) before other (strength=50)? 36#!:strength * 2 37# no FNAME and FCOMMENT bit implies no file name/comment. That means only binary 38>3 byte&0x18 =0 39# For binary gzipped no ASCII text should occur 40# mcd-monu-cad.trid.xml 41>>10 string MCD Monu-Cad Drawing, Component or Font 42#>>36 string Created\ with\ MONU-CAD 43#!:mime application/octet-stream 44# http://fileformats.archiveteam.org/wiki/Monu-CAD 45# http://www.monucad.com/downloads/FullDemo-2005.EXE 46# /HANDS96.MCC Component 47# /DEMO_DD01.MCD Drawing 48# /MCALF020.FNT Font 49!:ext mcc/mcd/fnt 50# http://www.generalcadd.com 51>>10 string GXD General CADD, Drawing or Component 52#!:mime application/octet-stream 53# /gxc/BUILDINGEDGE.gxc Component 54# /gxd/HOCKETT-STPAUL-WRHSE.gxd Drawing 55# /gxd/POWERLAND-MILL-ADD-11.gxd Drawing v9.1.06 56!:ext gxc/gxd 57#>>>13 ubyte 0 \b, version 0 58>>>13 string 09 \b, version 9 59# other gzipped binary like gzipped tar, VirtualBox extension package,... 60>>10 default x gzip compressed data 61!:mime application/gzip 62>>>0 use gzip-info 63# size of the original (uncompressed) input data modulo 2^32 64>>-0 offset >48 65>>>-4 ulelong x \b, original size modulo 2^32 %u 66>>-0 offset <48 \b, truncated 67# gzipped TAR or VirtualBox extension package 68#!:mime application/x-compressed-tar 69#!:mime application/x-virtualbox-vbox-extpack 70# https://www.w3.org/TR/SVG/mimereg.html 71#!:mime image/image/svg+xml-compressed 72# zlib.3.gz 73# microcode-20180312.tgz 74# tpz same as tgz 75# lua-md5_1.2-1_i386_i486.ipk https://en.wikipedia.org/wiki/Opkg 76# Oracle_VM_VirtualBox_Extension_Pack-5.0.12-104815.vbox-extpack 77!:ext gz/tgz/tpz/ipk/vbox-extpack/svgz 78# FNAME/FCOMMENT bit implies file name/comment as iso-8859-1 text 79>3 byte&0x18 >0 gzip compressed data 80!:mime application/gzip 81# gzipped tar, gzipped Abiword document 82#!:mime application/x-compressed-tar 83#!:mime application/x-abiword-compressed 84#!:mime image/image/svg+xml-compressed 85# kleopatra_splashscreen.svgz gzipped .svg 86!:ext gz/tgz/tpz/zabw/svgz 87>>0 use gzip-info 88# size of the original (uncompressed) input data modulo 2^32 89>>-0 offset >48 90>>>-4 ulelong x \b, original size modulo 2^32 %u 91>>-0 offset <48 \b, truncated 92# display information of gzip compressed files 930 name gzip-info 94#>2 byte x THIS iS GZIP 95>2 byte <8 \b, reserved method 96>2 byte >8 \b, unknown method 97>3 byte &0x01 \b, ASCII 98>3 byte &0x02 \b, has CRC 99>3 byte &0x04 \b, extra field 100>3 byte&0xC =0x08 101>>10 string x \b, was "%s" 102>3 byte &0x10 \b, has comment 103>3 byte &0x20 \b, encrypted 104>4 ledate >0 \b, last modified: %s 105>8 byte 2 \b, max compression 106>8 byte 4 \b, max speed 107>9 byte =0x00 \b, from FAT filesystem (MS-DOS, OS/2, NT) 108>9 byte =0x01 \b, from Amiga 109>9 byte =0x02 \b, from VMS 110>9 byte =0x03 \b, from Unix 111>9 byte =0x04 \b, from VM/CMS 112>9 byte =0x05 \b, from Atari 113>9 byte =0x06 \b, from HPFS filesystem (OS/2, NT) 114>9 byte =0x07 \b, from MacOS 115>9 byte =0x08 \b, from Z-System 116>9 byte =0x09 \b, from CP/M 117>9 byte =0x0A \b, from TOPS/20 118>9 byte =0x0B \b, from NTFS filesystem (NT) 119>9 byte =0x0C \b, from QDOS 120>9 byte =0x0D \b, from Acorn RISCOS 121# size of the original (uncompressed) input data modulo 2^32 122#>-4 ulelong x \b, original size modulo 2^32 %u 123#ERROR: line 114: non zero offset 1048572 at level 1 124 125# packed data, Huffman (minimum redundancy) codes on a byte-by-byte basis 1260 string \037\036 packed data 127!:mime application/octet-stream 128>2 belong >1 \b, %d characters originally 129>2 belong =1 \b, %d character originally 130# 131# This magic number is byte-order-independent. 1320 short 0x1f1f old packed data 133!:mime application/octet-stream 134 135# XXX - why *two* entries for "compacted data", one of which is 136# byte-order independent, and one of which is byte-order dependent? 137# 1380 short 0x1fff compacted data 139!:mime application/octet-stream 140# This string is valid for SunOS (BE) and a matching "short" is listed 141# in the Ultrix (LE) magic file. 1420 string \377\037 compacted data 143!:mime application/octet-stream 1440 short 0145405 huf output 145!:mime application/octet-stream 146 147# bzip2 1480 string BZh bzip2 compressed data 149!:mime application/x-bzip2 150!:ext bz2 151>3 byte >47 \b, block size = %c00k 152 153# bzip a block-sorting file compressor 154# by Julian Seward <sewardj@cs.man.ac.uk> and others 1550 string BZ0 bzip compressed data 156!:mime application/x-bzip 157>3 byte >47 \b, block size = %c00k 158 159# lzip 1600 string LZIP lzip compressed data 161!:mime application/x-lzip 162>4 byte x \b, version: %d 163 164# squeeze and crunch 165# Michael Haardt <michael@cantor.informatik.rwth-aachen.de> 1660 beshort 0x76FF squeezed data, 167>4 string x original name %s 1680 beshort 0x76FE crunched data, 169>2 string x original name %s 1700 beshort 0x76FD LZH compressed data, 171>2 string x original name %s 172 173# Freeze 1740 string \037\237 frozen file 2.1 1750 string \037\236 frozen file 1.0 (or gzip 0.5) 176 177# SCO compress -H (LZH) 1780 string \037\240 SCO compress -H (LZH) data 179 180# European GSM 06.10 is a provisional standard for full-rate speech 181# transcoding, prI-ETS 300 036, which uses RPE/LTP (residual pulse 182# excitation/long term prediction) coding at 13 kbit/s. 183# 184# There's only a magic nibble (4 bits); that nibble repeats every 33 185# bytes. This isn't suited for use, but maybe we can use it someday. 186# 187# This will cause very short GSM files to be declared as data and 188# mismatches to be declared as data too! 189#0 byte&0xF0 0xd0 data 190#>33 byte&0xF0 0xd0 191#>66 byte&0xF0 0xd0 192#>99 byte&0xF0 0xd0 193#>132 byte&0xF0 0xd0 GSM 06.10 compressed audio 194 195# lzop from <markus.oberhumer@jk.uni-linz.ac.at> 1960 string \x89\x4c\x5a\x4f\x00\x0d\x0a\x1a\x0a lzop compressed data 197>9 beshort <0x0940 198>>9 byte&0xf0 =0x00 - version 0. 199>>9 beshort&0x0fff x \b%03x, 200>>13 byte 1 LZO1X-1, 201>>13 byte 2 LZO1X-1(15), 202>>13 byte 3 LZO1X-999, 203## >>22 bedate >0 last modified: %s, 204>>14 byte =0x00 os: MS-DOS 205>>14 byte =0x01 os: Amiga 206>>14 byte =0x02 os: VMS 207>>14 byte =0x03 os: Unix 208>>14 byte =0x05 os: Atari 209>>14 byte =0x06 os: OS/2 210>>14 byte =0x07 os: MacOS 211>>14 byte =0x0A os: Tops/20 212>>14 byte =0x0B os: WinNT 213>>14 byte =0x0E os: Win32 214>9 beshort >0x0939 215>>9 byte&0xf0 =0x00 - version 0. 216>>9 byte&0xf0 =0x10 - version 1. 217>>9 byte&0xf0 =0x20 - version 2. 218>>9 beshort&0x0fff x \b%03x, 219>>15 byte 1 LZO1X-1, 220>>15 byte 2 LZO1X-1(15), 221>>15 byte 3 LZO1X-999, 222## >>25 bedate >0 last modified: %s, 223>>17 byte =0x00 os: MS-DOS 224>>17 byte =0x01 os: Amiga 225>>17 byte =0x02 os: VMS 226>>17 byte =0x03 os: Unix 227>>17 byte =0x05 os: Atari 228>>17 byte =0x06 os: OS/2 229>>17 byte =0x07 os: MacOS 230>>17 byte =0x0A os: Tops/20 231>>17 byte =0x0B os: WinNT 232>>17 byte =0x0E os: Win32 233 234# 4.3BSD-Quasijarus Strong Compression 235# https://minnie.tuhs.org/Quasijarus/compress.html 2360 string \037\241 Quasijarus strong compressed data 237 238# From: Cory Dikkers <cdikkers@swbell.net> 2390 string XPKF Amiga xpkf.library compressed data 2400 string PP11 Power Packer 1.1 compressed data 2410 string PP20 Power Packer 2.0 compressed data, 242>4 belong 0x09090909 fast compression 243>4 belong 0x090A0A0A mediocre compression 244>4 belong 0x090A0B0B good compression 245>4 belong 0x090A0C0C very good compression 246>4 belong 0x090A0C0D best compression 247 248# 7-zip archiver, from Thomas Klausner (wiz@danbala.tuwien.ac.at) 249# https://www.7-zip.org or DOC/7zFormat.txt 250# 2510 string 7z\274\257\047\034 7-zip archive data, 252>6 byte x version %d 253>7 byte x \b.%d 254!:mime application/x-7z-compressed 255!:ext 7z/cb7 256 257# Type: LZMA 2580 lelong&0xffffff =0x5d 259>12 leshort 0xff LZMA compressed data, 260!:mime application/x-lzma 261>>5 lequad =0xffffffffffffffff streamed 262>>5 lequad !0xffffffffffffffff non-streamed, size %lld 263>12 leshort 0 LZMA compressed data, 264>>5 lequad =0xffffffffffffffff streamed 265>>5 lequad !0xffffffffffffffff non-streamed, size %lld 266 267# http://tukaani.org/xz/xz-file-format.txt 2680 ustring \xFD7zXZ\x00 XZ compressed data, checksum 269!:strength * 2 270!:mime application/x-xz 271>7 byte&0xf 0x0 NONE 272>7 byte&0xf 0x1 CRC32 273>7 byte&0xf 0x4 CRC64 274>7 byte&0xf 0xa SHA-256 275 276# https://github.com/ckolivas/lrzip/blob/master/doc/magic.header.txt 2770 string LRZI LRZIP compressed data 278>4 byte x - version %d 279>5 byte x \b.%d 280>22 byte 1 \b, encrypted 281!:mime application/x-lrzip 282 283# https://fastcompression.blogspot.fi/2013/04/lz4-streaming-format-final.html 2840 lelong 0x184d2204 LZ4 compressed data (v1.4+) 285!:mime application/x-lz4 286# Added by osm0sis@xda-developers.com 2870 lelong 0x184c2103 LZ4 compressed data (v1.0-v1.3) 288!:mime application/x-lz4 2890 lelong 0x184c2102 LZ4 compressed data (v0.1-v0.9) 290!:mime application/x-lz4 291 292# Zstandard/LZ4 skippable frames 293# https://github.com/facebook/zstd/blob/dev/zstd_compression_format.md 2940 lelong&0xFFFFFFF0 0x184D2A50 295>(4.l+8) indirect x 296 297# Zstandard Dictionary ID subroutine 2980 name zstd-dictionary-id 299# Single Segment = True 300>0 byte &0x20 \b, Dictionary ID: 301>>0 byte&0x03 0 None 302>>0 byte&0x03 1 303>>>1 byte x %u 304>>0 byte&0x03 2 305>>>1 leshort x %u 306>>0 byte&0x03 3 307>>>1 lelong x %u 308# Single Segment = False 309>0 byte ^0x20 \b, Dictionary ID: 310>>0 byte&0x03 0 None 311>>0 byte&0x03 1 312>>>2 byte x %u 313>>0 byte&0x03 2 314>>>2 leshort x %u 315>>0 byte&0x03 3 316>>>2 lelong x %u 317 318# Zstandard compressed data 319# https://github.com/facebook/zstd/blob/dev/zstd_compression_format.md 3200 lelong 0xFD2FB522 Zstandard compressed data (v0.2) 321!:mime application/zstd 3220 lelong 0xFD2FB523 Zstandard compressed data (v0.3) 323!:mime application/zstd 3240 lelong 0xFD2FB524 Zstandard compressed data (v0.4) 325!:mime application/zstd 3260 lelong 0xFD2FB525 Zstandard compressed data (v0.5) 327!:mime application/zstd 3280 lelong 0xFD2FB526 Zstandard compressed data (v0.6) 329!:mime application/zstd 3300 lelong 0xFD2FB527 Zstandard compressed data (v0.7) 331!:mime application/zstd 332>4 use zstd-dictionary-id 3330 lelong 0xFD2FB528 Zstandard compressed data (v0.8+) 334!:mime application/zstd 335>4 use zstd-dictionary-id 336 337# https://github.com/facebook/zstd/blob/dev/zstd_compression_format.md 3380 lelong 0xEC30A437 Zstandard dictionary 339!:mime application/x-std-dictionary 340>4 lelong x (ID %u) 341 342# AFX compressed files (Wolfram Kleff) 3432 string -afx- AFX compressed file data 344 345# Supplementary magic data for the file(1) command to support 346# rzip(1). The format is described in magic(5). 347# 348# Copyright (C) 2003 by Andrew Tridgell. You may do whatever you want with 349# this file. 350# 3510 string RZIP rzip compressed data 352>4 byte x - version %d 353>5 byte x \b.%d 354>6 belong x (%d bytes) 355 3560 string ArC\x01 FreeArc archive <http://freearc.org> 357 358# Type: DACT compressed files 3590 long 0x444354C3 DACT compressed data 360>4 byte >-1 (version %i. 361>5 byte >-1 %i. 362>6 byte >-1 %i) 363>7 long >0 , original size: %i bytes 364>15 long >30 , block size: %i bytes 365 366# Valve Pack (VPK) files 3670 lelong 0x55aa1234 Valve Pak file 368>0x4 lelong x \b, version %u 369>0x8 lelong x \b, %u entries 370 371# Snappy framing format 372# https://code.google.com/p/snappy/source/browse/trunk/framing_format.txt 3730 string \377\006\0\0sNaPpY snappy framed data 374!:mime application/x-snappy-framed 375 376# qpress, https://www.quicklz.com/ 3770 string qpress10 qpress compressed data 378!:mime application/x-qpress 379 380# Zlib https://www.ietf.org/rfc/rfc6713.txt 3810 string/b x 382>0 beshort%31 =0 383>>0 byte&0xf =8 384>>>0 byte&0x80 =0 zlib compressed data 385!:mime application/zlib 386 387# BWC compression 3880 string BWC 389>3 byte 0 BWC compressed data 390 391# UCL compression 3920 bequad 0x00e955434cff011a UCL compressed data 393 394# Softlib archive 3950 string SLIB Softlib archive 396>4 leshort x \b, version %d 397>6 leshort x (contains %d files) 398 399# URL: https://github.com/lzfse/lzfse/blob/master/src/lzfse_internal.h#L276 400# From: Eric Hall <eric.hall@darkart.com> 4010 string bvx- lzfse encoded, no compression 4020 string bvx1 lzfse compressed, uncompressed tables 4030 string bvx2 lzfse compressed, compressed tables 4040 string bvxn lzfse encoded, lzvn compressed 405 406# pcxLib.exe compression program 407# http://www.shikadi.net/moddingwiki/PCX_Library 4080 string/b pcxLib 409>0x0A string/b Copyright\020(c)\020Genus\020Microprogramming,\020Inc. pcxLib compressed 410