Lines Matching +full:charge +full:- +full:integration
2 * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
5 * This source code is licensed under both the BSD-style license (found in the
8 * You may select, at your option, one of the above-listed licenses.
43 real-time compression scenarios at zlib-level and better compression ratios.
44 The zstd compression library provides in-memory compression and decompression
48 which is currently 22. Levels >= 20, labeled `--ultra`, should be used with
54 - a single step (described as Simple API)
55 - a single step, reusing a context (described as Explicit context)
56 - unbounded multiple steps (described as Streaming compression)
60 - a single step (described as Simple dictionary API)
61 - a single step, reusing a dictionary (described as Bulk-processing
67 Advanced experimental APIs should never be used with a dynamically-linked
72 /*------ Version ------*/
97 /* All magic numbers are supposed read/written to/from files/memory using little-endian convention …
133 * @return : - decompressed size of `src` frame content, if known
134 * - ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
135 …* - ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too…
143 …* note 3 : decompressed size is always present when compression is completed using single-pass f…
145 * note 4 : decompressed size can be very large (64-bits value),
152 #define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1)
153 #define ZSTD_CONTENTSIZE_ERROR (0ULL - 2)
174 …ize) ((srcSize) + ((srcSize)>>8) + (((srcSize) < (128<<10)) ? (((128<<10) - (srcSize)) >> 11) /*…
175 …D_compressBound(size_t srcSize); /*!< maximum compressed size in worst case single-pass scenario */
188 * and re-use it for each successive compression operation.
190 * Note : re-using context is just a speed / resource optimization.
192 * Note 2 : In multi-threaded environments,
215 * and re-use it for each successive compression operation.
241 * __They do not apply to "simple" one-shot variants such as ZSTD_compressCCtx()__ .
270 * See ZSTD_CCtx_refCDict() for more info (superseded-by-cdict). */
271 … ZSTD_c_compressionLevel=100, /* Set compression parameters according to pre-defined cLevel table.
284 ZSTD_c_windowLog=101, /* Maximum allowed back-reference distance, expressed as power of 2.
298 ZSTD_c_chainLog=103, /* Size of the multi-probe search table, as a power of 2.
342 * default: windowlog - 7.
353 … * Must be clamped between 0 and (ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN).
354 … * Default is MAX(0, (windowLog - ldmHashLog)), optimizing hash table usage.
364 …ZSTD_c_checksumFlag=201, /* A 32-bits checksum of content is written at end of frame (default:0) */
367 /* multi-threading parameters */
368 …/* These parameters are only useful if multi-threading is enabled (compiled with build macro ZSTD_…
377 …* Default value is `0`, aka "single-threaded mode" : no worker is spawned, compression is performe…
389 … * - 0 means "default" : value will be determined by the library, depending on strategy
390 * - 1 means "no overlap"
391 * - 9 means "full overlap", using a full window size.
429 * - an error status field, which must be tested using ZSTD_isError()
430 * - lower and upper bounds, both inclusive
439 * Exception : when using multi-threading mode (nbWorkers >= 1),
472 * - The session : will stop compressing current frame, and make CCtx ready to start a new one.
478 * - The parameters : changes all parameters back to "default".
482 * - Both : similar to resetting the session, followed by resetting parameters.
490 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
491 * - The function is always blocking, returns when compression is completed.
518 …ameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
539 * - an error status field, which must be tested using ZSTD_isError()
540 * - both lower and upper bounds, inclusive
580 /*-***********************************************************************
581 * Streaming compression - HowTo
586 * It is recommended to re-use ZSTD_CStream since it will play nicer with system's memory, by re-us…
593 * it will re-use the same sticky parameters as previous compression session.
616 * using ZSTD_compressStream2() with ZSTD_e_flush. `output->pos` will be updated.
617 * Note that, if `output->size` is too small, a single invocation with ZSTD_e_flush might not be en…
663 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
664 …* - Compression parameters cannot be changed once compression is started (save a list of exceptio…
665 * - output->pos must be <= dstCapacity, input->pos must be <= srcSize
666 …* - output->pos and input->pos will be updated. They are guaranteed to remain below their respect…
667 …* - When nbWorkers==0 (default), function is blocking : it completes its job before returning to …
668 …* - When nbWorkers>=1, function is non-blocking : it just acquires a copy of input, and distribut…
671 …* - Exception : if the first call requests a ZSTD_e_end directive and provides enough dstCapacity…
672 * - @return provides a minimum amount of data remaining to be flushed from internal buffers
677 * - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
724 * the next read size (if non-zero and not an error). ZSTD_compressStream2()
725 * returns the minimum nb of bytes left to flush (if non-zero and not an error).
734 /*-***************************************************************************
735 * Streaming decompression - HowTo
739 * ZSTD_DStream objects can be re-used multiple times.
820 …Dict can be created once and shared by multiple threads concurrently, since its usage is read-only.
872 * It can still be loaded, but as a content-only dictionary. */
878 * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */
885 * - The frame does not require a dictionary to be decoded (most common case).
886 …* - The frame was built with dictID intentionally removed. Whatever dictionary is necessary is a …
887 * Note : this use case also happens when using a non-conformant dictionary.
888 …* - `srcSize` is too small, and as a result, the frame header could not be decoded (only possible…
889 * - This is not a Zstandard frame.
900 * ZSTD_reset_session_and_parameters. Prefixes are single-use.
908 * Special: Loading a NULL (or 0-size) dictionary invalidates previous dictionary,
909 * meaning "return to no-dictionary mode".
911 * To return to "no-dictionary" situation, load a NULL dictionary (or reset parameters).
913 * It's also a CPU consuming operation, with non-negligible impact on latency.
927 * The parameters ignored are labled as "superseded-by-cdict" in the ZSTD_cParameter enum docs.
928 * The ignored parameters will be used again if the CCtx is returned to no-dictionary mode.
931 * Special : Referencing a NULL CDict means "return to no-dictionary mode".
938 * Reference a prefix (single-usage dictionary) for next compressed frame.
951 * It's a CPU consuming operation, with non-negligible impact on latency.
963 * Special : Adding a NULL (or 0-size) dictionary invalidates any previous dictionary,
964 * meaning "return to no-dictionary mode".
966 * which has a non-negligible impact on CPU usage and latency.
981 * Special: referencing a NULL DDict means "return to no-dictionary mode".
987 * Reference a prefix (single-usage dictionary) to decompress next frame.
1058 #define ZSTD_SEARCHLOG_MAX (ZSTD_WINDOWLOG_MAX-1)
1075 …* The limit does not apply for one-pass decoders (such as ZSTD_decompress()), since no additional …
1086 #define ZSTD_LDM_HASHRATELOG_MAX (ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN)
1098 /* --- Advanced types --- */
1104 /* If seqDef.offset > 3, then this is seqDef.offset - 3
1109 * most recent repeat offset - 1
1132 …int checksumFlag; /**< 1: generate a 32-bits checksum using XXH64 algorithm at end of frame, fo…
1149 …ZSTD_dlm_byRef = 1 /**< Reference dictionary content -- the dictionary buffer must outlive its …
1154 … ZSTD_f_zstd1_magicless = 1 /* Variant of zstd frame format, without initial 4-bytes magic number.
1167 * - The contents of the CDict can be copied into the working context. This
1170 * the compression faster per-byte of input. However, the initial copy of
1175 * - The CDict's tables can be used in-place. In this model, compression is
1177 * tables. However, this model incurs no start-up cost (as long as the
1181 * - The CDict's tables are not used at all, and instead we use the working
1216 * @return : - decompressed size of all data in all successive frames
1217 * - if the decompressed size cannot be determined: ZSTD_CONTENTSIZE_UNKNOWN
1218 * - if an error occurred: ZSTD_CONTENTSIZE_ERROR
1224 * note 3 : decompressed size can be very large (64-bits value),
1239 * @return : - upper-bound for the decompressed size of all data in all successive frames
1240 * - if an error occured: ZSTD_CONTENTSIZE_ERROR
1243 …* note 2 : the upper-bound is exact when the decompressed size field is available in every ZSTD …
1245 …* note 3 : when the decompressed size field isn't available, the upper-bound for that frame is c…
1246 * upper-bound = # blocks * min(128 KB, Window_Size)
1278 * Therefore, the estimation is only guaranteed for single-shot compressions, not streaming.
1290 * Note 2 : only single-threaded compression is supported.
1303 …ams() can be used in tandem with ZSTD_CCtxParams_setParameter(). Only single-threaded compression …
1304 * Note : CStream size estimation is only correct for single-threaded compression.
1327 * Initialize an object using a pre-allocated fixed-size buffer.
1329 * Provided pointer *must be 8-bytes aligned*.
1344 * Limitation 2 : static cctx currently not compatible with multi-threading.
1413 …* same as ZSTD_getCParams(), but @return a full `ZSTD_parameters` object instead of sub-component…
1494 /* Force back-reference distances to remain < windowSize,
1531 * - ZSTD_createCCtxParams() : Create a ZSTD_CCtx_params structure
1532 * - ZSTD_CCtxParams_setParameter() : Push parameters one by one into
1536 * - ZSTD_CCtx_setParametersUsingCCtxParams() : Apply parameters to
1540 * - ZSTD_compressStream2() : Do compression using the CCtx.
1541 * - ZSTD_freeCCtxParams() : Free the memory.
1544 * for static allocation of CCtx for single-threaded compression.
1644 …ameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
1672 * If you need to avoid the input buffer allocation use the buffer-less
1676 * this flag is ALWAYS memory safe, and will never access out-of-bounds
1689 * This instruction is mandatory to decode data without a fully-formed header,
1751 * // Pseudocode: Set each zstd parameter and leave the rest as-is.
1782 * // Pseudocode: Set each zstd frame parameter and leave the rest as-is.
1806 * This is typically useful to skip dictionary loading stage, since it will re-use it in-place.
1830 * Note : (ingested - consumed) is amount of input data buffered internally, not yet compressed.
1879 * re-use decompression parameters from previous init; saves dictionary loading
1886 * Buffer-less and synchronous inner streaming functions
1894 Buffer-less streaming compression (synchronous mode)
1898 ZSTD_CCtx object can be re-used multiple times within successive compression operations.
1907 - ZSTD_compressContinue() has no internal buffer. It uses externally provided buffers only.
1908 - Interface is synchronous : input is consumed entirely and produces 1+ compressed blocks.
1909 …- Caller must ensure there is enough space in `dst` to store compressed data under worst case scen…
1912 …- ZSTD_compressContinue() presumes prior input ***is still accessible and unmodified*** (up to max…
1914 …- ZSTD_compressContinue() detects that prior input has been overwritten when `src` buffer overlaps.
1921 `ZSTD_CCtx` object can be re-used (ZSTD_compressBegin()) to compress again.
1924 /*===== Buffer-less streaming compression functions =====*/
1936 /*-
1937 Buffer-less streaming decompression (synchronous mode)
1941 A ZSTD_DCtx object can be re-used multiple times.
1952 such as the dictionary ID, content size, or maximum back-reference distance (`windowSize`).
1962 …or that previous contiguous segment is large enough to properly handle maximum back-reference dist…
1967 …which can @return an error code if required value is too large for current system (in 32-bits mode…
1977 as long as the encoder and decoder progress in "lock-step",
1999 Skippable frames allow integration of user-defined data into a flow of concatenated frames.
2002 a) Skippable frame ID - 4 Bytes, Little endian format, any value from 0x184D2A50 to 0x184D2A5F
2003 b) Frame Size - 4 Bytes, Little endian format, unsigned 32-bits
2004 c) Frame Content - any content (User Data) of length equal to Frame Size
2005 For skippable frames ZSTD_getFrameHeader() returns zfhPtr->frameType==ZSTD_skippableFrame.
2009 /*===== Buffer-less streaming decompression functions =====*/
2054 …Frame metadata cost is typically ~12 bytes, which can be non-negligible for very small blocks (< 1…
2055 …But users will have to take in charge needed metadata to regenerate data, such as compressed and c…
2058 - Compressing and decompressing require a context structure
2060 - It is necessary to init context before starting
2064 - Block size is limited, it must be <= ZSTD_getBlockSize() <= ZSTD_BLOCKSIZE_MAX == 128 KB
2068 …- When a block is considered not compressible enough, ZSTD_compressBlock() result will be 0 (zero)…
2083 …Size); /**< insert uncompressed block into `dctx` history. Useful for multi-blocks decompression.…