docs: document determinism and decoder memory

README.md

@@ -142,6 +142,9 @@ For advanced use cases, specialized flags which control binary generation and in
 in [`lib/README.md`](lib/README.md#modular-build) for the `libzstd` library
 and in [`programs/README.md`](programs/README.md#compilation-variables) for the `zstd` CLI.
 
+See [`doc/determinism.md`](doc/determinism.md) for notes about reproducibility
+and byte-for-byte determinism of compressed frames.
+
 ### cmake
 
 A `cmake` project generator is available for generating Makefiles or other build scripts

doc/determinism.md

@@ -0,0 +1,44 @@
+# Zstandard determinism
+
+Zstandard is deterministic for a fixed implementation, input, dictionary, and
+set of compression parameters: running the same compressor build with the same
+options on the same bytes should produce the same compressed bytes.
+
+Zstandard does not promise a canonical compressed representation. Many different
+compressed frames can decompress to the same original content. As a result, users
+should not rely on compressed bytes being identical across:
+
+- different zstd versions,
+- different libraries or ports implementing the format,
+- different compression levels or advanced compression parameters,
+- different dictionaries or dictionary-training inputs,
+- different frame options, such as content size, checksum, or dictionary ID
+  fields, or
+- different build-time feature choices that change the compressor implementation.
+
+The stable compatibility property is decompression compatibility: any valid
+frame generated by one compliant encoder should decode to the original content
+with another compliant decoder. Applications that need content identity should
+compare the original content or a content hash of the decompressed bytes, not the
+compressed frame bytes.
+
+## Multithreaded compression
+
+For a fixed zstd version and command line, multithreaded compression is intended
+to be reproducible. However, the compressed output may differ from single-threaded
+compression because the work is split into jobs. If byte-for-byte reproducibility
+matters, keep the thread count and all other compression parameters fixed.
+
+## Dictionaries
+
+Compression with a dictionary is deterministic when the dictionary bytes are
+fixed. Dictionary training is a separate process: changing the training samples,
+their order, the trainer, or its parameters can produce a different dictionary,
+which can in turn produce different compressed frames.
+
+## Tests and archives
+
+When tests or build systems need stable outputs, prefer checking that
+decompression round-trips to the expected content. If comparing compressed bytes
+is unavoidable, pin the zstd version, build, dictionary, thread count, and full
+set of compression options.

lib/README.md

@@ -55,6 +55,38 @@ is compiled. To correctly generate a `.pc` for the multi-threaded static library
 Multithreading capabilities are exposed
 via the [advanced API defined in `lib/zstd.h`](https://github.com/facebook/zstd/blob/v1.4.3/lib/zstd.h#L351).
 
+#### Memory-constrained decompression
+
+For environments with limited or specialized allocation support, `libzstd`
+provides several mechanisms to control decompression memory usage:
+
+- `ZSTD_estimateDCtxSize()` estimates the fixed workspace needed for one-shot
+  decompression with a `ZSTD_DCtx`.
+- `ZSTD_estimateDStreamSize()` and `ZSTD_estimateDStreamSize_fromFrame()`
+  estimate streaming decompression memory from a maximum window size or from a
+  frame header.
+- `ZSTD_initStaticDCtx()` and `ZSTD_initStaticDStream()` initialize a
+  decompression context inside caller-provided memory. Static contexts never
+  call `malloc()` or `free()` and return an error if the provided workspace is
+  too small.
+- `ZSTD_createDCtx_advanced()` and `ZSTD_createDStream_advanced()` accept a
+  `ZSTD_customMem` allocator, allowing applications to route allocations through
+  a private arena.
+
+When dictionaries are used, prefer explicitly creating a `ZSTD_DDict` with the
+desired memory strategy and referencing it with `ZSTD_DCtx_refDDict()`. Loading a
+dictionary directly into a context can create additional internal state.
+
+The zstd decompression state is independent of the compression level used to
+create the frame. For one-shot decompression, the `ZSTD_DCtx` workspace is a
+fixed-size state plus the caller-provided source and destination buffers. For
+streaming decompression, the memory budget is primarily driven by the frame
+window size.
+
+For builds that need to minimize stack usage from generic entropy helpers, define
+`ZSTD_NO_UNUSED_FUNCTIONS`. To reduce the decoder context footprint further, tune
+`ZSTD_DECODER_INTERNAL_BUFFER`, noting that smaller values can have a small
+decompression speed cost.
 
 #### API