ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(rust): port legacy v0.1 decoder

Browse Source 
Port lib/legacy/zstd_v01.c (the frozen zstd v0.1 decoder) to
rust/src/legacy/zstd_v01.rs as the first legacy-format port on the new
scaffolding, and reduce the C file to a declaration-only shim that
keeps its header includes for configuration and platform preprocessor
behavior.

Frozen-decoder policy: zstd_v01.c embeds its own v0.1-era FSE and
Huff0 snapshot, distinct from every other release. The Rust port is a
line-by-line translation with the same table layouts (FSE_DTable as a
u32 header word plus packed newState/symbol/nbBits entries, the Huff0
u16 DTable with byte/nbBits pairs), the same arithmetic including
wrap-around and pointer-comparison quirks (e.g. the offset-vs-base
address check in ZSTD_execSequence), the same internal FSE error space
(size_t)-1..-7, and the same public ZSTD error codes. It reuses no
modern Rust entropy module; its only crate dependency is `errors`,
matching the C file's error_private.h include. The 32-bit-only reload
points are kept as compile-time conditions on usize::BITS.

Symbol takeover boundary: all nine ZSTDv01_* entry points from
zstd_v01.h now come from Rust as context-free #[no_mangle] extern "C"
functions (isError, decompress, decompressDCtx,
findFrameSizeInfoLegacy, createDCtx, freeDCtx, resetDCtx,
nextSrcSizeToDecompress, decompressContinue). zstd_legacy.h only uses
the first four for v0.1; streaming for v0.1-v0.3 intentionally returns
version_unsupported there, unchanged. The ZSTDv01_Dctx struct
definition moves entirely into Rust: C code only ever holds an opaque
pointer (zstd_v01.h forward-declares the type), and the context is
malloc/free-allocated exactly like the C version so create/free may
pair across the language boundary.

Byte-identity verification against the pristine pre-migration C build
(f8745da6, pure C, ZSTD_LEGACY_SUPPORT=1):

- Real v0.1 frames were generated by building the v0.1.0 git tag and
  compressing text, random, and 426 KB multi-block inputs. A one-shot
  ZSTD_decompress harness linked once against the pristine C libzstd.a
  and once against the Rust-backed libzstd.a produced bit-identical
  outputs for all frames.
- A direct ZSTDv01_* probe (one-shot decode, dst-too-small, truncated
  input, bad magic, findFrameSizeInfoLegacy, and the streaming
  continue loop) printed identical results, including exact error
  codes (-70 dstSize_tooSmall, -72 srcSize_wrong, -10 prefix_unknown)
  and identical dBound values.
- zstd -l -v on v0.1 files matches the pristine binary; CLI streaming
  decode of v0.1 fails with the same "Version not supported" in both,
  by design of zstd_legacy.h.

Unit tests embed three v0.1.0-generated fixtures (entropy-coded,
raw-block, and four-block frames) plus the truncation, bad-magic,
small-destination, and streaming-API cases, all asserting the exact C
error codes above. Note that `make -C tests test-legacy` only covers
v0.4+ frames, so the embedded fixtures and the harness comparison are
the actual v0.1 coverage.

Test plan:
- cd rust && cargo fmt --check && cargo clippy --all-targets
  --features legacy-v01 -- -D warnings && cargo test --all-targets
  --features legacy-v01 (127 tests, 9 for v0.1)
- cargo clippy/test --no-default-features --features
  decompression,legacy-v01 (module builds standalone)
- make -C tests fuzzer && ./tests/fuzzer -i1 --no-big-tests, also with
  ZSTD_LEGACY_SUPPORT=1 (mixed Rust v0.1 + C v0.2-0.7 link)
- make -C tests test-rust-lib-smoke && make -C tests test-legacy
- make -C programs zstd (default and ZSTD_LEGACY_SUPPORT=1); nm shows
  the nine ZSTDv01_* symbols provided by Rust at level 1
- make -C lib libzstd.a ZSTD_LEGACY_SUPPORT=0 (no legacy symbols) and
  meson -Dlegacy_level=1 shared library exporting all nine
This commit is contained in:
ddidderr
2026-07-11 14:25:23 +02:00
parent c0a1b0bae1
commit 24488e4eaa
4 changed files with 2367 additions and 2112 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rust/README.md
+7 -4
View File
@@ -48,8 +48,9 @@ zstd ABI:
- Dictionary support
- `zstd_ddict` owns, loads, copies, and references decode dictionaries.
- Legacy decoding
- `legacy` hosts one frozen module per historical format (v0.1 through
v0.7). No version has been ported yet; all decoders are still C.
- `legacy` hosts one frozen module per historical format; `legacy::zstd_v01`
ports the self-contained v0.1 decoder. Versions v0.2 through v0.7 are
still C.
- Block decompression
- `zstd_decompress_block` decodes literal and sequence sections, maintains
FSE/Huffman repeat state, and executes compressed-block sequences.
@@ -64,7 +65,7 @@ zstd ABI:
metadata, and streaming I/O.
The optimal block matcher, high-level frame compression, dictionary-building,
the legacy v0.1-v0.7 decoders, and the CLI file-I/O backend are still C. They
the legacy v0.2-v0.7 decoders, and the CLI file-I/O backend are still C. They
must move before the rewrite is complete. Keeping that boundary explicit
prevents a passing hybrid build from being mistaken for the final all-Rust
result.
@@ -101,7 +102,9 @@ A feature whose version has not been ported yet gates nothing; the original
C file still provides that decoder, so mixed C/Rust legacy levels link
cleanly. Porting a version means adding `src/legacy/zstd_v0N.rs`, registering
it in `src/legacy/mod.rs` behind its feature, and reducing
`lib/legacy/zstd_v0N.c` to a declaration-only shim.
`lib/legacy/zstd_v0N.c` to a declaration-only shim. For v0.1 the streaming
`ZSTDv01_Dctx` state lives entirely in Rust: C code only ever holds an opaque
pointer, so the C-side struct definition is gone.
## Compatibility boundary
rust/src/legacy/mod.rs
+2
View File
@@ -36,3 +36,5 @@
//! pub mod zstd_v0N;
//! ```
#[cfg(feature = "legacy-v01")]
pub mod zstd_v01;
rust/src/legacy/zstd_v01.rs
+2354
View File
@@ -0,0 +1,2354 @@
#![allow(non_snake_case)]
//! Frozen decoder for the zstd v0.1 format.
//!
//! This is a line-by-line port of `lib/legacy/zstd_v01.c`: the same table
//! layouts, the same arithmetic, and the same error codes. The C file is a
//! self-contained snapshot of the v0.1-era FSE and Huff0 coders, so this
//! module deliberately reimplements them instead of reusing the modern
//! `fse_decompress`/`huf_decompress` modules (see `legacy/mod.rs` for the
//! frozen-decoder policy). The only shared dependency is `crate::errors`,
//! mirroring the C file's `error_private.h` include.
//!
//! The streaming `ZSTDv01_Dctx` state lives entirely in Rust; C callers only
//! ever hold an opaque pointer to it. It is allocated with `libc::malloc`
//! and released with `libc::free`, exactly like the original C context.
use crate::errors::{ERR_isError, ZstdErrorCode, ERROR};
use std::os::raw::{c_uint, c_void};
use std::ptr;
/* ******************************************
* Error management (frozen v0.1 FSE codes)
********************************************/
/* FSE_LIST_ERRORS in zstd_v01.c; values are returned as `(size_t)-code`. */
const FSE_ERROR_GENERIC: usize = 1;
const FSE_ERROR_TABLELOG_TOO_LARGE: usize = 2;
const FSE_ERROR_MAX_SYMBOL_VALUE_TOO_LARGE: usize = 3;
const FSE_ERROR_MAX_SYMBOL_VALUE_TOO_SMALL: usize = 4;
const FSE_ERROR_DST_SIZE_TOO_SMALL: usize = 5;
const FSE_ERROR_SRC_SIZE_WRONG: usize = 6;
const FSE_ERROR_CORRUPTION_DETECTED: usize = 7;
const FSE_ERROR_MAX_CODE: usize = 8;
#[inline]
fn fse_error(code: usize) -> usize {
code.wrapping_neg()
}
#[inline]
fn fse_is_error(code: usize) -> bool {
code > fse_error(FSE_ERROR_MAX_CODE)
}
/* ******************************************
* Tuning parameters (frozen)
********************************************/
const FSE_MAX_MEMORY_USAGE: u32 = 14;
const FSE_MAX_SYMBOL_VALUE: u32 = 255;
const FSE_MAX_TABLELOG: u32 = FSE_MAX_MEMORY_USAGE - 2;
const FSE_MIN_TABLELOG: u32 = 5;
const FSE_TABLELOG_ABSOLUTE_MAX: u32 = 15;
const HUF_MAX_SYMBOL_VALUE: u32 = 255;
const HUF_MAX_TABLELOG: u32 = 12;
const HUF_ABSOLUTEMAX_TABLELOG: u32 = 16;
const IS_32BITS: bool = std::mem::size_of::<usize>() == 4;
const USIZE_BITS: u32 = usize::BITS;
/* ******************************************
* Memory I/O (FSE_read* / ZSTD_read* helpers)
********************************************/
#[inline]
unsafe fn fse_read_le16(mem_ptr: *const u8) -> u16 {
u16::from_le_bytes(ptr::read_unaligned(mem_ptr as *const [u8; 2]))
}
#[inline]
unsafe fn fse_read_le32(mem_ptr: *const u8) -> u32 {
u32::from_le_bytes(ptr::read_unaligned(mem_ptr as *const [u8; 4]))
}
#[inline]
unsafe fn fse_read_le64(mem_ptr: *const u8) -> u64 {
u64::from_le_bytes(ptr::read_unaligned(mem_ptr as *const [u8; 8]))
}
#[inline]
unsafe fn fse_read_lest(mem_ptr: *const u8) -> usize {
if IS_32BITS {
fse_read_le32(mem_ptr) as usize
} else {
fse_read_le64(mem_ptr) as usize
}
}
/// `FSE_highbit32`; the caller guarantees `val != 0`, as in C.
#[inline]
fn fse_highbit32(val: u32) -> u32 {
val.leading_zeros() ^ 31
}
/* ******************************************
* FSE structures
********************************************/
#[repr(C)]
#[derive(Clone, Copy)]
struct FseDecode {
new_state: u16,
symbol: u8,
nb_bits: u8,
}
#[repr(C)]
struct FseDTableHeader {
table_log: u16,
fast_mode: u16,
}
struct FseDStream {
bit_container: usize,
bits_consumed: u32,
ptr: *const u8,
start: *const u8,
}
struct FseDState {
state: usize,
table: *const FseDecode,
}
const FSE_DSTREAM_UNFINISHED: u32 = 0;
const FSE_DSTREAM_END_OF_BUFFER: u32 = 1;
const FSE_DSTREAM_COMPLETED: u32 = 2;
const FSE_DSTREAM_TOO_FAR: u32 = 3;
#[inline]
fn fse_table_step(table_size: u32) -> u32 {
(table_size >> 1) + (table_size >> 3) + 3
}
/* An FSE_DTable is an opaque u32 array: one header word followed by
* `1 << tableLog` FseDecode entries, exactly as in C. */
unsafe fn fse_build_dtable(
dt: *mut u32,
normalized_counter: *const i16,
max_symbol_value: u32,
table_log: u32,
) -> usize {
let dtable_h = dt as *mut FseDTableHeader;
let table_decode = dt.add(1) as *mut FseDecode;
/* Sanity checks */
if max_symbol_value > FSE_MAX_SYMBOL_VALUE {
return fse_error(FSE_ERROR_MAX_SYMBOL_VALUE_TOO_LARGE);
}
if table_log > FSE_MAX_TABLELOG {
return fse_error(FSE_ERROR_TABLELOG_TOO_LARGE);
}
let table_size: u32 = 1 << table_log;
let table_mask = table_size - 1;
let step = fse_table_step(table_size);
let mut symbol_next = [0u16; (FSE_MAX_SYMBOL_VALUE + 1) as usize];
let mut position: u32 = 0;
let mut high_threshold = table_size - 1;
let large_limit = (1i32 << (table_log - 1)) as i16;
let mut no_large: u32 = 1;
/* Init, lay down lowprob symbols */
(*dtable_h).table_log = table_log as u16;
for s in 0..=max_symbol_value {
let count = *normalized_counter.add(s as usize);
if count == -1 {
(*table_decode.add(high_threshold as usize)).symbol = s as u8;
high_threshold = high_threshold.wrapping_sub(1);
symbol_next[s as usize] = 1;
} else {
if count >= large_limit {
no_large = 0;
}
symbol_next[s as usize] = count as u16;
}
}
/* Spread symbols */
for s in 0..=max_symbol_value {
let count = *normalized_counter.add(s as usize);
let mut i = 0i32;
while i < count as i32 {
(*table_decode.add(position as usize)).symbol = s as u8;
position = (position + step) & table_mask;
while position > high_threshold {
position = (position + step) & table_mask; /* lowprob area */
}
i += 1;
}
}
if position != 0 {
/* position must reach all cells once, otherwise normalizedCounter is incorrect */
return fse_error(FSE_ERROR_GENERIC);
}
/* Build Decoding table */
for i in 0..table_size as usize {
let symbol = (*table_decode.add(i)).symbol;
let next_state = symbol_next[symbol as usize];
symbol_next[symbol as usize] = next_state.wrapping_add(1);
let nb_bits = (table_log - fse_highbit32(next_state as u32)) as u8;
(*table_decode.add(i)).nb_bits = nb_bits;
(*table_decode.add(i)).new_state =
(((next_state as u32) << nb_bits).wrapping_sub(table_size)) as u16;
}
(*dtable_h).fast_mode = no_large as u16;
0
}
/* ******************************************
* FSE header bitstream (FSE_readNCount)
********************************************/
unsafe fn fse_read_ncount(
normalized_counter: *mut i16,
max_sv_ptr: &mut u32,
table_log_ptr: &mut u32,
header_buffer: *const u8,
hb_size: usize,
) -> usize {
let istart = header_buffer;
let iend_addr = (istart as usize).wrapping_add(hb_size);
let mut ip = istart;
let mut charnum: u32 = 0;
let mut previous0 = false;
if hb_size < 4 {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
let mut bit_stream = fse_read_le32(ip);
let mut nb_bits: i32 = ((bit_stream & 0xF) + FSE_MIN_TABLELOG) as i32; /* extract tableLog */
if nb_bits > FSE_TABLELOG_ABSOLUTE_MAX as i32 {
return fse_error(FSE_ERROR_TABLELOG_TOO_LARGE);
}
bit_stream >>= 4;
let mut bit_count: i32 = 4;
*table_log_ptr = nb_bits as u32;
let mut remaining: i32 = (1 << nb_bits) + 1;
let mut threshold: i32 = 1 << nb_bits;
nb_bits += 1;
while remaining > 1 && charnum <= *max_sv_ptr {
if previous0 {
let mut n0 = charnum;
while (bit_stream & 0xFFFF) == 0xFFFF {
n0 += 24;
if (ip as usize) < iend_addr.wrapping_sub(5) {
ip = ip.add(2);
bit_stream = fse_read_le32(ip).wrapping_shr(bit_count as u32);
} else {
bit_stream >>= 16;
bit_count += 16;
}
}
while (bit_stream & 3) == 3 {
n0 += 3;
bit_stream >>= 2;
bit_count += 2;
}
n0 += bit_stream & 3;
bit_count += 2;
if n0 > *max_sv_ptr {
return fse_error(FSE_ERROR_MAX_SYMBOL_VALUE_TOO_SMALL);
}
while charnum < n0 {
*normalized_counter.add(charnum as usize) = 0;
charnum += 1;
}
if (ip as usize) <= iend_addr.wrapping_sub(7)
|| (ip as usize).wrapping_add((bit_count >> 3) as usize)
<= iend_addr.wrapping_sub(4)
{
ip = ip.add((bit_count >> 3) as usize);
bit_count &= 7;
bit_stream = fse_read_le32(ip).wrapping_shr(bit_count as u32);
} else {
bit_stream >>= 2;
}
}
{
let max: i16 = ((2 * threshold - 1) - remaining) as i16;
let mut count: i16;
if (bit_stream & (threshold - 1) as u32) < max as i32 as u32 {
count = (bit_stream & (threshold - 1) as u32) as u16 as i16;
bit_count += nb_bits - 1;
} else {
count = (bit_stream & (2 * threshold - 1) as u32) as u16 as i16;
if count as i32 >= threshold {
count = ((count as i32) - (max as i32)) as i16;
}
bit_count += nb_bits;
}
count = count.wrapping_sub(1); /* extra accuracy */
remaining -= (count as i32).abs();
*normalized_counter.add(charnum as usize) = count;
charnum += 1;
previous0 = count == 0;
while remaining < threshold {
nb_bits -= 1;
threshold >>= 1;
}
if (ip as usize) <= iend_addr.wrapping_sub(7)
|| (ip as usize).wrapping_add((bit_count >> 3) as usize)
<= iend_addr.wrapping_sub(4)
{
ip = ip.add((bit_count >> 3) as usize);
bit_count &= 7;
} else {
bit_count -=
(8 * (iend_addr.wrapping_sub(4) as isize - ip as usize as isize)) as i32;
ip = (iend_addr - 4) as *const u8;
}
bit_stream = fse_read_le32(ip).wrapping_shr((bit_count & 31) as u32);
}
}
if remaining != 1 {
return fse_error(FSE_ERROR_GENERIC);
}
*max_sv_ptr = charnum - 1;
ip = ip.wrapping_offset(((bit_count + 7) >> 3) as isize);
if (ip as usize).wrapping_sub(istart as usize) > hb_size {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
(ip as usize).wrapping_sub(istart as usize)
}
/* ******************************************
* FSE decompression, byte symbols
********************************************/
unsafe fn fse_build_dtable_rle(dt: *mut u32, symbol_value: u8) -> usize {
let dtable_h = dt as *mut FseDTableHeader;
let cell = dt.add(1) as *mut FseDecode;
(*dtable_h).table_log = 0;
(*dtable_h).fast_mode = 0;
(*cell).new_state = 0;
(*cell).symbol = symbol_value;
(*cell).nb_bits = 0;
0
}
unsafe fn fse_build_dtable_raw(dt: *mut u32, nb_bits: u32) -> usize {
let dtable_h = dt as *mut FseDTableHeader;
let dinfo = dt.add(1) as *mut FseDecode;
/* Sanity checks */
if nb_bits < 1 {
return fse_error(FSE_ERROR_GENERIC); /* min size */
}
let table_size: u32 = 1 << nb_bits;
let table_mask = table_size - 1;
let max_symbol_value = table_mask;
(*dtable_h).table_log = nb_bits as u16;
(*dtable_h).fast_mode = 1;
for s in 0..=max_symbol_value {
let cell = dinfo.add(s as usize);
(*cell).new_state = 0;
(*cell).symbol = s as u8;
(*cell).nb_bits = nb_bits as u8;
}
0
}
unsafe fn fse_init_dstream(
bit_d: &mut FseDStream,
src_buffer: *const u8,
src_size: usize,
) -> usize {
let word = std::mem::size_of::<usize>();
if src_size < 1 {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
if src_size >= word {
bit_d.start = src_buffer;
bit_d.ptr = src_buffer.add(src_size - word);
bit_d.bit_container = fse_read_lest(bit_d.ptr);
let contain32 = *src_buffer.add(src_size - 1) as u32;
if contain32 == 0 {
return fse_error(FSE_ERROR_GENERIC); /* stop bit not present */
}
bit_d.bits_consumed = 8 - fse_highbit32(contain32);
} else {
bit_d.start = src_buffer;
bit_d.ptr = bit_d.start;
bit_d.bit_container = *bit_d.start as usize;
/* C switch with fallthrough over srcSize 7..2 */
if src_size >= 7 {
bit_d.bit_container += (*src_buffer.add(6) as usize) << (USIZE_BITS as usize - 16);
}
if src_size >= 6 {
bit_d.bit_container += (*src_buffer.add(5) as usize) << (USIZE_BITS as usize - 24);
}
if src_size >= 5 {
bit_d.bit_container += (*src_buffer.add(4) as usize) << (USIZE_BITS as usize - 32);
}
if src_size >= 4 {
bit_d.bit_container += (*src_buffer.add(3) as usize) << 24;
}
if src_size >= 3 {
bit_d.bit_container += (*src_buffer.add(2) as usize) << 16;
}
if src_size >= 2 {
bit_d.bit_container += (*src_buffer.add(1) as usize) << 8;
}
let contain32 = *src_buffer.add(src_size - 1) as u32;
if contain32 == 0 {
return fse_error(FSE_ERROR_GENERIC); /* stop bit not present */
}
bit_d.bits_consumed = 8 - fse_highbit32(contain32);
bit_d.bits_consumed += ((word - src_size) * 8) as u32;
}
src_size
}
#[inline]
unsafe fn fse_look_bits(bit_d: &FseDStream, nb_bits: u32) -> usize {
let bit_mask = USIZE_BITS - 1;
((bit_d.bit_container << (bit_d.bits_consumed & bit_mask)) >> 1)
>> (bit_mask.wrapping_sub(nb_bits) & bit_mask)
}
#[inline]
unsafe fn fse_look_bits_fast(bit_d: &FseDStream, nb_bits: u32) -> usize {
/* only if nb_bits >= 1 */
let bit_mask = USIZE_BITS - 1;
(bit_d.bit_container << (bit_d.bits_consumed & bit_mask))
>> ((bit_mask + 1).wrapping_sub(nb_bits) & bit_mask)
}
#[inline]
fn fse_skip_bits(bit_d: &mut FseDStream, nb_bits: u32) {
bit_d.bits_consumed = bit_d.bits_consumed.wrapping_add(nb_bits);
}
#[inline]
unsafe fn fse_read_bits(bit_d: &mut FseDStream, nb_bits: u32) -> usize {
let value = fse_look_bits(bit_d, nb_bits);
fse_skip_bits(bit_d, nb_bits);
value
}
#[inline]
unsafe fn fse_read_bits_fast(bit_d: &mut FseDStream, nb_bits: u32) -> usize {
/* only if nb_bits >= 1 */
let value = fse_look_bits_fast(bit_d, nb_bits);
fse_skip_bits(bit_d, nb_bits);
value
}
unsafe fn fse_reload_dstream(bit_d: &mut FseDStream) -> u32 {
let word = std::mem::size_of::<usize>();
if bit_d.bits_consumed > (word * 8) as u32 {
/* should never happen */
return FSE_DSTREAM_TOO_FAR;
}
if (bit_d.ptr as usize) >= (bit_d.start as usize).wrapping_add(word) {
bit_d.ptr = bit_d.ptr.sub((bit_d.bits_consumed >> 3) as usize);
bit_d.bits_consumed &= 7;
bit_d.bit_container = fse_read_lest(bit_d.ptr);
return FSE_DSTREAM_UNFINISHED;
}
if bit_d.ptr == bit_d.start {
if bit_d.bits_consumed < (word * 8) as u32 {
return FSE_DSTREAM_END_OF_BUFFER;
}
return FSE_DSTREAM_COMPLETED;
}
{
let mut nb_bytes = bit_d.bits_consumed >> 3;
let mut result = FSE_DSTREAM_UNFINISHED;
if (bit_d.ptr as usize).wrapping_sub(nb_bytes as usize) < (bit_d.start as usize) {
nb_bytes = ((bit_d.ptr as usize) - (bit_d.start as usize)) as u32; /* ptr > start */
result = FSE_DSTREAM_END_OF_BUFFER;
}
bit_d.ptr = bit_d.ptr.sub(nb_bytes as usize);
bit_d.bits_consumed -= nb_bytes * 8;
bit_d.bit_container = fse_read_lest(bit_d.ptr); /* reminder : srcSize > sizeof(bitD) */
result
}
}
unsafe fn fse_init_dstate(d_state: &mut FseDState, bit_d: &mut FseDStream, dt: *const u32) {
let dtable_h = dt as *const FseDTableHeader;
d_state.state = fse_read_bits(bit_d, (*dtable_h).table_log as u32);
fse_reload_dstream(bit_d);
d_state.table = dt.add(1) as *const FseDecode;
}
unsafe fn fse_decode_symbol(d_state: &mut FseDState, bit_d: &mut FseDStream) -> u8 {
let d_info = *d_state.table.add(d_state.state);
let low_bits = fse_read_bits(bit_d, d_info.nb_bits as u32);
d_state.state = (d_info.new_state as usize).wrapping_add(low_bits);
d_info.symbol
}
unsafe fn fse_decode_symbol_fast(d_state: &mut FseDState, bit_d: &mut FseDStream) -> u8 {
let d_info = *d_state.table.add(d_state.state);
let low_bits = fse_read_bits_fast(bit_d, d_info.nb_bits as u32);
d_state.state = (d_info.new_state as usize).wrapping_add(low_bits);
d_info.symbol
}
#[inline]
fn fse_end_of_dstream(bit_d: &FseDStream) -> bool {
bit_d.ptr == bit_d.start && bit_d.bits_consumed == USIZE_BITS
}
#[inline]
fn fse_end_of_dstate(d_state: &FseDState) -> bool {
d_state.state == 0
}
unsafe fn fse_decompress_using_dtable_generic(
dst: *mut u8,
max_dst_size: usize,
c_src: *const u8,
c_src_size: usize,
dt: *const u32,
fast: bool,
) -> usize {
let ostart = dst;
let mut op = ostart;
let omax_addr = (op as usize).wrapping_add(max_dst_size);
let olimit_addr = omax_addr.wrapping_sub(3);
let mut bit_d = FseDStream {
bit_container: 0,
bits_consumed: 0,
ptr: ptr::null(),
start: ptr::null(),
};
let mut state1 = FseDState {
state: 0,
table: ptr::null(),
};
let mut state2 = FseDState {
state: 0,
table: ptr::null(),
};
/* Init */
let error_code = fse_init_dstream(&mut bit_d, c_src, c_src_size);
if fse_is_error(error_code) {
return error_code;
}
fse_init_dstate(&mut state1, &mut bit_d, dt);
fse_init_dstate(&mut state2, &mut bit_d, dt);
macro_rules! fse_getsymbol {
($state:expr) => {
if fast {
fse_decode_symbol_fast($state, &mut bit_d)
} else {
fse_decode_symbol($state, &mut bit_d)
}
};
}
/* Constant conditions from the C source; on 64-bit both are false, on
* 32-bit only the *4 variant reloads. */
const RELOAD_2: bool = FSE_MAX_TABLELOG * 2 + 7 > USIZE_BITS;
const RELOAD_4: bool = FSE_MAX_TABLELOG * 4 + 7 > USIZE_BITS;
/* 4 symbols per loop */
while fse_reload_dstream(&mut bit_d) == FSE_DSTREAM_UNFINISHED && (op as usize) < olimit_addr {
*op = fse_getsymbol!(&mut state1);
if RELOAD_2 {
/* This test must be static */
fse_reload_dstream(&mut bit_d);
}
*op.add(1) = fse_getsymbol!(&mut state2);
if RELOAD_4 {
/* This test must be static */
if fse_reload_dstream(&mut bit_d) > FSE_DSTREAM_UNFINISHED {
op = op.add(2);
break;
}
}
*op.add(2) = fse_getsymbol!(&mut state1);
if RELOAD_2 {
/* This test must be static */
fse_reload_dstream(&mut bit_d);
}
*op.add(3) = fse_getsymbol!(&mut state2);
op = op.add(4);
}
/* tail */
loop {
if fse_reload_dstream(&mut bit_d) > FSE_DSTREAM_COMPLETED
|| (op as usize) == omax_addr
|| (fse_end_of_dstream(&bit_d) && (fast || fse_end_of_dstate(&state1)))
{
break;
}
*op = fse_getsymbol!(&mut state1);
op = op.add(1);
if fse_reload_dstream(&mut bit_d) > FSE_DSTREAM_COMPLETED
|| (op as usize) == omax_addr
|| (fse_end_of_dstream(&bit_d) && (fast || fse_end_of_dstate(&state2)))
{
break;
}
*op = fse_getsymbol!(&mut state2);
op = op.add(1);
}
/* end ? */
if fse_end_of_dstream(&bit_d) && fse_end_of_dstate(&state1) && fse_end_of_dstate(&state2) {
return (op as usize) - (ostart as usize);
}
if (op as usize) == omax_addr {
/* dst buffer is full, but cSrc unfinished */
return fse_error(FSE_ERROR_DST_SIZE_TOO_SMALL);
}
fse_error(FSE_ERROR_CORRUPTION_DETECTED)
}
unsafe fn fse_decompress_using_dtable(
dst: *mut u8,
original_size: usize,
c_src: *const u8,
c_src_size: usize,
dt: *const u32,
) -> usize {
let fast_mode = (*(dt as *const FseDTableHeader)).fast_mode;
/* select fast mode (static) */
if fast_mode != 0 {
return fse_decompress_using_dtable_generic(
dst,
original_size,
c_src,
c_src_size,
dt,
true,
);
}
fse_decompress_using_dtable_generic(dst, original_size, c_src, c_src_size, dt, false)
}
unsafe fn fse_decompress(
dst: *mut u8,
max_dst_size: usize,
c_src: *const u8,
c_src_size: usize,
) -> usize {
let istart = c_src;
let mut ip = istart;
let mut counting = [0i16; (FSE_MAX_SYMBOL_VALUE + 1) as usize];
let mut dt = [0u32; 1 + (1 << FSE_MAX_TABLELOG)]; /* DTable_max_t */
let mut table_log: u32 = 0;
let mut max_symbol_value: u32 = FSE_MAX_SYMBOL_VALUE;
let mut remaining_size = c_src_size;
if c_src_size < 2 {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG); /* too small input size */
}
/* normal FSE decoding mode */
let error_code = fse_read_ncount(
counting.as_mut_ptr(),
&mut max_symbol_value,
&mut table_log,
istart,
c_src_size,
);
if fse_is_error(error_code) {
return error_code;
}
if error_code >= c_src_size {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG); /* too small input size */
}
ip = ip.add(error_code);
remaining_size -= error_code;
let error_code = fse_build_dtable(
dt.as_mut_ptr(),
counting.as_ptr(),
max_symbol_value,
table_log,
);
if fse_is_error(error_code) {
return error_code;
}
/* always return, even if it is an error code */
fse_decompress_using_dtable(dst, max_dst_size, ip, remaining_size, dt.as_ptr())
}
/* ******************************************
* Huff0 : Huffman block decompression
********************************************/
#[repr(C)]
#[derive(Clone, Copy)]
struct HufDElt {
byte: u8,
nb_bits: u8,
}
/* Loop shapes and arithmetic below intentionally mirror the frozen C. */
#[allow(clippy::needless_range_loop, clippy::manual_div_ceil)]
unsafe fn huf_read_dtable(dtable: *mut u16, src: *const u8, src_size: usize) -> usize {
let mut huff_weight = [0u8; (HUF_MAX_SYMBOL_VALUE + 1) as usize];
let mut rank_val = [0u32; (HUF_ABSOLUTEMAX_TABLELOG + 1) as usize];
let ip = src;
let dt = dtable.add(1) as *mut HufDElt;
if src_size == 0 {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
let mut i_size = *ip as usize;
let o_size: usize;
if i_size >= 128 {
/* special header */
if i_size >= 242 {
/* RLE */
const L: [usize; 14] = [1, 2, 3, 4, 7, 8, 15, 16, 31, 32, 63, 64, 127, 128];
o_size = L[i_size - 242];
huff_weight = [1u8; (HUF_MAX_SYMBOL_VALUE + 1) as usize];
i_size = 0;
} else {
/* Incompressible */
o_size = i_size - 127;
i_size = (o_size + 1) / 2;
if i_size + 1 > src_size {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
let ip = ip.add(1);
let mut n = 0usize;
while n < o_size {
huff_weight[n] = *ip.add(n / 2) >> 4;
huff_weight[n + 1] = *ip.add(n / 2) & 15;
n += 2;
}
}
} else {
/* header compressed with FSE (normal case) */
if i_size + 1 > src_size {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
/* max 255 values decoded, last one is implied */
let decoded = fse_decompress(
huff_weight.as_mut_ptr(),
HUF_MAX_SYMBOL_VALUE as usize,
ip.add(1),
i_size,
);
if fse_is_error(decoded) {
return decoded;
}
o_size = decoded;
}
/* collect weight stats */
let mut weight_total: u32 = 0;
for n in 0..o_size {
if huff_weight[n] >= HUF_ABSOLUTEMAX_TABLELOG as u8 {
return fse_error(FSE_ERROR_CORRUPTION_DETECTED);
}
rank_val[huff_weight[n] as usize] += 1;
weight_total += (1u32 << huff_weight[n]) >> 1;
}
if weight_total == 0 {
return fse_error(FSE_ERROR_CORRUPTION_DETECTED);
}
/* get last non-null symbol weight (implied, total must be 2^n) */
let max_bits = fse_highbit32(weight_total) + 1;
if max_bits > *dtable as u32 {
return fse_error(FSE_ERROR_TABLELOG_TOO_LARGE); /* DTable is too small */
}
*dtable = max_bits as u16;
{
let total = 1u32 << max_bits;
let rest = total - weight_total;
let verif = 1u32 << fse_highbit32(rest);
let last_weight = fse_highbit32(rest) + 1;
if verif != rest {
return fse_error(FSE_ERROR_CORRUPTION_DETECTED); /* last value must be a clean power of 2 */
}
huff_weight[o_size] = last_weight as u8;
rank_val[last_weight as usize] += 1;
}
/* check tree construction validity */
if rank_val[1] < 2 || (rank_val[1] & 1) != 0 {
/* by construction : at least 2 elts of rank 1, must be even */
return fse_error(FSE_ERROR_CORRUPTION_DETECTED);
}
/* Prepare ranks */
let mut next_rank_start: u32 = 0;
for n in 1..=(max_bits as usize) {
let current = next_rank_start;
next_rank_start += rank_val[n] << (n - 1);
rank_val[n] = current;
}
/* fill DTable */
for n in 0..=o_size {
let w = huff_weight[n] as usize;
let length = (1u32 << w) >> 1;
let d = HufDElt {
byte: n as u8,
nb_bits: (max_bits + 1 - w as u32) as u8,
};
for i in rank_val[w]..(rank_val[w] + length) {
*dt.add(i as usize) = d;
}
rank_val[w] += length;
}
i_size + 1
}
unsafe fn huf_decode_symbol(d_stream: &mut FseDStream, dt: *const HufDElt, dt_log: u32) -> u8 {
let val = fse_look_bits_fast(d_stream, dt_log); /* note : dtLog >= 1 */
let entry = *dt.add(val);
fse_skip_bits(d_stream, entry.nb_bits as u32);
entry.byte
}
/* The C decode macros: SYMBOL_1 reloads on 32-bit only when
* HUF_MAX_TABLELOG > 12 (never here); SYMBOL_2 reloads on 32-bit. */
#[inline]
unsafe fn huf_decode_symbol_1(
op: *mut u8,
d_stream: &mut FseDStream,
dt: *const HufDElt,
dt_log: u32,
) {
*op = huf_decode_symbol(d_stream, dt, dt_log);
if IS_32BITS && HUF_MAX_TABLELOG > 12 {
fse_reload_dstream(d_stream);
}
}
#[inline]
unsafe fn huf_decode_symbol_2(
op: *mut u8,
d_stream: &mut FseDStream,
dt: *const HufDElt,
dt_log: u32,
) {
*op = huf_decode_symbol(d_stream, dt, dt_log);
if IS_32BITS {
fse_reload_dstream(d_stream);
}
}
unsafe fn huf_decompress_using_dtable(
dst: *mut u8,
max_dst_size: usize,
c_src: *const u8,
c_src_size: usize,
dtable: *const u16,
) -> usize {
if c_src_size < 6 {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
let ostart = dst;
let mut op = ostart;
let omax_addr = (op as usize).wrapping_add(max_dst_size);
let olimit_addr = if max_dst_size < 15 {
op as usize
} else {
omax_addr - 15
};
let dt = dtable.add(1) as *const HufDElt;
let dt_log = *dtable as u32;
/* Init */
let length1 = fse_read_le16(c_src) as usize;
let length2 = fse_read_le16(c_src.add(2)) as usize;
let length3 = fse_read_le16(c_src.add(4)) as usize;
let length4 = c_src_size
.wrapping_sub(6)
.wrapping_sub(length1)
.wrapping_sub(length2)
.wrapping_sub(length3); /* check coherency !! */
let start1 = c_src.add(6);
let start2 = start1.wrapping_add(length1);
let start3 = start2.wrapping_add(length2);
let start4 = start3.wrapping_add(length3);
if length1 + length2 + length3 + 6 >= c_src_size {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
let mut bit_d1 = FseDStream {
bit_container: 0,
bits_consumed: 0,
ptr: ptr::null(),
start: ptr::null(),
};
let mut bit_d2 = FseDStream {
bit_container: 0,
bits_consumed: 0,
ptr: ptr::null(),
start: ptr::null(),
};
let mut bit_d3 = FseDStream {
bit_container: 0,
bits_consumed: 0,
ptr: ptr::null(),
start: ptr::null(),
};
let mut bit_d4 = FseDStream {
bit_container: 0,
bits_consumed: 0,
ptr: ptr::null(),
start: ptr::null(),
};
let error_code = fse_init_dstream(&mut bit_d1, start1, length1);
if fse_is_error(error_code) {
return error_code;
}
let error_code = fse_init_dstream(&mut bit_d2, start2, length2);
if fse_is_error(error_code) {
return error_code;
}
let error_code = fse_init_dstream(&mut bit_d3, start3, length3);
if fse_is_error(error_code) {
return error_code;
}
let error_code = fse_init_dstream(&mut bit_d4, start4, length4);
if fse_is_error(error_code) {
return error_code;
}
let mut reload_status = fse_reload_dstream(&mut bit_d2);
/* 16 symbols per loop; D2-3-4 are supposed to be synchronized and finish together */
while reload_status < FSE_DSTREAM_COMPLETED && (op as usize) < olimit_addr {
huf_decode_symbol_1(op, &mut bit_d1, dt, dt_log);
huf_decode_symbol_1(op.add(1), &mut bit_d2, dt, dt_log);
huf_decode_symbol_1(op.add(2), &mut bit_d3, dt, dt_log);
huf_decode_symbol_1(op.add(3), &mut bit_d4, dt, dt_log);
huf_decode_symbol_2(op.add(4), &mut bit_d1, dt, dt_log);
huf_decode_symbol_2(op.add(5), &mut bit_d2, dt, dt_log);
huf_decode_symbol_2(op.add(6), &mut bit_d3, dt, dt_log);
huf_decode_symbol_2(op.add(7), &mut bit_d4, dt, dt_log);
huf_decode_symbol_1(op.add(8), &mut bit_d1, dt, dt_log);
huf_decode_symbol_1(op.add(9), &mut bit_d2, dt, dt_log);
huf_decode_symbol_1(op.add(10), &mut bit_d3, dt, dt_log);
huf_decode_symbol_1(op.add(11), &mut bit_d4, dt, dt_log);
*op.add(12) = huf_decode_symbol(&mut bit_d1, dt, dt_log);
*op.add(13) = huf_decode_symbol(&mut bit_d2, dt, dt_log);
*op.add(14) = huf_decode_symbol(&mut bit_d3, dt, dt_log);
*op.add(15) = huf_decode_symbol(&mut bit_d4, dt, dt_log);
op = op.add(16);
reload_status = fse_reload_dstream(&mut bit_d2)
| fse_reload_dstream(&mut bit_d3)
| fse_reload_dstream(&mut bit_d4);
fse_reload_dstream(&mut bit_d1);
}
if reload_status != FSE_DSTREAM_COMPLETED {
/* not complete : some bitStream might be FSE_DStream_unfinished */
return fse_error(FSE_ERROR_CORRUPTION_DETECTED);
}
/* tail */
{
let mut bit_tail = FseDStream {
bit_container: bit_d1.bit_container, /* required in case of FSE_DStream_endOfBuffer */
bits_consumed: bit_d1.bits_consumed,
ptr: bit_d1.ptr,
start: start1,
};
while fse_reload_dstream(&mut bit_tail) < FSE_DSTREAM_COMPLETED && (op as usize) < omax_addr
{
*op = huf_decode_symbol(&mut bit_tail, dt, dt_log);
op = op.add(1);
}
if fse_end_of_dstream(&bit_tail) {
return (op as usize) - (ostart as usize);
}
}
if (op as usize) == omax_addr {
return fse_error(FSE_ERROR_DST_SIZE_TOO_SMALL); /* dst buffer is full, but cSrc unfinished */
}
fse_error(FSE_ERROR_CORRUPTION_DETECTED)
}
unsafe fn huf_decompress(
dst: *mut u8,
max_dst_size: usize,
c_src: *const u8,
c_src_size: usize,
) -> usize {
/* HUF_CREATE_STATIC_DTABLE(DTable, HUF_MAX_TABLELOG) */
let mut dtable = [0u16; 1 + (1 << HUF_MAX_TABLELOG)];
dtable[0] = HUF_MAX_TABLELOG as u16;
let mut ip = c_src;
let error_code = huf_read_dtable(dtable.as_mut_ptr(), c_src, c_src_size);
if fse_is_error(error_code) {
return error_code;
}
if error_code >= c_src_size {
return fse_error(FSE_ERROR_SRC_SIZE_WRONG);
}
ip = ip.add(error_code);
huf_decompress_using_dtable(
dst,
max_dst_size,
ip,
c_src_size - error_code,
dtable.as_ptr(),
)
}
/* ******************************************
* zstd v0.1 frame decoding
********************************************/
const ZSTD_MAGIC_NUMBER: u32 = 0xFD2FB51E; /* 3rd version : seqNb header */
const KB: usize = 1 << 10;
const BLOCKSIZE: usize = 128 * KB; /* define, for static allocation */
const MINMATCH: usize = 4;
const MLBITS: u32 = 7;
const LLBITS: u32 = 6;
const OFFBITS: u32 = 5;
const MAX_ML: u32 = (1 << MLBITS) - 1;
const MAX_LL: u32 = (1 << LLBITS) - 1;
const MAX_OFF: u32 = (1 << OFFBITS) - 1;
#[allow(dead_code)] /* part of the frozen v0.1 constant set; used only by the compressor */
const LIT_FSE_LOG: u32 = 11;
const ML_FSE_LOG: u32 = 10;
const LL_FSE_LOG: u32 = 10;
const OFF_FSE_LOG: u32 = 9;
const ZSTD_CONTENTSIZE_ERROR: u64 = 0u64.wrapping_sub(2);
const ZSTD_BLOCK_HEADER_SIZE: usize = 3;
const ZSTD_FRAME_HEADER_SIZE: usize = 4;
/* FSE_DTABLE_SIZE_U32(maxTableLog) == 1 + (1 << maxTableLog) */
const LL_DTABLE_SIZE_U32: usize = 1 + (1 << LL_FSE_LOG as usize);
const OFF_DTABLE_SIZE_U32: usize = 1 + (1 << OFF_FSE_LOG as usize);
const ML_DTABLE_SIZE_U32: usize = 1 + (1 << ML_FSE_LOG as usize);
#[inline]
unsafe fn zstd_copy4(dst: *mut u8, src: *const u8) {
ptr::copy_nonoverlapping(src, dst, 4);
}
#[inline]
unsafe fn zstd_copy8(dst: *mut u8, src: *const u8) {
ptr::copy_nonoverlapping(src, dst, 8);
}
unsafe fn zstd_wildcopy(dst: *mut u8, src: *const u8, length: isize) {
let mut ip = src;
let mut op = dst;
let oend_addr = (op as usize).wrapping_add(length as usize);
while (op as usize) < oend_addr {
zstd_copy8(op, ip);
op = op.add(8);
ip = ip.add(8);
}
}
#[inline]
unsafe fn zstd_read_le16(mem_ptr: *const u8) -> u16 {
u16::from_le_bytes(ptr::read_unaligned(mem_ptr as *const [u8; 2]))
}
#[inline]
unsafe fn zstd_read_le24(mem_ptr: *const u8) -> u32 {
zstd_read_le16(mem_ptr) as u32 + ((*mem_ptr.add(2) as u32) << 16)
}
#[inline]
unsafe fn zstd_read_be32(mem_ptr: *const u8) -> u32 {
u32::from_be_bytes(ptr::read_unaligned(mem_ptr as *const [u8; 4]))
}
/* blockType_t */
const BT_COMPRESSED: u32 = 0;
const BT_RAW: u32 = 1;
const BT_RLE: u32 = 2;
const BT_END: u32 = 3;
struct BlockProperties {
block_type: u32,
orig_size: u32,
}
/// The v0.1 streaming decompression context (`dctx_t` in C). The struct is
/// opaque to C: `zstd_v01.h` only forward-declares it, so the definition now
/// lives here.
#[repr(C)]
pub struct ZSTDv01_Dctx {
ll_table: [u32; LL_DTABLE_SIZE_U32],
off_table: [u32; OFF_DTABLE_SIZE_U32],
ml_table: [u32; ML_DTABLE_SIZE_U32],
previous_dst_end: *const u8,
base: *const u8,
expected: usize,
b_type: u32,
phase: u32,
}
unsafe fn zstdv01_getc_block_size(
src: *const u8,
src_size: usize,
bp_ptr: &mut BlockProperties,
) -> usize {
if src_size < 3 {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
let header_flags = *src;
let c_size =
(*src.add(2) as u32) + ((*src.add(1) as u32) << 8) + (((header_flags & 7) as u32) << 16);
bp_ptr.block_type = (header_flags >> 6) as u32;
bp_ptr.orig_size = if bp_ptr.block_type == BT_RLE {
c_size
} else {
0
};
if bp_ptr.block_type == BT_END {
return 0;
}
if bp_ptr.block_type == BT_RLE {
return 1;
}
c_size as usize
}
unsafe fn zstd_copy_uncompressed_block(
dst: *mut u8,
max_dst_size: usize,
src: *const u8,
src_size: usize,
) -> usize {
if src_size > max_dst_size {
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
if src_size > 0 {
ptr::copy_nonoverlapping(src, dst, src_size);
}
src_size
}
unsafe fn zstd_decompress_literals(
dst: *mut u8,
max_dst_size: usize,
src: *const u8,
src_size: usize,
) -> usize {
let oend = dst.add(max_dst_size);
let ip = src;
/* check : minimum 2, for litSize, +1, for content */
if src_size <= 3 {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
let mut lit_size = (*ip.add(1) as usize) + ((*ip as usize) << 8);
lit_size += (((*ip.offset(-3) as usize) >> 3) & 7) << 16; /* mmmmh.... */
if lit_size > max_dst_size {
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
let op = oend.sub(lit_size);
let error_code = huf_decompress(op, lit_size, ip.add(2), src_size - 2);
if fse_is_error(error_code) {
return ERROR(ZstdErrorCode::Generic);
}
lit_size
}
unsafe fn zstdv01_decode_literals_block(
dst: *mut u8,
max_dst_size: usize,
lit_start: &mut *const u8,
lit_size: &mut usize,
src: *const u8,
src_size: usize,
) -> usize {
let istart = src;
let mut ip = istart;
let ostart = dst;
let oend = ostart.add(max_dst_size);
let mut litbp = BlockProperties {
block_type: 0,
orig_size: 0,
};
let litc_size = zstdv01_getc_block_size(src, src_size, &mut litbp);
if ERR_isError(litc_size) {
return litc_size;
}
if litc_size > src_size - ZSTD_BLOCK_HEADER_SIZE {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
ip = ip.add(ZSTD_BLOCK_HEADER_SIZE);
match litbp.block_type {
BT_RAW => {
*lit_start = ip;
ip = ip.add(litc_size);
*lit_size = litc_size;
}
BT_RLE => {
let rle_size = litbp.orig_size as usize;
if rle_size > max_dst_size {
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
if src_size == 0 {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
if rle_size > 0 {
ptr::write_bytes(oend.sub(rle_size), *ip, rle_size);
}
*lit_start = oend.sub(rle_size);
*lit_size = rle_size;
ip = ip.add(1);
}
BT_COMPRESSED => {
let decoded_lit_size = zstd_decompress_literals(dst, max_dst_size, ip, litc_size);
if ERR_isError(decoded_lit_size) {
return decoded_lit_size;
}
*lit_start = oend.sub(decoded_lit_size);
*lit_size = decoded_lit_size;
ip = ip.add(litc_size);
}
_ => {
/* bt_end and impossible values */
return ERROR(ZstdErrorCode::Generic);
}
}
(ip as usize) - (istart as usize)
}
#[allow(clippy::too_many_arguments)]
unsafe fn zstdv01_decode_seq_headers(
nb_seq: &mut i32,
dumps_ptr: &mut *const u8,
dumps_length_ptr: &mut usize,
dtable_ll: *mut u32,
dtable_ml: *mut u32,
dtable_offb: *mut u32,
src: *const u8,
src_size: usize,
) -> usize {
let istart = src;
let mut ip = istart;
let iend_addr = (istart as usize).wrapping_add(src_size);
/* check */
if src_size < 5 {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
/* SeqHead */
*nb_seq = zstd_read_le16(ip) as i32;
ip = ip.add(2);
let ll_type = (*ip >> 6) as u32;
let off_type = ((*ip >> 4) & 3) as u32;
let ml_type = ((*ip >> 2) & 3) as u32;
let dumps_length: usize;
if (*ip & 2) != 0 {
dumps_length = (*ip.add(2) as usize) + ((*ip.add(1) as usize) << 8);
ip = ip.add(3);
} else {
dumps_length = (*ip.add(1) as usize) + (((*ip as usize) & 1) << 8);
ip = ip.add(2);
}
*dumps_ptr = ip;
ip = ip.wrapping_add(dumps_length);
*dumps_length_ptr = dumps_length;
/* check */
if (ip as usize) > iend_addr.wrapping_sub(3) {
/* min : all 3 are "raw", hence no header, but at least xxLog bits per type */
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
/* sequences */
{
let mut norm = [0i16; (MAX_ML + 1) as usize]; /* assumption : MaxML >= MaxLL and MaxOff */
/* Build DTables */
match ll_type {
BT_RLE => {
fse_build_dtable_rle(dtable_ll, *ip);
ip = ip.add(1);
}
BT_RAW => {
fse_build_dtable_raw(dtable_ll, LLBITS);
}
_ => {
let mut max = MAX_LL;
let mut ll_log = 0u32;
let header_size = fse_read_ncount(
norm.as_mut_ptr(),
&mut max,
&mut ll_log,
ip,
iend_addr - ip as usize,
);
if fse_is_error(header_size) {
return ERROR(ZstdErrorCode::Generic);
}
if ll_log > LL_FSE_LOG {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
ip = ip.add(header_size);
fse_build_dtable(dtable_ll, norm.as_ptr(), max, ll_log);
}
}
match off_type {
BT_RLE => {
if (ip as usize) > iend_addr.wrapping_sub(2) {
/* min : "raw", hence no header, but at least xxLog bits */
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
fse_build_dtable_rle(dtable_offb, *ip);
ip = ip.add(1);
}
BT_RAW => {
fse_build_dtable_raw(dtable_offb, OFFBITS);
}
_ => {
let mut max = MAX_OFF;
let mut off_log = 0u32;
let header_size = fse_read_ncount(
norm.as_mut_ptr(),
&mut max,
&mut off_log,
ip,
iend_addr - ip as usize,
);
if fse_is_error(header_size) {
return ERROR(ZstdErrorCode::Generic);
}
if off_log > OFF_FSE_LOG {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
ip = ip.add(header_size);
fse_build_dtable(dtable_offb, norm.as_ptr(), max, off_log);
}
}
match ml_type {
BT_RLE => {
if (ip as usize) > iend_addr.wrapping_sub(2) {
/* min : "raw", hence no header, but at least xxLog bits */
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
fse_build_dtable_rle(dtable_ml, *ip);
ip = ip.add(1);
}
BT_RAW => {
fse_build_dtable_raw(dtable_ml, MLBITS);
}
_ => {
let mut max = MAX_ML;
let mut ml_log = 0u32;
let header_size = fse_read_ncount(
norm.as_mut_ptr(),
&mut max,
&mut ml_log,
ip,
iend_addr - ip as usize,
);
if fse_is_error(header_size) {
return ERROR(ZstdErrorCode::Generic);
}
if ml_log > ML_FSE_LOG {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
ip = ip.add(header_size);
fse_build_dtable(dtable_ml, norm.as_ptr(), max, ml_log);
}
}
}
(ip as usize) - (istart as usize)
}
#[derive(Clone, Copy)]
struct Seq {
lit_length: usize,
offset: usize,
match_length: usize,
}
struct SeqState {
d_stream: FseDStream,
state_ll: FseDState,
state_offb: FseDState,
state_ml: FseDState,
prev_offset: usize,
dumps: *const u8,
dumps_end: *const u8,
}
unsafe fn zstd_decode_sequence(seq: &mut Seq, seq_state: &mut SeqState) {
let mut dumps = seq_state.dumps;
let de = seq_state.dumps_end;
/* Literal length */
let mut lit_length =
fse_decode_symbol(&mut seq_state.state_ll, &mut seq_state.d_stream) as usize;
let prev_offset = if lit_length != 0 {
seq.offset
} else {
seq_state.prev_offset
};
seq_state.prev_offset = seq.offset;
if lit_length == MAX_LL as usize {
let add = if (dumps as usize) < (de as usize) {
let v = *dumps as u32;
dumps = dumps.add(1);
v
} else {
0
};
if add < 255 {
lit_length += add as usize;
} else if (dumps as usize) <= (de as usize).wrapping_sub(3) {
lit_length = zstd_read_le24(dumps) as usize;
dumps = dumps.add(3);
}
}
/* Offset */
let mut offset: usize;
{
let offset_code =
fse_decode_symbol(&mut seq_state.state_offb, &mut seq_state.d_stream) as u32;
if IS_32BITS {
fse_reload_dstream(&mut seq_state.d_stream);
}
let mut nb_bits = offset_code.wrapping_sub(1);
if offset_code == 0 {
nb_bits = 0; /* cmove */
}
offset = (1usize << (nb_bits & (USIZE_BITS - 1)))
.wrapping_add(fse_read_bits(&mut seq_state.d_stream, nb_bits));
if IS_32BITS {
fse_reload_dstream(&mut seq_state.d_stream);
}
if offset_code == 0 {
offset = prev_offset;
}
}
/* MatchLength */
let mut match_length =
fse_decode_symbol(&mut seq_state.state_ml, &mut seq_state.d_stream) as usize;
if match_length == MAX_ML as usize {
let add = if (dumps as usize) < (de as usize) {
let v = *dumps as u32;
dumps = dumps.add(1);
v
} else {
0
};
if add < 255 {
match_length += add as usize;
} else if (dumps as usize) <= (de as usize).wrapping_sub(3) {
match_length = zstd_read_le24(dumps) as usize;
dumps = dumps.add(3);
}
}
match_length += MINMATCH;
/* save result */
seq.lit_length = lit_length;
seq.offset = offset;
seq.match_length = match_length;
seq_state.dumps = dumps;
}
unsafe fn zstd_exec_sequence(
op: *mut u8,
sequence: Seq,
lit_ptr: &mut *const u8,
lit_limit: *const u8,
base: *const u8,
oend: *mut u8,
) -> usize {
static DEC32TABLE: [usize; 8] = [0, 1, 2, 1, 4, 4, 4, 4]; /* added */
static DEC64TABLE: [usize; 8] = [8, 8, 8, 7, 8, 9, 10, 11]; /* subtracted */
let ostart = op;
let mut op = op;
let o_lit_end = op.wrapping_add(sequence.lit_length);
let lit_length = sequence.lit_length;
/* risk : address space overflow (32-bits) */
let end_match = op
.wrapping_add(lit_length)
.wrapping_add(sequence.match_length);
let lit_end = (*lit_ptr).wrapping_add(lit_length);
/* checks */
let seq_length = sequence.lit_length.wrapping_add(sequence.match_length);
if seq_length > (oend as usize).wrapping_sub(op as usize) {
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
if sequence.lit_length > (lit_limit as usize).wrapping_sub(*lit_ptr as usize) {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
/* Now we know there are no overflow in literal nor match lengths, can use pointer checks */
if sequence.offset > ((o_lit_end as usize).wrapping_sub(base as usize)) as u32 as usize {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
if (end_match as usize) > (oend as usize) {
/* overwrite beyond dst buffer */
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
if (lit_end as usize) > (lit_limit as usize) {
/* overRead beyond lit buffer */
return ERROR(ZstdErrorCode::CorruptionDetected);
}
if sequence.match_length > (*lit_ptr as usize).wrapping_sub(op as usize) {
/* overwrite literal segment */
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
/* copy Literals */
/* note : v0.1 seems to allow scenarios where output or input are close to end of buffer */
ptr::copy(*lit_ptr, op, sequence.lit_length);
op = op.add(lit_length);
*lit_ptr = lit_end; /* update for next sequence */
/* check : last match must be at a minimum distance of 8 from end of dest buffer */
if (oend as usize) - (op as usize) < 8 {
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
/* copy Match */
{
let overlap_risk = (lit_end as usize).wrapping_sub(end_match as usize) < 12;
/* possible underflow at op - offset ? */
let mut match_ptr = (op as usize).wrapping_sub(sequence.offset) as *const u8;
let mut qutt: usize = 12;
let mut saved = [0u8; 16]; /* U64 saved[2] */
/* check */
if (match_ptr as usize) < (base as usize) {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
if sequence.offset > base as usize {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
/* save beginning of literal sequence, in case of write overlap */
if overlap_risk {
if (end_match as usize).wrapping_add(qutt) > (oend as usize) {
qutt = (oend as usize) - (end_match as usize);
}
ptr::copy_nonoverlapping(end_match as *const u8, saved.as_mut_ptr(), qutt);
}
if sequence.offset < 8 {
let dec64 = DEC64TABLE[sequence.offset];
*op = *match_ptr;
*op.add(1) = *match_ptr.add(1);
*op.add(2) = *match_ptr.add(2);
*op.add(3) = *match_ptr.add(3);
match_ptr = match_ptr.add(DEC32TABLE[sequence.offset]);
zstd_copy4(op.add(4), match_ptr);
match_ptr = match_ptr.sub(dec64);
} else {
zstd_copy8(op, match_ptr);
}
op = op.add(8);
match_ptr = match_ptr.add(8);
if (end_match as usize) > (oend as usize).wrapping_sub(16 - MINMATCH) {
if (op as usize) < (oend as usize).wrapping_sub(8) {
let dist = ((oend as usize) - 8) - (op as usize);
zstd_wildcopy(op, match_ptr, dist as isize);
match_ptr = match_ptr.add(dist);
op = oend.sub(8);
}
while (op as usize) < (end_match as usize) {
*op = *match_ptr;
op = op.add(1);
match_ptr = match_ptr.add(1);
}
} else {
/* works even if matchLength < 8 */
zstd_wildcopy(op, match_ptr, sequence.match_length as isize - 8);
}
/* restore, in case of overlap */
if overlap_risk {
ptr::copy_nonoverlapping(saved.as_ptr(), end_match, qutt);
}
}
(end_match as usize) - (ostart as usize)
}
unsafe fn zstd_decompress_sequences(
ctx: *mut ZSTDv01_Dctx,
dst: *mut u8,
max_dst_size: usize,
seq_start: *const u8,
seq_size: usize,
lit_start: *const u8,
lit_size: usize,
) -> usize {
let dctx = ctx;
let mut ip = seq_start;
let iend_addr = (ip as usize).wrapping_add(seq_size);
let ostart = dst;
let mut op = ostart;
let oend = ostart.add(max_dst_size);
let mut lit_ptr = lit_start;
let lit_end = lit_start.add(lit_size);
let mut nb_seq: i32 = 0;
let mut dumps: *const u8 = ptr::null();
let mut dumps_length: usize = 0;
let dtable_ll = (*dctx).ll_table.as_mut_ptr();
let dtable_ml = (*dctx).ml_table.as_mut_ptr();
let dtable_offb = (*dctx).off_table.as_mut_ptr();
let base = (*dctx).base;
/* Build Decoding Tables */
let error_code = zstdv01_decode_seq_headers(
&mut nb_seq,
&mut dumps,
&mut dumps_length,
dtable_ll,
dtable_ml,
dtable_offb,
ip,
iend_addr - ip as usize,
);
if ERR_isError(error_code) {
return error_code;
}
ip = ip.add(error_code);
/* Regen sequences */
{
let mut sequence = Seq {
lit_length: 0,
offset: 0,
match_length: 0,
};
let mut seq_state = SeqState {
d_stream: FseDStream {
bit_container: 0,
bits_consumed: 0,
ptr: ptr::null(),
start: ptr::null(),
},
state_ll: FseDState {
state: 0,
table: ptr::null(),
},
state_offb: FseDState {
state: 0,
table: ptr::null(),
},
state_ml: FseDState {
state: 0,
table: ptr::null(),
},
prev_offset: 1,
dumps,
dumps_end: dumps.wrapping_add(dumps_length),
};
let error_code = fse_init_dstream(&mut seq_state.d_stream, ip, iend_addr - ip as usize);
if fse_is_error(error_code) {
return ERROR(ZstdErrorCode::CorruptionDetected);
}
fse_init_dstate(&mut seq_state.state_ll, &mut seq_state.d_stream, dtable_ll);
fse_init_dstate(
&mut seq_state.state_offb,
&mut seq_state.d_stream,
dtable_offb,
);
fse_init_dstate(&mut seq_state.state_ml, &mut seq_state.d_stream, dtable_ml);
while fse_reload_dstream(&mut seq_state.d_stream) <= FSE_DSTREAM_COMPLETED && nb_seq > 0 {
nb_seq -= 1;
zstd_decode_sequence(&mut sequence, &mut seq_state);
let one_seq_size = zstd_exec_sequence(op, sequence, &mut lit_ptr, lit_end, base, oend);
if ERR_isError(one_seq_size) {
return one_seq_size;
}
op = op.add(one_seq_size);
}
/* check if reached exact end */
if !fse_end_of_dstream(&seq_state.d_stream) {
/* requested too much : data is corrupted */
return ERROR(ZstdErrorCode::CorruptionDetected);
}
if nb_seq < 0 {
/* requested too many sequences : data is corrupted */
return ERROR(ZstdErrorCode::CorruptionDetected);
}
/* last literal segment */
{
let last_ll_size = (lit_end as usize) - (lit_ptr as usize);
if (op as usize).wrapping_add(last_ll_size) > (oend as usize) {
return ERROR(ZstdErrorCode::DstSizeTooSmall);
}
if last_ll_size > 0 {
if !std::ptr::eq(op as *const u8, lit_ptr) {
ptr::copy(lit_ptr, op, last_ll_size);
}
op = op.add(last_ll_size);
}
}
}
(op as usize) - (ostart as usize)
}
unsafe fn zstd_decompress_block(
ctx: *mut ZSTDv01_Dctx,
dst: *mut u8,
max_dst_size: usize,
src: *const u8,
src_size: usize,
) -> usize {
/* blockType == blockCompressed, srcSize is trusted */
let mut ip = src;
let mut lit_ptr: *const u8 = ptr::null();
let mut lit_size: usize = 0;
/* Decode literals sub-block */
let error_code = zstdv01_decode_literals_block(
dst,
max_dst_size,
&mut lit_ptr,
&mut lit_size,
src,
src_size,
);
if ERR_isError(error_code) {
return error_code;
}
ip = ip.add(error_code);
let src_size = src_size - error_code;
zstd_decompress_sequences(ctx, dst, max_dst_size, ip, src_size, lit_ptr, lit_size)
}
unsafe fn zstdv01_decompress_dctx(
ctx: *mut ZSTDv01_Dctx,
dst: *mut u8,
max_dst_size: usize,
src: *const u8,
src_size: usize,
) -> usize {
let mut ip = src;
let iend_addr = (ip as usize).wrapping_add(src_size);
let ostart = dst;
let mut op = ostart;
let oend_addr = (ostart as usize).wrapping_add(max_dst_size);
let mut remaining_size = src_size;
let mut error_code: usize = 0;
/* Frame Header */
if src_size < ZSTD_FRAME_HEADER_SIZE + ZSTD_BLOCK_HEADER_SIZE {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
let magic_number = zstd_read_be32(src);
if magic_number != ZSTD_MAGIC_NUMBER {
return ERROR(ZstdErrorCode::PrefixUnknown);
}
ip = ip.add(ZSTD_FRAME_HEADER_SIZE);
remaining_size -= ZSTD_FRAME_HEADER_SIZE;
/* Loop on each block */
loop {
let mut block_properties = BlockProperties {
block_type: 0,
orig_size: 0,
};
let block_size =
zstdv01_getc_block_size(ip, iend_addr - ip as usize, &mut block_properties);
if ERR_isError(block_size) {
return block_size;
}
ip = ip.add(ZSTD_BLOCK_HEADER_SIZE);
remaining_size -= ZSTD_BLOCK_HEADER_SIZE;
if block_size > remaining_size {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
match block_properties.block_type {
BT_COMPRESSED => {
error_code =
zstd_decompress_block(ctx, op, oend_addr - op as usize, ip, block_size);
}
BT_RAW => {
error_code =
zstd_copy_uncompressed_block(op, oend_addr - op as usize, ip, block_size);
}
BT_RLE => {
return ERROR(ZstdErrorCode::Generic); /* not yet supported */
}
BT_END => {
/* end of frame */
if remaining_size != 0 {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
}
_ => {
return ERROR(ZstdErrorCode::Generic);
}
}
if block_size == 0 {
break; /* bt_end */
}
if ERR_isError(error_code) {
return error_code;
}
op = op.add(error_code);
ip = ip.add(block_size);
remaining_size -= block_size;
}
(op as usize) - (ostart as usize)
}
/* ZSTD_errorFrameSizeInfoLegacy() :
assumes `cSize` and `dBound` are _not_ NULL */
unsafe fn zstd_error_frame_size_info_legacy(c_size: *mut usize, d_bound: *mut u64, ret: usize) {
*c_size = ret;
*d_bound = ZSTD_CONTENTSIZE_ERROR;
}
/* ******************************************
* Exported C ABI (zstd_v01.h)
********************************************/
/// C ABI: `ZSTDv01_isError`.
#[no_mangle]
pub extern "C" fn ZSTDv01_isError(code: usize) -> c_uint {
ERR_isError(code) as c_uint
}
/// C ABI: `ZSTDv01_decompressDCtx`.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_decompressDCtx(
ctx: *mut c_void,
dst: *mut c_void,
max_original_size: usize,
src: *const c_void,
compressed_size: usize,
) -> usize {
zstdv01_decompress_dctx(
ctx as *mut ZSTDv01_Dctx,
dst as *mut u8,
max_original_size,
src as *const u8,
compressed_size,
)
}
/// C ABI: `ZSTDv01_decompress`.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_decompress(
dst: *mut c_void,
max_original_size: usize,
src: *const c_void,
compressed_size: usize,
) -> usize {
/* The C version uses an uninitialized on-stack dctx_t; only `base` is
* read before being written, so a zeroed context is equivalent. */
let mut ctx = std::mem::MaybeUninit::<ZSTDv01_Dctx>::zeroed();
let ctx_ptr = ctx.as_mut_ptr();
(*ctx_ptr).base = dst as *const u8;
zstdv01_decompress_dctx(
ctx_ptr,
dst as *mut u8,
max_original_size,
src as *const u8,
compressed_size,
)
}
/// C ABI: `ZSTDv01_findFrameSizeInfoLegacy`.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_findFrameSizeInfoLegacy(
src: *const c_void,
src_size: usize,
c_size: *mut usize,
d_bound: *mut u64,
) {
let mut ip = src as *const u8;
let mut remaining_size = src_size;
let mut nb_blocks: usize = 0;
/* Frame Header */
if src_size < ZSTD_FRAME_HEADER_SIZE + ZSTD_BLOCK_HEADER_SIZE {
zstd_error_frame_size_info_legacy(c_size, d_bound, ERROR(ZstdErrorCode::SrcSizeWrong));
return;
}
let magic_number = zstd_read_be32(src as *const u8);
if magic_number != ZSTD_MAGIC_NUMBER {
zstd_error_frame_size_info_legacy(c_size, d_bound, ERROR(ZstdErrorCode::PrefixUnknown));
return;
}
ip = ip.add(ZSTD_FRAME_HEADER_SIZE);
remaining_size -= ZSTD_FRAME_HEADER_SIZE;
/* Loop on each block */
loop {
let mut block_properties = BlockProperties {
block_type: 0,
orig_size: 0,
};
let block_size = zstdv01_getc_block_size(ip, remaining_size, &mut block_properties);
if ERR_isError(block_size) {
zstd_error_frame_size_info_legacy(c_size, d_bound, block_size);
return;
}
ip = ip.add(ZSTD_BLOCK_HEADER_SIZE);
remaining_size -= ZSTD_BLOCK_HEADER_SIZE;
if block_size > remaining_size {
zstd_error_frame_size_info_legacy(c_size, d_bound, ERROR(ZstdErrorCode::SrcSizeWrong));
return;
}
if block_size == 0 {
break; /* bt_end */
}
ip = ip.add(block_size);
remaining_size -= block_size;
nb_blocks += 1;
}
*c_size = (ip as usize) - (src as usize);
*d_bound = (nb_blocks * BLOCKSIZE) as u64;
}
/* ******************************************
* Streaming Decompression API
********************************************/
/// C ABI: `ZSTDv01_resetDCtx`.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_resetDCtx(dctx: *mut ZSTDv01_Dctx) -> usize {
(*dctx).expected = ZSTD_FRAME_HEADER_SIZE;
(*dctx).phase = 0;
(*dctx).previous_dst_end = ptr::null();
(*dctx).base = ptr::null();
0
}
/// C ABI: `ZSTDv01_createDCtx`. Allocated with `malloc` exactly like the C
/// implementation, so create/free may be paired across the C/Rust boundary.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_createDCtx() -> *mut ZSTDv01_Dctx {
let dctx = libc::malloc(std::mem::size_of::<ZSTDv01_Dctx>()) as *mut ZSTDv01_Dctx;
if dctx.is_null() {
return ptr::null_mut();
}
ZSTDv01_resetDCtx(dctx);
dctx
}
/// C ABI: `ZSTDv01_freeDCtx`.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_freeDCtx(dctx: *mut ZSTDv01_Dctx) -> usize {
libc::free(dctx as *mut c_void);
0
}
/// C ABI: `ZSTDv01_nextSrcSizeToDecompress`.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_nextSrcSizeToDecompress(dctx: *mut ZSTDv01_Dctx) -> usize {
(*dctx).expected
}
/// C ABI: `ZSTDv01_decompressContinue`.
#[no_mangle]
pub unsafe extern "C" fn ZSTDv01_decompressContinue(
dctx: *mut ZSTDv01_Dctx,
dst: *mut c_void,
max_dst_size: usize,
src: *const c_void,
src_size: usize,
) -> usize {
let ctx = dctx;
let src = src as *const u8;
let dst = dst as *mut u8;
/* Sanity check */
if src_size != (*ctx).expected {
return ERROR(ZstdErrorCode::SrcSizeWrong);
}
if !std::ptr::eq(dst as *const u8, (*ctx).previous_dst_end) {
/* not contiguous */
(*ctx).base = dst as *const u8;
}
/* Decompress : frame header */
if (*ctx).phase == 0 {
/* Check frame magic header */
let magic_number = zstd_read_be32(src);
if magic_number != ZSTD_MAGIC_NUMBER {
return ERROR(ZstdErrorCode::PrefixUnknown);
}
(*ctx).phase = 1;
(*ctx).expected = ZSTD_BLOCK_HEADER_SIZE;
return 0;
}
/* Decompress : block header */
if (*ctx).phase == 1 {
let mut bp = BlockProperties {
block_type: 0,
orig_size: 0,
};
let block_size = zstdv01_getc_block_size(src, ZSTD_BLOCK_HEADER_SIZE, &mut bp);
if ERR_isError(block_size) {
return block_size;
}
if bp.block_type == BT_END {
(*ctx).expected = 0;
(*ctx).phase = 0;
} else {
(*ctx).expected = block_size;
(*ctx).b_type = bp.block_type;
(*ctx).phase = 2;
}
return 0;
}
/* Decompress : block content */
{
let r_size = match (*ctx).b_type {
BT_COMPRESSED => zstd_decompress_block(ctx, dst, max_dst_size, src, src_size),
BT_RAW => zstd_copy_uncompressed_block(dst, max_dst_size, src, src_size),
BT_RLE => {
return ERROR(ZstdErrorCode::Generic); /* not yet handled */
}
BT_END => {
/* should never happen (filtered at phase 1) */
0
}
_ => {
return ERROR(ZstdErrorCode::Generic);
}
};
(*ctx).phase = 1;
(*ctx).expected = ZSTD_BLOCK_HEADER_SIZE;
if ERR_isError(r_size) {
return r_size;
}
(*ctx).previous_dst_end = (dst as usize + r_size) as *const u8;
r_size
}
}
#[cfg(test)]
mod tests {
use super::*;
/// 237-byte base text; the fixtures compress repetitions of it.
const SAMPLE: &str = "snowden is snowed in / he's now then in his snow den / when does the snow end?\ngoodbye little dog / you dug some holes in your day / they'll be hard to fill.\nwhen life shuts a door, / just open it. it's a door. / that is how doors work.\n";
/// `SAMPLE` repeated 3 times (711 bytes), compressed by a zstd binary
/// built from the v0.1.0 tag. Single compressed block (FSE + Huff0),
/// verified byte-identical against the pristine C decoder.
const FRAME_ENTROPY: &[u8] = &[
0xFD, 0x2F, 0xB5, 0x1E, 0x00, 0x00, 0xB0, 0x00, 0x00, 0x93, 0x00, 0xD6, 0x19, 0x80, 0x87,
0xB1, 0x00, 0xC0, 0x73, 0xC5, 0x8A, 0xA5, 0x6C, 0x6F, 0x4B, 0xF2, 0x9F, 0x94, 0xC4, 0x2B,
0xE4, 0x6B, 0x2C, 0x96, 0xAE, 0x5F, 0xC8, 0x9C, 0x1F, 0x00, 0x1C, 0x00, 0x1B, 0x00, 0x43,
0x31, 0x70, 0xE4, 0xC9, 0xA3, 0x46, 0xD9, 0xAD, 0xFD, 0x78, 0x37, 0x66, 0xA6, 0x4D, 0x15,
0xCA, 0x33, 0xF5, 0xA7, 0x31, 0x83, 0x02, 0x96, 0x7E, 0xD0, 0x5E, 0xAC, 0xE9, 0x48, 0x03,
0xC7, 0xEF, 0x44, 0x81, 0x0B, 0x00, 0x90, 0xEA, 0x55, 0x41, 0xD8, 0x4E, 0x36, 0x10, 0xAC,
0x95, 0x92, 0x42, 0x5D, 0x92, 0xF4, 0x64, 0x7F, 0x96, 0x44, 0x59, 0x0B, 0x03, 0x4A, 0xF6,
0x16, 0x9F, 0xC8, 0xB9, 0x11, 0x70, 0xB8, 0x35, 0x06, 0xC4, 0x26, 0xC7, 0x6A, 0x89, 0xE4,
0x6F, 0x65, 0xC0, 0x9B, 0x55, 0x08, 0xEB, 0xE8, 0x37, 0x66, 0x67, 0x9E, 0x7D, 0x57, 0xA0,
0x25, 0x0D, 0xA4, 0x3E, 0x26, 0x8F, 0x32, 0xF6, 0xC9, 0x49, 0x50, 0x40, 0x0A, 0xED, 0x34,
0x94, 0x15, 0x5E, 0xE3, 0x1B, 0xD6, 0x27, 0x05, 0x00, 0x54, 0x05, 0x41, 0xFF, 0xD6, 0x01,
0x00, 0x80, 0x36, 0x00, 0xFE, 0x8D, 0x38, 0x42, 0x51, 0x7E, 0x40, 0x38, 0xD5, 0x60, 0x46,
0x03, 0x90, 0x25, 0xC0, 0x00, 0x00,
];
/// 48 random bytes compressed by v0.1.0: a raw (uncompressed) block.
const FRAME_RAW: &[u8] = &[
0xFD, 0x2F, 0xB5, 0x1E, 0x40, 0x00, 0x30, 0x77, 0xE9, 0x9A, 0x70, 0x2F, 0x77, 0x16, 0x9A,
0x92, 0xD4, 0xEE, 0x72, 0xFD, 0xDD, 0x0F, 0x39, 0x33, 0x8B, 0x8B, 0x3C, 0x6A, 0x91, 0xB2,
0x2B, 0x3A, 0xF0, 0x0E, 0xA6, 0x4C, 0x51, 0xA0, 0xC9, 0x5F, 0x69, 0x1A, 0xCE, 0x93, 0x5C,
0x7F, 0x43, 0x0C, 0xD9, 0x25, 0xBC, 0x91, 0xDD, 0x6E, 0xD7, 0xC0, 0x00, 0x00,
];
const RAW_PAYLOAD: &[u8] = &[
0x77, 0xE9, 0x9A, 0x70, 0x2F, 0x77, 0x16, 0x9A, 0x92, 0xD4, 0xEE, 0x72, 0xFD, 0xDD, 0x0F,
0x39, 0x33, 0x8B, 0x8B, 0x3C, 0x6A, 0x91, 0xB2, 0x2B, 0x3A, 0xF0, 0x0E, 0xA6, 0x4C, 0x51,
0xA0, 0xC9, 0x5F, 0x69, 0x1A, 0xCE, 0x93, 0x5C, 0x7F, 0x43, 0x0C, 0xD9, 0x25, 0xBC, 0x91,
0xDD, 0x6E, 0xD7,
];
/// `SAMPLE` repeated 1800 times (426600 bytes) compressed by v0.1.0:
/// four data blocks, so it exercises multi-block frames and repeated
/// offsets.
const FRAME_MULTI_BLOCK: &[u8] = &[
0xFD, 0x2F, 0xB5, 0x1E, 0x00, 0x00, 0xB0, 0x00, 0x00, 0x93, 0x00, 0xD6, 0x19, 0x80, 0x87,
0xB1, 0x00, 0xC0, 0x73, 0xC5, 0x8A, 0xA5, 0x6C, 0x6F, 0x4B, 0xF2, 0x9F, 0x94, 0xC4, 0x2B,
0xE4, 0x6B, 0x2C, 0x96, 0xAE, 0x5F, 0xC8, 0x9C, 0x1F, 0x00, 0x1C, 0x00, 0x1B, 0x00, 0x43,
0x31, 0x70, 0xE4, 0xC9, 0xA3, 0x46, 0xD9, 0xAD, 0xFD, 0x78, 0x37, 0x66, 0xA6, 0x4D, 0x15,
0xCA, 0x33, 0xF5, 0xA7, 0x31, 0x83, 0x02, 0x96, 0x7E, 0xD0, 0x5E, 0xAC, 0xE9, 0x48, 0x03,
0xC7, 0xEF, 0x44, 0x81, 0x0B, 0x00, 0x90, 0xEA, 0x55, 0x41, 0xD8, 0x4E, 0x36, 0x10, 0xAC,
0x95, 0x92, 0x42, 0x5D, 0x92, 0xF4, 0x64, 0x7F, 0x96, 0x44, 0x59, 0x0B, 0x03, 0x4A, 0xF6,
0x16, 0x9F, 0xC8, 0xB9, 0x11, 0x70, 0xB8, 0x35, 0x06, 0xC4, 0x26, 0xC7, 0x6A, 0x89, 0xE4,
0x6F, 0x65, 0xC0, 0x9B, 0x55, 0x08, 0xEB, 0xE8, 0x37, 0x66, 0x67, 0x9E, 0x7D, 0x57, 0xA0,
0x25, 0x0D, 0xA4, 0x3E, 0x26, 0x8F, 0x32, 0xF6, 0xC9, 0x49, 0x50, 0x40, 0x0A, 0xED, 0x34,
0x94, 0x15, 0x5E, 0xE3, 0x1B, 0xD6, 0x27, 0x05, 0x00, 0x54, 0x05, 0x41, 0xFF, 0x0F, 0xFF,
0x01, 0x80, 0x36, 0x00, 0xFE, 0x8D, 0x38, 0x42, 0x51, 0x7E, 0x40, 0x38, 0xD5, 0x60, 0x46,
0x03, 0x90, 0x25, 0x00, 0x00, 0x12, 0x40, 0x00, 0x00, 0x01, 0x00, 0x54, 0x04, 0xFF, 0xFC,
0xFF, 0x01, 0x80, 0xFA, 0x7F, 0x00, 0xFC, 0x23, 0x10, 0x00, 0x00, 0x12, 0x40, 0x00, 0x00,
0x01, 0x00, 0x54, 0x04, 0xFF, 0xFC, 0xFF, 0x01, 0x00, 0xF5, 0xFF, 0x00, 0xF8, 0x4B, 0x20,
0x00, 0x00, 0x12, 0x40, 0x00, 0x00, 0x01, 0x00, 0x54, 0x04, 0xFF, 0x64, 0x82, 0x00, 0x80,
0xEF, 0xFF, 0x00, 0xF0, 0x9F, 0x40, 0xC0, 0x00, 0x00,
];
fn decompress(frame: &[u8], capacity: usize) -> Result<Vec<u8>, usize> {
let mut out = vec![0u8; capacity];
let code = unsafe {
ZSTDv01_decompress(
out.as_mut_ptr() as *mut c_void,
capacity,
frame.as_ptr() as *const c_void,
frame.len(),
)
};
if ZSTDv01_isError(code) != 0 {
return Err(code);
}
out.truncate(code);
Ok(out)
}
fn frame_size_info(frame: &[u8]) -> (usize, u64) {
let mut c_size = 0usize;
let mut d_bound = 0u64;
unsafe {
ZSTDv01_findFrameSizeInfoLegacy(
frame.as_ptr() as *const c_void,
frame.len(),
&mut c_size,
&mut d_bound,
);
}
(c_size, d_bound)
}
#[test]
fn decodes_entropy_frame_byte_identically() {
let expected = SAMPLE.repeat(3).into_bytes();
let out = decompress(FRAME_ENTROPY, expected.len()).unwrap();
assert_eq!(out, expected);
}
#[test]
fn decodes_raw_block_frame_byte_identically() {
let out = decompress(FRAME_RAW, RAW_PAYLOAD.len()).unwrap();
assert_eq!(out, RAW_PAYLOAD);
}
#[test]
fn decodes_multi_block_frame_byte_identically() {
let expected = SAMPLE.repeat(1800).into_bytes();
assert_eq!(expected.len(), 426600);
let out = decompress(FRAME_MULTI_BLOCK, expected.len()).unwrap();
assert_eq!(out, expected);
}
#[test]
fn reports_frame_size_info_like_c() {
/* Values verified against the pristine C implementation. */
assert_eq!(
frame_size_info(FRAME_ENTROPY),
(FRAME_ENTROPY.len(), 131072)
);
assert_eq!(frame_size_info(FRAME_RAW), (FRAME_RAW.len(), 131072));
assert_eq!(
frame_size_info(FRAME_MULTI_BLOCK),
(FRAME_MULTI_BLOCK.len(), 524288)
);
}
#[test]
fn rejects_bad_magic_with_prefix_unknown() {
let mut frame = FRAME_ENTROPY.to_vec();
frame[0] ^= 0x55;
assert_eq!(
decompress(&frame, 1024).unwrap_err(),
ERROR(ZstdErrorCode::PrefixUnknown)
);
let (c_size, d_bound) = frame_size_info(&frame);
assert_eq!(c_size, ERROR(ZstdErrorCode::PrefixUnknown));
assert_eq!(d_bound, ZSTD_CONTENTSIZE_ERROR);
}
#[test]
fn rejects_truncated_frames_with_src_size_wrong() {
for frame in [FRAME_ENTROPY, FRAME_RAW, FRAME_MULTI_BLOCK] {
let truncated = &frame[..frame.len() - 1];
assert_eq!(
decompress(truncated, 1 << 20).unwrap_err(),
ERROR(ZstdErrorCode::SrcSizeWrong)
);
assert_eq!(
decompress(&frame[..5], 1 << 20).unwrap_err(),
ERROR(ZstdErrorCode::SrcSizeWrong)
);
}
}
#[test]
fn rejects_small_destination_with_dst_size_too_small() {
/* Error codes verified against the pristine C implementation. */
let expected_len = SAMPLE.len() * 3;
assert_eq!(
decompress(FRAME_ENTROPY, expected_len - 1).unwrap_err(),
ERROR(ZstdErrorCode::DstSizeTooSmall)
);
assert_eq!(
decompress(FRAME_ENTROPY, 0).unwrap_err(),
ERROR(ZstdErrorCode::DstSizeTooSmall)
);
assert_eq!(
decompress(FRAME_RAW, RAW_PAYLOAD.len() - 1).unwrap_err(),
ERROR(ZstdErrorCode::DstSizeTooSmall)
);
}
#[test]
fn streaming_api_decodes_the_frame() {
let expected = SAMPLE.repeat(3).into_bytes();
let mut out = vec![0u8; expected.len()];
let mut in_pos = 0usize;
let mut out_pos = 0usize;
unsafe {
let dctx = ZSTDv01_createDCtx();
assert!(!dctx.is_null());
loop {
let needed = ZSTDv01_nextSrcSizeToDecompress(dctx);
if needed == 0 {
break;
}
assert!(in_pos + needed <= FRAME_ENTROPY.len());
let produced = ZSTDv01_decompressContinue(
dctx,
out.as_mut_ptr().add(out_pos) as *mut c_void,
out.len() - out_pos,
FRAME_ENTROPY.as_ptr().add(in_pos) as *const c_void,
needed,
);
assert_eq!(ZSTDv01_isError(produced), 0);
in_pos += needed;
out_pos += produced;
}
assert_eq!(ZSTDv01_freeDCtx(dctx), 0);
}
assert_eq!(in_pos, FRAME_ENTROPY.len());
assert_eq!(out_pos, expected.len());
assert_eq!(out, expected);
}
#[test]
fn streaming_context_reset_and_null_free() {
unsafe {
let dctx = ZSTDv01_createDCtx();
assert_eq!(ZSTDv01_nextSrcSizeToDecompress(dctx), 4);
/* bad magic through the streaming entry point */
let bad = [0u8; 4];
assert_eq!(
ZSTDv01_decompressContinue(
dctx,
ptr::null_mut(),
0,
bad.as_ptr() as *const c_void,
4
),
ERROR(ZstdErrorCode::PrefixUnknown)
);
/* wrong srcSize */
assert_eq!(
ZSTDv01_decompressContinue(
dctx,
ptr::null_mut(),
0,
bad.as_ptr() as *const c_void,
3
),
ERROR(ZstdErrorCode::SrcSizeWrong)
);
assert_eq!(ZSTDv01_resetDCtx(dctx), 0);
assert_eq!(ZSTDv01_nextSrcSizeToDecompress(dctx), 4);
assert_eq!(ZSTDv01_freeDCtx(dctx), 0);
assert_eq!(ZSTDv01_freeDCtx(ptr::null_mut()), 0);
}
}
}