mirror of
https://github.com/ddidderr/wuffs.git
synced 2026-11-18 00:32:11 +01:00
This is the first milestone of a byte-exact rewrite of brotli (reference implementation tag v1.2.1) in Wuffs. The decoder mirrors c/dec/decode.c function by function, including its validation checks: the same output for valid input and the same verdict, with each BROTLI_DECODER_ERROR_FORMAT_* code mapped to a Wuffs status (see std/brotli/README.md). The C state machine is replaced by Wuffs coroutines, so decoding works with arbitrarily small src and dst buffers. The history window lives in the caller's workbuf. Huffman table groups, context maps and the static dictionary scratch space live in the decoder struct. The static dictionary, transforms, context lookup tables and prefix code tables in common_tables.wuffs are generated from the brotli source by script/print-brotli-tables.go. Tests: golden tests on the upstream tests/testdata vectors and on streams produced by the v1.2.1 CLI at several qualities and window sizes, 1-byte I/O limits, truncation at every prefix, malformed inputs per error code, trailing data and workbuf boundaries. With -mimic, every vector and every malformed input is also compared against libbrotlidec, including the exact error code. A differential run over about a million mutated inputs found no divergence. Not yet supported (left as TODOs): large-window brotli, custom/shared dictionaries, metadata callbacks and fast paths. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
68 lines
2.0 KiB
Markdown
68 lines
2.0 KiB
Markdown
# Compression Decoders
|
|
|
|
Compression decoders read from one input stream (an `io_reader` called `src`)
|
|
and write to an output stream (an `io_writer` called `dst`). Wuffs'
|
|
implementations have one key method: `transform_io`. It incrementally
|
|
decompresses the source data.
|
|
|
|
This method is a [coroutine](/doc/note/coroutines.md), and does not require
|
|
either all of the input or all of the output to fit in a single contiguous
|
|
buffer. It can suspend with the `$short_read` or `$short_write`
|
|
[statuses](/doc/note/statuses.md) when the `src` buffer needs re-filling or the
|
|
`dst` buffer needs flushing. For an example, look at the
|
|
[example/zcat](/example/zcat/zcat.c) program, which uses fixed size buffers,
|
|
but reads arbitrarily long compressed input from stdin and writes arbitrarily
|
|
long decompressed output to stdout.
|
|
|
|
|
|
## Dictionaries
|
|
|
|
TODO: standardize the [various dictionary
|
|
APIs](https://github.com/google/wuffs/issues/73), after Wuffs v0.2 is released.
|
|
|
|
|
|
## API Listing
|
|
|
|
In Wuffs syntax, the `base.io_transformer` methods are:
|
|
|
|
- `dst_history_retain_length() u64`
|
|
- `get_quirk(key: u32) u64`
|
|
- `set_quirk!(key: u32, value: u64) status`
|
|
- `transform_io?(dst: io_writer, src: io_reader, workbuf: slice u8)`
|
|
- `workbuf_len() range_ii_u64`
|
|
|
|
|
|
## Implementations
|
|
|
|
- [std/brotli](/std/brotli)
|
|
- [std/bzip2](/std/bzip2)
|
|
- [std/deflate](/std/deflate)
|
|
- [std/gzip](/std/gzip)
|
|
- [std/lzip](/std/lzip)
|
|
- [std/lzma](/std/lzma)
|
|
- [std/lzw](/std/lzw)
|
|
- [std/xz](/std/xz)
|
|
- [std/zlib](/std/zlib)
|
|
|
|
|
|
## Examples
|
|
|
|
- [example/mzcat](/example/mzcat)
|
|
- [example/toy-genlib](/example/toy-genlib)
|
|
- [example/zcat](/example/zcat)
|
|
|
|
|
|
## [Quirks](/doc/note/quirks.md)
|
|
|
|
- [LZMA decoder quirks](/std/lzma/decode_quirks.wuffs)
|
|
- [LZW decoder quirks](/std/lzw/decode_quirks.wuffs)
|
|
- [XZ decoder quirks](/std/xz/decode_quirks.wuffs)
|
|
- [Zlib decoder quirks](/std/zlib/decode_quirks.wuffs)
|
|
|
|
|
|
## Related Documentation
|
|
|
|
- [I/O (Input / Output)](/doc/note/io-input-output.md)
|
|
|
|
See also the general remarks on [Wuffs' standard library](/doc/std/README.md).
|