Files
wuffs/doc/std/compression-encoders.md
Claude 6568bc1b75 std/brotli: document compound dictionaries at qualities 10 and 11
Describe the Zopfli path's dictionary lookups, the extra quality 11
BackwardMatch storage in workbuf_len and drop the TODO items.
2026-10-27 09:08:15 +00:00

53 lines
2.5 KiB
Markdown

# Compression Encoders
Compression encoders read from one input stream (an `io_reader` called `src`)
and write to an output stream (an `io_writer` called `dst`). Like
[compression decoders](/doc/std/compression-decoders.md), they implement the
`base.io_transformer` interface, whose key method is `transform_io`. It
incrementally compresses the source data.
This method is a [coroutine](/doc/note/coroutines.md), and does not require
either all of the input or all of the output to fit in a single contiguous
buffer. It suspends with the `$short_read` status when it has consumed all of
`src` (and `src` is not closed) and with `$short_write` when `dst` is full. It
returns ok once `src` is closed and all of the compressed data has been
written. Encoder parameters (e.g. the compression quality) are set with
`set_quirk` before the first `transform_io` call.
## API Listing
In Wuffs syntax, the `base.io_transformer` methods are:
- `dst_history_retain_length() u64`
- `get_quirk(key: u32) u64`
- `set_quirk!(key: u32, value: u64) status`
- `transform_io?(dst: io_writer, src: io_reader, workbuf: slice u8)`
- `workbuf_len() range_ii_u64`
A workbuf of `workbuf_len().max_incl` bytes always suffices. Some encoders
(e.g. brotli at qualities 10 and 11) have a smaller `min_incl`, which is
enough for realistic inputs: with a workbuf shorter than `max_incl`, an
unusual input can make `transform_io` return `"#bad workbuf length"`.
Encoders may have further, format-specific methods. For example, the brotli
encoder has `request_flush`, `request_emit_metadata`, `max_compressed_size`
and, for raw (LZ77 prefix) dictionaries, `add_dictionary` and
`add_dictionary_workbuf_len`. Like quirks, dictionaries are added before the
first `transform_io` call.
## Implementations
- [std/brotli](/std/brotli/README.md#encoder) (qualities 0 to 11,
including large-window brotli, with windows of up to 1 GiB, at qualities 3
to 11). Its `workbuf_len` is the worst case for the parameters (more than
2 GiB for large-window brotli at lgwin 30; at qualities 10 and 11,
`min_incl` is a smaller practical bound), but a shorter workbuf works for
short inputs: see the README's "Encoder Workbuf Layout" section. Up to 15
raw dictionaries can be attached (like libbrotlienc's
`BrotliEncoderAttachPreparedDictionary`; like the C code, qualities 0
and 1 ignore them), see the README's "Encoder Dictionaries" section. They live at the start of the
workbuf, with their prepared hash tables, and `workbuf_len` grows
accordingly.