diff --git a/doc/std/compression-encoders.md b/doc/std/compression-encoders.md index f26433f4..7ebd9f0a 100644 --- a/doc/std/compression-encoders.md +++ b/doc/std/compression-encoders.md @@ -32,4 +32,4 @@ encoder has `request_flush`, `request_emit_metadata` and ## Implementations -- [std/brotli](/std/brotli/README.md#encoder) (qualities 0 and 1 only). +- [std/brotli](/std/brotli/README.md#encoder) (qualities 0 to 9 only). diff --git a/std/brotli/README.md b/std/brotli/README.md index db513a40..dc5cb81a 100644 --- a/std/brotli/README.md +++ b/std/brotli/README.md @@ -154,8 +154,8 @@ invariants for memory safety but Wuffs requires explicit bounds checks. - Metadata callbacks (metadata blocks are currently skipped). See the "Metadata Blocks" section below. - Fast paths. -- Encoder quality levels 2 to 11, encoder dictionaries and large-window - encoding (see the "Encoder" section below). +- Encoder quality levels 10 and 11, encoder dictionaries and large-window + encoding at qualities 3 and above (see the "Encoder" section below). ## Encoder @@ -169,11 +169,20 @@ the Wuffs code mirrors the C code function by function, with comments like point arithmetic (the C code's `double`s) uses `std/softfloat`, so that the results are bit-exact regardless of the compiler and CPU. -Only qualities 0 and 1 (the C code's `FAST_ONE_PASS_COMPRESSION_QUALITY` and -`FAST_TWO_PASS_COMPRESSION_QUALITY`, implemented by `compress_fragment.c` and -`compress_fragment_two_pass.c`) are implemented. For qualities 2 to 11 -(including the default quality, 11), `transform_io` returns -`"#unsupported encoder quality"`. +Qualities 0 to 9 are implemented: qualities 0 and 1 (the C code's +`FAST_ONE_PASS_COMPRESSION_QUALITY` and `FAST_TWO_PASS_COMPRESSION_QUALITY`, +implemented by `compress_fragment.c` and `compress_fragment_two_pass.c`) and +qualities 2 to 9 (the C code's `EncodeData` path: a ring buffer, hashers, +`BrotliCreateBackwardReferences` with static dictionary matching, and +`WriteMetaBlockInternal`, which picks `BrotliStoreMetaBlockFast` at quality +2, `BrotliStoreMetaBlockTrivial` at quality 3 and, at qualities 4 to 9, the +literal context modeling heuristics, `BrotliBuildMetaBlockGreedy`, +`BrotliOptimizeHistograms` and `BrotliStoreMetaBlock`). For qualities 10 and +11 (including the default quality, 11), and for large-window brotli at +qualities 3 and above, `transform_io` returns +`"#unsupported encoder quality"` (and `workbuf_len` is 0). Large-window +brotli works at quality 2 (and is ignored at qualities 0 and 1, like the C +code). The source files are: @@ -195,11 +204,41 @@ The source files are: - `encode_histogram.wuffs`: `histogram.h`, `histogram_inc.h` and `histogram.c` (histogram helpers and `BrotliBuildHistogramsWithContext`), also as `bit_cost` methods. +- `encode_data.wuffs`: the rest of `encode.c` that qualities 2 to 9 use + (`EncodeData`, `WriteMetaBlockInternal`, `ShouldCompress`, + `DecideOverLiteralContextModeling`, `ChooseContextMap`, `UpdateSizeHint`, + `ExtendLastCommand`, etc), `ringbuffer.h`, the `quality.h` helpers and the + workbuf layout. +- `encode_hasher.wuffs`: `hash.h` and the `hash_*_inc.h` hashers (H2, H3, H4, + H54, H5, H6, H58, H68, H40, H41 and H42) and `SearchInStaticDictionary` + (with `static_dict.c`'s matching), as one set of functions selected by the + hasher type at run time. +- `encode_backward_references.wuffs`: `backward_references.c` and + `backward_references_inc.h` (`BrotliCreateBackwardReferences`), + `command.h` and `ComputeDistanceCode`. +- `encode_bit_stream.wuffs`: `brotli_bit_stream.c` (`StoreMetaBlockHeader`, + `BrotliStoreUncompressedMetaBlock`, `BrotliStoreMetaBlockTrivial`, + `BrotliStoreMetaBlockFast`, `BrotliStoreMetaBlock`, block split codes, + `EncodeContextMap` with move-to-front and run-length coding, + `BuildAndStoreEntropyCodes`, etc). +- `encode_metablock_greedy.wuffs`: `BrotliBuildMetaBlockGreedy` (and its + greedy block splitters, `metablock_inc.h`) and `BrotliOptimizeHistograms` + for its output. +- `encode_util.wuffs`: little-endian workbuf accessors. +- `encode_block_splitter.wuffs`, `encode_cluster.wuffs` and + `encode_metablock.wuffs`: the `metablock_builder` struct, with + `block_splitter.c`, `cluster.c` and `BrotliBuildMetaBlock` (which only + quality 10 and above use, and so the encoder does not call it yet). The + encoder uses some of its smaller functions (`BrotliInitDistanceParams`, + `PrefixEncodeCopyDistance`, `CommandRestoreDistanceCode` and + `BrotliOptimizeHuffmanCountsForRle`). - `encode_tester.wuffs`: TEST ONLY. The `metablock_tester` struct exposes the internal functions above to C test programs. It is not part of the encoder API. - `common_encoder_tables.wuffs`: tables generated by `go run - script/print-brotli-tables.go -encoder` from the C code. + script/print-brotli-tables.go -encoder` from the C code (including the + static dictionary hash tables of `dictionary_hash.c` and the + `static_dict_lut.h` buckets and words). ### Encoder API @@ -247,7 +286,14 @@ The quality quirk is not named `QUIRK_QUALITY` because that name would clash with `base.QUIRK_QUALITY`. At qualities 0 and 1, only the quality, lgwin and stream offset affect the output (lgblock, mode, size hint, large window, npostfix, ndirect and literal context modeling are stored, sanitized and then -ignored, as in the C code). +ignored, as in the C code). At qualities 2 to 9, they have the same effect as +in the C code: lgblock (at qualities 4 and above; the C code's +`ComputeLgBlock` fixes it to 14 below that), npostfix, ndirect and the mode +(only `BROTLI_MODE_FONT` matters, through `ChooseDistanceParams`, at +qualities 4 and above), disabling literal context modeling (at qualities 5 +and above) and the size hint (the hasher choice at qualities 4 and above and +the complex static context map at qualities 5 and above). An unset (zero) +size hint is replaced by the C code's `UpdateSizeHint` estimate. ### Encoder Workbuf Layout @@ -257,15 +303,77 @@ At qualities 0 and 1, the workbuf is the C code's metablock storage (its (the storage for a `1 << lgwin` byte block plus 16 bytes of slack for 8-byte bit writes), where lgwin is the sanitized value. A shorter workbuf still works if every input block is shorter: a block of `n` bytes needs `(2 * n) + 519` -bytes, otherwise `transform_io` returns `"#bad workbuf length"`. At other -qualities, `workbuf_len` is 0. +bytes, otherwise `transform_io` returns `"#bad workbuf length"`. -The hash table (up to `1 << 17` entries), the quality 1 command and literal -buffers (`1 << 17` entries each) and the Huffman scratch space are arrays in -the `encoder` struct (which is therefore more than 1 MiB). They do not need -zero-initializing: `WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED` is -fine. Unlike the C code, the input block is not copied: the encoder compresses -directly from `src`. +At qualities 2 to 9, the workbuf holds everything whose size depends on the +input or on the parameters, in this order (see `compute_layout_offset` in +`encode_data.wuffs`). Here `rb_bits` is `1 + max(lgwin, lgblock)` (the C +code's `ComputeRbBits`), `max_len` is `1 << min(rb_bits, 24)` (the C code's +`MaxMetablockSize`) and `cmds_cap` is the commands capacity (see below): + +1. The metablock storage: `(2 * max_len) + 503 + 16` bytes, like at + qualities 0 and 1 but for the largest metablock. +2. The ring buffer (the C code's `RingBuffer.buffer_`): `2 + (1 << rb_bits) + + (1 << lgblock) + 7` bytes. The first 2 bytes are the copy of the last two + bytes that precede `data_`, the tail (`1 << lgblock` bytes) mirrors the + start of the buffer and the 7 trailing bytes are the C code's slack for + 8-byte loads. +3. The hasher's memory (the C code's `common->extra[0 .. 3]`): the sum of + `FN(HashMemAllocInBytes)` for the hasher that `ChooseHasher` picks. When + the size hint quirk is unset (zero), that choice depends on the input's + length (through `UpdateSizeHint`), so this is the larger of the two + possible choices (for a size hint below or at least 1 MiB). +4. The commands: `16 * cmds_cap` bytes, in the C struct order: `insert_len_` + (u32le), `copy_len_` (u32le, the copy length in the low 25 bits and + `copy_code - copy_len` in the high 7 bits), `dist_extra_` (u32le), + `cmd_prefix_` (u16le) and `dist_prefix_` (u16le). `cmds_cap` is + `min(prior + ((1 << lgblock) / 2) + 1, (max_len / 2) + 1)`, where `prior` + is `(max_len / 8) - 1` (and at most `MAX_NUM_DELAYED_SYMBOLS - 1`, which + is `0x2FFE`, below quality 4): `EncodeData` keeps (merges with the next + input block) fewer than that many commands and, like the C code's + `num_commands_ + bytes / 2 + 1` allocation, each input block of `n` bytes + adds at most `(n / 2) + 1` commands. +5. At qualities 4 and above, `BrotliBuildMetaBlockGreedy`'s `MetaBlockSplit` + and `BrotliStoreMetaBlock`'s scratch space: + - the block types (u8) and then the block lengths (u32le) of the literal, + command and distance block splits, with room for `(max_len / 512) + 1`, + `(cmds_cap / 1024) + 1` and `(cmds_cap / 512) + 1` blocks (the C + code's `max_num_blocks`, given that a metablock has at most `max_len` + literals and `cmds_cap` commands), + - the histograms (u32le counts, without the C code's `total_count_` and + `bit_cost_` fields, which are not needed here): up to 260 literal + histograms (`(256 / 13) + 1` block types of 13 contexts), 257 command + histograms and 257 distance histograms of 256, 704 and 544 counts, + - the block splitters' `combined_histo` (2 * 13 * 256 u32le counts), + - the literal context map (256 * 64 bytes), + - `EncodeContextMap`'s `rle_symbols` (256 * 64 u32le values) and + - the block encoders' depths (u8) and then bits (u16le) for up to 256 + histograms per category. + +Unlike the C code, which allocates its storage and command buffers as it +goes (for the actual metablock and block sizes), `workbuf_len` is the worst +case for the sanitized parameters, so that one workbuf works for any input. +At qualities 2 to 9, `transform_io` returns `"#bad workbuf length"` if the +workbuf is shorter than that. Some examples (in MiB, with the default +lgblock and an unset size hint): + +| quality | lgwin 10 | lgwin 16 | lgwin 18 | lgwin 22 | lgwin 24 | +| ------- | -------- | -------- | -------- | -------- | -------- | +| 0, 1 | 0.0 | 0.1 | 0.5 | 8.0 | 32.0 | +| 2, 3 | 0.5 | 1.0 | 2.1 | 24.6 | 64.6 | +| 4 | 7.9 | 7.9 | 9.8 | 47.3 | 103.4 | +| 5 | 4.4 | 4.4 | 7.0 | 45.9 | 102.0 | +| 7 | 4.4 | 4.4 | 13.8 | 51.4 | 107.5 | +| 9 | 5.1 | 5.1 | 39.5 | 77.1 | 133.2 | + +Fixed-size state (the hash table and command and literal buffers of +qualities 0 and 1, the Huffman scratch space, the greedy block splitters' +state, the block encoders' block type and length codes, etc) lives in the `encoder` struct (which is therefore +about 2.5 MiB). It does not need zero-initializing: +`WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED` is fine. At +qualities 0 and 1, the input block is not copied: the encoder compresses +directly from `src`. At qualities 2 to 9, like the C code, the input is +copied into the ring buffer. ### Encoder Operation Mapping @@ -304,23 +412,53 @@ on how the input is split into chunks. The test programs' encode drivers `mimic_brotli_encode` in `test/c/mimiclib/brotli.c`) run the same "plan" of (operation, length) steps with each library. +At qualities 2 to 9 the C code copies the input into its ring buffer and +only compresses it (`EncodeData`) when an input block (`1 << lgblock` bytes) +is full, or on a flush or finish, so the output depends on the chunking only +through the flushes (and the `UpdateSizeHint` estimate, which depends on how +much input is available when the first block is compressed). The Wuffs +encoder makes the same `EncodeData` calls, at the same points. One +`transform_io` call can correspond to several C calls: with a stream offset +(the C code's "flint"), a C call can return early, with input left, after +emitting the flint, and its callers then call it again. + ### Encoder Testing `wuffs test -mimic std/brotli` checks byte-exact parity with libbrotlienc -over a grid of qualities, lgwins, modes, input chunkings, flushes, metadata -blocks, size hints and other parameters, as well as round trips through the -decoder. `script/manual-test-brotli-encoder-parity.c` runs a larger grid -(about 86 thousand configurations over every file in `test/data/brotli` and -some synthetic inputs). +over a grid of qualities (0 to 9), lgwins, modes, input chunkings, flushes, +metadata blocks, size hints and other parameters (lgblock, npostfix, ndirect, +disabling literal context modeling, stream offset, large window at quality 2), +as well as round trips through the decoder. +`script/manual-test-brotli-encoder-parity.c` runs a larger grid over every +file in `test/data/brotli`, some larger files in `test/data` (up to about +400 KiB) and some synthetic inputs (up to 1 MiB). Its `-q=LO-HI` and +`-shard=K/N` options select the qualities and split the inputs over several +processes. For qualities 0 and 1 that is about 86 thousand configurations +and for qualities 2 to 9 several hundred thousand, all byte-exact. ### Encoder Differences from the C Code -- The input block is not copied (see the workbuf layout above), and output is - always staged in the workbuf (or a small internal buffer) and then copied to - `dst`. The C code sometimes writes directly to `next_out`. The bytes are the - same. +- At qualities 0 and 1, the input block is not copied (see the workbuf + layout above). Output is always staged in the workbuf (or a small internal + buffer) and then copied to `dst`. The C code sometimes writes directly to + `next_out`. The bytes are the same. +- The C code instantiates each hasher (and the block splitters) from + templates. Here, each template is one set of functions, parameterized at + run time (e.g. by the hasher type). The H58 and H68 hashers' SIMD tag + matching (`GetMatchingTagMask` in `matching_tag_mask.h`) is computed one + tag at a time, which finds the same candidates in the same order. Like the + x86-64 GCC reference build (where `BROTLI_MAX_SIMD_QUALITY` is 6, as in + any build with a 64-bit `__builtin_ctzll`), qualities 5 and 6 with lgwin + above 16 use H58 or H68 instead of H5 or H6. libbrotlienc builds where + `BROTLI_MAX_SIMD_QUALITY` is undefined (no 64-bit count-trailing-zeros + builtin) or 7 (MSVC on x86-64) choose differently and so produce + different (but valid) output at those qualities. +- `BrotliBuildMetaBlockGreedy`'s two kinds of block splitter (one for a + single histogram per block type, `metablock_inc.h`, and one for several + literal contexts, `ContextBlockSplitter` in `metablock.c`) are one + implementation here, as they make the same decisions with one context. - Wuffs' C code generator does not support `slice base.u16` or `slice base.u32` arguments (and mishandles `copy_from_slice` on `base.u16` arrays), so some functions (e.g. `create_huffman_tree`, @@ -332,9 +470,11 @@ some synthetic inputs). ### Encoder TODO -- Qualities 2 to 11 (the C code's `EncodeData`, hashers, block splitting, - context modeling, Zopfli, etc). -- Large-window encoding (only used at qualities 3 and above). +- Qualities 10 and 11 (Zopfli, the H10 binary tree hasher and + `BrotliBuildMetaBlock`, which is already ported in `encode_metablock.wuffs`). +- Large-window encoding at qualities 3 and above (it works at quality 2). +- `workbuf_len` is a worst case. A future option could take the input length + (like the C code's size hint) to give a tighter bound. - Custom and shared dictionaries (`BrotliEncoderAttachPreparedDictionary`). - Changing parameters mid-stream. diff --git a/std/brotli/encode_brotli.wuffs b/std/brotli/encode_brotli.wuffs index f283b173..9c299228 100644 --- a/std/brotli/encode_brotli.wuffs +++ b/std/brotli/encode_brotli.wuffs @@ -19,14 +19,15 @@ // The C code's floating point (double) arithmetic is done with the // std/softfloat package, which is bit-exact. FastLog2's log2 matches glibc's. // -// Only qualities 0 and 1 (the C code's BrotliCompressFragmentFast and -// BrotliCompressFragmentTwoPass) are implemented so far. See README.md for -// the API, including how the C code's FLUSH and EMIT_METADATA operations map -// to the request_flush and request_emit_metadata methods. +// Qualities 0 and 1 (the C code's BrotliCompressFragmentFast and +// BrotliCompressFragmentTwoPass) and qualities 2 to 9 (the C code's +// EncodeData, see encode_data.wuffs) are implemented so far. See README.md +// for the API, including how the C code's FLUSH and EMIT_METADATA operations +// map to the request_flush and request_emit_metadata methods. // -// TODO: qualities 2 to 11 (the C code's EncodeData, WriteMetaBlockInternal, -// hashers, block splitting, context modeling and Zopfli), large-window -// brotli at those qualities and custom (prepared or shared) dictionaries. +// TODO: qualities 10 and 11 (Zopfli and BrotliBuildMetaBlock), large-window +// brotli at qualities 3 and above and custom (prepared or shared) +// dictionaries. use "std/softfloat"