Merge claude/project-thread-0izcgq (large window at qualities 3 to 9)

Put the q10/q11 variable region before the ring buffer (now the last
region), growing it (and moving the ring buffer) at the first
transform_io call to use a workbuf longer than workbuf_len's minimum.
Large window now also works at qualities 10 and 11.
This commit is contained in:
Claude committed 2026-10-27 04:46:52 +00:00
commit 7d58caf91c
11 files changed
+1664 -216

No files matched your search

+154 -36
View File
@@ -154,8 +154,7 @@ invariants for memory safety but Wuffs requires explicit bounds checks.
- Metadata callbacks (metadata blocks are currently skipped). See the
"Metadata Blocks" section below.
- Fast paths.
- Encoder dictionaries and large-window encoding at qualities 3 and above
(see the "Encoder" section below).
- Encoder dictionaries (see the "Encoder" section below).
## Encoder
@@ -182,11 +181,9 @@ literal context modeling heuristics, `BrotliBuildMetaBlockGreedy`,
hasher, `BrotliCreateZopfliBackwardReferences` at quality 10 and
`BrotliCreateHqZopfliBackwardReferences` at quality 11, and then
`ChooseContextMode`, `BrotliBuildMetaBlock`, `BrotliOptimizeHistograms` and
`BrotliStoreMetaBlock`). For large-window brotli at qualities 3 and above,
`transform_io` returns `"#unsupported encoder quality"` (and `workbuf_len`
is 0). Large-window
brotli works at quality 2 (and is ignored at qualities 0 and 1, like the C
code).
`BrotliStoreMetaBlock`). Large-window brotli (see "Encoder Large Window"
below) works at qualities 3 to 11 with lgwin 10 to 30. Like the C code,
qualities 0 to 2 ignore the large-window quirk.
The source files are:
@@ -217,6 +214,10 @@ The source files are:
H54, H5, H6, H58, H68, H40, H41 and H42) and `SearchInStaticDictionary`
(with `static_dict.c`'s matching), as one set of functions selected by the
hasher type at run time.
- `encode_hasher_composite.wuffs`: the large-window hashers:
`hash_composite_inc.h` (H35, H55 and H65, which combine H3, H54 or H6 with
a rolling hasher) and `hash_rolling_inc.h` (`HROLLING_FAST` and
`HROLLING`).
- `encode_backward_references.wuffs`: `backward_references.c` and
`backward_references_inc.h` (`BrotliCreateBackwardReferences`),
`command.h` and `ComputeDistanceCode`.
@@ -306,7 +307,7 @@ first `transform_io` call, after which `set_quirk` returns
| `QUIRK_MODE` | `BROTLI_PARAM_MODE` |
| `QUIRK_DISABLE_LITERAL_CONTEXT_MODELING` | `BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING` (0 or 1) |
| `QUIRK_SIZE_HINT` | `BROTLI_PARAM_SIZE_HINT` |
| `QUIRK_LARGE_WINDOW` (shared with the decoder) | `BROTLI_PARAM_LARGE_WINDOW` |
| `QUIRK_LARGE_WINDOW` (shared with the decoder) | `BROTLI_PARAM_LARGE_WINDOW` (0 or 1) |
| `QUIRK_NPOSTFIX` | `BROTLI_PARAM_NPOSTFIX` |
| `QUIRK_NDIRECT` | `BROTLI_PARAM_NDIRECT` |
| `QUIRK_STREAM_OFFSET` | `BROTLI_PARAM_STREAM_OFFSET` (at most `1 << 30`) |
@@ -316,7 +317,8 @@ with `base.QUIRK_QUALITY`. At qualities 0 and 1, only the quality, lgwin and
stream offset affect the output (lgblock, mode, size hint, large window,
npostfix, ndirect and literal context modeling are stored, sanitized and then
ignored, as in the C code). At qualities 2 to 9, they have the same effect as
in the C code: lgblock (at qualities 4 and above; the C code's
in the C code: large window (see "Encoder Large Window" below), lgblock (at
qualities 4 and above; the C code's
`ComputeLgBlock` fixes it to 14 below that), npostfix, ndirect and the mode
(only `BROTLI_MODE_FONT` matters, through `ChooseDistanceParams`, at
qualities 4 and above), disabling literal context modeling (at qualities 5
@@ -342,17 +344,15 @@ code's `ComputeRbBits`), `max_len` is `1 << min(rb_bits, 24)` (the C code's
1. The metablock storage: `(2 * max_len) + 503 + 16` bytes, like at
qualities 0 and 1 but for the largest metablock.
2. The ring buffer (the C code's `RingBuffer.buffer_`): `2 + (1 << rb_bits) +
(1 << lgblock) + 7` bytes. The first 2 bytes are the copy of the last two
bytes that precede `data_`, the tail (`1 << lgblock` bytes) mirrors the
start of the buffer and the 7 trailing bytes are the C code's slack for
8-byte loads.
3. The hasher's memory (the C code's `common->extra[0 .. 3]`): the sum of
2. The hasher's memory (the C code's `common->extra[0 .. 3]`): the sum of
`FN(HashMemAllocInBytes)` for the hasher that `ChooseHasher` picks. When
the size hint quirk is unset (zero), that choice depends on the input's
length (through `UpdateSizeHint`), so this is the larger of the two
possible choices (for a size hint below or at least 1 MiB).
4. The commands: `16 * cmds_cap` bytes, in the C struct order: `insert_len_`
possible choices (for a size hint below or at least 1 MiB). With lgwin
above 24 (large window), the composite hashers H35, H55 and H65 add their
rolling hasher's table (`(1 << 24)` u32le values, 64 MiB) after H3's,
H54's or H6's memory.
3. The commands: `16 * cmds_cap` bytes, in the C struct order: `insert_len_`
(u32le), `copy_len_` (u32le, the copy length in the low 25 bits and
`copy_code - copy_len` in the high 7 bits), `dist_extra_` (u32le),
`cmd_prefix_` (u16le) and `dist_prefix_` (u16le). `cmds_cap` is
@@ -362,7 +362,7 @@ code's `ComputeRbBits`), `max_len` is `1 << min(rb_bits, 24)` (the C code's
input block) fewer than that many commands and, like the C code's
`num_commands_ + bytes / 2 + 1` allocation, each input block of `n` bytes
adds at most `(n / 2) + 1` commands.
5. At qualities 4 and above, `BrotliBuildMetaBlockGreedy`'s `MetaBlockSplit`
4. At qualities 4 and above, `BrotliBuildMetaBlockGreedy`'s `MetaBlockSplit`
and `BrotliStoreMetaBlock`'s scratch space:
- the block types (u8) and then the block lengths (u32le) of the literal,
command and distance block splits, with room for `(max_len / 512) + 1`,
@@ -378,13 +378,39 @@ code's `ComputeRbBits`), `max_len` is `1 << min(rb_bits, 24)` (the C code's
- `EncodeContextMap`'s `rle_symbols` (256 * 64 u32le values) and
- the block encoders' depths (u8) and then bits (u16le) for up to 256
histograms per category.
5. The ring buffer (the C code's `RingBuffer.buffer_`): `2 + (1 << rb_bits) +
(1 << lgblock) + 7` bytes. The first 2 bytes are the copy of the last two
bytes that precede `data_`, the tail (`1 << lgblock` bytes) mirrors the
start of the buffer and the 7 trailing bytes are the C code's slack for
8-byte loads.
Unlike the C code, which allocates its storage and command buffers as it
goes (for the actual metablock and block sizes), `workbuf_len` is the worst
case for the sanitized parameters, so that one workbuf works for any input.
At qualities 2 to 9, `transform_io` returns `"#bad workbuf length"` if the
workbuf is shorter than that. Some examples (in MiB, with the default
lgblock and an unset size hint):
Unlike the C code, which allocates its storage, ring buffer and command
buffers as it goes (for the actual metablock and block sizes, and the actual
input length), `workbuf_len` is the worst case for the sanitized parameters,
so that one workbuf works for any input. A shorter workbuf still works for
short inputs, because the ring buffer is the last region and the encoder
only touches its first `2 + n + 7` bytes for an input of `n` bytes, as long
as the ring buffer never wraps around. Precisely, at qualities 2 to 11
(where, at qualities 10 and 11, `workbuf_len` means its `min_incl`):
- `transform_io` returns `"#bad workbuf length"` if the workbuf is shorter
than `workbuf_len - R + 18`, where `R` is the ring buffer region's length
(`2 + (1 << rb_bits) + (1 << lgblock) + 7`), i.e. if it does not reach 16
bytes into the ring buffer's data.
- If any `transform_io` call gets a workbuf shorter than `workbuf_len`, then
from then on (for the rest of the stream), before copying input into the
ring buffer, `transform_io` returns `"#bad workbuf length"` (without
consuming that input) if the total input so far (not counting metadata)
would be `n` bytes with `n + 16 > (1 << rb_bits)` (the ring buffer would
wrap, or its tail would be needed) or with the workbuf shorter than
`workbuf_len - R + 2 + n + 16`.
So, for a total input of `n` bytes with `n + 16 <= (1 << rb_bits)`, a
workbuf of `workbuf_len - R + n + 18` bytes produces the same output as one
of `workbuf_len` bytes. The parity test checks this with exactly that length
(and that one byte less fails). This matters most for large-window brotli,
whose ring buffer is up to 2 GiB. Some examples of `workbuf_len` (in MiB,
with the default lgblock and an unset size hint):
| quality | lgwin 10 | lgwin 16 | lgwin 18 | lgwin 22 | lgwin 24 |
| ------- | -------- | -------- | -------- | -------- | -------- |
@@ -395,6 +421,21 @@ lgblock and an unset size hint):
| 7 | 4.4 | 4.4 | 13.8 | 51.4 | 107.5 |
| 9 | 5.1 | 5.1 | 39.5 | 77.1 | 133.2 |
With the large-window quirk (at qualities 3 and above; lgwin up to 24 gives
the same numbers as without it), larger lgwin values double the ring buffer
for each extra bit, and lgwin above 24 adds the composite hashers' 64 MiB
rolling hash table (at quality 4 and at qualities 5 to 9 with lgwin of at
least 19, only for a size hint of at least 1 MiB, or an unset one, as
`ChooseHasher` otherwise keeps H3, H5 or H58):
| quality | lgwin 25 | lgwin 26 | lgwin 28 | lgwin 30 |
| ------- | -------- | -------- | -------- | -------- |
| 3 | 160.6 | 224.6 | 608.6 | 2144.6 |
| 4 | 199.4 | 263.4 | 647.4 | 2183.4 |
| 5 | 197.5 | 261.5 | 645.5 | 2181.5 |
| 7 | 203.5 | 267.5 | 651.5 | 2187.5 |
| 9 | 229.2 | 293.2 | 677.2 | 2213.2 |
At qualities 10 and 11, `workbuf_len` is a range instead (see the next
section).
@@ -413,11 +454,19 @@ copied into the ring buffer.
At qualities 10 and 11, the workbuf has the same regions as at quality 9
(with `cmds_cap` computed the same way), except that:
- region 3 is the H10 hasher: `4 << 17` bytes of buckets (u32le) and then
the binary tree forest (`8 << lgwin` bytes, u32le), and
- region 5 keeps only the literal context map, `rle_symbols` and the block
encoders' depths and bits, followed by a variable region, which runs from
there to the end of the workbuf (however long that is).
- region 2 is the H10 hasher: `4 << 17` bytes of buckets (u32le) and then
the binary tree forest (`8 << lgwin` bytes, u32le),
- region 4 keeps only the literal context map, `rle_symbols` and the block
encoders' depths and bits, followed by a variable region, and
- the ring buffer (region 5) is still the last region, after the variable
region.
The variable region's length is set at the first `transform_io` call: it is
its minimum length (see below) plus however much of the workbuf beyond
`workbuf_len().min_incl` there is (up to the worst case), and the ring
buffer moves along with it. Later `transform_io` calls should pass a
workbuf of the same length. A workbuf shorter than `min_incl` follows the
short workbuf rules above (with the minimum variable region).
The variable region holds, at different times, the Zopfli scratch space
(while `EncodeData` computes the backward references: `(n + 1)` 16-byte
@@ -431,8 +480,8 @@ The C code allocates these as it goes, for the actual input, and their worst
cases are huge (see `workbuf_len_worst_case` below), so `workbuf_len` returns
a range whose `max_incl` is the C code's worst case for the sanitized
parameters (saturating at `0xFFFF_FFFF_FFFF_FFFF`) and whose `min_incl` is a
practical bound. Both have the same fixed regions, followed by a variable
region of `max(Z, M)` bytes, where `L` is the most literals that a metablock
practical bound. Both have the same fixed regions, with a variable region of
`max(Z, M)` bytes, where `L` is the most literals that a metablock
can hold (`min(max_len, (max_len / 8) + (1 << lgblock))`, since `EncodeData`
merges another input block only while there are fewer than `max_len / 8`
literals) and:
@@ -531,20 +580,73 @@ encoder makes the same `EncodeData` calls, at the same points. One
emitting the flint, and its callers then call it again.
### Encoder Large Window
The `QUIRK_LARGE_WINDOW` quirk mirrors `BROTLI_PARAM_LARGE_WINDOW`
(large-window brotli is an incompatible extension of RFC 7932, decodable by
this package's decoder with the same quirk, see "Large Window" above, and by
libbrotlidec with `BROTLI_DECODER_PARAM_LARGE_WINDOW`). Like the C code's
`SanitizeParams`, `EnsureInitialized` and `ChooseHasher`:
- At qualities 0 to 2 (the C code's `MAX_QUALITY_FOR_STATIC_ENTROPY_CODES`
and below), the quirk is ignored: lgwin is clamped to 24 and the output is
standard brotli.
- At qualities 3 and above, lgwin is clamped to `[10 ..= 30]` (instead of
`[10 ..= 24]`) and the stream starts with the 14-bit large-window header
(`EncodeWindowBits`), even for lgwin 24 and below. The distance parameters
(`BrotliInitDistanceParams` with `large_window`) allow distances up to
`BROTLI_MAX_ALLOWED_DISTANCE` with an alphabet of up to
`BROTLI_LARGE_MAX_DISTANCE_BITS` (62) distance bits, so the distance
Huffman codes are stored with a larger alphabet size than without the
quirk, even when no distance is large. Backward distances are at most
`(1 << lgwin) - 16`, so their extra bits (at most 30) fit in a command's
32-bit `dist_extra_`, as in the C code.
- lgwin above 24 changes the hasher (only at qualities 3 to 9; the C code's
qualities 10 and 11 use H10 regardless): H3 becomes H35 (H3 plus
`HROLLING_FAST`), H54 becomes H55 (H54 plus `HROLLING_FAST`) and H6 and
H68 become H65 (H6, not H68, plus `HROLLING`). H5, H58 and H40 to H42 stay
as they are, so qualities 5 to 9 only use H65 when the size hint (set or
estimated by `UpdateSizeHint`) is at least 1 MiB (and lgwin is at least 19,
which it is). The rolling hashers find matches of at least 32 bytes at any
distance in the window (`HROLLING_FAST` only at positions and distances
that are multiples of 4) and do not search the static dictionary.
- lgblock is unaffected (it is at most 24), but `rb_bits` (and so the ring
buffer) grows with lgwin. Metablocks are still at most `1 << 24` bytes.
- At qualities 10 and 11, the H10 hasher's binary tree forest grows with
lgwin (`8 << lgwin` bytes, so 8 GiB at lgwin 30, as in the C code, which
allocates the same), and the Zopfli and static dictionary distance limits
use the large-window `max_distance`.
Positions (the C code's `size_t` and the `WrapPosition`'ed `uint32_t`
values) and distances are `base.u64` throughout the encoder, so nothing is
truncated for windows up to `1 << 30`. The rolling hashers' table holds
32-bit positions, like the C code's.
### Encoder Testing
`wuffs test -mimic std/brotli` checks byte-exact parity with libbrotlienc
over a grid of qualities (0 to 11), lgwins, modes, input chunkings, flushes,
metadata blocks, size hints and other parameters (lgblock, npostfix, ndirect,
disabling literal context modeling, stream offset, large window at quality 2),
as well as round trips through the decoder.
disabling literal context modeling, stream offset, large window), as well as
round trips through the decoder. Its large-window tests cover qualities 3 to
9 with lgwin 18 to 30 (and the H35, H55 and H65 hashers) on small files and
on a 46 MiB input with repeats at distances of about 20 and 40 MiB, decoded
by both this package's decoder and libbrotlidec.
`script/manual-test-brotli-encoder-parity.c` runs a larger grid over every
file in `test/data/brotli`, some larger files in `test/data` (up to about
400 KiB) and some synthetic inputs (up to 1 MiB). Its `-q=LO-HI` and
`-shard=K/N` options select the qualities and split the inputs over several
processes. With `-q=0-9` (the default) it runs 718,555 configurations
(about 86 thousand of them at qualities 0 and 1), all byte-exact against
libbrotlienc v1.2.1 on x86-64.
processes, `-lw` runs only its large-window grid (lgwin 10 to 30 at
qualities 2 to 9, with various chunkings, flushes, size hints and other
parameters, each also with the shortest workbuf that should work) and `-big`
replaces the inputs with synthetic ones of 3 to 70 MiB with repeats at
distances of up to 40 MiB. With `-q=0-9` (the default) it runs 814,706
configurations (96,688 of them in the large-window grid), all byte-exact
against libbrotlienc v1.2.1 on x86-64. With `-lw -big` it runs 720 more
(on 3, 6 and 12 MiB inputs with the large-window grid and on 24, 40 and
70 MiB inputs with one-shot and streaming large-window configurations at
lgwin 24 to 30), also all byte-exact.
### Encoder Differences from the C Code
@@ -564,6 +666,22 @@ libbrotlienc v1.2.1 on x86-64.
`BROTLI_MAX_SIMD_QUALITY` is undefined (no 64-bit count-trailing-zeros
builtin) or 7 (MSVC on x86-64) choose differently and so produce
different (but valid) output at those qualities.
- The composite hashers (H35, H55 and H65) are not separate hasher types
here: `hasher_type` is their first hasher (H3, H54 or H6) and a non-zero
`hr_jump` (4 for `HROLLING_FAST`, 1 for `HROLLING`) adds the rolling
hasher, whose functions run after the first hasher's, like the C code's
composite `FN(Prepare)`, `FN(StitchToPreviousBlock)` and
`FN(FindLongestMatch)`. The C code's composite hasher defers its inner
hashers' `FN(Initialize)` to its first `FN(Prepare)`. Here, the first
hasher's `FN(Initialize)` runs in `hasher_setup` (it does not touch the
hasher's memory, so this is equivalent) and the rolling hasher's at the
first `hasher_prepare`. The C code's composite gives its first hasher a
copy of its `HasherCommon` (with zeroed static dictionary statistics);
here, the first hasher uses the encoder's statistics, which the composite
itself never uses, so the result is the same.
- The workbuf's regions are in a different order than the C code's
allocations and the ring buffer is fully reserved from the start (see the
workbuf layout above). This does not change the output.
- `BrotliBuildMetaBlockGreedy`'s two kinds of block splitter (one for a
single histogram per block type, `metablock_inc.h`, and one for several
literal contexts, `ContextBlockSplitter` in `metablock.c`) are one
+40 -7
View File
@@ -157,6 +157,12 @@ pub struct encoder? implements base.io_transformer(
rb_cur_size : base.u32,
rb_pos : base.u32,
// rb_short_workbuf is whether any transform_io call (at qualities 2
// to 9) had a workbuf shorter than workbuf_len. If so, the ring
// buffer's tail may be incomplete, so the input must stay short
// enough for the ring buffer never to wrap.
rb_short_workbuf : base.bool,
// The C code's BrotliDistanceParams fields (other than
// distance_postfix_bits and num_direct_distance_codes).
dist_alphabet_size_max : base.u32,
@@ -178,6 +184,16 @@ pub struct encoder? implements base.io_transformer(
dict_num_matches : base.u64,
fc_max_hops : base.u64,
// The large-window composite hashers' rolling hasher (the C code's
// HashRolling). See encode_hasher_composite.wuffs. hr_jump is 0 when
// the hasher is not a composite one.
hr_jump : base.u32[..= 4],
hr_fresh : base.bool,
hr_state : base.u32,
hr_factor_remove : base.u32,
hr_next_ix : base.u64,
hr_off : base.u64,
// The C code's HasherSearchResult.
sr_len : base.u64,
sr_distance : base.u64,
@@ -205,6 +221,8 @@ pub struct encoder? implements base.io_transformer(
// and the StartPosQueue's idx_ and the PosData being pushed.
literal_context_mode : base.u32,
var_off : base.u64,
var_end : base.u64,
hq_layout_fixed : base.bool,
hq_insufficient : base.bool,
hq_wb_high : base.u64,
h10_window_mask : base.u64,
@@ -464,10 +482,10 @@ pub func encoder.workbuf_len() base.range_ii_u64 {
lgwin = this.sanitized_lgwin(quality: q)
if q <= 1 {
n = ((2 as base.u64) << lgwin) + 503 + STORAGE_SLACK
} else if (q <= 9) and ((q == 2) or (this.p_large_window == 0)) {
} else if q <= 9 {
n = this.compute_layout_offset(quality: q, lgwin: lgwin,
lgblock: this.computed_lgblock(quality: q, lgwin: lgwin), idx: LAYOUT_END)
} else if (q >= 10) and (this.p_large_window == 0) {
} else {
return this.workbuf_len_hq(quality: q, lgwin: lgwin)
}
return this.util.make_range_ii_u64(min_incl: n, max_incl: n)
@@ -847,13 +865,18 @@ pub func encoder.transform_io?(dst: base.io_writer, src: base.io_reader, workbuf
var injected : base.bool
var copy_input_size : base.u64
var total : base.u64
this.ensure_initialized!()
if this.large_window and (this.quality > 2) {
// TODO: implement large-window brotli at qualities 3 and above.
return "#unsupported encoder quality"
} else if (this.quality >= 2) and (args.workbuf.length() < this.workbuf_end) {
return base."#bad workbuf length"
this.hq_extend_var_region!(wb_len: args.workbuf.length())
if (this.quality >= 2) and (args.workbuf.length() < this.workbuf_end) {
// A workbuf shorter than workbuf_len works if it is long enough for
// the input so far (see the rb_short_workbuf check below), as long
// as the ring buffer never wraps.
this.rb_short_workbuf = true
if args.workbuf.length() < (this.rb_buf_off ~mod+ RB_SHORT_SLACK) {
return base."#bad workbuf length"
}
}
while true {
@@ -953,6 +976,16 @@ pub func encoder.transform_io?(dst: base.io_writer, src: base.io_reader, workbuf
if (remaining <> 0) and (args.src.length() > 0) {
copy_input_size = remaining.min(no_more_than: args.src.length())
if this.rb_short_workbuf {
// The ring buffer must not wrap (or reach its tail) and
// the workbuf must hold the input so far plus some
// slack.
total = (this.input_pos ~mod+ copy_input_size) ~mod+ RB_SHORT_SLACK
if (total > (this.rb_size as base.u64)) or
((this.rb_buf_off ~mod+ total) > args.workbuf.length()) {
return base."#bad workbuf length"
}
}
mark = args.src.mark()
args.src.skip?(n: copy_input_size)
this.copy_input_to_ring_buffer!(wb: args.workbuf, bytes: args.src.since(mark: mark))
+40 -18
View File
@@ -17,30 +17,37 @@
// The workbuf holds, in this order (see README.md's "Encoder Workbuf Layout"
// section, which compute_layout_offset mirrors):
// - the storage (the C code's GetBrotliStorage) for one metablock,
// - the ring buffer (the C code's RingBuffer.data_),
// - the hasher's memory (the C code's common->extra),
// - the commands (16 bytes each), and
// - the commands (16 bytes each),
// - at qualities 4 and above, the metablock split (see
// encode_metablock_greedy.wuffs), the literal context map, the
// EncodeContextMap symbols and the block encoders' depths and bits.
// EncodeContextMap symbols and the block encoders' depths and bits, and
// - the ring buffer (the C code's RingBuffer.data_).
// The workbuf layout's region indexes. LAYOUT_END's offset is the total
// length.
// The ring buffer is last, so that a workbuf shorter than workbuf_len works
// for short inputs (see transform_io's rb_short_workbuf check).
pri const LAYOUT_STORAGE : base.u32 = 0
pri const LAYOUT_RB : base.u32 = 1
pri const LAYOUT_HASH : base.u32 = 2
pri const LAYOUT_CMDS : base.u32 = 3
pri const LAYOUT_TYPES_0 : base.u32 = 4
pri const LAYOUT_LENGTHS_0 : base.u32 = 7
pri const LAYOUT_HISTO_0 : base.u32 = 10
pri const LAYOUT_COMB : base.u32 = 13
pri const LAYOUT_CMAP : base.u32 = 14
pri const LAYOUT_RLE : base.u32 = 15
pri const LAYOUT_BE_DEPTH_0 : base.u32 = 16
pri const LAYOUT_BE_BITS_0 : base.u32 = 19
pri const LAYOUT_VAR : base.u32 = 22
pri const LAYOUT_HASH : base.u32 = 1
pri const LAYOUT_CMDS : base.u32 = 2
pri const LAYOUT_TYPES_0 : base.u32 = 3
pri const LAYOUT_LENGTHS_0 : base.u32 = 6
pri const LAYOUT_HISTO_0 : base.u32 = 9
pri const LAYOUT_COMB : base.u32 = 12
pri const LAYOUT_CMAP : base.u32 = 13
pri const LAYOUT_RLE : base.u32 = 14
pri const LAYOUT_BE_DEPTH_0 : base.u32 = 15
pri const LAYOUT_BE_BITS_0 : base.u32 = 18
pri const LAYOUT_VAR : base.u32 = 21
pri const LAYOUT_RB : base.u32 = 22
pri const LAYOUT_END : base.u32 = 23
// RB_SHORT_SLACK is how many bytes past the input's end (in the ring buffer)
// a short workbuf must have: the C code reads at most 7 bytes past the
// input's end (its kSlackForEightByteHashingEverywhere), rounded up.
pri const RB_SHORT_SLACK : base.u64 = 16
// The C code's MAX_NUM_DELAYED_SYMBOLS (in quality.h).
pri const MAX_NUM_DELAYED_SYMBOLS : base.u64 = 0x2FFF
@@ -91,11 +98,13 @@ pri func encoder.max_metablock_size(lgwin: base.u32[..= 30], lgblock: base.u32[.
// hasher_total_size returns the sum of FN(HashMemAllocInBytes)'s
// alloc_size[i] for the hasher that ChooseHasher picks for the given
// quality, lgwin and size_hint.
// quality, lgwin and size_hint (including, for lgwin above 24, the
// large-window composite hashers' rolling hasher table).
pri func encoder.hasher_total_size(quality: base.u32[..= 11], lgwin: base.u32[..= 30], size_hint: base.u32) base.u64 {
var htype : base.u32
var bucket_bits : base.u32[..= 24]
var block_bits : base.u32[..= 16]
var rolling : base.u64
if args.quality >= 10 {
return (4 * H10_BUCKET_SIZE) + this.h10_alloc_size_1(lgwin: args.lgwin)
@@ -126,9 +135,21 @@ pri func encoder.hasher_total_size(quality: base.u32[..= 11], lgwin: base.u32[..
}
block_bits = args.quality - 1
}
return (this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 0) ~mod+
rolling = 0
if args.lgwin > 24 {
// The large-window composite hashers H35, H55 and H65 (see
// choose_hasher): HASHER_A plus the rolling hasher's table.
if (htype == 3) or (htype == 54) {
rolling = this.hasher_rolling_alloc_size()
} else if (htype == 6) or (htype == 68) {
htype = 6
rolling = this.hasher_rolling_alloc_size()
}
}
return ((this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 0) ~mod+
this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 1)) ~mod+
this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 2)
this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 2)) ~mod+
rolling
}
// hash_region_size returns the workbuf length reserved for the hasher's
@@ -279,6 +300,7 @@ pri func encoder.compute_layout!() {
this.mb_cmap_off = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_CMAP)
this.mb_rle_off = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_RLE)
this.var_off = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_VAR)
this.var_end = this.rb_off
this.workbuf_end = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_END)
// RingBufferSetup.
+44 -1
View File
@@ -102,7 +102,7 @@ pri func encoder.fc_bank_bits() base.u32[..= 16] {
}
// Mirrors ChooseHasher in quality.h (with BROTLI_MAX_SIMD_QUALITY defined as
// 6), for qualities 2 to 9 and lgwin up to 24.
// 6), for qualities 2 to 9.
pri func encoder.choose_hasher!() {
var q : base.u32[..= 11]
@@ -149,6 +149,25 @@ pri func encoder.choose_hasher!() {
this.h_num_last_distances = 10
}
}
this.hr_jump = 0
if this.lgwin > 24 {
// Different hashers for large window brotli: not for qualities <= 2,
// these are too fast for large window. Not for qualities >= 10:
// their hasher already works well with large window. So the changes
// are:
// H3 --> H35: for quality 3.
// H54 --> H55: for quality 4 with size hint > 1MB
// H6/H68 --> H65: for qualities 5, 6, 7, 8, 9.
// See encode_hasher_composite.wuffs: the composite hasher's
// HASHER_A stays in this.hasher_type.
if (this.hasher_type == 3) or (this.hasher_type == 54) {
this.hr_jump = 4
} else if (this.hasher_type == 6) or (this.hasher_type == 68) {
this.hasher_type = 6
this.hr_jump = 1
}
}
}
// hasher_alloc_size mirrors FN(HashMemAllocInBytes) (alloc_size[i], for i in
@@ -226,6 +245,9 @@ pri func encoder.hasher_setup!(wb: slice base.u8, position: base.u64, input_size
htype: this.hasher_type, bucket_bits: this.h_bucket_bits, block_bits: this.h_block_bits, i: 0)
this.h_off2 = this.h_off1 ~mod+ this.hasher_alloc_size(
htype: this.hasher_type, bucket_bits: this.h_bucket_bits, block_bits: this.h_block_bits, i: 1)
// The composite hashers' HASHER_B memory follows HASHER_A's.
this.hr_off = this.h_off2
this.hr_fresh = true
this.h_is_prepared = false
this.h_is_setup = true
}
@@ -314,6 +336,18 @@ pri func encoder.hasher_prepare!(wb: slice base.u8, one_shot: base.bool, input_s
j ~mod+= 1
}
}
if this.hr_jump <> 0 {
// The composite hasher's FN(Prepare) (in hash_composite_inc.h), for
// HASHER_B. HASHER_B's FN(Initialize) is deferred to the first
// FN(Prepare). HASHER_A's FN(Initialize) does not touch the hasher
// memory, so doing it earlier (in hasher_setup) is equivalent.
if this.hr_fresh {
this.hr_fresh = false
this.hasher_rolling_initialize!(wb: args.wb)
}
this.hasher_rolling_prepare!(wb: args.wb, input_size: args.input_size, data_off: this.rb_buf_off)
}
}
// Mirrors FN(HashBytes) in hash_longest_match_quickly_inc.h.
@@ -449,6 +483,10 @@ pri func encoder.hasher_stitch_to_previous_block!(wb: slice base.u8, num_bytes:
this.hasher_store!(wb: args.wb, ix: args.position ~mod- 2)
this.hasher_store!(wb: args.wb, ix: args.position ~mod- 1)
}
if this.hr_jump <> 0 {
// The composite hasher's HASHER_B.
this.hasher_rolling_stitch_to_previous_block!(wb: args.wb, num_bytes: args.num_bytes, position: args.position)
}
}
// Mirrors PrepareDistanceCache in hash.h, via each hasher's
@@ -614,6 +652,11 @@ pri func encoder.find_longest_match!(wb: slice base.u8, cur_ix: base.u64, max_le
this.find_longest_match_forgetful!(wb: args.wb, cur_ix: args.cur_ix, max_length: args.max_length,
max_backward: args.max_backward, dictionary_distance: args.dictionary_distance, max_distance: args.max_distance)
}
if this.hr_jump <> 0 {
// The composite hasher's HASHER_B.
this.hasher_rolling_find_longest_match!(wb: args.wb, cur_ix: args.cur_ix, max_length: args.max_length,
max_backward: args.max_backward)
}
}
// Mirrors FN(FindLongestMatch) in hash_longest_match_quickly_inc.h.
+208
View File
@@ -0,0 +1,208 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// This file mirrors the C code's large-window hashers: the composite hashers
// (hash_composite_inc.h) H35 (H3 + HROLLING_FAST), H55 (H54 + HROLLING_FAST)
// and H65 (H6 + HROLLING), and the rolling hashers (hash_rolling_inc.h)
// HROLLING_FAST (JUMP 4) and HROLLING (JUMP 1), both with CHUNKLEN 32,
// NUMBUCKETS (1 << 24) and MASK ((NUMBUCKETS * 64) - 1).
//
// A composite hasher is represented by its first hasher (HASHER_A, H3, H54 or
// H6) in this.hasher_type and its rolling hasher (HASHER_B) by a non-zero
// this.hr_jump (4 for HROLLING_FAST, 1 for HROLLING). The composite's FN(etc)
// functions call HASHER_A's and then HASHER_B's, so the encode_hasher.wuffs
// functions (which implement HASHER_A) call the hasher_rolling_etc functions
// below after their own work when this.hr_jump is non-zero.
//
// The composite hasher's memory is HASHER_A's alloc_size[0] and [1] followed
// by HASHER_B's alloc_size[0] (the rolling table of NUMBUCKETS u32le values),
// at the this.hr_off workbuf offset. HASHER_A's HasherCommon is a copy of the
// composite's (the C code's ha_common), whose dict_num_lookups and
// dict_num_matches start at zero, like the composite's own, so HASHER_A can
// keep using this.dict_num_lookups and this.dict_num_matches.
// Mirrors CHUNKLEN in hash.h, for the rolling hashers.
pri const ROLLING_CHUNKLEN : base.u64 = 32
// Mirrors NUMBUCKETS in hash.h, for the rolling hashers.
pri const ROLLING_NUMBUCKETS : base.u32 = 0x100_0000
// Mirrors MASK in hash.h, for the rolling hashers.
pri const ROLLING_MASK : base.u32 = 0x3FFF_FFFF
// Mirrors FN(kRollingHashMul32) in hash_rolling_inc.h.
pri const ROLLING_HASH_MUL_32 : base.u32 = 69069
// Mirrors FN(kInvalidPos) in hash_rolling_inc.h.
pri const ROLLING_INVALID_POS : base.u32 = 0xFFFF_FFFF
// hasher_rolling_alloc_size mirrors FN(HashMemAllocInBytes) in
// hash_rolling_inc.h: alloc_size[0], the table.
pri func encoder.hasher_rolling_alloc_size() base.u64 {
return 4 * (ROLLING_NUMBUCKETS as base.u64)
}
// Mirrors FN(Initialize) in hash_rolling_inc.h. The C code's composite
// hashers call it (via their deferred initialization) in their first
// FN(Prepare), which HasherSetup calls right after FN(Initialize).
pri func encoder.hasher_rolling_initialize!(wb: slice base.u8) {
var i : base.u64
this.hr_state = 0
this.hr_next_ix = 0
// Compute the factor of the oldest byte to remove: factor**steps modulo
// 0xffffffff (the multiplications rely on 32-bit overflow).
this.hr_factor_remove = 1
i = 0
while i < ROLLING_CHUNKLEN {
this.hr_factor_remove ~mod*= ROLLING_HASH_MUL_32
i ~mod+= this.hr_jump as base.u64
}
// Every table entry is ROLLING_INVALID_POS.
this.fill_bytes!(s: args.wb, i: this.hr_off, n: this.hasher_rolling_alloc_size(), v: 0xFF)
}
// Mirrors FN(Prepare) in hash_rolling_inc.h, where data is the workbuf's
// bytes starting at offset data_off.
pri func encoder.hasher_rolling_prepare!(wb: slice base.u8, input_size: base.u64, data_off: base.u64) {
var i : base.u64
// Too small size, cannot use this hasher.
if args.input_size < ROLLING_CHUNKLEN {
return nothing
}
this.hr_state = 0
i = 0
while i < ROLLING_CHUNKLEN {
// FN(HashRollingFunctionInitial).
this.hr_state = (ROLLING_HASH_MUL_32 ~mod* this.hr_state) ~mod+
((this.byte_at(s: args.wb, i: args.data_off ~mod+ i) as base.u32) + 1)
i ~mod+= this.hr_jump as base.u64
}
}
// Mirrors FN(StitchToPreviousBlock) in hash_rolling_inc.h.
pri func encoder.hasher_rolling_stitch_to_previous_block!(wb: slice base.u8, num_bytes: base.u64, position: base.u64) {
var position : base.u64
var position_masked : base.u64
var available : base.u64
var jm1 : base.u64
var diff : base.u64
var mask : base.u64
// In this case we must re-initialize the hasher from scratch from the
// current position.
position = args.position
available = args.num_bytes
jm1 = (this.hr_jump as base.u64) ~mod- 1
if (position & jm1) <> 0 {
diff = (this.hr_jump as base.u64) ~mod- (position & jm1)
if diff > available {
available = 0
} else {
available ~mod-= diff
}
position ~mod+= diff
}
mask = this.rb_mask as base.u64
position_masked = position & mask
// Wrapping around ringbuffer not handled.
if available > (mask ~mod- position_masked) {
available = mask ~mod- position_masked
}
this.hasher_rolling_prepare!(wb: args.wb, input_size: available,
data_off: this.rb_buf_off ~mod+ position_masked)
this.hr_next_ix = position
}
// Mirrors FN(FindLongestMatch) in hash_rolling_inc.h. The search result is in
// the this.sr_etc fields.
pri func encoder.hasher_rolling_find_longest_match!(wb: slice base.u8, cur_ix: base.u64, max_length: base.u64, max_backward: base.u64) {
var data : roslice base.u8
var mask : base.u64
var cur_ix_masked : base.u64
var jump : base.u64
var pos : base.u64
var code : base.u32
var rem : base.u8
var add : base.u8
var found_ix : base.u64
var backward : base.u64
var found_ix_masked : base.u64
var len : base.u64
var score : base.u64
var t : base.u64
if this.rb_buf_off > args.wb.length() {
return nothing
}
data = args.wb[this.rb_buf_off ..]
mask = this.rb_mask as base.u64
cur_ix_masked = args.cur_ix & mask
jump = this.hr_jump as base.u64
if (args.cur_ix & (jump ~mod- 1)) <> 0 {
return nothing
}
// Not enough lookahead.
if args.max_length < ROLLING_CHUNKLEN {
return nothing
}
pos = this.hr_next_ix
while pos <= args.cur_ix {
code = this.hr_state & ROLLING_MASK
rem = this.byte_at(s: data, i: pos & mask)
add = this.byte_at(s: data, i: (pos ~mod+ ROLLING_CHUNKLEN) & mask)
found_ix = ROLLING_INVALID_POS as base.u64
// FN(HashRollingFunction).
this.hr_state = ((ROLLING_HASH_MUL_32 ~mod* this.hr_state) ~mod+ ((add as base.u32) + 1)) ~mod-
(this.hr_factor_remove ~mod* ((rem as base.u32) + 1))
if code < ROLLING_NUMBUCKETS {
t = this.hr_off ~mod+ (4 * (code as base.u64))
found_ix = this.load32le(s: args.wb, i: t) as base.u64
this.poke_u32!(s: args.wb, i: t, v: (pos & 0xFFFF_FFFF) as base.u32)
if (pos == args.cur_ix) and (found_ix <> (ROLLING_INVALID_POS as base.u64)) {
// The cast to 32-bit makes backward distances up to 4GB work
// even if cur_ix is above 4GB, despite using 32-bit values in
// the table.
backward = (args.cur_ix ~mod- found_ix) & 0xFFFF_FFFF
if backward <= args.max_backward {
found_ix_masked = found_ix & mask
len = this.find_match_length_with_limit(s: data,
i1: found_ix_masked, i2: cur_ix_masked, limit: args.max_length)
if (len >= 4) and (len > this.sr_len) {
score = this.backward_reference_score(copy_length: len, backward_reference_offset: backward)
if score > this.sr_score {
this.sr_len = len
this.sr_distance = backward
this.sr_score = score
this.sr_len_code_delta = 0
}
}
}
}
}
pos ~mod+= jump
}
this.hr_next_ix = args.cur_ix ~mod+ jump
// NOTE: this hasher does not search in the dictionary. It is used as
// backup-hasher, the main hasher already searches in it.
}
+49 -7
View File
@@ -105,9 +105,11 @@ pri func encoder.write_meta_block_hq!(wb: slice base.u8, wrapped_last_flush_pos:
var cmds_hi : base.u64
var c : base.u32
var ds : base.u32[..= 704]
var var_hi : base.u64
cmds_hi = this.cmds_off ~mod+ (this.num_cmds ~mod* 16)
if (this.var_off > args.wb.length()) or (this.rb_buf_off > args.wb.length()) or
var_hi = this.var_end.min(no_more_than: args.wb.length())
if (this.var_off > var_hi) or (this.rb_buf_off > args.wb.length()) or
(this.cmds_off > cmds_hi) or (cmds_hi > args.wb.length()) {
this.hq_insufficient = true
return nothing
@@ -117,7 +119,7 @@ pri func encoder.write_meta_block_hq!(wb: slice base.u8, wrapped_last_flush_pos:
disable_literal_context_modeling: this.p_disable_lcm <> 0)
this.mb.init_distance_params!(npostfix: this.distance_postfix_bits,
ndirect: this.num_direct_distance_codes, large_window: this.large_window)
status = this.mb.build_meta_block!(wb: args.wb[this.var_off ..], ringbuffer: args.wb[this.rb_buf_off ..],
status = this.mb.build_meta_block!(wb: args.wb[this.var_off .. var_hi], ringbuffer: args.wb[this.rb_buf_off ..],
pos: args.wrapped_last_flush_pos, mask: this.rb_mask as base.u64,
prev_byte: args.prev_byte, prev_byte2: args.prev_byte2,
cmds: args.wb[this.cmds_off .. cmds_hi], num_commands: this.num_cmds,
@@ -131,7 +133,7 @@ pri func encoder.write_meta_block_hq!(wb: slice base.u8, wrapped_last_flush_pos:
}
// MIN_QUALITY_FOR_OPTIMIZE_HISTOGRAMS is 4. The number of distance
// symbols effectively used for distance histograms.
this.mb.optimize_histograms!(wb: args.wb[this.var_off ..], num_distance_codes: this.mb.dist_alphabet_size_limit)
this.mb.optimize_histograms!(wb: args.wb[this.var_off .. var_hi], num_distance_codes: this.mb.dist_alphabet_size_limit)
// Point the block encoders' inputs (the gbs_etc fields, which
// build_meta_block_greedy sets at qualities 4 to 9) at the
@@ -328,11 +330,51 @@ pri func encoder.store_meta_block_hq!(wb: slice base.u8, start_pos: base.u64, le
// fixed regions plus the minimum or the worst case variable region.
pri func encoder.workbuf_len_hq(quality: base.u32[..= 11], lgwin: base.u32[..= 30]) base.range_ii_u64 {
var lgblock : base.u32[..= 30]
var off : base.u64
var n : base.u64
lgblock = this.computed_lgblock(quality: args.quality, lgwin: args.lgwin)
off = this.compute_layout_offset(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, idx: LAYOUT_VAR)
n = this.compute_layout_offset(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, idx: LAYOUT_END)
return this.util.make_range_ii_u64(
min_incl: off ~sat+ this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, worst_case: false),
max_incl: off ~sat+ this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, worst_case: true))
min_incl: n,
max_incl: n ~sat+ this.hq_var_extra_max(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock))
}
// hq_var_extra_max returns how much longer than its minimum length (the
// LAYOUT_VAR region's length) the variable region can usefully be: the
// difference between its worst case and minimum lengths.
pri func encoder.hq_var_extra_max(quality: base.u32[..= 11], lgwin: base.u32[..= 30], lgblock: base.u32[..= 30]) base.u64 {
var lo : base.u64
var hi : base.u64
lo = this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: args.lgblock, worst_case: false)
hi = this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: args.lgblock, worst_case: true)
if hi <= lo {
return 0
}
return hi - lo
}
// hq_extend_var_region grows the variable region (and moves the ring buffer,
// which is the last region, along with this.workbuf_end) to use as much of a
// workbuf of wb_len bytes as the worst case can need. It only does this
// once, at the first transform_io call (before the ring buffer is used), at
// qualities 10 and 11. Later calls should pass a workbuf of the same length
// (a shorter one is handled like any short workbuf, and the variable region
// is then cut short).
pri func encoder.hq_extend_var_region!(wb_len: base.u64) {
var extra : base.u64
if (this.quality < 10) or this.hq_layout_fixed {
return nothing
}
this.hq_layout_fixed = true
if args.wb_len <= this.workbuf_end {
return nothing
}
extra = this.hq_var_extra_max(quality: this.quality, lgwin: this.lgwin, lgblock: this.lgblock)
extra = extra.min(no_more_than: args.wb_len - this.workbuf_end)
this.var_end ~mod+= extra
this.rb_off ~mod+= extra
this.rb_buf_off ~mod+= extra
this.workbuf_end ~mod+= extra
}
+2 -2
View File
@@ -1007,7 +1007,7 @@ pri func encoder.create_zopfli_backward_references!(wb: slice base.u8, num_bytes
var n : base.u64
var ok_offsets : base.bool
ok_offsets = this.z_set_offsets!(wb_len: args.wb.length(), num_bytes: args.num_bytes)
ok_offsets = this.z_set_offsets!(wb_len: this.var_end.min(no_more_than: args.wb.length()), num_bytes: args.num_bytes)
if not ok_offsets {
return base."#bad workbuf length"
}
@@ -1047,7 +1047,7 @@ pri func encoder.create_hq_zopfli_backward_references!(wb: slice base.u8, num_by
var n : base.u64
var ok_offsets : base.bool
ok_offsets = this.z_set_offsets!(wb_len: args.wb.length(), num_bytes: args.num_bytes)
ok_offsets = this.z_set_offsets!(wb_len: this.var_end.min(no_more_than: args.wb.length()), num_bytes: args.num_bytes)
if not ok_offsets {
return base."#bad workbuf length"
}