mirror of
https://github.com/ddidderr/wuffs.git
synced 2026-11-19 08:38:14 +01:00
Merge claude/project-thread-0izcgq (large window at qualities 3 to 9)
Put the q10/q11 variable region before the ring buffer (now the last region), growing it (and moving the ring buffer) at the first transform_io call to use a workbuf longer than workbuf_len's minimum. Large window now also works at qualities 10 and 11.
This commit is contained in:
11 files changed
+1664
-216
No files matched your search
+154
-36
@@ -154,8 +154,7 @@ invariants for memory safety but Wuffs requires explicit bounds checks.
|
||||
- Metadata callbacks (metadata blocks are currently skipped). See the
|
||||
"Metadata Blocks" section below.
|
||||
- Fast paths.
|
||||
- Encoder dictionaries and large-window encoding at qualities 3 and above
|
||||
(see the "Encoder" section below).
|
||||
- Encoder dictionaries (see the "Encoder" section below).
|
||||
|
||||
|
||||
## Encoder
|
||||
@@ -182,11 +181,9 @@ literal context modeling heuristics, `BrotliBuildMetaBlockGreedy`,
|
||||
hasher, `BrotliCreateZopfliBackwardReferences` at quality 10 and
|
||||
`BrotliCreateHqZopfliBackwardReferences` at quality 11, and then
|
||||
`ChooseContextMode`, `BrotliBuildMetaBlock`, `BrotliOptimizeHistograms` and
|
||||
`BrotliStoreMetaBlock`). For large-window brotli at qualities 3 and above,
|
||||
`transform_io` returns `"#unsupported encoder quality"` (and `workbuf_len`
|
||||
is 0). Large-window
|
||||
brotli works at quality 2 (and is ignored at qualities 0 and 1, like the C
|
||||
code).
|
||||
`BrotliStoreMetaBlock`). Large-window brotli (see "Encoder Large Window"
|
||||
below) works at qualities 3 to 11 with lgwin 10 to 30. Like the C code,
|
||||
qualities 0 to 2 ignore the large-window quirk.
|
||||
|
||||
The source files are:
|
||||
|
||||
@@ -217,6 +214,10 @@ The source files are:
|
||||
H54, H5, H6, H58, H68, H40, H41 and H42) and `SearchInStaticDictionary`
|
||||
(with `static_dict.c`'s matching), as one set of functions selected by the
|
||||
hasher type at run time.
|
||||
- `encode_hasher_composite.wuffs`: the large-window hashers:
|
||||
`hash_composite_inc.h` (H35, H55 and H65, which combine H3, H54 or H6 with
|
||||
a rolling hasher) and `hash_rolling_inc.h` (`HROLLING_FAST` and
|
||||
`HROLLING`).
|
||||
- `encode_backward_references.wuffs`: `backward_references.c` and
|
||||
`backward_references_inc.h` (`BrotliCreateBackwardReferences`),
|
||||
`command.h` and `ComputeDistanceCode`.
|
||||
@@ -306,7 +307,7 @@ first `transform_io` call, after which `set_quirk` returns
|
||||
| `QUIRK_MODE` | `BROTLI_PARAM_MODE` |
|
||||
| `QUIRK_DISABLE_LITERAL_CONTEXT_MODELING` | `BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING` (0 or 1) |
|
||||
| `QUIRK_SIZE_HINT` | `BROTLI_PARAM_SIZE_HINT` |
|
||||
| `QUIRK_LARGE_WINDOW` (shared with the decoder) | `BROTLI_PARAM_LARGE_WINDOW` |
|
||||
| `QUIRK_LARGE_WINDOW` (shared with the decoder) | `BROTLI_PARAM_LARGE_WINDOW` (0 or 1) |
|
||||
| `QUIRK_NPOSTFIX` | `BROTLI_PARAM_NPOSTFIX` |
|
||||
| `QUIRK_NDIRECT` | `BROTLI_PARAM_NDIRECT` |
|
||||
| `QUIRK_STREAM_OFFSET` | `BROTLI_PARAM_STREAM_OFFSET` (at most `1 << 30`) |
|
||||
@@ -316,7 +317,8 @@ with `base.QUIRK_QUALITY`. At qualities 0 and 1, only the quality, lgwin and
|
||||
stream offset affect the output (lgblock, mode, size hint, large window,
|
||||
npostfix, ndirect and literal context modeling are stored, sanitized and then
|
||||
ignored, as in the C code). At qualities 2 to 9, they have the same effect as
|
||||
in the C code: lgblock (at qualities 4 and above; the C code's
|
||||
in the C code: large window (see "Encoder Large Window" below), lgblock (at
|
||||
qualities 4 and above; the C code's
|
||||
`ComputeLgBlock` fixes it to 14 below that), npostfix, ndirect and the mode
|
||||
(only `BROTLI_MODE_FONT` matters, through `ChooseDistanceParams`, at
|
||||
qualities 4 and above), disabling literal context modeling (at qualities 5
|
||||
@@ -342,17 +344,15 @@ code's `ComputeRbBits`), `max_len` is `1 << min(rb_bits, 24)` (the C code's
|
||||
|
||||
1. The metablock storage: `(2 * max_len) + 503 + 16` bytes, like at
|
||||
qualities 0 and 1 but for the largest metablock.
|
||||
2. The ring buffer (the C code's `RingBuffer.buffer_`): `2 + (1 << rb_bits) +
|
||||
(1 << lgblock) + 7` bytes. The first 2 bytes are the copy of the last two
|
||||
bytes that precede `data_`, the tail (`1 << lgblock` bytes) mirrors the
|
||||
start of the buffer and the 7 trailing bytes are the C code's slack for
|
||||
8-byte loads.
|
||||
3. The hasher's memory (the C code's `common->extra[0 .. 3]`): the sum of
|
||||
2. The hasher's memory (the C code's `common->extra[0 .. 3]`): the sum of
|
||||
`FN(HashMemAllocInBytes)` for the hasher that `ChooseHasher` picks. When
|
||||
the size hint quirk is unset (zero), that choice depends on the input's
|
||||
length (through `UpdateSizeHint`), so this is the larger of the two
|
||||
possible choices (for a size hint below or at least 1 MiB).
|
||||
4. The commands: `16 * cmds_cap` bytes, in the C struct order: `insert_len_`
|
||||
possible choices (for a size hint below or at least 1 MiB). With lgwin
|
||||
above 24 (large window), the composite hashers H35, H55 and H65 add their
|
||||
rolling hasher's table (`(1 << 24)` u32le values, 64 MiB) after H3's,
|
||||
H54's or H6's memory.
|
||||
3. The commands: `16 * cmds_cap` bytes, in the C struct order: `insert_len_`
|
||||
(u32le), `copy_len_` (u32le, the copy length in the low 25 bits and
|
||||
`copy_code - copy_len` in the high 7 bits), `dist_extra_` (u32le),
|
||||
`cmd_prefix_` (u16le) and `dist_prefix_` (u16le). `cmds_cap` is
|
||||
@@ -362,7 +362,7 @@ code's `ComputeRbBits`), `max_len` is `1 << min(rb_bits, 24)` (the C code's
|
||||
input block) fewer than that many commands and, like the C code's
|
||||
`num_commands_ + bytes / 2 + 1` allocation, each input block of `n` bytes
|
||||
adds at most `(n / 2) + 1` commands.
|
||||
5. At qualities 4 and above, `BrotliBuildMetaBlockGreedy`'s `MetaBlockSplit`
|
||||
4. At qualities 4 and above, `BrotliBuildMetaBlockGreedy`'s `MetaBlockSplit`
|
||||
and `BrotliStoreMetaBlock`'s scratch space:
|
||||
- the block types (u8) and then the block lengths (u32le) of the literal,
|
||||
command and distance block splits, with room for `(max_len / 512) + 1`,
|
||||
@@ -378,13 +378,39 @@ code's `ComputeRbBits`), `max_len` is `1 << min(rb_bits, 24)` (the C code's
|
||||
- `EncodeContextMap`'s `rle_symbols` (256 * 64 u32le values) and
|
||||
- the block encoders' depths (u8) and then bits (u16le) for up to 256
|
||||
histograms per category.
|
||||
5. The ring buffer (the C code's `RingBuffer.buffer_`): `2 + (1 << rb_bits) +
|
||||
(1 << lgblock) + 7` bytes. The first 2 bytes are the copy of the last two
|
||||
bytes that precede `data_`, the tail (`1 << lgblock` bytes) mirrors the
|
||||
start of the buffer and the 7 trailing bytes are the C code's slack for
|
||||
8-byte loads.
|
||||
|
||||
Unlike the C code, which allocates its storage and command buffers as it
|
||||
goes (for the actual metablock and block sizes), `workbuf_len` is the worst
|
||||
case for the sanitized parameters, so that one workbuf works for any input.
|
||||
At qualities 2 to 9, `transform_io` returns `"#bad workbuf length"` if the
|
||||
workbuf is shorter than that. Some examples (in MiB, with the default
|
||||
lgblock and an unset size hint):
|
||||
Unlike the C code, which allocates its storage, ring buffer and command
|
||||
buffers as it goes (for the actual metablock and block sizes, and the actual
|
||||
input length), `workbuf_len` is the worst case for the sanitized parameters,
|
||||
so that one workbuf works for any input. A shorter workbuf still works for
|
||||
short inputs, because the ring buffer is the last region and the encoder
|
||||
only touches its first `2 + n + 7` bytes for an input of `n` bytes, as long
|
||||
as the ring buffer never wraps around. Precisely, at qualities 2 to 11
|
||||
(where, at qualities 10 and 11, `workbuf_len` means its `min_incl`):
|
||||
|
||||
- `transform_io` returns `"#bad workbuf length"` if the workbuf is shorter
|
||||
than `workbuf_len - R + 18`, where `R` is the ring buffer region's length
|
||||
(`2 + (1 << rb_bits) + (1 << lgblock) + 7`), i.e. if it does not reach 16
|
||||
bytes into the ring buffer's data.
|
||||
- If any `transform_io` call gets a workbuf shorter than `workbuf_len`, then
|
||||
from then on (for the rest of the stream), before copying input into the
|
||||
ring buffer, `transform_io` returns `"#bad workbuf length"` (without
|
||||
consuming that input) if the total input so far (not counting metadata)
|
||||
would be `n` bytes with `n + 16 > (1 << rb_bits)` (the ring buffer would
|
||||
wrap, or its tail would be needed) or with the workbuf shorter than
|
||||
`workbuf_len - R + 2 + n + 16`.
|
||||
|
||||
So, for a total input of `n` bytes with `n + 16 <= (1 << rb_bits)`, a
|
||||
workbuf of `workbuf_len - R + n + 18` bytes produces the same output as one
|
||||
of `workbuf_len` bytes. The parity test checks this with exactly that length
|
||||
(and that one byte less fails). This matters most for large-window brotli,
|
||||
whose ring buffer is up to 2 GiB. Some examples of `workbuf_len` (in MiB,
|
||||
with the default lgblock and an unset size hint):
|
||||
|
||||
| quality | lgwin 10 | lgwin 16 | lgwin 18 | lgwin 22 | lgwin 24 |
|
||||
| ------- | -------- | -------- | -------- | -------- | -------- |
|
||||
@@ -395,6 +421,21 @@ lgblock and an unset size hint):
|
||||
| 7 | 4.4 | 4.4 | 13.8 | 51.4 | 107.5 |
|
||||
| 9 | 5.1 | 5.1 | 39.5 | 77.1 | 133.2 |
|
||||
|
||||
With the large-window quirk (at qualities 3 and above; lgwin up to 24 gives
|
||||
the same numbers as without it), larger lgwin values double the ring buffer
|
||||
for each extra bit, and lgwin above 24 adds the composite hashers' 64 MiB
|
||||
rolling hash table (at quality 4 and at qualities 5 to 9 with lgwin of at
|
||||
least 19, only for a size hint of at least 1 MiB, or an unset one, as
|
||||
`ChooseHasher` otherwise keeps H3, H5 or H58):
|
||||
|
||||
| quality | lgwin 25 | lgwin 26 | lgwin 28 | lgwin 30 |
|
||||
| ------- | -------- | -------- | -------- | -------- |
|
||||
| 3 | 160.6 | 224.6 | 608.6 | 2144.6 |
|
||||
| 4 | 199.4 | 263.4 | 647.4 | 2183.4 |
|
||||
| 5 | 197.5 | 261.5 | 645.5 | 2181.5 |
|
||||
| 7 | 203.5 | 267.5 | 651.5 | 2187.5 |
|
||||
| 9 | 229.2 | 293.2 | 677.2 | 2213.2 |
|
||||
|
||||
At qualities 10 and 11, `workbuf_len` is a range instead (see the next
|
||||
section).
|
||||
|
||||
@@ -413,11 +454,19 @@ copied into the ring buffer.
|
||||
At qualities 10 and 11, the workbuf has the same regions as at quality 9
|
||||
(with `cmds_cap` computed the same way), except that:
|
||||
|
||||
- region 3 is the H10 hasher: `4 << 17` bytes of buckets (u32le) and then
|
||||
the binary tree forest (`8 << lgwin` bytes, u32le), and
|
||||
- region 5 keeps only the literal context map, `rle_symbols` and the block
|
||||
encoders' depths and bits, followed by a variable region, which runs from
|
||||
there to the end of the workbuf (however long that is).
|
||||
- region 2 is the H10 hasher: `4 << 17` bytes of buckets (u32le) and then
|
||||
the binary tree forest (`8 << lgwin` bytes, u32le),
|
||||
- region 4 keeps only the literal context map, `rle_symbols` and the block
|
||||
encoders' depths and bits, followed by a variable region, and
|
||||
- the ring buffer (region 5) is still the last region, after the variable
|
||||
region.
|
||||
|
||||
The variable region's length is set at the first `transform_io` call: it is
|
||||
its minimum length (see below) plus however much of the workbuf beyond
|
||||
`workbuf_len().min_incl` there is (up to the worst case), and the ring
|
||||
buffer moves along with it. Later `transform_io` calls should pass a
|
||||
workbuf of the same length. A workbuf shorter than `min_incl` follows the
|
||||
short workbuf rules above (with the minimum variable region).
|
||||
|
||||
The variable region holds, at different times, the Zopfli scratch space
|
||||
(while `EncodeData` computes the backward references: `(n + 1)` 16-byte
|
||||
@@ -431,8 +480,8 @@ The C code allocates these as it goes, for the actual input, and their worst
|
||||
cases are huge (see `workbuf_len_worst_case` below), so `workbuf_len` returns
|
||||
a range whose `max_incl` is the C code's worst case for the sanitized
|
||||
parameters (saturating at `0xFFFF_FFFF_FFFF_FFFF`) and whose `min_incl` is a
|
||||
practical bound. Both have the same fixed regions, followed by a variable
|
||||
region of `max(Z, M)` bytes, where `L` is the most literals that a metablock
|
||||
practical bound. Both have the same fixed regions, with a variable region of
|
||||
`max(Z, M)` bytes, where `L` is the most literals that a metablock
|
||||
can hold (`min(max_len, (max_len / 8) + (1 << lgblock))`, since `EncodeData`
|
||||
merges another input block only while there are fewer than `max_len / 8`
|
||||
literals) and:
|
||||
@@ -531,20 +580,73 @@ encoder makes the same `EncodeData` calls, at the same points. One
|
||||
emitting the flint, and its callers then call it again.
|
||||
|
||||
|
||||
### Encoder Large Window
|
||||
|
||||
The `QUIRK_LARGE_WINDOW` quirk mirrors `BROTLI_PARAM_LARGE_WINDOW`
|
||||
(large-window brotli is an incompatible extension of RFC 7932, decodable by
|
||||
this package's decoder with the same quirk, see "Large Window" above, and by
|
||||
libbrotlidec with `BROTLI_DECODER_PARAM_LARGE_WINDOW`). Like the C code's
|
||||
`SanitizeParams`, `EnsureInitialized` and `ChooseHasher`:
|
||||
|
||||
- At qualities 0 to 2 (the C code's `MAX_QUALITY_FOR_STATIC_ENTROPY_CODES`
|
||||
and below), the quirk is ignored: lgwin is clamped to 24 and the output is
|
||||
standard brotli.
|
||||
- At qualities 3 and above, lgwin is clamped to `[10 ..= 30]` (instead of
|
||||
`[10 ..= 24]`) and the stream starts with the 14-bit large-window header
|
||||
(`EncodeWindowBits`), even for lgwin 24 and below. The distance parameters
|
||||
(`BrotliInitDistanceParams` with `large_window`) allow distances up to
|
||||
`BROTLI_MAX_ALLOWED_DISTANCE` with an alphabet of up to
|
||||
`BROTLI_LARGE_MAX_DISTANCE_BITS` (62) distance bits, so the distance
|
||||
Huffman codes are stored with a larger alphabet size than without the
|
||||
quirk, even when no distance is large. Backward distances are at most
|
||||
`(1 << lgwin) - 16`, so their extra bits (at most 30) fit in a command's
|
||||
32-bit `dist_extra_`, as in the C code.
|
||||
- lgwin above 24 changes the hasher (only at qualities 3 to 9; the C code's
|
||||
qualities 10 and 11 use H10 regardless): H3 becomes H35 (H3 plus
|
||||
`HROLLING_FAST`), H54 becomes H55 (H54 plus `HROLLING_FAST`) and H6 and
|
||||
H68 become H65 (H6, not H68, plus `HROLLING`). H5, H58 and H40 to H42 stay
|
||||
as they are, so qualities 5 to 9 only use H65 when the size hint (set or
|
||||
estimated by `UpdateSizeHint`) is at least 1 MiB (and lgwin is at least 19,
|
||||
which it is). The rolling hashers find matches of at least 32 bytes at any
|
||||
distance in the window (`HROLLING_FAST` only at positions and distances
|
||||
that are multiples of 4) and do not search the static dictionary.
|
||||
- lgblock is unaffected (it is at most 24), but `rb_bits` (and so the ring
|
||||
buffer) grows with lgwin. Metablocks are still at most `1 << 24` bytes.
|
||||
- At qualities 10 and 11, the H10 hasher's binary tree forest grows with
|
||||
lgwin (`8 << lgwin` bytes, so 8 GiB at lgwin 30, as in the C code, which
|
||||
allocates the same), and the Zopfli and static dictionary distance limits
|
||||
use the large-window `max_distance`.
|
||||
|
||||
Positions (the C code's `size_t` and the `WrapPosition`'ed `uint32_t`
|
||||
values) and distances are `base.u64` throughout the encoder, so nothing is
|
||||
truncated for windows up to `1 << 30`. The rolling hashers' table holds
|
||||
32-bit positions, like the C code's.
|
||||
|
||||
|
||||
### Encoder Testing
|
||||
|
||||
`wuffs test -mimic std/brotli` checks byte-exact parity with libbrotlienc
|
||||
over a grid of qualities (0 to 11), lgwins, modes, input chunkings, flushes,
|
||||
metadata blocks, size hints and other parameters (lgblock, npostfix, ndirect,
|
||||
disabling literal context modeling, stream offset, large window at quality 2),
|
||||
as well as round trips through the decoder.
|
||||
disabling literal context modeling, stream offset, large window), as well as
|
||||
round trips through the decoder. Its large-window tests cover qualities 3 to
|
||||
9 with lgwin 18 to 30 (and the H35, H55 and H65 hashers) on small files and
|
||||
on a 46 MiB input with repeats at distances of about 20 and 40 MiB, decoded
|
||||
by both this package's decoder and libbrotlidec.
|
||||
`script/manual-test-brotli-encoder-parity.c` runs a larger grid over every
|
||||
file in `test/data/brotli`, some larger files in `test/data` (up to about
|
||||
400 KiB) and some synthetic inputs (up to 1 MiB). Its `-q=LO-HI` and
|
||||
`-shard=K/N` options select the qualities and split the inputs over several
|
||||
processes. With `-q=0-9` (the default) it runs 718,555 configurations
|
||||
(about 86 thousand of them at qualities 0 and 1), all byte-exact against
|
||||
libbrotlienc v1.2.1 on x86-64.
|
||||
processes, `-lw` runs only its large-window grid (lgwin 10 to 30 at
|
||||
qualities 2 to 9, with various chunkings, flushes, size hints and other
|
||||
parameters, each also with the shortest workbuf that should work) and `-big`
|
||||
replaces the inputs with synthetic ones of 3 to 70 MiB with repeats at
|
||||
distances of up to 40 MiB. With `-q=0-9` (the default) it runs 814,706
|
||||
configurations (96,688 of them in the large-window grid), all byte-exact
|
||||
against libbrotlienc v1.2.1 on x86-64. With `-lw -big` it runs 720 more
|
||||
(on 3, 6 and 12 MiB inputs with the large-window grid and on 24, 40 and
|
||||
70 MiB inputs with one-shot and streaming large-window configurations at
|
||||
lgwin 24 to 30), also all byte-exact.
|
||||
|
||||
|
||||
### Encoder Differences from the C Code
|
||||
@@ -564,6 +666,22 @@ libbrotlienc v1.2.1 on x86-64.
|
||||
`BROTLI_MAX_SIMD_QUALITY` is undefined (no 64-bit count-trailing-zeros
|
||||
builtin) or 7 (MSVC on x86-64) choose differently and so produce
|
||||
different (but valid) output at those qualities.
|
||||
- The composite hashers (H35, H55 and H65) are not separate hasher types
|
||||
here: `hasher_type` is their first hasher (H3, H54 or H6) and a non-zero
|
||||
`hr_jump` (4 for `HROLLING_FAST`, 1 for `HROLLING`) adds the rolling
|
||||
hasher, whose functions run after the first hasher's, like the C code's
|
||||
composite `FN(Prepare)`, `FN(StitchToPreviousBlock)` and
|
||||
`FN(FindLongestMatch)`. The C code's composite hasher defers its inner
|
||||
hashers' `FN(Initialize)` to its first `FN(Prepare)`. Here, the first
|
||||
hasher's `FN(Initialize)` runs in `hasher_setup` (it does not touch the
|
||||
hasher's memory, so this is equivalent) and the rolling hasher's at the
|
||||
first `hasher_prepare`. The C code's composite gives its first hasher a
|
||||
copy of its `HasherCommon` (with zeroed static dictionary statistics);
|
||||
here, the first hasher uses the encoder's statistics, which the composite
|
||||
itself never uses, so the result is the same.
|
||||
- The workbuf's regions are in a different order than the C code's
|
||||
allocations and the ring buffer is fully reserved from the start (see the
|
||||
workbuf layout above). This does not change the output.
|
||||
- `BrotliBuildMetaBlockGreedy`'s two kinds of block splitter (one for a
|
||||
single histogram per block type, `metablock_inc.h`, and one for several
|
||||
literal contexts, `ContextBlockSplitter` in `metablock.c`) are one
|
||||
|
||||
@@ -157,6 +157,12 @@ pub struct encoder? implements base.io_transformer(
|
||||
rb_cur_size : base.u32,
|
||||
rb_pos : base.u32,
|
||||
|
||||
// rb_short_workbuf is whether any transform_io call (at qualities 2
|
||||
// to 9) had a workbuf shorter than workbuf_len. If so, the ring
|
||||
// buffer's tail may be incomplete, so the input must stay short
|
||||
// enough for the ring buffer never to wrap.
|
||||
rb_short_workbuf : base.bool,
|
||||
|
||||
// The C code's BrotliDistanceParams fields (other than
|
||||
// distance_postfix_bits and num_direct_distance_codes).
|
||||
dist_alphabet_size_max : base.u32,
|
||||
@@ -178,6 +184,16 @@ pub struct encoder? implements base.io_transformer(
|
||||
dict_num_matches : base.u64,
|
||||
fc_max_hops : base.u64,
|
||||
|
||||
// The large-window composite hashers' rolling hasher (the C code's
|
||||
// HashRolling). See encode_hasher_composite.wuffs. hr_jump is 0 when
|
||||
// the hasher is not a composite one.
|
||||
hr_jump : base.u32[..= 4],
|
||||
hr_fresh : base.bool,
|
||||
hr_state : base.u32,
|
||||
hr_factor_remove : base.u32,
|
||||
hr_next_ix : base.u64,
|
||||
hr_off : base.u64,
|
||||
|
||||
// The C code's HasherSearchResult.
|
||||
sr_len : base.u64,
|
||||
sr_distance : base.u64,
|
||||
@@ -205,6 +221,8 @@ pub struct encoder? implements base.io_transformer(
|
||||
// and the StartPosQueue's idx_ and the PosData being pushed.
|
||||
literal_context_mode : base.u32,
|
||||
var_off : base.u64,
|
||||
var_end : base.u64,
|
||||
hq_layout_fixed : base.bool,
|
||||
hq_insufficient : base.bool,
|
||||
hq_wb_high : base.u64,
|
||||
h10_window_mask : base.u64,
|
||||
@@ -464,10 +482,10 @@ pub func encoder.workbuf_len() base.range_ii_u64 {
|
||||
lgwin = this.sanitized_lgwin(quality: q)
|
||||
if q <= 1 {
|
||||
n = ((2 as base.u64) << lgwin) + 503 + STORAGE_SLACK
|
||||
} else if (q <= 9) and ((q == 2) or (this.p_large_window == 0)) {
|
||||
} else if q <= 9 {
|
||||
n = this.compute_layout_offset(quality: q, lgwin: lgwin,
|
||||
lgblock: this.computed_lgblock(quality: q, lgwin: lgwin), idx: LAYOUT_END)
|
||||
} else if (q >= 10) and (this.p_large_window == 0) {
|
||||
} else {
|
||||
return this.workbuf_len_hq(quality: q, lgwin: lgwin)
|
||||
}
|
||||
return this.util.make_range_ii_u64(min_incl: n, max_incl: n)
|
||||
@@ -847,13 +865,18 @@ pub func encoder.transform_io?(dst: base.io_writer, src: base.io_reader, workbuf
|
||||
var injected : base.bool
|
||||
|
||||
var copy_input_size : base.u64
|
||||
var total : base.u64
|
||||
|
||||
this.ensure_initialized!()
|
||||
if this.large_window and (this.quality > 2) {
|
||||
// TODO: implement large-window brotli at qualities 3 and above.
|
||||
return "#unsupported encoder quality"
|
||||
} else if (this.quality >= 2) and (args.workbuf.length() < this.workbuf_end) {
|
||||
return base."#bad workbuf length"
|
||||
this.hq_extend_var_region!(wb_len: args.workbuf.length())
|
||||
if (this.quality >= 2) and (args.workbuf.length() < this.workbuf_end) {
|
||||
// A workbuf shorter than workbuf_len works if it is long enough for
|
||||
// the input so far (see the rb_short_workbuf check below), as long
|
||||
// as the ring buffer never wraps.
|
||||
this.rb_short_workbuf = true
|
||||
if args.workbuf.length() < (this.rb_buf_off ~mod+ RB_SHORT_SLACK) {
|
||||
return base."#bad workbuf length"
|
||||
}
|
||||
}
|
||||
|
||||
while true {
|
||||
@@ -953,6 +976,16 @@ pub func encoder.transform_io?(dst: base.io_writer, src: base.io_reader, workbuf
|
||||
|
||||
if (remaining <> 0) and (args.src.length() > 0) {
|
||||
copy_input_size = remaining.min(no_more_than: args.src.length())
|
||||
if this.rb_short_workbuf {
|
||||
// The ring buffer must not wrap (or reach its tail) and
|
||||
// the workbuf must hold the input so far plus some
|
||||
// slack.
|
||||
total = (this.input_pos ~mod+ copy_input_size) ~mod+ RB_SHORT_SLACK
|
||||
if (total > (this.rb_size as base.u64)) or
|
||||
((this.rb_buf_off ~mod+ total) > args.workbuf.length()) {
|
||||
return base."#bad workbuf length"
|
||||
}
|
||||
}
|
||||
mark = args.src.mark()
|
||||
args.src.skip?(n: copy_input_size)
|
||||
this.copy_input_to_ring_buffer!(wb: args.workbuf, bytes: args.src.since(mark: mark))
|
||||
|
||||
@@ -17,30 +17,37 @@
|
||||
// The workbuf holds, in this order (see README.md's "Encoder Workbuf Layout"
|
||||
// section, which compute_layout_offset mirrors):
|
||||
// - the storage (the C code's GetBrotliStorage) for one metablock,
|
||||
// - the ring buffer (the C code's RingBuffer.data_),
|
||||
// - the hasher's memory (the C code's common->extra),
|
||||
// - the commands (16 bytes each), and
|
||||
// - the commands (16 bytes each),
|
||||
// - at qualities 4 and above, the metablock split (see
|
||||
// encode_metablock_greedy.wuffs), the literal context map, the
|
||||
// EncodeContextMap symbols and the block encoders' depths and bits.
|
||||
// EncodeContextMap symbols and the block encoders' depths and bits, and
|
||||
// - the ring buffer (the C code's RingBuffer.data_).
|
||||
|
||||
// The workbuf layout's region indexes. LAYOUT_END's offset is the total
|
||||
// length.
|
||||
// The ring buffer is last, so that a workbuf shorter than workbuf_len works
|
||||
// for short inputs (see transform_io's rb_short_workbuf check).
|
||||
pri const LAYOUT_STORAGE : base.u32 = 0
|
||||
pri const LAYOUT_RB : base.u32 = 1
|
||||
pri const LAYOUT_HASH : base.u32 = 2
|
||||
pri const LAYOUT_CMDS : base.u32 = 3
|
||||
pri const LAYOUT_TYPES_0 : base.u32 = 4
|
||||
pri const LAYOUT_LENGTHS_0 : base.u32 = 7
|
||||
pri const LAYOUT_HISTO_0 : base.u32 = 10
|
||||
pri const LAYOUT_COMB : base.u32 = 13
|
||||
pri const LAYOUT_CMAP : base.u32 = 14
|
||||
pri const LAYOUT_RLE : base.u32 = 15
|
||||
pri const LAYOUT_BE_DEPTH_0 : base.u32 = 16
|
||||
pri const LAYOUT_BE_BITS_0 : base.u32 = 19
|
||||
pri const LAYOUT_VAR : base.u32 = 22
|
||||
pri const LAYOUT_HASH : base.u32 = 1
|
||||
pri const LAYOUT_CMDS : base.u32 = 2
|
||||
pri const LAYOUT_TYPES_0 : base.u32 = 3
|
||||
pri const LAYOUT_LENGTHS_0 : base.u32 = 6
|
||||
pri const LAYOUT_HISTO_0 : base.u32 = 9
|
||||
pri const LAYOUT_COMB : base.u32 = 12
|
||||
pri const LAYOUT_CMAP : base.u32 = 13
|
||||
pri const LAYOUT_RLE : base.u32 = 14
|
||||
pri const LAYOUT_BE_DEPTH_0 : base.u32 = 15
|
||||
pri const LAYOUT_BE_BITS_0 : base.u32 = 18
|
||||
pri const LAYOUT_VAR : base.u32 = 21
|
||||
pri const LAYOUT_RB : base.u32 = 22
|
||||
pri const LAYOUT_END : base.u32 = 23
|
||||
|
||||
// RB_SHORT_SLACK is how many bytes past the input's end (in the ring buffer)
|
||||
// a short workbuf must have: the C code reads at most 7 bytes past the
|
||||
// input's end (its kSlackForEightByteHashingEverywhere), rounded up.
|
||||
pri const RB_SHORT_SLACK : base.u64 = 16
|
||||
|
||||
// The C code's MAX_NUM_DELAYED_SYMBOLS (in quality.h).
|
||||
pri const MAX_NUM_DELAYED_SYMBOLS : base.u64 = 0x2FFF
|
||||
|
||||
@@ -91,11 +98,13 @@ pri func encoder.max_metablock_size(lgwin: base.u32[..= 30], lgblock: base.u32[.
|
||||
|
||||
// hasher_total_size returns the sum of FN(HashMemAllocInBytes)'s
|
||||
// alloc_size[i] for the hasher that ChooseHasher picks for the given
|
||||
// quality, lgwin and size_hint.
|
||||
// quality, lgwin and size_hint (including, for lgwin above 24, the
|
||||
// large-window composite hashers' rolling hasher table).
|
||||
pri func encoder.hasher_total_size(quality: base.u32[..= 11], lgwin: base.u32[..= 30], size_hint: base.u32) base.u64 {
|
||||
var htype : base.u32
|
||||
var bucket_bits : base.u32[..= 24]
|
||||
var block_bits : base.u32[..= 16]
|
||||
var rolling : base.u64
|
||||
|
||||
if args.quality >= 10 {
|
||||
return (4 * H10_BUCKET_SIZE) + this.h10_alloc_size_1(lgwin: args.lgwin)
|
||||
@@ -126,9 +135,21 @@ pri func encoder.hasher_total_size(quality: base.u32[..= 11], lgwin: base.u32[..
|
||||
}
|
||||
block_bits = args.quality - 1
|
||||
}
|
||||
return (this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 0) ~mod+
|
||||
rolling = 0
|
||||
if args.lgwin > 24 {
|
||||
// The large-window composite hashers H35, H55 and H65 (see
|
||||
// choose_hasher): HASHER_A plus the rolling hasher's table.
|
||||
if (htype == 3) or (htype == 54) {
|
||||
rolling = this.hasher_rolling_alloc_size()
|
||||
} else if (htype == 6) or (htype == 68) {
|
||||
htype = 6
|
||||
rolling = this.hasher_rolling_alloc_size()
|
||||
}
|
||||
}
|
||||
return ((this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 0) ~mod+
|
||||
this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 1)) ~mod+
|
||||
this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 2)
|
||||
this.hasher_alloc_size(htype: htype, bucket_bits: bucket_bits, block_bits: block_bits, i: 2)) ~mod+
|
||||
rolling
|
||||
}
|
||||
|
||||
// hash_region_size returns the workbuf length reserved for the hasher's
|
||||
@@ -279,6 +300,7 @@ pri func encoder.compute_layout!() {
|
||||
this.mb_cmap_off = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_CMAP)
|
||||
this.mb_rle_off = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_RLE)
|
||||
this.var_off = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_VAR)
|
||||
this.var_end = this.rb_off
|
||||
this.workbuf_end = this.compute_layout_offset(quality: q, lgwin: lw, lgblock: lb, idx: LAYOUT_END)
|
||||
|
||||
// RingBufferSetup.
|
||||
|
||||
@@ -102,7 +102,7 @@ pri func encoder.fc_bank_bits() base.u32[..= 16] {
|
||||
}
|
||||
|
||||
// Mirrors ChooseHasher in quality.h (with BROTLI_MAX_SIMD_QUALITY defined as
|
||||
// 6), for qualities 2 to 9 and lgwin up to 24.
|
||||
// 6), for qualities 2 to 9.
|
||||
pri func encoder.choose_hasher!() {
|
||||
var q : base.u32[..= 11]
|
||||
|
||||
@@ -149,6 +149,25 @@ pri func encoder.choose_hasher!() {
|
||||
this.h_num_last_distances = 10
|
||||
}
|
||||
}
|
||||
|
||||
this.hr_jump = 0
|
||||
if this.lgwin > 24 {
|
||||
// Different hashers for large window brotli: not for qualities <= 2,
|
||||
// these are too fast for large window. Not for qualities >= 10:
|
||||
// their hasher already works well with large window. So the changes
|
||||
// are:
|
||||
// H3 --> H35: for quality 3.
|
||||
// H54 --> H55: for quality 4 with size hint > 1MB
|
||||
// H6/H68 --> H65: for qualities 5, 6, 7, 8, 9.
|
||||
// See encode_hasher_composite.wuffs: the composite hasher's
|
||||
// HASHER_A stays in this.hasher_type.
|
||||
if (this.hasher_type == 3) or (this.hasher_type == 54) {
|
||||
this.hr_jump = 4
|
||||
} else if (this.hasher_type == 6) or (this.hasher_type == 68) {
|
||||
this.hasher_type = 6
|
||||
this.hr_jump = 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// hasher_alloc_size mirrors FN(HashMemAllocInBytes) (alloc_size[i], for i in
|
||||
@@ -226,6 +245,9 @@ pri func encoder.hasher_setup!(wb: slice base.u8, position: base.u64, input_size
|
||||
htype: this.hasher_type, bucket_bits: this.h_bucket_bits, block_bits: this.h_block_bits, i: 0)
|
||||
this.h_off2 = this.h_off1 ~mod+ this.hasher_alloc_size(
|
||||
htype: this.hasher_type, bucket_bits: this.h_bucket_bits, block_bits: this.h_block_bits, i: 1)
|
||||
// The composite hashers' HASHER_B memory follows HASHER_A's.
|
||||
this.hr_off = this.h_off2
|
||||
this.hr_fresh = true
|
||||
this.h_is_prepared = false
|
||||
this.h_is_setup = true
|
||||
}
|
||||
@@ -314,6 +336,18 @@ pri func encoder.hasher_prepare!(wb: slice base.u8, one_shot: base.bool, input_s
|
||||
j ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
if this.hr_jump <> 0 {
|
||||
// The composite hasher's FN(Prepare) (in hash_composite_inc.h), for
|
||||
// HASHER_B. HASHER_B's FN(Initialize) is deferred to the first
|
||||
// FN(Prepare). HASHER_A's FN(Initialize) does not touch the hasher
|
||||
// memory, so doing it earlier (in hasher_setup) is equivalent.
|
||||
if this.hr_fresh {
|
||||
this.hr_fresh = false
|
||||
this.hasher_rolling_initialize!(wb: args.wb)
|
||||
}
|
||||
this.hasher_rolling_prepare!(wb: args.wb, input_size: args.input_size, data_off: this.rb_buf_off)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors FN(HashBytes) in hash_longest_match_quickly_inc.h.
|
||||
@@ -449,6 +483,10 @@ pri func encoder.hasher_stitch_to_previous_block!(wb: slice base.u8, num_bytes:
|
||||
this.hasher_store!(wb: args.wb, ix: args.position ~mod- 2)
|
||||
this.hasher_store!(wb: args.wb, ix: args.position ~mod- 1)
|
||||
}
|
||||
if this.hr_jump <> 0 {
|
||||
// The composite hasher's HASHER_B.
|
||||
this.hasher_rolling_stitch_to_previous_block!(wb: args.wb, num_bytes: args.num_bytes, position: args.position)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors PrepareDistanceCache in hash.h, via each hasher's
|
||||
@@ -614,6 +652,11 @@ pri func encoder.find_longest_match!(wb: slice base.u8, cur_ix: base.u64, max_le
|
||||
this.find_longest_match_forgetful!(wb: args.wb, cur_ix: args.cur_ix, max_length: args.max_length,
|
||||
max_backward: args.max_backward, dictionary_distance: args.dictionary_distance, max_distance: args.max_distance)
|
||||
}
|
||||
if this.hr_jump <> 0 {
|
||||
// The composite hasher's HASHER_B.
|
||||
this.hasher_rolling_find_longest_match!(wb: args.wb, cur_ix: args.cur_ix, max_length: args.max_length,
|
||||
max_backward: args.max_backward)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors FN(FindLongestMatch) in hash_longest_match_quickly_inc.h.
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// This file mirrors the C code's large-window hashers: the composite hashers
|
||||
// (hash_composite_inc.h) H35 (H3 + HROLLING_FAST), H55 (H54 + HROLLING_FAST)
|
||||
// and H65 (H6 + HROLLING), and the rolling hashers (hash_rolling_inc.h)
|
||||
// HROLLING_FAST (JUMP 4) and HROLLING (JUMP 1), both with CHUNKLEN 32,
|
||||
// NUMBUCKETS (1 << 24) and MASK ((NUMBUCKETS * 64) - 1).
|
||||
//
|
||||
// A composite hasher is represented by its first hasher (HASHER_A, H3, H54 or
|
||||
// H6) in this.hasher_type and its rolling hasher (HASHER_B) by a non-zero
|
||||
// this.hr_jump (4 for HROLLING_FAST, 1 for HROLLING). The composite's FN(etc)
|
||||
// functions call HASHER_A's and then HASHER_B's, so the encode_hasher.wuffs
|
||||
// functions (which implement HASHER_A) call the hasher_rolling_etc functions
|
||||
// below after their own work when this.hr_jump is non-zero.
|
||||
//
|
||||
// The composite hasher's memory is HASHER_A's alloc_size[0] and [1] followed
|
||||
// by HASHER_B's alloc_size[0] (the rolling table of NUMBUCKETS u32le values),
|
||||
// at the this.hr_off workbuf offset. HASHER_A's HasherCommon is a copy of the
|
||||
// composite's (the C code's ha_common), whose dict_num_lookups and
|
||||
// dict_num_matches start at zero, like the composite's own, so HASHER_A can
|
||||
// keep using this.dict_num_lookups and this.dict_num_matches.
|
||||
|
||||
// Mirrors CHUNKLEN in hash.h, for the rolling hashers.
|
||||
pri const ROLLING_CHUNKLEN : base.u64 = 32
|
||||
|
||||
// Mirrors NUMBUCKETS in hash.h, for the rolling hashers.
|
||||
pri const ROLLING_NUMBUCKETS : base.u32 = 0x100_0000
|
||||
|
||||
// Mirrors MASK in hash.h, for the rolling hashers.
|
||||
pri const ROLLING_MASK : base.u32 = 0x3FFF_FFFF
|
||||
|
||||
// Mirrors FN(kRollingHashMul32) in hash_rolling_inc.h.
|
||||
pri const ROLLING_HASH_MUL_32 : base.u32 = 69069
|
||||
|
||||
// Mirrors FN(kInvalidPos) in hash_rolling_inc.h.
|
||||
pri const ROLLING_INVALID_POS : base.u32 = 0xFFFF_FFFF
|
||||
|
||||
// hasher_rolling_alloc_size mirrors FN(HashMemAllocInBytes) in
|
||||
// hash_rolling_inc.h: alloc_size[0], the table.
|
||||
pri func encoder.hasher_rolling_alloc_size() base.u64 {
|
||||
return 4 * (ROLLING_NUMBUCKETS as base.u64)
|
||||
}
|
||||
|
||||
// Mirrors FN(Initialize) in hash_rolling_inc.h. The C code's composite
|
||||
// hashers call it (via their deferred initialization) in their first
|
||||
// FN(Prepare), which HasherSetup calls right after FN(Initialize).
|
||||
pri func encoder.hasher_rolling_initialize!(wb: slice base.u8) {
|
||||
var i : base.u64
|
||||
|
||||
this.hr_state = 0
|
||||
this.hr_next_ix = 0
|
||||
|
||||
// Compute the factor of the oldest byte to remove: factor**steps modulo
|
||||
// 0xffffffff (the multiplications rely on 32-bit overflow).
|
||||
this.hr_factor_remove = 1
|
||||
i = 0
|
||||
while i < ROLLING_CHUNKLEN {
|
||||
this.hr_factor_remove ~mod*= ROLLING_HASH_MUL_32
|
||||
i ~mod+= this.hr_jump as base.u64
|
||||
}
|
||||
|
||||
// Every table entry is ROLLING_INVALID_POS.
|
||||
this.fill_bytes!(s: args.wb, i: this.hr_off, n: this.hasher_rolling_alloc_size(), v: 0xFF)
|
||||
}
|
||||
|
||||
// Mirrors FN(Prepare) in hash_rolling_inc.h, where data is the workbuf's
|
||||
// bytes starting at offset data_off.
|
||||
pri func encoder.hasher_rolling_prepare!(wb: slice base.u8, input_size: base.u64, data_off: base.u64) {
|
||||
var i : base.u64
|
||||
|
||||
// Too small size, cannot use this hasher.
|
||||
if args.input_size < ROLLING_CHUNKLEN {
|
||||
return nothing
|
||||
}
|
||||
this.hr_state = 0
|
||||
i = 0
|
||||
while i < ROLLING_CHUNKLEN {
|
||||
// FN(HashRollingFunctionInitial).
|
||||
this.hr_state = (ROLLING_HASH_MUL_32 ~mod* this.hr_state) ~mod+
|
||||
((this.byte_at(s: args.wb, i: args.data_off ~mod+ i) as base.u32) + 1)
|
||||
i ~mod+= this.hr_jump as base.u64
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors FN(StitchToPreviousBlock) in hash_rolling_inc.h.
|
||||
pri func encoder.hasher_rolling_stitch_to_previous_block!(wb: slice base.u8, num_bytes: base.u64, position: base.u64) {
|
||||
var position : base.u64
|
||||
var position_masked : base.u64
|
||||
var available : base.u64
|
||||
var jm1 : base.u64
|
||||
var diff : base.u64
|
||||
var mask : base.u64
|
||||
|
||||
// In this case we must re-initialize the hasher from scratch from the
|
||||
// current position.
|
||||
position = args.position
|
||||
available = args.num_bytes
|
||||
jm1 = (this.hr_jump as base.u64) ~mod- 1
|
||||
if (position & jm1) <> 0 {
|
||||
diff = (this.hr_jump as base.u64) ~mod- (position & jm1)
|
||||
if diff > available {
|
||||
available = 0
|
||||
} else {
|
||||
available ~mod-= diff
|
||||
}
|
||||
position ~mod+= diff
|
||||
}
|
||||
mask = this.rb_mask as base.u64
|
||||
position_masked = position & mask
|
||||
// Wrapping around ringbuffer not handled.
|
||||
if available > (mask ~mod- position_masked) {
|
||||
available = mask ~mod- position_masked
|
||||
}
|
||||
this.hasher_rolling_prepare!(wb: args.wb, input_size: available,
|
||||
data_off: this.rb_buf_off ~mod+ position_masked)
|
||||
this.hr_next_ix = position
|
||||
}
|
||||
|
||||
// Mirrors FN(FindLongestMatch) in hash_rolling_inc.h. The search result is in
|
||||
// the this.sr_etc fields.
|
||||
pri func encoder.hasher_rolling_find_longest_match!(wb: slice base.u8, cur_ix: base.u64, max_length: base.u64, max_backward: base.u64) {
|
||||
var data : roslice base.u8
|
||||
var mask : base.u64
|
||||
var cur_ix_masked : base.u64
|
||||
var jump : base.u64
|
||||
var pos : base.u64
|
||||
var code : base.u32
|
||||
var rem : base.u8
|
||||
var add : base.u8
|
||||
var found_ix : base.u64
|
||||
var backward : base.u64
|
||||
var found_ix_masked : base.u64
|
||||
var len : base.u64
|
||||
var score : base.u64
|
||||
var t : base.u64
|
||||
|
||||
if this.rb_buf_off > args.wb.length() {
|
||||
return nothing
|
||||
}
|
||||
data = args.wb[this.rb_buf_off ..]
|
||||
mask = this.rb_mask as base.u64
|
||||
cur_ix_masked = args.cur_ix & mask
|
||||
jump = this.hr_jump as base.u64
|
||||
|
||||
if (args.cur_ix & (jump ~mod- 1)) <> 0 {
|
||||
return nothing
|
||||
}
|
||||
|
||||
// Not enough lookahead.
|
||||
if args.max_length < ROLLING_CHUNKLEN {
|
||||
return nothing
|
||||
}
|
||||
|
||||
pos = this.hr_next_ix
|
||||
while pos <= args.cur_ix {
|
||||
code = this.hr_state & ROLLING_MASK
|
||||
|
||||
rem = this.byte_at(s: data, i: pos & mask)
|
||||
add = this.byte_at(s: data, i: (pos ~mod+ ROLLING_CHUNKLEN) & mask)
|
||||
found_ix = ROLLING_INVALID_POS as base.u64
|
||||
|
||||
// FN(HashRollingFunction).
|
||||
this.hr_state = ((ROLLING_HASH_MUL_32 ~mod* this.hr_state) ~mod+ ((add as base.u32) + 1)) ~mod-
|
||||
(this.hr_factor_remove ~mod* ((rem as base.u32) + 1))
|
||||
|
||||
if code < ROLLING_NUMBUCKETS {
|
||||
t = this.hr_off ~mod+ (4 * (code as base.u64))
|
||||
found_ix = this.load32le(s: args.wb, i: t) as base.u64
|
||||
this.poke_u32!(s: args.wb, i: t, v: (pos & 0xFFFF_FFFF) as base.u32)
|
||||
if (pos == args.cur_ix) and (found_ix <> (ROLLING_INVALID_POS as base.u64)) {
|
||||
// The cast to 32-bit makes backward distances up to 4GB work
|
||||
// even if cur_ix is above 4GB, despite using 32-bit values in
|
||||
// the table.
|
||||
backward = (args.cur_ix ~mod- found_ix) & 0xFFFF_FFFF
|
||||
if backward <= args.max_backward {
|
||||
found_ix_masked = found_ix & mask
|
||||
len = this.find_match_length_with_limit(s: data,
|
||||
i1: found_ix_masked, i2: cur_ix_masked, limit: args.max_length)
|
||||
if (len >= 4) and (len > this.sr_len) {
|
||||
score = this.backward_reference_score(copy_length: len, backward_reference_offset: backward)
|
||||
if score > this.sr_score {
|
||||
this.sr_len = len
|
||||
this.sr_distance = backward
|
||||
this.sr_score = score
|
||||
this.sr_len_code_delta = 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
pos ~mod+= jump
|
||||
}
|
||||
|
||||
this.hr_next_ix = args.cur_ix ~mod+ jump
|
||||
|
||||
// NOTE: this hasher does not search in the dictionary. It is used as
|
||||
// backup-hasher, the main hasher already searches in it.
|
||||
}
|
||||
@@ -105,9 +105,11 @@ pri func encoder.write_meta_block_hq!(wb: slice base.u8, wrapped_last_flush_pos:
|
||||
var cmds_hi : base.u64
|
||||
var c : base.u32
|
||||
var ds : base.u32[..= 704]
|
||||
var var_hi : base.u64
|
||||
|
||||
cmds_hi = this.cmds_off ~mod+ (this.num_cmds ~mod* 16)
|
||||
if (this.var_off > args.wb.length()) or (this.rb_buf_off > args.wb.length()) or
|
||||
var_hi = this.var_end.min(no_more_than: args.wb.length())
|
||||
if (this.var_off > var_hi) or (this.rb_buf_off > args.wb.length()) or
|
||||
(this.cmds_off > cmds_hi) or (cmds_hi > args.wb.length()) {
|
||||
this.hq_insufficient = true
|
||||
return nothing
|
||||
@@ -117,7 +119,7 @@ pri func encoder.write_meta_block_hq!(wb: slice base.u8, wrapped_last_flush_pos:
|
||||
disable_literal_context_modeling: this.p_disable_lcm <> 0)
|
||||
this.mb.init_distance_params!(npostfix: this.distance_postfix_bits,
|
||||
ndirect: this.num_direct_distance_codes, large_window: this.large_window)
|
||||
status = this.mb.build_meta_block!(wb: args.wb[this.var_off ..], ringbuffer: args.wb[this.rb_buf_off ..],
|
||||
status = this.mb.build_meta_block!(wb: args.wb[this.var_off .. var_hi], ringbuffer: args.wb[this.rb_buf_off ..],
|
||||
pos: args.wrapped_last_flush_pos, mask: this.rb_mask as base.u64,
|
||||
prev_byte: args.prev_byte, prev_byte2: args.prev_byte2,
|
||||
cmds: args.wb[this.cmds_off .. cmds_hi], num_commands: this.num_cmds,
|
||||
@@ -131,7 +133,7 @@ pri func encoder.write_meta_block_hq!(wb: slice base.u8, wrapped_last_flush_pos:
|
||||
}
|
||||
// MIN_QUALITY_FOR_OPTIMIZE_HISTOGRAMS is 4. The number of distance
|
||||
// symbols effectively used for distance histograms.
|
||||
this.mb.optimize_histograms!(wb: args.wb[this.var_off ..], num_distance_codes: this.mb.dist_alphabet_size_limit)
|
||||
this.mb.optimize_histograms!(wb: args.wb[this.var_off .. var_hi], num_distance_codes: this.mb.dist_alphabet_size_limit)
|
||||
|
||||
// Point the block encoders' inputs (the gbs_etc fields, which
|
||||
// build_meta_block_greedy sets at qualities 4 to 9) at the
|
||||
@@ -328,11 +330,51 @@ pri func encoder.store_meta_block_hq!(wb: slice base.u8, start_pos: base.u64, le
|
||||
// fixed regions plus the minimum or the worst case variable region.
|
||||
pri func encoder.workbuf_len_hq(quality: base.u32[..= 11], lgwin: base.u32[..= 30]) base.range_ii_u64 {
|
||||
var lgblock : base.u32[..= 30]
|
||||
var off : base.u64
|
||||
var n : base.u64
|
||||
|
||||
lgblock = this.computed_lgblock(quality: args.quality, lgwin: args.lgwin)
|
||||
off = this.compute_layout_offset(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, idx: LAYOUT_VAR)
|
||||
n = this.compute_layout_offset(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, idx: LAYOUT_END)
|
||||
return this.util.make_range_ii_u64(
|
||||
min_incl: off ~sat+ this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, worst_case: false),
|
||||
max_incl: off ~sat+ this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock, worst_case: true))
|
||||
min_incl: n,
|
||||
max_incl: n ~sat+ this.hq_var_extra_max(quality: args.quality, lgwin: args.lgwin, lgblock: lgblock))
|
||||
}
|
||||
|
||||
// hq_var_extra_max returns how much longer than its minimum length (the
|
||||
// LAYOUT_VAR region's length) the variable region can usefully be: the
|
||||
// difference between its worst case and minimum lengths.
|
||||
pri func encoder.hq_var_extra_max(quality: base.u32[..= 11], lgwin: base.u32[..= 30], lgblock: base.u32[..= 30]) base.u64 {
|
||||
var lo : base.u64
|
||||
var hi : base.u64
|
||||
|
||||
lo = this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: args.lgblock, worst_case: false)
|
||||
hi = this.hq_var_len(quality: args.quality, lgwin: args.lgwin, lgblock: args.lgblock, worst_case: true)
|
||||
if hi <= lo {
|
||||
return 0
|
||||
}
|
||||
return hi - lo
|
||||
}
|
||||
|
||||
// hq_extend_var_region grows the variable region (and moves the ring buffer,
|
||||
// which is the last region, along with this.workbuf_end) to use as much of a
|
||||
// workbuf of wb_len bytes as the worst case can need. It only does this
|
||||
// once, at the first transform_io call (before the ring buffer is used), at
|
||||
// qualities 10 and 11. Later calls should pass a workbuf of the same length
|
||||
// (a shorter one is handled like any short workbuf, and the variable region
|
||||
// is then cut short).
|
||||
pri func encoder.hq_extend_var_region!(wb_len: base.u64) {
|
||||
var extra : base.u64
|
||||
|
||||
if (this.quality < 10) or this.hq_layout_fixed {
|
||||
return nothing
|
||||
}
|
||||
this.hq_layout_fixed = true
|
||||
if args.wb_len <= this.workbuf_end {
|
||||
return nothing
|
||||
}
|
||||
extra = this.hq_var_extra_max(quality: this.quality, lgwin: this.lgwin, lgblock: this.lgblock)
|
||||
extra = extra.min(no_more_than: args.wb_len - this.workbuf_end)
|
||||
this.var_end ~mod+= extra
|
||||
this.rb_off ~mod+= extra
|
||||
this.rb_buf_off ~mod+= extra
|
||||
this.workbuf_end ~mod+= extra
|
||||
}
|
||||
@@ -1007,7 +1007,7 @@ pri func encoder.create_zopfli_backward_references!(wb: slice base.u8, num_bytes
|
||||
var n : base.u64
|
||||
var ok_offsets : base.bool
|
||||
|
||||
ok_offsets = this.z_set_offsets!(wb_len: args.wb.length(), num_bytes: args.num_bytes)
|
||||
ok_offsets = this.z_set_offsets!(wb_len: this.var_end.min(no_more_than: args.wb.length()), num_bytes: args.num_bytes)
|
||||
if not ok_offsets {
|
||||
return base."#bad workbuf length"
|
||||
}
|
||||
@@ -1047,7 +1047,7 @@ pri func encoder.create_hq_zopfli_backward_references!(wb: slice base.u8, num_by
|
||||
var n : base.u64
|
||||
var ok_offsets : base.bool
|
||||
|
||||
ok_offsets = this.z_set_offsets!(wb_len: args.wb.length(), num_bytes: args.num_bytes)
|
||||
ok_offsets = this.z_set_offsets!(wb_len: this.var_end.min(no_more_than: args.wb.length()), num_bytes: args.num_bytes)
|
||||
if not ok_offsets {
|
||||
return base."#bad workbuf length"
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user