std/brotli: add an encoder for quality levels 0 and 1

Adds a brotli encoder (std/brotli's encoder struct, a base.io_transformer)
whose output is byte-identical to libbrotlienc v1.2.1 for quality levels 0
and 1, mirroring encode.c's BrotliEncoderCompressStream(Fast),
compress_fragment.c and compress_fragment_two_pass.c, plus the entropy
coding and bit stream helpers they need. Every double operation goes
through std/softfloat, following the C evaluation order exactly.

Parameters are quirks (encode_quirks.wuffs) that mirror
BrotliEncoderSetParameter, sanitized at the first transform_io call.
Each transform_io resumption corresponds to one C call with available_in
equal to the readable src length; a closed src is FINISH; request_flush!
and request_emit_metadata! make the next resumption a FLUSH or
EMIT_METADATA call. Qualities 2 to 11 return "#unsupported encoder
quality" for now. Encoder tables are generated by
script/print-brotli-tables.go -encoder.

Tests: round trips through the Wuffs decoder over the corpus and synthetic
inputs; with -mimic, byte-exact comparisons against libbrotlienc over
quality × window × mode × input chunking × flush schedule × metadata ×
size hint. script/manual-test-brotli-encoder-parity.c runs the full
grid: 86,268 configurations, zero mismatches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
This commit is contained in:
Claude committed 2026-10-26 23:55:56 +00:00
1 parent 4978207531
commit ca1ba9d134
17 files changed
+13990 -2772

No files matched your search

+172
View File
@@ -152,3 +152,175 @@ invariants for memory safety but Wuffs requires explicit bounds checks.
- Fast paths.
- Fuzzers.
- An encoder.
## Encoder
This package also has an `encoder`, which implements the `base.io_transformer`
interface. It aims for byte-exact parity with the reference implementation's
encoder (libbrotlienc), version v1.2.1: for the same parameters and the same
sequence of operations, it produces exactly the same bytes. Like the decoder,
the Wuffs code mirrors the C code function by function, with comments like
"Mirrors BrotliCompressFragmentFast in compress_fragment.c". Its floating
point arithmetic (the C code's `double`s) uses `std/softfloat`, so that the
results are bit-exact regardless of the compiler and CPU.
Only qualities 0 and 1 (the C code's `FAST_ONE_PASS_COMPRESSION_QUALITY` and
`FAST_TWO_PASS_COMPRESSION_QUALITY`, implemented by `compress_fragment.c` and
`compress_fragment_two_pass.c`) are implemented. For qualities 2 to 11
(including the default quality, 11), `transform_io` returns
`"#unsupported encoder quality"`.
The source files are:
- `encode_brotli.wuffs`: the `encoder` struct, its public methods and the
`encode.c` streaming logic (`BrotliEncoderCompressStream`,
`BrotliEncoderCompressStreamFast`, `ProcessMetadata`, `EnsureInitialized`,
`InjectBytePaddingBlock`, etc).
- `encode_quirks.wuffs`: the encoder's quirk keys.
- `encode_bit_writer.wuffs`: `write_bits.h`, `fast_log.h`,
`find_match_length.h` and small `brotli_bit_stream.c` helpers.
- `encode_entropy.wuffs`: `entropy_encode.c` (Huffman tree construction) and
the `brotli_bit_stream.c` functions that store Huffman codes.
- `encode_fragment_fast.wuffs`: `compress_fragment.c` (quality 0).
- `encode_fragment_two_pass.wuffs`: `compress_fragment_two_pass.c` (quality
1).
- `common_encoder_tables.wuffs`: tables generated by `go run
script/print-brotli-tables.go -encoder` from the C code.
### Encoder API
In addition to the `base.io_transformer` methods, the encoder has:
- `request_flush!() status`: the next `transform_io` call acts like the C
code's `BROTLI_OPERATION_FLUSH` (instead of `PROCESS` or `FINISH`) until
the flush is complete. It returns `"#bad call sequence"` while a metadata
request is pending.
- `request_emit_metadata!(length: u64) status`: the next `transform_io` call
acts like `BROTLI_OPERATION_EMIT_METADATA`, with the next `length` bytes of
`src` as the metadata block's contents. `length` can be at most `1 << 24`
(otherwise `"#bad argument"`). It returns `"#bad call sequence"` while
another request is pending.
- `max_compressed_size(input_size: u64) u64`: mirrors
`BrotliEncoderMaxCompressedSize` (0 means overflow).
`dst_history_retain_length` is 0: the encoder never re-reads `dst`.
### Encoder Quirks
Each quirk mirrors one `BrotliEncoderParameter`, with the value semantics of
`BrotliEncoderSetParameter`. Values are stored as is (and must fit in a u32,
otherwise `set_quirk` returns `"#bad argument"`). Like the C code, they are
sanitized (`SanitizeParams`, `ComputeLgBlock`, `ChooseDistanceParams`) at the
first `transform_io` call, after which `set_quirk` returns
`"#bad call sequence"`.
| Quirk | C parameter |
| ---------------------------------------- | ---------------------------------------------- |
| `QUIRK_ENCODER_QUALITY` | `BROTLI_PARAM_QUALITY` (default 11) |
| `QUIRK_LGWIN` | `BROTLI_PARAM_LGWIN` (default 22) |
| `QUIRK_LGBLOCK` | `BROTLI_PARAM_LGBLOCK` |
| `QUIRK_MODE` | `BROTLI_PARAM_MODE` |
| `QUIRK_DISABLE_LITERAL_CONTEXT_MODELING` | `BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING` (0 or 1) |
| `QUIRK_SIZE_HINT` | `BROTLI_PARAM_SIZE_HINT` |
| `QUIRK_LARGE_WINDOW` (shared with the decoder) | `BROTLI_PARAM_LARGE_WINDOW` |
| `QUIRK_NPOSTFIX` | `BROTLI_PARAM_NPOSTFIX` |
| `QUIRK_NDIRECT` | `BROTLI_PARAM_NDIRECT` |
| `QUIRK_STREAM_OFFSET` | `BROTLI_PARAM_STREAM_OFFSET` (at most `1 << 30`) |
The quality quirk is not named `QUIRK_QUALITY` because that name would clash
with `base.QUIRK_QUALITY`. At qualities 0 and 1, only the quality, lgwin and
stream offset affect the output (lgblock, mode, size hint, large window,
npostfix, ndirect and literal context modeling are stored, sanitized and then
ignored, as in the C code).
### Encoder Workbuf Layout
At qualities 0 and 1, the workbuf is the C code's metablock storage (its
`GetBrotliStorage` buffer): `workbuf_len` is `(2 << lgwin) + 503 + 16` bytes
(the storage for a `1 << lgwin` byte block plus 16 bytes of slack for 8-byte
bit writes), where lgwin is the sanitized value. A shorter workbuf still works
if every input block is shorter: a block of `n` bytes needs `(2 * n) + 519`
bytes, otherwise `transform_io` returns `"#bad workbuf length"`. At other
qualities, `workbuf_len` is 0.
The hash table (up to `1 << 17` entries), the quality 1 command and literal
buffers (`1 << 17` entries each) and the Huffman scratch space are arrays in
the `encoder` struct (which is therefore more than 1 MiB). They do not need
zero-initializing: `WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED` is
fine. Unlike the C code, the input block is not copied: the encoder compresses
directly from `src`.
### Encoder Operation Mapping
Each `transform_io` call (initial or resumed) corresponds to one or more
`BrotliEncoderCompressStream` calls, with `src`'s readable bytes as the C
code's `available_in` and `dst`'s writable bytes as `available_out`. The
operation is:
- `EMIT_METADATA` if `request_emit_metadata` is pending (until the metadata
block is complete),
- otherwise `FLUSH` if `request_flush` is pending (until the flush is
complete),
- otherwise `FINISH` if `src` is closed,
- otherwise `PROCESS`.
`transform_io` suspends with `"$short write"` when there is pending output
(the C code's `BrotliEncoderHasMoreOutput`) but `dst` is full, suspends with
`"$short read"` when all of `src` is consumed (and, for a flush, the flush is
complete) and returns ok once the stream is finished (the C code's
`BrotliEncoderIsFinished`). So these two loops produce the same bytes:
- C: for each chunk, call `BrotliEncoderCompressStream(op)` until all of the
chunk is consumed and there is no more output (or, for `FINISH`, until
finished).
- Wuffs: for each chunk, call `request_flush` (for `FLUSH`) or
`request_emit_metadata(len)` (for `EMIT_METADATA`), make `src` hold exactly
that chunk (closed for `FINISH`) and call `transform_io` until it returns
`"$short read"` (or ok for `FINISH`), giving it more `dst` room after each
`"$short write"`.
At qualities 0 and 1 the C code compresses each call's available input
immediately (in blocks of at most `1 << lgwin` bytes), so the output depends
on how the input is split into chunks. The test programs' encode drivers
(`wuffs_brotli_encode_plan` in `test/c/std/brotli.c` and
`mimic_brotli_encode` in `test/c/mimiclib/brotli.c`) run the same "plan" of
(operation, length) steps with each library.
### Encoder Testing
`wuffs test -mimic std/brotli` checks byte-exact parity with libbrotlienc
over a grid of qualities, lgwins, modes, input chunkings, flushes, metadata
blocks, size hints and other parameters, as well as round trips through the
decoder. `script/manual-test-brotli-encoder-parity.c` runs a larger grid
(about 86 thousand configurations over every file in `test/data/brotli` and
some synthetic inputs).
### Encoder Differences from the C Code
- The input block is not copied (see the workbuf layout above), and output is
always staged in the workbuf (or a small internal buffer) and then copied to
`dst`. The C code sometimes writes directly to `next_out`. The bytes are the
same.
- Wuffs' C code generator does not support `slice base.u16` or
`slice base.u32` arguments (and mishandles `copy_from_slice` on `base.u16`
arrays), so some functions (e.g. `create_huffman_tree`,
`convert_bit_depths_to_symbols`) take a selector naming one of the
encoder's arrays instead of a slice.
- Like the C code, there is no fallback to an uncompressed stream (that is
only in the C code's one-shot `BrotliEncoderCompress`).
### Encoder TODO
- Qualities 2 to 11 (the C code's `EncodeData`, hashers, block splitting,
context modeling, Zopfli, etc).
- Large-window encoding (only used at qualities 3 and above).
- Custom and shared dictionaries (`BrotliEncoderAttachPreparedDictionary`).
- Changing parameters mid-stream.
+558
View File
@@ -0,0 +1,558 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// The tables in this file were generated by script/print-brotli-tables.go
// (with the -encoder flag) from the brotli (C) source code. Do not edit by
// hand. These tables are only used by the encoder.
// From enc/fast_log.c kBrotliLog2Table, as the bit patterns of IEEE 754
// binary64 values. The C literals are float literals widened to double.
pri const LOG2_TABLE : roarray[256] base.u64 = [
0x0000_0000_0000_0000, 0x0000_0000_0000_0000, 0x3FF0_0000_0000_0000, 0x3FF9_5C01_A000_0000,
0x4000_0000_0000_0000, 0x4002_934F_0000_0000, 0x4004_AE00_E000_0000, 0x4006_7576_8000_0000,
0x4008_0000_0000_0000, 0x4009_5C01_A000_0000, 0x400A_934F_0000_0000, 0x400B_ACEA_8000_0000,
0x400C_AE00_E000_0000, 0x400D_9A80_2000_0000, 0x400E_7576_8000_0000, 0x400F_414F_E000_0000,
0x4010_0000_0000_0000, 0x4010_598F_E000_0000, 0x4010_AE00_E000_0000, 0x4010_FDE0_C000_0000,
0x4011_49A7_8000_0000, 0x4011_91BB_A000_0000, 0x4011_D675_4000_0000, 0x4012_1820_A000_0000,
0x4012_5700_6000_0000, 0x4012_934F_0000_0000, 0x4012_CD40_2000_0000, 0x4013_0501_4000_0000,
0x4013_3ABB_4000_0000, 0x4013_6E92_A000_0000, 0x4013_A0A7_E000_0000, 0x4013_D118_E000_0000,
0x4014_0000_0000_0000, 0x4014_2D75_A000_0000, 0x4014_598F_E000_0000, 0x4014_8462_C000_0000,
0x4014_AE00_E000_0000, 0x4014_D67B_0000_0000, 0x4014_FDE0_C000_0000, 0x4015_2440_8000_0000,
0x4015_49A7_8000_0000, 0x4015_6E22_2000_0000, 0x4015_91BB_A000_0000, 0x4015_B47E_C000_0000,
0x4015_D675_4000_0000, 0x4015_F7A8_6000_0000, 0x4016_1820_A000_0000, 0x4016_37E6_2000_0000,
0x4016_5700_6000_0000, 0x4016_7576_8000_0000, 0x4016_934F_0000_0000, 0x4016_B090_4000_0000,
0x4016_CD40_2000_0000, 0x4016_E964_0000_0000, 0x4017_0501_4000_0000, 0x4017_201C_C000_0000,
0x4017_3ABB_4000_0000, 0x4017_54E1_2000_0000, 0x4017_6E92_A000_0000, 0x4017_87D3_A000_0000,
0x4017_A0A7_E000_0000, 0x4017_B913_4000_0000, 0x4017_D118_E000_0000, 0x4017_E8BC_2000_0000,
0x4018_0000_0000_0000, 0x4018_16E7_A000_0000, 0x4018_2D75_A000_0000, 0x4018_43AC_E000_0000,
0x4018_598F_E000_0000, 0x4018_6F21_0000_0000, 0x4018_8462_C000_0000, 0x4018_9957_4000_0000,
0x4018_AE00_E000_0000, 0x4018_C261_6000_0000, 0x4018_D67B_0000_0000, 0x4018_EA4F_8000_0000,
0x4018_FDE0_C000_0000, 0x4019_1130_8000_0000, 0x4019_2440_8000_0000, 0x4019_3712_4000_0000,
0x4019_49A7_8000_0000, 0x4019_5C01_A000_0000, 0x4019_6E22_2000_0000, 0x4019_800A_6000_0000,
0x4019_91BB_A000_0000, 0x4019_A337_6000_0000, 0x4019_B47E_C000_0000, 0x4019_C593_0000_0000,
0x4019_D675_4000_0000, 0x4019_E726_A000_0000, 0x4019_F7A8_6000_0000, 0x401A_07FB_6000_0000,
0x401A_1820_A000_0000, 0x401A_2819_4000_0000, 0x401A_37E6_2000_0000, 0x401A_4788_4000_0000,
0x401A_5700_6000_0000, 0x401A_664F_8000_0000, 0x401A_7576_8000_0000, 0x401A_8476_0000_0000,
0x401A_934F_0000_0000, 0x401A_A202_4000_0000, 0x401A_B090_4000_0000, 0x401A_BEFA_0000_0000,
0x401A_CD40_2000_0000, 0x401A_DB63_2000_0000, 0x401A_E964_0000_0000, 0x401A_F743_2000_0000,
0x401B_0501_4000_0000, 0x401B_129E_E000_0000, 0x401B_201C_C000_0000, 0x401B_2D7B_6000_0000,
0x401B_3ABB_4000_0000, 0x401B_47DD_0000_0000, 0x401B_54E1_2000_0000, 0x401B_61C8_2000_0000,
0x401B_6E92_A000_0000, 0x401B_7B40_E000_0000, 0x401B_87D3_A000_0000, 0x401B_944B_2000_0000,
0x401B_A0A7_E000_0000, 0x401B_ACEA_8000_0000, 0x401B_B913_4000_0000, 0x401B_C522_8000_0000,
0x401B_D118_E000_0000, 0x401B_DCF6_8000_0000, 0x401B_E8BC_2000_0000, 0x401B_F469_C000_0000,
0x401C_0000_0000_0000, 0x401C_0B7F_2000_0000, 0x401C_16E7_A000_0000, 0x401C_2239_A000_0000,
0x401C_2D75_A000_0000, 0x401C_389C_0000_0000, 0x401C_43AC_E000_0000, 0x401C_4EA8_C000_0000,
0x401C_598F_E000_0000, 0x401C_6462_8000_0000, 0x401C_6F21_0000_0000, 0x401C_79CB_C000_0000,
0x401C_8462_C000_0000, 0x401C_8EE6_8000_0000, 0x401C_9957_4000_0000, 0x401C_A3B5_4000_0000,
0x401C_AE00_E000_0000, 0x401C_B83A_2000_0000, 0x401C_C261_6000_0000, 0x401C_CC76_E000_0000,
0x401C_D67B_0000_0000, 0x401C_E06D_C000_0000, 0x401C_EA4F_8000_0000, 0x401C_F420_6000_0000,
0x401C_FDE0_C000_0000, 0x401D_0790_A000_0000, 0x401D_1130_8000_0000, 0x401D_1AC0_6000_0000,
0x401D_2440_8000_0000, 0x401D_2DB1_0000_0000, 0x401D_3712_4000_0000, 0x401D_4064_6000_0000,
0x401D_49A7_8000_0000, 0x401D_52DB_E000_0000, 0x401D_5C01_A000_0000, 0x401D_6519_0000_0000,
0x401D_6E22_2000_0000, 0x401D_771D_2000_0000, 0x401D_800A_6000_0000, 0x401D_88E9_C000_0000,
0x401D_91BB_A000_0000, 0x401D_9A80_2000_0000, 0x401D_A337_6000_0000, 0x401D_ABE1_8000_0000,
0x401D_B47E_C000_0000, 0x401D_BD0F_2000_0000, 0x401D_C593_0000_0000, 0x401D_CE0A_4000_0000,
0x401D_D675_4000_0000, 0x401D_DED4_0000_0000, 0x401D_E726_A000_0000, 0x401D_EF6D_6000_0000,
0x401D_F7A8_6000_0000, 0x401D_FFD7_A000_0000, 0x401E_07FB_6000_0000, 0x401E_1013_A000_0000,
0x401E_1820_A000_0000, 0x401E_2022_8000_0000, 0x401E_2819_4000_0000, 0x401E_3005_2000_0000,
0x401E_37E6_2000_0000, 0x401E_3FBC_8000_0000, 0x401E_4788_4000_0000, 0x401E_4F49_8000_0000,
0x401E_5700_6000_0000, 0x401E_5EAD_0000_0000, 0x401E_664F_8000_0000, 0x401E_6DE8_0000_0000,
0x401E_7576_8000_0000, 0x401E_7CFB_2000_0000, 0x401E_8476_0000_0000, 0x401E_8BE7_6000_0000,
0x401E_934F_0000_0000, 0x401E_9AAD_4000_0000, 0x401E_A202_4000_0000, 0x401E_A94D_E000_0000,
0x401E_B090_4000_0000, 0x401E_B7C9_A000_0000, 0x401E_BEFA_0000_0000, 0x401E_C621_8000_0000,
0x401E_CD40_2000_0000, 0x401E_D456_0000_0000, 0x401E_DB63_2000_0000, 0x401E_E267_E000_0000,
0x401E_E964_0000_0000, 0x401E_F057_C000_0000, 0x401E_F743_2000_0000, 0x401E_FE26_4000_0000,
0x401F_0501_4000_0000, 0x401F_0BD4_2000_0000, 0x401F_129E_E000_0000, 0x401F_1961_C000_0000,
0x401F_201C_C000_0000, 0x401F_26CF_E000_0000, 0x401F_2D7B_6000_0000, 0x401F_341F_2000_0000,
0x401F_3ABB_4000_0000, 0x401F_414F_E000_0000, 0x401F_47DD_0000_0000, 0x401F_4E62_C000_0000,
0x401F_54E1_2000_0000, 0x401F_5B58_4000_0000, 0x401F_61C8_2000_0000, 0x401F_6830_E000_0000,
0x401F_6E92_A000_0000, 0x401F_74ED_4000_0000, 0x401F_7B40_E000_0000, 0x401F_818D_A000_0000,
0x401F_87D3_A000_0000, 0x401F_8E12_C000_0000, 0x401F_944B_2000_0000, 0x401F_9A7C_E000_0000,
0x401F_A0A7_E000_0000, 0x401F_A6CC_8000_0000, 0x401F_ACEA_8000_0000, 0x401F_B302_0000_0000,
0x401F_B913_4000_0000, 0x401F_BF1E_0000_0000, 0x401F_C522_8000_0000, 0x401F_CB20_C000_0000,
0x401F_D118_E000_0000, 0x401F_D70A_C000_0000, 0x401F_DCF6_8000_0000, 0x401F_E2DC_6000_0000,
0x401F_E8BC_2000_0000, 0x401F_EE95_E000_0000, 0x401F_F469_C000_0000, 0x401F_FA37_C000_0000,
]
// From enc/encode.c kDefaultCommandCodeNumBits.
pri const DEFAULT_COMMAND_CODE_NUM_BITS : base.u32 = 448
// From enc/encode.c kDefaultCommandDepths.
pri const DEFAULT_COMMAND_DEPTHS : roarray[128] base.u8[..= 15] = [
0, 4, 4, 5, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8,
0, 0, 0, 4, 4, 4, 4, 4, 5, 5, 6, 6, 6, 6, 7, 7,
7, 7, 10, 10, 10, 10, 10, 10, 0, 4, 4, 5, 5, 5, 6, 6,
7, 8, 8, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
6, 6, 6, 6, 6, 6, 5, 5, 5, 5, 5, 5, 4, 4, 4, 4,
4, 4, 4, 5, 5, 5, 5, 5, 5, 6, 6, 7, 7, 7, 8, 10,
12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 0, 0, 0, 0,
]
// From enc/encode.c kDefaultCommandBits.
pri const DEFAULT_COMMAND_BITS : roarray[128] base.u16 = [
0, 0, 8, 9, 3, 35, 7, 71, 39, 103, 23, 47,
175, 111, 239, 31, 0, 0, 0, 4, 12, 2, 10, 6,
13, 29, 11, 43, 27, 59, 87, 55, 15, 79, 319, 831,
191, 703, 447, 959, 0, 14, 1, 25, 5, 21, 19, 51,
119, 159, 95, 223, 479, 991, 63, 575, 127, 639, 383, 895,
255, 767, 511, 1023, 14, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 27, 59, 7, 39,
23, 55, 30, 1, 17, 9, 25, 5, 0, 8, 4, 12,
2, 10, 6, 21, 13, 29, 3, 19, 11, 15, 47, 31,
95, 63, 127, 255, 767, 2815, 1791, 3839, 511, 2559, 1535, 3583,
1023, 3071, 2047, 4095, 0, 0, 0, 0,
]
// From enc/encode.c kDefaultCommandCode.
pri const DEFAULT_COMMAND_CODE : roarray[57] base.u8 = [
0xFF, 0x77, 0xD5, 0xBF, 0xE7, 0xDE, 0xEA, 0x9E, 0x51, 0x5D, 0xDE, 0xC6,
0x70, 0x57, 0xBC, 0x58, 0x58, 0x58, 0xD8, 0xD8, 0x58, 0xD5, 0xCB, 0x8C,
0xEA, 0xE0, 0xC3, 0x87, 0x1F, 0x83, 0xC1, 0x60, 0x1C, 0x67, 0xB2, 0xAA,
0x06, 0x83, 0xC1, 0x60, 0x30, 0x18, 0xCC, 0xA1, 0xCE, 0x88, 0x54, 0x94,
0x46, 0xE1, 0xB0, 0xD0, 0x4E, 0xB2, 0xF7, 0x04, 0x00,
]
// From enc/compress_fragment.c kCmdHistoSeed.
pri const CMD_HISTO_SEED : roarray[128] base.u32[..= 1] = [
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0,
]
// From enc/compress_fragment_two_pass.c kNumExtraBits.
pri const TWO_PASS_NUM_EXTRA_BITS : roarray[128] base.u32[..= 24] = [
0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5,
6, 7, 8, 9, 10, 12, 14, 24, 0, 0, 0, 0, 0, 0, 0, 0,
1, 1, 2, 2, 3, 3, 4, 4, 0, 0, 0, 0, 0, 0, 0, 0,
1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 7, 8, 9, 10, 24,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8,
9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16,
17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 24,
]
// From enc/compress_fragment_two_pass.c kInsertOffset.
pri const TWO_PASS_INSERT_OFFSET : roarray[24] base.u32[..= 22594] = [
0, 1, 2, 3, 4, 5, 6, 8, 10, 14, 18, 26,
34, 50, 66, 98, 130, 194, 322, 578, 1090, 2114, 6210, 22594,
]
// From enc/entropy_encode.c kBrotliShellGaps.
pri const SHELL_GAPS : roarray[6] base.u32[..= 132] = [
132, 57, 23, 10, 4, 1,
]
// From enc/entropy_encode.c kLut (in BrotliReverseBits).
pri const REVERSE_BITS_LUT : roarray[16] base.u32[..= 15] = [
0x00, 0x08, 0x04, 0x0C, 0x02, 0x0A, 0x06, 0x0E, 0x01, 0x09, 0x05, 0x0D,
0x03, 0x0B, 0x07, 0x0F,
]
// From enc/entropy_encode_static.h StoreStaticCodeLengthCode: 40 bits.
pri const STATIC_CODE_LENGTH_CODE : base.u64 = 0xFF_5555_5554
// From enc/entropy_encode_static.h kCodeLengthDepth.
pri const CODE_LENGTH_DEPTH : roarray[18] base.u32[..= 56] = [
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 0, 4, 4,
]
// From enc/entropy_encode_static.h kCodeLengthBits.
pri const CODE_LENGTH_BITS : roarray[18] base.u64 = [
0, 8, 4, 12, 2, 10, 6, 14, 1, 9, 5, 13, 3, 15, 31, 0, 11, 7,
]
// From enc/entropy_encode_static.h kZeroRepsBits.
pri const ZERO_REPS_BITS : roarray[704] base.u64 = [
0x0000, 0x0000, 0x0000, 0x0007, 0x0017, 0x0027, 0x0037, 0x0047,
0x0057, 0x0067, 0x0077, 0x0770, 0x0B87, 0x1387, 0x1B87, 0x2387,
0x2B87, 0x3387, 0x3B87, 0x0397, 0x0B97, 0x1397, 0x1B97, 0x2397,
0x2B97, 0x3397, 0x3B97, 0x03A7, 0x0BA7, 0x13A7, 0x1BA7, 0x23A7,
0x2BA7, 0x33A7, 0x3BA7, 0x03B7, 0x0BB7, 0x13B7, 0x1BB7, 0x23B7,
0x2BB7, 0x33B7, 0x3BB7, 0x03C7, 0x0BC7, 0x13C7, 0x1BC7, 0x23C7,
0x2BC7, 0x33C7, 0x3BC7, 0x03D7, 0x0BD7, 0x13D7, 0x1BD7, 0x23D7,
0x2BD7, 0x33D7, 0x3BD7, 0x03E7, 0x0BE7, 0x13E7, 0x1BE7, 0x23E7,
0x2BE7, 0x33E7, 0x3BE7, 0x03F7, 0x0BF7, 0x13F7, 0x1BF7, 0x23F7,
0x2BF7, 0x33F7, 0x3BF7, 0x1_C387, 0x5_C387, 0x9_C387, 0xD_C387, 0x11_C387,
0x15_C387, 0x19_C387, 0x1D_C387, 0x1_CB87, 0x5_CB87, 0x9_CB87, 0xD_CB87, 0x11_CB87,
0x15_CB87, 0x19_CB87, 0x1D_CB87, 0x1_D387, 0x5_D387, 0x9_D387, 0xD_D387, 0x11_D387,
0x15_D387, 0x19_D387, 0x1D_D387, 0x1_DB87, 0x5_DB87, 0x9_DB87, 0xD_DB87, 0x11_DB87,
0x15_DB87, 0x19_DB87, 0x1D_DB87, 0x1_E387, 0x5_E387, 0x9_E387, 0xD_E387, 0x11_E387,
0x15_E387, 0x19_E387, 0x1D_E387, 0x1_EB87, 0x5_EB87, 0x9_EB87, 0xD_EB87, 0x11_EB87,
0x15_EB87, 0x19_EB87, 0x1D_EB87, 0x1_F387, 0x5_F387, 0x9_F387, 0xD_F387, 0x11_F387,
0x15_F387, 0x19_F387, 0x1D_F387, 0x1_FB87, 0x5_FB87, 0x9_FB87, 0xD_FB87, 0x11_FB87,
0x15_FB87, 0x19_FB87, 0x1D_FB87, 0x1_C397, 0x5_C397, 0x9_C397, 0xD_C397, 0x11_C397,
0x15_C397, 0x19_C397, 0x1D_C397, 0x1_CB97, 0x5_CB97, 0x9_CB97, 0xD_CB97, 0x11_CB97,
0x15_CB97, 0x19_CB97, 0x1D_CB97, 0x1_D397, 0x5_D397, 0x9_D397, 0xD_D397, 0x11_D397,
0x15_D397, 0x19_D397, 0x1D_D397, 0x1_DB97, 0x5_DB97, 0x9_DB97, 0xD_DB97, 0x11_DB97,
0x15_DB97, 0x19_DB97, 0x1D_DB97, 0x1_E397, 0x5_E397, 0x9_E397, 0xD_E397, 0x11_E397,
0x15_E397, 0x19_E397, 0x1D_E397, 0x1_EB97, 0x5_EB97, 0x9_EB97, 0xD_EB97, 0x11_EB97,
0x15_EB97, 0x19_EB97, 0x1D_EB97, 0x1_F397, 0x5_F397, 0x9_F397, 0xD_F397, 0x11_F397,
0x15_F397, 0x19_F397, 0x1D_F397, 0x1_FB97, 0x5_FB97, 0x9_FB97, 0xD_FB97, 0x11_FB97,
0x15_FB97, 0x19_FB97, 0x1D_FB97, 0x1_C3A7, 0x5_C3A7, 0x9_C3A7, 0xD_C3A7, 0x11_C3A7,
0x15_C3A7, 0x19_C3A7, 0x1D_C3A7, 0x1_CBA7, 0x5_CBA7, 0x9_CBA7, 0xD_CBA7, 0x11_CBA7,
0x15_CBA7, 0x19_CBA7, 0x1D_CBA7, 0x1_D3A7, 0x5_D3A7, 0x9_D3A7, 0xD_D3A7, 0x11_D3A7,
0x15_D3A7, 0x19_D3A7, 0x1D_D3A7, 0x1_DBA7, 0x5_DBA7, 0x9_DBA7, 0xD_DBA7, 0x11_DBA7,
0x15_DBA7, 0x19_DBA7, 0x1D_DBA7, 0x1_E3A7, 0x5_E3A7, 0x9_E3A7, 0xD_E3A7, 0x11_E3A7,
0x15_E3A7, 0x19_E3A7, 0x1D_E3A7, 0x1_EBA7, 0x5_EBA7, 0x9_EBA7, 0xD_EBA7, 0x11_EBA7,
0x15_EBA7, 0x19_EBA7, 0x1D_EBA7, 0x1_F3A7, 0x5_F3A7, 0x9_F3A7, 0xD_F3A7, 0x11_F3A7,
0x15_F3A7, 0x19_F3A7, 0x1D_F3A7, 0x1_FBA7, 0x5_FBA7, 0x9_FBA7, 0xD_FBA7, 0x11_FBA7,
0x15_FBA7, 0x19_FBA7, 0x1D_FBA7, 0x1_C3B7, 0x5_C3B7, 0x9_C3B7, 0xD_C3B7, 0x11_C3B7,
0x15_C3B7, 0x19_C3B7, 0x1D_C3B7, 0x1_CBB7, 0x5_CBB7, 0x9_CBB7, 0xD_CBB7, 0x11_CBB7,
0x15_CBB7, 0x19_CBB7, 0x1D_CBB7, 0x1_D3B7, 0x5_D3B7, 0x9_D3B7, 0xD_D3B7, 0x11_D3B7,
0x15_D3B7, 0x19_D3B7, 0x1D_D3B7, 0x1_DBB7, 0x5_DBB7, 0x9_DBB7, 0xD_DBB7, 0x11_DBB7,
0x15_DBB7, 0x19_DBB7, 0x1D_DBB7, 0x1_E3B7, 0x5_E3B7, 0x9_E3B7, 0xD_E3B7, 0x11_E3B7,
0x15_E3B7, 0x19_E3B7, 0x1D_E3B7, 0x1_EBB7, 0x5_EBB7, 0x9_EBB7, 0xD_EBB7, 0x11_EBB7,
0x15_EBB7, 0x19_EBB7, 0x1D_EBB7, 0x1_F3B7, 0x5_F3B7, 0x9_F3B7, 0xD_F3B7, 0x11_F3B7,
0x15_F3B7, 0x19_F3B7, 0x1D_F3B7, 0x1_FBB7, 0x5_FBB7, 0x9_FBB7, 0xD_FBB7, 0x11_FBB7,
0x15_FBB7, 0x19_FBB7, 0x1D_FBB7, 0x1_C3C7, 0x5_C3C7, 0x9_C3C7, 0xD_C3C7, 0x11_C3C7,
0x15_C3C7, 0x19_C3C7, 0x1D_C3C7, 0x1_CBC7, 0x5_CBC7, 0x9_CBC7, 0xD_CBC7, 0x11_CBC7,
0x15_CBC7, 0x19_CBC7, 0x1D_CBC7, 0x1_D3C7, 0x5_D3C7, 0x9_D3C7, 0xD_D3C7, 0x11_D3C7,
0x15_D3C7, 0x19_D3C7, 0x1D_D3C7, 0x1_DBC7, 0x5_DBC7, 0x9_DBC7, 0xD_DBC7, 0x11_DBC7,
0x15_DBC7, 0x19_DBC7, 0x1D_DBC7, 0x1_E3C7, 0x5_E3C7, 0x9_E3C7, 0xD_E3C7, 0x11_E3C7,
0x15_E3C7, 0x19_E3C7, 0x1D_E3C7, 0x1_EBC7, 0x5_EBC7, 0x9_EBC7, 0xD_EBC7, 0x11_EBC7,
0x15_EBC7, 0x19_EBC7, 0x1D_EBC7, 0x1_F3C7, 0x5_F3C7, 0x9_F3C7, 0xD_F3C7, 0x11_F3C7,
0x15_F3C7, 0x19_F3C7, 0x1D_F3C7, 0x1_FBC7, 0x5_FBC7, 0x9_FBC7, 0xD_FBC7, 0x11_FBC7,
0x15_FBC7, 0x19_FBC7, 0x1D_FBC7, 0x1_C3D7, 0x5_C3D7, 0x9_C3D7, 0xD_C3D7, 0x11_C3D7,
0x15_C3D7, 0x19_C3D7, 0x1D_C3D7, 0x1_CBD7, 0x5_CBD7, 0x9_CBD7, 0xD_CBD7, 0x11_CBD7,
0x15_CBD7, 0x19_CBD7, 0x1D_CBD7, 0x1_D3D7, 0x5_D3D7, 0x9_D3D7, 0xD_D3D7, 0x11_D3D7,
0x15_D3D7, 0x19_D3D7, 0x1D_D3D7, 0x1_DBD7, 0x5_DBD7, 0x9_DBD7, 0xD_DBD7, 0x11_DBD7,
0x15_DBD7, 0x19_DBD7, 0x1D_DBD7, 0x1_E3D7, 0x5_E3D7, 0x9_E3D7, 0xD_E3D7, 0x11_E3D7,
0x15_E3D7, 0x19_E3D7, 0x1D_E3D7, 0x1_EBD7, 0x5_EBD7, 0x9_EBD7, 0xD_EBD7, 0x11_EBD7,
0x15_EBD7, 0x19_EBD7, 0x1D_EBD7, 0x1_F3D7, 0x5_F3D7, 0x9_F3D7, 0xD_F3D7, 0x11_F3D7,
0x15_F3D7, 0x19_F3D7, 0x1D_F3D7, 0x1_FBD7, 0x5_FBD7, 0x9_FBD7, 0xD_FBD7, 0x11_FBD7,
0x15_FBD7, 0x19_FBD7, 0x1D_FBD7, 0x1_C3E7, 0x5_C3E7, 0x9_C3E7, 0xD_C3E7, 0x11_C3E7,
0x15_C3E7, 0x19_C3E7, 0x1D_C3E7, 0x1_CBE7, 0x5_CBE7, 0x9_CBE7, 0xD_CBE7, 0x11_CBE7,
0x15_CBE7, 0x19_CBE7, 0x1D_CBE7, 0x1_D3E7, 0x5_D3E7, 0x9_D3E7, 0xD_D3E7, 0x11_D3E7,
0x15_D3E7, 0x19_D3E7, 0x1D_D3E7, 0x1_DBE7, 0x5_DBE7, 0x9_DBE7, 0xD_DBE7, 0x11_DBE7,
0x15_DBE7, 0x19_DBE7, 0x1D_DBE7, 0x1_E3E7, 0x5_E3E7, 0x9_E3E7, 0xD_E3E7, 0x11_E3E7,
0x15_E3E7, 0x19_E3E7, 0x1D_E3E7, 0x1_EBE7, 0x5_EBE7, 0x9_EBE7, 0xD_EBE7, 0x11_EBE7,
0x15_EBE7, 0x19_EBE7, 0x1D_EBE7, 0x1_F3E7, 0x5_F3E7, 0x9_F3E7, 0xD_F3E7, 0x11_F3E7,
0x15_F3E7, 0x19_F3E7, 0x1D_F3E7, 0x1_FBE7, 0x5_FBE7, 0x9_FBE7, 0xD_FBE7, 0x11_FBE7,
0x15_FBE7, 0x19_FBE7, 0x1D_FBE7, 0x1_C3F7, 0x5_C3F7, 0x9_C3F7, 0xD_C3F7, 0x11_C3F7,
0x15_C3F7, 0x19_C3F7, 0x1D_C3F7, 0x1_CBF7, 0x5_CBF7, 0x9_CBF7, 0xD_CBF7, 0x11_CBF7,
0x15_CBF7, 0x19_CBF7, 0x1D_CBF7, 0x1_D3F7, 0x5_D3F7, 0x9_D3F7, 0xD_D3F7, 0x11_D3F7,
0x15_D3F7, 0x19_D3F7, 0x1D_D3F7, 0x1_DBF7, 0x5_DBF7, 0x9_DBF7, 0xD_DBF7, 0x11_DBF7,
0x15_DBF7, 0x19_DBF7, 0x1D_DBF7, 0x1_E3F7, 0x5_E3F7, 0x9_E3F7, 0xD_E3F7, 0x11_E3F7,
0x15_E3F7, 0x19_E3F7, 0x1D_E3F7, 0x1_EBF7, 0x5_EBF7, 0x9_EBF7, 0xD_EBF7, 0x11_EBF7,
0x15_EBF7, 0x19_EBF7, 0x1D_EBF7, 0x1_F3F7, 0x5_F3F7, 0x9_F3F7, 0xD_F3F7, 0x11_F3F7,
0x15_F3F7, 0x19_F3F7, 0x1D_F3F7, 0x1_FBF7, 0x5_FBF7, 0x9_FBF7, 0xD_FBF7, 0x11_FBF7,
0x15_FBF7, 0x19_FBF7, 0x1D_FBF7, 0xE1_C387, 0x2E1_C387, 0x4E1_C387, 0x6E1_C387, 0x8E1_C387,
0xAE1_C387, 0xCE1_C387, 0xEE1_C387, 0xE5_C387, 0x2E5_C387, 0x4E5_C387, 0x6E5_C387, 0x8E5_C387,
0xAE5_C387, 0xCE5_C387, 0xEE5_C387, 0xE9_C387, 0x2E9_C387, 0x4E9_C387, 0x6E9_C387, 0x8E9_C387,
0xAE9_C387, 0xCE9_C387, 0xEE9_C387, 0xED_C387, 0x2ED_C387, 0x4ED_C387, 0x6ED_C387, 0x8ED_C387,
0xAED_C387, 0xCED_C387, 0xEED_C387, 0xF1_C387, 0x2F1_C387, 0x4F1_C387, 0x6F1_C387, 0x8F1_C387,
0xAF1_C387, 0xCF1_C387, 0xEF1_C387, 0xF5_C387, 0x2F5_C387, 0x4F5_C387, 0x6F5_C387, 0x8F5_C387,
0xAF5_C387, 0xCF5_C387, 0xEF5_C387, 0xF9_C387, 0x2F9_C387, 0x4F9_C387, 0x6F9_C387, 0x8F9_C387,
0xAF9_C387, 0xCF9_C387, 0xEF9_C387, 0xFD_C387, 0x2FD_C387, 0x4FD_C387, 0x6FD_C387, 0x8FD_C387,
0xAFD_C387, 0xCFD_C387, 0xEFD_C387, 0xE1_CB87, 0x2E1_CB87, 0x4E1_CB87, 0x6E1_CB87, 0x8E1_CB87,
0xAE1_CB87, 0xCE1_CB87, 0xEE1_CB87, 0xE5_CB87, 0x2E5_CB87, 0x4E5_CB87, 0x6E5_CB87, 0x8E5_CB87,
0xAE5_CB87, 0xCE5_CB87, 0xEE5_CB87, 0xE9_CB87, 0x2E9_CB87, 0x4E9_CB87, 0x6E9_CB87, 0x8E9_CB87,
0xAE9_CB87, 0xCE9_CB87, 0xEE9_CB87, 0xED_CB87, 0x2ED_CB87, 0x4ED_CB87, 0x6ED_CB87, 0x8ED_CB87,
0xAED_CB87, 0xCED_CB87, 0xEED_CB87, 0xF1_CB87, 0x2F1_CB87, 0x4F1_CB87, 0x6F1_CB87, 0x8F1_CB87,
0xAF1_CB87, 0xCF1_CB87, 0xEF1_CB87, 0xF5_CB87, 0x2F5_CB87, 0x4F5_CB87, 0x6F5_CB87, 0x8F5_CB87,
0xAF5_CB87, 0xCF5_CB87, 0xEF5_CB87, 0xF9_CB87, 0x2F9_CB87, 0x4F9_CB87, 0x6F9_CB87, 0x8F9_CB87,
]
// From enc/entropy_encode_static.h kZeroRepsDepth.
pri const ZERO_REPS_DEPTH : roarray[704] base.u32[..= 56] = [
0, 4, 8, 7, 7, 7, 7, 7, 7, 7, 7, 11, 14, 14, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
]
// From enc/entropy_encode_static.h kNonZeroRepsBits.
pri const NON_ZERO_REPS_BITS : roarray[704] base.u64 = [
0x000B, 0x001B, 0x002B, 0x003B,
0x02CB, 0x06CB, 0x0ACB, 0x0ECB,
0x02DB, 0x06DB, 0x0ADB, 0x0EDB,
0x02EB, 0x06EB, 0x0AEB, 0x0EEB,
0x02FB, 0x06FB, 0x0AFB, 0x0EFB,
0xB2CB, 0x1_B2CB, 0x2_B2CB, 0x3_B2CB,
0xB6CB, 0x1_B6CB, 0x2_B6CB, 0x3_B6CB,
0xBACB, 0x1_BACB, 0x2_BACB, 0x3_BACB,
0xBECB, 0x1_BECB, 0x2_BECB, 0x3_BECB,
0xB2DB, 0x1_B2DB, 0x2_B2DB, 0x3_B2DB,
0xB6DB, 0x1_B6DB, 0x2_B6DB, 0x3_B6DB,
0xBADB, 0x1_BADB, 0x2_BADB, 0x3_BADB,
0xBEDB, 0x1_BEDB, 0x2_BEDB, 0x3_BEDB,
0xB2EB, 0x1_B2EB, 0x2_B2EB, 0x3_B2EB,
0xB6EB, 0x1_B6EB, 0x2_B6EB, 0x3_B6EB,
0xBAEB, 0x1_BAEB, 0x2_BAEB, 0x3_BAEB,
0xBEEB, 0x1_BEEB, 0x2_BEEB, 0x3_BEEB,
0xB2FB, 0x1_B2FB, 0x2_B2FB, 0x3_B2FB,
0xB6FB, 0x1_B6FB, 0x2_B6FB, 0x3_B6FB,
0xBAFB, 0x1_BAFB, 0x2_BAFB, 0x3_BAFB,
0xBEFB, 0x1_BEFB, 0x2_BEFB, 0x3_BEFB,
0x2C_B2CB, 0x6C_B2CB, 0xAC_B2CB, 0xEC_B2CB,
0x2D_B2CB, 0x6D_B2CB, 0xAD_B2CB, 0xED_B2CB,
0x2E_B2CB, 0x6E_B2CB, 0xAE_B2CB, 0xEE_B2CB,
0x2F_B2CB, 0x6F_B2CB, 0xAF_B2CB, 0xEF_B2CB,
0x2C_B6CB, 0x6C_B6CB, 0xAC_B6CB, 0xEC_B6CB,
0x2D_B6CB, 0x6D_B6CB, 0xAD_B6CB, 0xED_B6CB,
0x2E_B6CB, 0x6E_B6CB, 0xAE_B6CB, 0xEE_B6CB,
0x2F_B6CB, 0x6F_B6CB, 0xAF_B6CB, 0xEF_B6CB,
0x2C_BACB, 0x6C_BACB, 0xAC_BACB, 0xEC_BACB,
0x2D_BACB, 0x6D_BACB, 0xAD_BACB, 0xED_BACB,
0x2E_BACB, 0x6E_BACB, 0xAE_BACB, 0xEE_BACB,
0x2F_BACB, 0x6F_BACB, 0xAF_BACB, 0xEF_BACB,
0x2C_BECB, 0x6C_BECB, 0xAC_BECB, 0xEC_BECB,
0x2D_BECB, 0x6D_BECB, 0xAD_BECB, 0xED_BECB,
0x2E_BECB, 0x6E_BECB, 0xAE_BECB, 0xEE_BECB,
0x2F_BECB, 0x6F_BECB, 0xAF_BECB, 0xEF_BECB,
0x2C_B2DB, 0x6C_B2DB, 0xAC_B2DB, 0xEC_B2DB,
0x2D_B2DB, 0x6D_B2DB, 0xAD_B2DB, 0xED_B2DB,
0x2E_B2DB, 0x6E_B2DB, 0xAE_B2DB, 0xEE_B2DB,
0x2F_B2DB, 0x6F_B2DB, 0xAF_B2DB, 0xEF_B2DB,
0x2C_B6DB, 0x6C_B6DB, 0xAC_B6DB, 0xEC_B6DB,
0x2D_B6DB, 0x6D_B6DB, 0xAD_B6DB, 0xED_B6DB,
0x2E_B6DB, 0x6E_B6DB, 0xAE_B6DB, 0xEE_B6DB,
0x2F_B6DB, 0x6F_B6DB, 0xAF_B6DB, 0xEF_B6DB,
0x2C_BADB, 0x6C_BADB, 0xAC_BADB, 0xEC_BADB,
0x2D_BADB, 0x6D_BADB, 0xAD_BADB, 0xED_BADB,
0x2E_BADB, 0x6E_BADB, 0xAE_BADB, 0xEE_BADB,
0x2F_BADB, 0x6F_BADB, 0xAF_BADB, 0xEF_BADB,
0x2C_BEDB, 0x6C_BEDB, 0xAC_BEDB, 0xEC_BEDB,
0x2D_BEDB, 0x6D_BEDB, 0xAD_BEDB, 0xED_BEDB,
0x2E_BEDB, 0x6E_BEDB, 0xAE_BEDB, 0xEE_BEDB,
0x2F_BEDB, 0x6F_BEDB, 0xAF_BEDB, 0xEF_BEDB,
0x2C_B2EB, 0x6C_B2EB, 0xAC_B2EB, 0xEC_B2EB,
0x2D_B2EB, 0x6D_B2EB, 0xAD_B2EB, 0xED_B2EB,
0x2E_B2EB, 0x6E_B2EB, 0xAE_B2EB, 0xEE_B2EB,
0x2F_B2EB, 0x6F_B2EB, 0xAF_B2EB, 0xEF_B2EB,
0x2C_B6EB, 0x6C_B6EB, 0xAC_B6EB, 0xEC_B6EB,
0x2D_B6EB, 0x6D_B6EB, 0xAD_B6EB, 0xED_B6EB,
0x2E_B6EB, 0x6E_B6EB, 0xAE_B6EB, 0xEE_B6EB,
0x2F_B6EB, 0x6F_B6EB, 0xAF_B6EB, 0xEF_B6EB,
0x2C_BAEB, 0x6C_BAEB, 0xAC_BAEB, 0xEC_BAEB,
0x2D_BAEB, 0x6D_BAEB, 0xAD_BAEB, 0xED_BAEB,
0x2E_BAEB, 0x6E_BAEB, 0xAE_BAEB, 0xEE_BAEB,
0x2F_BAEB, 0x6F_BAEB, 0xAF_BAEB, 0xEF_BAEB,
0x2C_BEEB, 0x6C_BEEB, 0xAC_BEEB, 0xEC_BEEB,
0x2D_BEEB, 0x6D_BEEB, 0xAD_BEEB, 0xED_BEEB,
0x2E_BEEB, 0x6E_BEEB, 0xAE_BEEB, 0xEE_BEEB,
0x2F_BEEB, 0x6F_BEEB, 0xAF_BEEB, 0xEF_BEEB,
0x2C_B2FB, 0x6C_B2FB, 0xAC_B2FB, 0xEC_B2FB,
0x2D_B2FB, 0x6D_B2FB, 0xAD_B2FB, 0xED_B2FB,
0x2E_B2FB, 0x6E_B2FB, 0xAE_B2FB, 0xEE_B2FB,
0x2F_B2FB, 0x6F_B2FB, 0xAF_B2FB, 0xEF_B2FB,
0x2C_B6FB, 0x6C_B6FB, 0xAC_B6FB, 0xEC_B6FB,
0x2D_B6FB, 0x6D_B6FB, 0xAD_B6FB, 0xED_B6FB,
0x2E_B6FB, 0x6E_B6FB, 0xAE_B6FB, 0xEE_B6FB,
0x2F_B6FB, 0x6F_B6FB, 0xAF_B6FB, 0xEF_B6FB,
0x2C_BAFB, 0x6C_BAFB, 0xAC_BAFB, 0xEC_BAFB,
0x2D_BAFB, 0x6D_BAFB, 0xAD_BAFB, 0xED_BAFB,
0x2E_BAFB, 0x6E_BAFB, 0xAE_BAFB, 0xEE_BAFB,
0x2F_BAFB, 0x6F_BAFB, 0xAF_BAFB, 0xEF_BAFB,
0x2C_BEFB, 0x6C_BEFB, 0xAC_BEFB, 0xEC_BEFB,
0x2D_BEFB, 0x6D_BEFB, 0xAD_BEFB, 0xED_BEFB,
0x2E_BEFB, 0x6E_BEFB, 0xAE_BEFB, 0xEE_BEFB,
0x2F_BEFB, 0x6F_BEFB, 0xAF_BEFB, 0xEF_BEFB,
0xB2C_B2CB, 0x1B2C_B2CB, 0x2B2C_B2CB, 0x3B2C_B2CB,
0xB6C_B2CB, 0x1B6C_B2CB, 0x2B6C_B2CB, 0x3B6C_B2CB,
0xBAC_B2CB, 0x1BAC_B2CB, 0x2BAC_B2CB, 0x3BAC_B2CB,
0xBEC_B2CB, 0x1BEC_B2CB, 0x2BEC_B2CB, 0x3BEC_B2CB,
0xB2D_B2CB, 0x1B2D_B2CB, 0x2B2D_B2CB, 0x3B2D_B2CB,
0xB6D_B2CB, 0x1B6D_B2CB, 0x2B6D_B2CB, 0x3B6D_B2CB,
0xBAD_B2CB, 0x1BAD_B2CB, 0x2BAD_B2CB, 0x3BAD_B2CB,
0xBED_B2CB, 0x1BED_B2CB, 0x2BED_B2CB, 0x3BED_B2CB,
0xB2E_B2CB, 0x1B2E_B2CB, 0x2B2E_B2CB, 0x3B2E_B2CB,
0xB6E_B2CB, 0x1B6E_B2CB, 0x2B6E_B2CB, 0x3B6E_B2CB,
0xBAE_B2CB, 0x1BAE_B2CB, 0x2BAE_B2CB, 0x3BAE_B2CB,
0xBEE_B2CB, 0x1BEE_B2CB, 0x2BEE_B2CB, 0x3BEE_B2CB,
0xB2F_B2CB, 0x1B2F_B2CB, 0x2B2F_B2CB, 0x3B2F_B2CB,
0xB6F_B2CB, 0x1B6F_B2CB, 0x2B6F_B2CB, 0x3B6F_B2CB,
0xBAF_B2CB, 0x1BAF_B2CB, 0x2BAF_B2CB, 0x3BAF_B2CB,
0xBEF_B2CB, 0x1BEF_B2CB, 0x2BEF_B2CB, 0x3BEF_B2CB,
0xB2C_B6CB, 0x1B2C_B6CB, 0x2B2C_B6CB, 0x3B2C_B6CB,
0xB6C_B6CB, 0x1B6C_B6CB, 0x2B6C_B6CB, 0x3B6C_B6CB,
0xBAC_B6CB, 0x1BAC_B6CB, 0x2BAC_B6CB, 0x3BAC_B6CB,
0xBEC_B6CB, 0x1BEC_B6CB, 0x2BEC_B6CB, 0x3BEC_B6CB,
0xB2D_B6CB, 0x1B2D_B6CB, 0x2B2D_B6CB, 0x3B2D_B6CB,
0xB6D_B6CB, 0x1B6D_B6CB, 0x2B6D_B6CB, 0x3B6D_B6CB,
0xBAD_B6CB, 0x1BAD_B6CB, 0x2BAD_B6CB, 0x3BAD_B6CB,
0xBED_B6CB, 0x1BED_B6CB, 0x2BED_B6CB, 0x3BED_B6CB,
0xB2E_B6CB, 0x1B2E_B6CB, 0x2B2E_B6CB, 0x3B2E_B6CB,
0xB6E_B6CB, 0x1B6E_B6CB, 0x2B6E_B6CB, 0x3B6E_B6CB,
0xBAE_B6CB, 0x1BAE_B6CB, 0x2BAE_B6CB, 0x3BAE_B6CB,
0xBEE_B6CB, 0x1BEE_B6CB, 0x2BEE_B6CB, 0x3BEE_B6CB,
0xB2F_B6CB, 0x1B2F_B6CB, 0x2B2F_B6CB, 0x3B2F_B6CB,
0xB6F_B6CB, 0x1B6F_B6CB, 0x2B6F_B6CB, 0x3B6F_B6CB,
0xBAF_B6CB, 0x1BAF_B6CB, 0x2BAF_B6CB, 0x3BAF_B6CB,
0xBEF_B6CB, 0x1BEF_B6CB, 0x2BEF_B6CB, 0x3BEF_B6CB,
0xB2C_BACB, 0x1B2C_BACB, 0x2B2C_BACB, 0x3B2C_BACB,
0xB6C_BACB, 0x1B6C_BACB, 0x2B6C_BACB, 0x3B6C_BACB,
0xBAC_BACB, 0x1BAC_BACB, 0x2BAC_BACB, 0x3BAC_BACB,
0xBEC_BACB, 0x1BEC_BACB, 0x2BEC_BACB, 0x3BEC_BACB,
0xB2D_BACB, 0x1B2D_BACB, 0x2B2D_BACB, 0x3B2D_BACB,
0xB6D_BACB, 0x1B6D_BACB, 0x2B6D_BACB, 0x3B6D_BACB,
0xBAD_BACB, 0x1BAD_BACB, 0x2BAD_BACB, 0x3BAD_BACB,
0xBED_BACB, 0x1BED_BACB, 0x2BED_BACB, 0x3BED_BACB,
0xB2E_BACB, 0x1B2E_BACB, 0x2B2E_BACB, 0x3B2E_BACB,
0xB6E_BACB, 0x1B6E_BACB, 0x2B6E_BACB, 0x3B6E_BACB,
0xBAE_BACB, 0x1BAE_BACB, 0x2BAE_BACB, 0x3BAE_BACB,
0xBEE_BACB, 0x1BEE_BACB, 0x2BEE_BACB, 0x3BEE_BACB,
0xB2F_BACB, 0x1B2F_BACB, 0x2B2F_BACB, 0x3B2F_BACB,
0xB6F_BACB, 0x1B6F_BACB, 0x2B6F_BACB, 0x3B6F_BACB,
0xBAF_BACB, 0x1BAF_BACB, 0x2BAF_BACB, 0x3BAF_BACB,
0xBEF_BACB, 0x1BEF_BACB, 0x2BEF_BACB, 0x3BEF_BACB,
0xB2C_BECB, 0x1B2C_BECB, 0x2B2C_BECB, 0x3B2C_BECB,
0xB6C_BECB, 0x1B6C_BECB, 0x2B6C_BECB, 0x3B6C_BECB,
0xBAC_BECB, 0x1BAC_BECB, 0x2BAC_BECB, 0x3BAC_BECB,
0xBEC_BECB, 0x1BEC_BECB, 0x2BEC_BECB, 0x3BEC_BECB,
0xB2D_BECB, 0x1B2D_BECB, 0x2B2D_BECB, 0x3B2D_BECB,
0xB6D_BECB, 0x1B6D_BECB, 0x2B6D_BECB, 0x3B6D_BECB,
0xBAD_BECB, 0x1BAD_BECB, 0x2BAD_BECB, 0x3BAD_BECB,
0xBED_BECB, 0x1BED_BECB, 0x2BED_BECB, 0x3BED_BECB,
0xB2E_BECB, 0x1B2E_BECB, 0x2B2E_BECB, 0x3B2E_BECB,
0xB6E_BECB, 0x1B6E_BECB, 0x2B6E_BECB, 0x3B6E_BECB,
0xBAE_BECB, 0x1BAE_BECB, 0x2BAE_BECB, 0x3BAE_BECB,
0xBEE_BECB, 0x1BEE_BECB, 0x2BEE_BECB, 0x3BEE_BECB,
0xB2F_BECB, 0x1B2F_BECB, 0x2B2F_BECB, 0x3B2F_BECB,
0xB6F_BECB, 0x1B6F_BECB, 0x2B6F_BECB, 0x3B6F_BECB,
0xBAF_BECB, 0x1BAF_BECB, 0x2BAF_BECB, 0x3BAF_BECB,
0xBEF_BECB, 0x1BEF_BECB, 0x2BEF_BECB, 0x3BEF_BECB,
0xB2C_B2DB, 0x1B2C_B2DB, 0x2B2C_B2DB, 0x3B2C_B2DB,
0xB6C_B2DB, 0x1B6C_B2DB, 0x2B6C_B2DB, 0x3B6C_B2DB,
0xBAC_B2DB, 0x1BAC_B2DB, 0x2BAC_B2DB, 0x3BAC_B2DB,
0xBEC_B2DB, 0x1BEC_B2DB, 0x2BEC_B2DB, 0x3BEC_B2DB,
0xB2D_B2DB, 0x1B2D_B2DB, 0x2B2D_B2DB, 0x3B2D_B2DB,
0xB6D_B2DB, 0x1B6D_B2DB, 0x2B6D_B2DB, 0x3B6D_B2DB,
0xBAD_B2DB, 0x1BAD_B2DB, 0x2BAD_B2DB, 0x3BAD_B2DB,
0xBED_B2DB, 0x1BED_B2DB, 0x2BED_B2DB, 0x3BED_B2DB,
0xB2E_B2DB, 0x1B2E_B2DB, 0x2B2E_B2DB, 0x3B2E_B2DB,
0xB6E_B2DB, 0x1B6E_B2DB, 0x2B6E_B2DB, 0x3B6E_B2DB,
0xBAE_B2DB, 0x1BAE_B2DB, 0x2BAE_B2DB, 0x3BAE_B2DB,
0xBEE_B2DB, 0x1BEE_B2DB, 0x2BEE_B2DB, 0x3BEE_B2DB,
0xB2F_B2DB, 0x1B2F_B2DB, 0x2B2F_B2DB, 0x3B2F_B2DB,
0xB6F_B2DB, 0x1B6F_B2DB, 0x2B6F_B2DB, 0x3B6F_B2DB,
0xBAF_B2DB, 0x1BAF_B2DB, 0x2BAF_B2DB, 0x3BAF_B2DB,
0xBEF_B2DB, 0x1BEF_B2DB, 0x2BEF_B2DB, 0x3BEF_B2DB,
0xB2C_B6DB, 0x1B2C_B6DB, 0x2B2C_B6DB, 0x3B2C_B6DB,
0xB6C_B6DB, 0x1B6C_B6DB, 0x2B6C_B6DB, 0x3B6C_B6DB,
0xBAC_B6DB, 0x1BAC_B6DB, 0x2BAC_B6DB, 0x3BAC_B6DB,
0xBEC_B6DB, 0x1BEC_B6DB, 0x2BEC_B6DB, 0x3BEC_B6DB,
0xB2D_B6DB, 0x1B2D_B6DB, 0x2B2D_B6DB, 0x3B2D_B6DB,
0xB6D_B6DB, 0x1B6D_B6DB, 0x2B6D_B6DB, 0x3B6D_B6DB,
0xBAD_B6DB, 0x1BAD_B6DB, 0x2BAD_B6DB, 0x3BAD_B6DB,
0xBED_B6DB, 0x1BED_B6DB, 0x2BED_B6DB, 0x3BED_B6DB,
0xB2E_B6DB, 0x1B2E_B6DB, 0x2B2E_B6DB, 0x3B2E_B6DB,
0xB6E_B6DB, 0x1B6E_B6DB, 0x2B6E_B6DB, 0x3B6E_B6DB,
0xBAE_B6DB, 0x1BAE_B6DB, 0x2BAE_B6DB, 0x3BAE_B6DB,
]
// From enc/entropy_encode_static.h kNonZeroRepsDepth.
pri const NON_ZERO_REPS_DEPTH : roarray[704] base.u32[..= 56] = [
6, 6, 6, 6, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
12, 12, 12, 12, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
18, 18, 18, 18, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
]
// From enc/brotli_bit_stream.c kStorageOrder.
pri const STORAGE_ORDER : roarray[18] base.u8[..= 17] = [
1, 2, 3, 4, 0, 5, 17, 6, 16, 7, 8, 9, 10, 11, 12, 13, 14, 15,
]
// From enc/brotli_bit_stream.c kHuffmanBitLengthHuffmanCodeSymbols.
pri const HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_SYMBOLS : roarray[6] base.u64[..= 15] = [
0, 7, 3, 2, 1, 15,
]
// From enc/brotli_bit_stream.c kHuffmanBitLengthHuffmanCodeBitLengths.
pri const HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_BIT_LENGTHS : roarray[6] base.u32[..= 4] = [
2, 4, 3, 2, 2, 4,
]
+32
View File
@@ -0,0 +1,32 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// Quirks are discussed in (/doc/note/quirks.md).
//
// The base38 encoding of "brot" is 0x0A_ADE4. Left shifting by 10 gives
// 0x2AB7_9000.
pri const QUIRKS_BASE : base.u32 = 0x2AB7_9000
// --------
// When this quirk is set, a positive value enables large-window brotli
// (https://datatracker.ietf.org/doc/draft-vandevenne-shared-brotli-format/),
// with window sizes up to (1 << 30) bytes. Zero means standard (RFC 7932)
// brotli. It mirrors the C code's BROTLI_DECODER_PARAM_LARGE_WINDOW.
//
// It must be set before decoding starts. Setting it also increases the
// workbuf_len.
//
// Without this quirk, a stream that starts with the large-window WBITS escape
// is rejected with "#unsupported large window". With this quirk, a stream that
// does not start with that escape is decoded as standard brotli.
pub const QUIRK_LARGE_WINDOW : base.u32 = 0x2AB7_9000 | 0x00
+259
View File
@@ -0,0 +1,259 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// This file holds the encoder's bit writer and other small helpers shared by
// the quality 0 and quality 1 code paths. The bit position (the C code's
// *storage_ix) is the encoder.bw_ix field. The storage (the C code's uint8_t*
// storage) is a slice argument.
// Mirrors BrotliWriteBits in write_bits.h (its BROTLI_LITTLE_ENDIAN branch).
//
// Like the C code, this ORs the bits into the byte at (bw_ix >> 3) and then
// stores 8 bytes, little-endian, zeroing any bits above the newly written
// ones. The byte-exact output depends on that zeroing, as later code reads
// back (and ORs into) those bytes. The storage slice always has enough slack
// that the 8-byte store fits. If it ever did not, the bits are written byte
// by byte, as far as they fit.
pri func encoder.write_bits!(n_bits: base.u32, bits: base.u64, s: slice base.u8) {
var p : base.u64
var v : base.u64
var q : slice base.u8
p = this.bw_ix >> 3
if p < args.s.length() {
q = args.s[p ..]
v = (args.s[p] as base.u64) | (args.bits ~mod<< (this.bw_ix & 7))
if q.length() >= 8 {
q.poke_u64le!(a: v)
} else {
while q.length() > 0 {
q[0] = (v & 0xFF) as base.u8
v >>= 8
q = q[1 ..]
}
}
}
this.bw_ix ~mod+= args.n_bits as base.u64
}
// Mirrors JumpToByteBoundary in brotli_bit_stream.c, without the zeroing
// (which the C code's callers in this file rely on the 8-byte stores for).
// The "(*storage_ix + 7u) & ~7u" expressions in compress_fragment.c and
// compress_fragment_two_pass.c.
pri func encoder.align_to_byte!() {
this.bw_ix = (this.bw_ix ~mod+ 7) & 0xFFFF_FFFF_FFFF_FFF8
}
// Mirrors RewindBitPosition in compress_fragment.c and
// compress_fragment_two_pass.c.
pri func encoder.rewind_bit_position!(new_ix: base.u64, s: slice base.u8) {
var p : base.u64
var mask : base.u8
p = args.new_ix >> 3
mask = (((1 as base.u32) << (args.new_ix & 7)) - 1) as base.u8
if p < args.s.length() {
args.s[p] &= mask
}
this.bw_ix = args.new_ix
}
// Mirrors UpdateBits in compress_fragment.c.
pri func encoder.update_bits!(n_bits: base.u32, bits: base.u32, pos: base.u64, s: slice base.u8) {
var n_bits : base.u32
var bits : base.u32
var pos : base.u64
var byte_pos : base.u64
var n_unchanged_bits : base.u32[..= 7]
var n_changed_bits : base.u32[..= 8]
var total_bits : base.u32[..= 15]
var mask : base.u32
var unchanged_bits : base.u32
var changed_bits : base.u32
n_bits = args.n_bits
bits = args.bits
pos = args.pos
while n_bits > 0 {
byte_pos = pos >> 3
n_unchanged_bits = (pos & 7) as base.u32
n_changed_bits = n_bits.min(no_more_than: 8 - n_unchanged_bits)
total_bits = n_unchanged_bits + n_changed_bits
mask = (0xFFFF_FFFF ^ (((1 as base.u32) << total_bits) - 1)) |
(((1 as base.u32) << n_unchanged_bits) - 1)
if byte_pos >= args.s.length() {
return nothing
}
unchanged_bits = (args.s[byte_pos] as base.u32) & mask
changed_bits = bits & (((1 as base.u32) << n_changed_bits) - 1)
args.s[byte_pos] = (((changed_bits << n_unchanged_bits) | unchanged_bits) & 0xFF) as base.u8
n_bits ~mod-= n_changed_bits
bits = bits >> n_changed_bits
pos ~mod+= n_changed_bits as base.u64
}
}
// Mirrors BrotliStoreMetaBlockHeader in compress_fragment.c (and the identical
// function in compress_fragment_two_pass.c).
pri func encoder.store_meta_block_header!(len: base.u64, is_uncompressed: base.bool, s: slice base.u8) {
var nibbles_minus_4 : base.u32[..= 2]
nibbles_minus_4 = 2
// ISLAST.
this.write_bits!(n_bits: 1, bits: 0, s: args.s)
if args.len <= 0x1_0000 {
nibbles_minus_4 = 0
} else if args.len <= 0x10_0000 {
nibbles_minus_4 = 1
}
this.write_bits!(n_bits: 2, bits: nibbles_minus_4 as base.u64, s: args.s)
this.write_bits!(n_bits: (nibbles_minus_4 + 4) * 4, bits: args.len ~mod- 1, s: args.s)
// ISUNCOMPRESSED.
if args.is_uncompressed {
this.write_bits!(n_bits: 1, bits: 1, s: args.s)
} else {
this.write_bits!(n_bits: 1, bits: 0, s: args.s)
}
}
// Mirrors Log2FloorNonZero in fast_log.h, which (with BROTLI_BSR32) truncates
// its argument to 32 bits. It returns 0 for an argument of 0.
pri func encoder.log2_floor_non_zero(n: base.u64) base.u32[..= 31] {
var x : base.u32
var r : base.u32
x = (args.n & 0xFFFF_FFFF) as base.u32
if x >= 0x1_0000 {
x >>= 16
r = 16
}
if x >= 0x100 {
x >>= 8
r ~mod+= 8
}
if x >= 0x10 {
x >>= 4
r ~mod+= 4
}
if x >= 4 {
x >>= 2
r ~mod+= 2
}
if x >= 2 {
r ~mod+= 1
}
return r & 31
}
// Mirrors FastLog2 in fast_log.h. The result is the bit pattern of an IEEE
// 754 binary64 value.
pri func encoder.fast_log2(v: base.u64) base.u64 {
if args.v < 256 {
return LOG2_TABLE[args.v]
}
return this.sf.log2_u64(v: args.v)
}
// Mirrors BrotliBitsEntropy in bit_cost.c, over this.histogram (with size
// 256). The result is the bit pattern of an IEEE 754 binary64 value.
pri func encoder.bits_entropy_of_histogram() base.u64 {
var sum : base.u64
var retval : base.u64
var i : base.u32
var p : base.u64
i = 0
while i < 256 {
p = this.histogram[i] as base.u64
sum ~mod+= p
retval = this.sf.f64_sub(a: retval, b: this.sf.f64_mul(
a: this.sf.f64_from_u64(a: p), b: this.fast_log2(v: p)))
i += 1
}
if sum <> 0 {
retval = this.sf.f64_add(a: retval, b: this.sf.f64_mul(
a: this.sf.f64_from_u64(a: sum), b: this.fast_log2(v: sum)))
}
if this.sf.f64_lt(a: retval, b: this.sf.f64_from_u64(a: sum)) <> 0 {
retval = this.sf.f64_from_u64(a: sum)
}
return retval
}
// load64le returns the 8 bytes (little-endian) of s starting at i, like the C
// code's BROTLI_UNALIGNED_LOAD64LE. The C code never reads out of bounds, so
// the zero result for an out of bounds read should never be used.
pri func encoder.load64le(s: roslice base.u8, i: base.u64) base.u64 {
var q : roslice base.u8
if args.i < args.s.length() {
q = args.s[args.i ..]
if q.length() >= 8 {
return q.peek_u64le()
}
}
return 0
}
// load32le is like load64le but for 4 bytes.
pri func encoder.load32le(s: roslice base.u8, i: base.u64) base.u32 {
var q : roslice base.u8
if args.i < args.s.length() {
q = args.s[args.i ..]
if q.length() >= 4 {
return q.peek_u32le()
}
}
return 0
}
// Mirrors FindMatchLengthWithLimit in find_match_length.h: the length of the
// common prefix of s[i1 ..] and s[i2 ..], up to limit bytes.
pri func encoder.find_match_length_with_limit(s: roslice base.u8, i1: base.u64, i2: base.u64, limit: base.u64) base.u64 {
var matched : base.u64
var n : base.u64
var a : roslice base.u8
var b : roslice base.u8
var x : base.u64
if (args.i1 > args.s.length()) or (args.i2 > args.s.length()) {
return 0
}
a = args.s[args.i1 ..]
b = args.s[args.i2 ..]
n = args.limit
while (n >= 8) and (a.length() >= 8) and (b.length() >= 8) {
x = a.peek_u64le() ^ b.peek_u64le()
if x <> 0 {
while (x & 0xFF) == 0 {
matched ~mod+= 1
x >>= 8
}
return matched
}
matched ~mod+= 8
n -= 8
a = a[8 ..]
b = b[8 ..]
}
while (n > 0) and (a.length() > 0) and (b.length() > 0) {
if a[0] <> b[0] {
break
}
matched ~mod+= 1
n -= 1
a = a[1 ..]
b = b[1 ..]
}
return matched
}
+833
View File
@@ -0,0 +1,833 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// The encoder aims for parity with the reference C implementation's encoder
// (libbrotlienc, tag v1.2.1): for the same parameters and the same sequence of
// BrotliEncoderCompressStream operations (with the same input chunking), it
// produces byte-for-byte identical output. Comments like "Mirrors Foo in
// encode.c" name the C function whose logic a Wuffs function mirrors.
//
// The C code's floating point (double) arithmetic is done with the
// std/softfloat package, which is bit-exact. FastLog2's log2 matches glibc's.
//
// Only qualities 0 and 1 (the C code's BrotliCompressFragmentFast and
// BrotliCompressFragmentTwoPass) are implemented so far. See README.md for
// the API, including how the C code's FLUSH and EMIT_METADATA operations map
// to the request_flush and request_emit_metadata methods.
//
// TODO: qualities 2 to 11 (the C code's EncodeData, WriteMetaBlockInternal,
// hashers, block splitting, context modeling and Zopfli), large-window
// brotli at those qualities and custom (prepared or shared) dictionaries.
use "std/softfloat"
pub status "#unsupported encoder quality"
pri status "#internal error: inconsistent encoder state"
pub const ENCODER_DST_HISTORY_RETAIN_LENGTH_MAX_INCL_WORST_CASE : base.u64 = 0
// The C code's BrotliEncoderStreamState values.
pri const STREAM_PROCESSING : base.u32 = 0
pri const STREAM_FLUSH_REQUESTED : base.u32 = 1
pri const STREAM_FINISHED : base.u32 = 2
pri const STREAM_METADATA_HEAD : base.u32 = 3
pri const STREAM_METADATA_BODY : base.u32 = 4
// The C code's BrotliEncoderOperation values.
pri const OP_PROCESS : base.u32 = 0
pri const OP_FLUSH : base.u32 = 1
pri const OP_FINISH : base.u32 = 2
pri const OP_EMIT_METADATA : base.u32 = 3
// The C code's BrotliEncoderFlintState values, as two's complement u32s.
pri const FLINT_NEEDS_2_BYTES : base.u32 = 2
pri const FLINT_DONE : base.u32 = 0xFFFF_FFFF
// STORAGE_SLACK is how many bytes the storage (the workbuf) needs on top of
// the C code's (2 * block_size + 503) bytes for a metablock: the 8-byte
// stores of write_bits and up to 3 bytes for InjectBytePaddingBlock.
pri const STORAGE_SLACK : base.u64 = 16
// Mirrors kCompressFragmentTwoPassBlockSize in compress_fragment_two_pass.h.
pri const TWO_PASS_BLOCK_SIZE : base.u32 = 0x2_0000
pub struct encoder? implements base.io_transformer(
// The raw parameters, as set by set_quirk (the C code's
// BrotliEncoderSetParameter). The quality and lgwin defaults (11 and
// 22) apply unless the p_quality_set and p_lgwin_set bools are set.
p_mode : base.u32,
p_quality : base.u32,
p_quality_set : base.bool,
p_lgwin : base.u32,
p_lgwin_set : base.bool,
p_lgblock : base.u32,
p_disable_lcm : base.u32,
p_size_hint : base.u32,
p_large_window : base.u32,
p_npostfix : base.u32,
p_ndirect : base.u32,
p_stream_offset : base.u32,
// The parameters after EnsureInitialized (SanitizeParams,
// ComputeLgBlock and ChooseDistanceParams).
is_initialized : base.bool,
quality : base.u32[..= 11],
lgwin : base.u32[..= 30],
lgblock : base.u32[..= 30],
large_window : base.bool,
size_hint : base.u32,
stream_offset : base.u32,
distance_postfix_bits : base.u32,
num_direct_distance_codes : base.u32,
// The C code's BrotliEncoderState fields of the same name.
dist_cache : array[4] base.u32,
saved_dist_cache : array[4] base.u32,
flint : base.u32,
last_bytes : base.u32[..= 0xFFFF],
last_bytes_bits : base.u32[..= 15],
stream_state : base.u32,
remaining_metadata_bytes : base.u32,
// The pending output: out_len bytes starting at out_pos, in either
// the tiny_buf array (if out_in_tiny_buf) or in the storage (the
// workbuf). These are the C code's next_out_ and available_out_.
out_in_tiny_buf : base.bool,
out_pos : base.u64,
out_len : base.u64,
// The pending requests. See request_flush and request_emit_metadata.
flush_requested : base.bool,
metadata_requested : base.bool,
metadata_length : base.u32,
// bw_ix is the bit writer's position, the C code's *storage_ix.
bw_ix : base.u64,
// cmd_code_numbits is the C code's BrotliOnePassArena.cmd_code_numbits.
cmd_code_numbits : base.u64,
// huffman_tree_size is BrotliStoreHuffmanTree's huffman_tree_size.
huffman_tree_size : base.u32,
// num_commands and num_literals are the lengths of command_buf and
// literal_buf, as filled by CreateCommands.
num_commands : base.u32,
num_literals : base.u32,
sf : softfloat.arith,
util : base.utility,
) + (
// tiny_buf is the C code's tiny_buf_ (plus slack for the 8-byte
// stores of write_bits).
tiny_buf : array[32] base.u8,
// These are the fields of the C code's BrotliOnePassArena and
// BrotliTwoPassArena (an encoder uses only one of the two). The
// histogram field is the C code's BrotliOnePassArena.histogram and
// BrotliTwoPassArena.lit_histo.
lit_depth : array[256] base.u8,
lit_bits : array[256] base.u16,
cmd_depth : array[128] base.u8,
cmd_bits : array[128] base.u16,
cmd_histo : array[128] base.u32,
cmd_code : array[520] base.u8,
histogram : array[256] base.u32,
tmp_depth : array[704] base.u8,
tmp_bits : array[64] base.u16,
// The C code's HuffmanTree pool. See encode_entropy.wuffs.
tree_count : array[1024] base.u32,
tree_left : array[1024] base.u16,
tree_right : array[1024] base.u16,
// BrotliStoreHuffmanTree's local variables: huffman_tree,
// huffman_tree_extra_bits, huffman_tree_histogram,
// code_length_bitdepth and code_length_bitdepth_symbols.
huffman_tree : array[1024] base.u8,
huffman_tree_extra_bits : array[1024] base.u8,
cl_histo : array[32] base.u32,
cl_depth : array[32] base.u8,
cl_bits : array[32] base.u16,
// hash_table is the C code's small_table_ or large_table_ (the
// GetHashTable result), with up to (1 << 17) elements.
hash_table : array[131072] base.u32,
// command_buf and literal_buf are the C code's command_buf_ and
// literal_buf_ (quality 1 only).
command_buf : array[131072] base.u32,
literal_buf : array[131072] base.u8,
)
pub func encoder.get_quirk(key: base.u32) base.u64 {
if args.key == QUIRK_MODE {
return this.p_mode as base.u64
} else if args.key == QUIRK_ENCODER_QUALITY {
if not this.p_quality_set {
return 11
}
return this.p_quality as base.u64
} else if args.key == QUIRK_LGWIN {
if not this.p_lgwin_set {
return 22
}
return this.p_lgwin as base.u64
} else if args.key == QUIRK_LGBLOCK {
return this.p_lgblock as base.u64
} else if args.key == QUIRK_DISABLE_LITERAL_CONTEXT_MODELING {
return this.p_disable_lcm as base.u64
} else if args.key == QUIRK_SIZE_HINT {
return this.p_size_hint as base.u64
} else if args.key == QUIRK_LARGE_WINDOW {
return this.p_large_window as base.u64
} else if args.key == QUIRK_NPOSTFIX {
return this.p_npostfix as base.u64
} else if args.key == QUIRK_NDIRECT {
return this.p_ndirect as base.u64
} else if args.key == QUIRK_STREAM_OFFSET {
return this.p_stream_offset as base.u64
}
return 0
}
// Mirrors BrotliEncoderSetParameter in encode.c.
pub func encoder.set_quirk!(key: base.u32, value: base.u64) base.status {
var v : base.u32
// Changing parameters on the fly is not implemented (in the C code).
if this.is_initialized {
return base."#bad call sequence"
} else if args.value > 0xFFFF_FFFF {
return base."#bad argument"
}
v = (args.value & 0xFFFF_FFFF) as base.u32
if args.key == QUIRK_MODE {
this.p_mode = v
} else if args.key == QUIRK_ENCODER_QUALITY {
this.p_quality = v
this.p_quality_set = true
} else if args.key == QUIRK_LGWIN {
this.p_lgwin = v
this.p_lgwin_set = true
} else if args.key == QUIRK_LGBLOCK {
this.p_lgblock = v
} else if args.key == QUIRK_DISABLE_LITERAL_CONTEXT_MODELING {
if v > 1 {
return base."#bad argument"
}
this.p_disable_lcm = v
} else if args.key == QUIRK_SIZE_HINT {
this.p_size_hint = v
} else if args.key == QUIRK_LARGE_WINDOW {
if v <> 0 {
this.p_large_window = 1
} else {
this.p_large_window = 0
}
} else if args.key == QUIRK_NPOSTFIX {
this.p_npostfix = v
} else if args.key == QUIRK_NDIRECT {
this.p_ndirect = v
} else if args.key == QUIRK_STREAM_OFFSET {
if v > 0x4000_0000 {
return base."#bad argument"
}
this.p_stream_offset = v
} else {
return base."#unsupported option"
}
return ok
}
pub func encoder.dst_history_retain_length() base.optional_u63 {
return this.util.make_optional_u63(has_value: true, value: 0)
}
// workbuf_len returns the workbuf length needed for the current parameters
// (after the C code's SanitizeParams). At qualities 0 and 1 the workbuf is
// the storage for one metablock's output: ((2 << lgwin) + 503) bytes (the C
// code's GetBrotliStorage size for a (1 << lgwin) byte block) plus 16 bytes
// of slack. A shorter workbuf still works if every transform_io input block
// is short enough: a block of n bytes needs ((2 * n) + 519) bytes.
pub func encoder.workbuf_len() base.range_ii_u64 {
var q : base.u32[..= 11]
var n : base.u64
q = this.sanitized_quality()
if q <= 1 {
n = ((2 as base.u64) << this.sanitized_lgwin(quality: q)) + 503 + STORAGE_SLACK
}
return this.util.make_range_ii_u64(min_incl: n, max_incl: n)
}
// max_compressed_size mirrors BrotliEncoderMaxCompressedSize in encode.c:
// an upper bound on the compressed size of input_size bytes (with an
// uncompressed fallback, which this encoder does not do itself), or 0 on
// overflow.
pub func encoder.max_compressed_size(input_size: base.u64) base.u64 {
var num_large_blocks : base.u64
var overhead : base.u64
var result : base.u64
num_large_blocks = args.input_size >> 14
overhead = 2 + (4 * num_large_blocks) + 3 + 1
result = args.input_size ~mod+ overhead
if args.input_size == 0 {
return 2
} else if result < args.input_size {
return 0
}
return result
}
// request_flush makes the next transform_io call act like the C code's
// BrotliEncoderCompressStream with BROTLI_OPERATION_FLUSH (instead of
// PROCESS or FINISH), until the flush is complete. It is complete once a
// transform_io call returns "$short read" (or ok).
pub func encoder.request_flush!() base.status {
if this.metadata_requested {
return base."#bad call sequence"
}
this.flush_requested = true
return ok
}
// request_emit_metadata makes the next transform_io call act like the C
// code's BrotliEncoderCompressStream with BROTLI_OPERATION_EMIT_METADATA,
// with the next length bytes of src as the metadata. Once those length bytes
// have been read (transform_io returns "$short read" if src has fewer of
// them), and the metadata block written, transform_io continues as usual.
// Like the C code, length can be at most (1 << 24).
pub func encoder.request_emit_metadata!(length: base.u64) base.status {
if args.length > 0x100_0000 {
return base."#bad argument"
} else if this.flush_requested or this.metadata_requested {
return base."#bad call sequence"
}
this.metadata_requested = true
this.metadata_length = (args.length & 0xFFFF_FFFF) as base.u32
return ok
}
// sanitized_quality mirrors SanitizeParams in quality.h, for the quality.
pri func encoder.sanitized_quality() base.u32[..= 11] {
if not this.p_quality_set {
return 11
} else if this.p_quality >= 0x8000_0000 {
// A negative (int) value.
return 0
}
return this.p_quality.min(no_more_than: 11)
}
// sanitized_lgwin mirrors SanitizeParams in quality.h, for the lgwin.
pri func encoder.sanitized_lgwin(quality: base.u32[..= 11]) base.u32[..= 30] {
var lgwin : base.u32
lgwin = 22
if this.p_lgwin_set {
lgwin = this.p_lgwin
}
if (lgwin >= 0x8000_0000) or (lgwin < 10) {
return 10
} else if (this.p_large_window <> 0) and (args.quality > 2) {
return lgwin.min(no_more_than: 30)
}
return lgwin.min(no_more_than: 24)
}
// Mirrors EnsureInitialized (and BrotliEncoderInitState's distance cache
// initialization) in encode.c.
pri func encoder.ensure_initialized!() {
var lgblock : base.u32
var lgwin : base.u32[..= 30]
var limit : base.u32
var npostfix : base.u32
var ndirect : base.u32
var ndirect_msb : base.u32
var i : base.u32
if this.is_initialized {
return nothing
}
// BrotliEncoderInitState.
this.dist_cache[0] = 4
this.dist_cache[1] = 11
this.dist_cache[2] = 15
this.dist_cache[3] = 16
this.saved_dist_cache[0] = 4
this.saved_dist_cache[1] = 11
this.saved_dist_cache[2] = 15
this.saved_dist_cache[3] = 16
this.stream_state = STREAM_PROCESSING
this.last_bytes_bits = 0
this.last_bytes = 0
this.flint = FLINT_DONE
this.remaining_metadata_bytes = 0xFFFF_FFFF
// SanitizeParams.
this.quality = this.sanitized_quality()
this.large_window = (this.p_large_window <> 0) and (this.quality > 2)
this.lgwin = this.sanitized_lgwin(quality: this.quality)
this.size_hint = this.p_size_hint
this.stream_offset = this.p_stream_offset
// ComputeLgBlock.
if this.quality <= 1 {
this.lgblock = this.lgwin
} else if this.quality < 4 {
this.lgblock = 14
} else if this.p_lgblock == 0 {
this.lgblock = 16
if (this.quality >= 9) and (this.lgwin > 16) {
this.lgblock = this.lgwin.min(no_more_than: 18)
}
} else {
lgblock = this.p_lgblock
if lgblock >= 0x8000_0000 {
// A negative (int) value.
lgblock = 16
}
this.lgblock = lgblock.max(no_less_than: 16).min(no_more_than: 24)
}
// ChooseDistanceParams. BrotliInitDistanceParams' other fields are not
// needed at qualities 0 and 1.
if this.quality >= 4 {
if this.p_mode == 2 {
npostfix = 1
ndirect = 12
} else {
npostfix = this.p_npostfix
ndirect = this.p_ndirect
}
if npostfix > 3 {
npostfix = 0
ndirect = 0
} else {
ndirect_msb = (ndirect >> npostfix) & 0x0F
if (ndirect > 120) or ((ndirect_msb << npostfix) <> ndirect) {
npostfix = 0
ndirect = 0
}
}
}
this.distance_postfix_bits = npostfix
this.num_direct_distance_codes = ndirect
if this.stream_offset <> 0 {
this.flint = FLINT_NEEDS_2_BYTES
// Poison the distance cache. -16 +- 3 is still less than zero
// (invalid).
i = 0
while i < 4 {
this.dist_cache[i] = 0xFFFF_FFF0
this.saved_dist_cache[i] = 0xFFFF_FFF0
i += 1
}
}
// Initialize last byte with stream header.
lgwin = this.lgwin
if this.quality <= 1 {
lgwin = lgwin.max(no_less_than: 18)
}
if this.stream_offset == 0 {
this.encode_window_bits!(lgwin: lgwin)
} else {
// Bigger values have the same effect, but could cause overflows.
limit = ((1 as base.u32) << lgwin) ~mod- 16
this.stream_offset = this.stream_offset.min(no_more_than: limit)
}
if this.quality == 0 {
this.init_command_prefix_codes!()
}
this.is_initialized = true
}
// Mirrors EncodeWindowBits in encode.c.
pri func encoder.encode_window_bits!(lgwin: base.u32[..= 30]) {
if this.large_window {
this.last_bytes = ((args.lgwin & 0x3F) << 8) | 0x11
this.last_bytes_bits = 14
} else if args.lgwin == 16 {
this.last_bytes = 0
this.last_bytes_bits = 1
} else if args.lgwin == 17 {
this.last_bytes = 1
this.last_bytes_bits = 7
} else if args.lgwin > 17 {
this.last_bytes = ((args.lgwin - 17) << 1) | 0x01
this.last_bytes_bits = 4
} else if args.lgwin >= 8 {
this.last_bytes = ((args.lgwin - 8) << 4) | 0x01
this.last_bytes_bits = 7
}
}
// Mirrors InitCommandPrefixCodes in encode.c.
pri func encoder.init_command_prefix_codes!() {
var i : base.u32
i = 0
while i < 128 {
this.cmd_depth[i] = DEFAULT_COMMAND_DEPTHS[i]
this.cmd_bits[i] = DEFAULT_COMMAND_BITS[i]
i += 1
}
i = 0
while i < 57 {
this.cmd_code[i] = DEFAULT_COMMAND_CODE[i]
i += 1
}
this.cmd_code_numbits = DEFAULT_COMMAND_CODE_NUM_BITS as base.u64
}
// Mirrors HashTableSize and GetHashTable in encode.c. It zeroes the first
// htsize elements of this.hash_table and returns log2(htsize), the C code's
// table_bits (Log2FloorNonZero(table_size) in BrotliCompressFragmentFast and
// BrotliCompressFragmentTwoPass).
pri func encoder.get_hash_table!(input_size: base.u64) base.u32 {
var max_table_size : base.u64
var htsize : base.u64
var table_bits : base.u32
// MaxHashTableSize in quality.h.
max_table_size = 0x2_0000
if this.quality == 0 {
max_table_size = 0x8000
}
htsize = 256
table_bits = 8
while (htsize < max_table_size) and (htsize < args.input_size) {
htsize ~mod<<= 1
table_bits ~mod+= 1
}
if this.quality == 0 {
// Only odd shifts are supported by fast-one-pass.
if (htsize & 0xA_AAAA) == 0 {
htsize ~mod<<= 1
table_bits ~mod+= 1
}
}
if htsize <= 0x2_0000 {
this.hash_table[.. htsize].bulk_memset!(byte_value: 0)
}
return table_bits
}
// Mirrors InjectBytePaddingBlock in encode.c. Its destination is right after
// the pending output (in the storage or in tiny_buf).
pri func encoder.inject_byte_padding_block!(workbuf: slice base.u8) {
var seal : base.u32
var seal_bits : base.u32
var p : base.u64
var n : base.u32
seal = this.last_bytes
seal_bits = this.last_bytes_bits
this.last_bytes = 0
this.last_bytes_bits = 0
// is_last = 0, data_nibbles = 11, reserved = 0, meta_nibbles = 00.
seal |= (6 as base.u32) << seal_bits
seal_bits += 6
n = (seal_bits + 7) >> 3
if this.out_len == 0 {
this.out_in_tiny_buf = true
this.out_pos = 0
}
p = this.out_pos ~mod+ this.out_len
if this.out_in_tiny_buf {
if p <= 24 {
this.tiny_buf[p] = (seal & 0xFF) as base.u8
this.tiny_buf[p + 1] = ((seal >> 8) & 0xFF) as base.u8
this.tiny_buf[p + 2] = ((seal >> 16) & 0xFF) as base.u8
}
} else if p < args.workbuf.length() {
args.workbuf[p] = (seal & 0xFF) as base.u8
if seal_bits > 8 {
p ~mod+= 1
if p < args.workbuf.length() {
args.workbuf[p] = ((seal >> 8) & 0xFF) as base.u8
}
}
if seal_bits > 16 {
p ~mod+= 1
if p < args.workbuf.length() {
args.workbuf[p] = ((seal >> 16) & 0xFF) as base.u8
}
}
}
this.out_len ~mod+= n as base.u64
}
// push_output copies as much pending output to args.dst as fits, like the
// second half of InjectFlushOrPushOutput in encode.c.
pri func encoder.push_output!(dst: base.io_writer, workbuf: slice base.u8) {
var n : base.u64
var end : base.u64
end = this.out_pos ~mod+ this.out_len
if this.out_in_tiny_buf {
if (this.out_pos <= end) and (end <= 32) {
n = args.dst.copy_from_slice!(s: this.tiny_buf[this.out_pos .. end])
}
} else if (this.out_pos <= end) and (end <= args.workbuf.length()) {
n = args.dst.copy_from_slice!(s: args.workbuf[this.out_pos .. end])
}
if n == 0 {
// Should be unreachable, since the caller checks that args.dst is
// not full. Discard the output so that the caller makes progress.
this.out_len = 0
return nothing
}
n = n.min(no_more_than: this.out_len)
this.out_pos ~mod+= n
this.out_len ~mod-= n
}
// Mirrors CheckFlushComplete in encode.c.
pri func encoder.check_flush_complete!() {
if (this.stream_state == STREAM_FLUSH_REQUESTED) and (this.out_len == 0) {
this.stream_state = STREAM_PROCESSING
this.out_in_tiny_buf = false
this.out_pos = 0
}
}
// Mirrors WriteMetadataHeader in encode.c, writing to tiny_buf.
pri func encoder.write_metadata_header!(block_size: base.u32) base.u64 {
var nbits : base.u32[..= 32]
var nbytes : base.u32[..= 4]
this.bw_ix = this.last_bytes_bits as base.u64
this.tiny_buf[0] = (this.last_bytes & 0xFF) as base.u8
this.tiny_buf[1] = (this.last_bytes >> 8) as base.u8
this.last_bytes = 0
this.last_bytes_bits = 0
this.write_bits!(n_bits: 1, bits: 0, s: this.tiny_buf[..])
this.write_bits!(n_bits: 2, bits: 3, s: this.tiny_buf[..])
this.write_bits!(n_bits: 1, bits: 0, s: this.tiny_buf[..])
if args.block_size == 0 {
this.write_bits!(n_bits: 2, bits: 0, s: this.tiny_buf[..])
} else {
nbits = 1
if args.block_size > 1 {
nbits = this.log2_floor_non_zero(n: (args.block_size - 1) as base.u64) + 1
}
nbytes = (nbits + 7) / 8
this.write_bits!(n_bits: 2, bits: nbytes as base.u64, s: this.tiny_buf[..])
this.write_bits!(n_bits: 8 * nbytes, bits: (args.block_size - 1) as base.u64, s: this.tiny_buf[..])
}
return (this.bw_ix ~mod+ 7) >> 3
}
// transform_io mirrors BrotliEncoderCompressStream in encode.c (and, for
// qualities 0 and 1, BrotliEncoderCompressStreamFast). Each (initial or
// resumed) transform_io call corresponds to one or more C calls. See
// README.md for the details.
pub func encoder.transform_io?(dst: base.io_writer, src: base.io_reader, workbuf: slice base.u8) {
var op : base.u32
var avail : base.u64
var block_size : base.u64
var is_last : base.bool
var force_flush : base.bool
var storage_len : base.u64
var table_bits : base.u32
var mark : base.u64
var n32 : base.u32
var ix : base.u64
this.ensure_initialized!()
if this.quality > 1 {
// TODO: implement the C code's EncodeData (and its callees) for
// qualities 2 to 11.
return "#unsupported encoder quality"
}
while true {
// ---- The start of a C BrotliEncoderCompressStream call.
if this.metadata_requested {
// op == BROTLI_OPERATION_EMIT_METADATA. UpdateSizeHint(s, 0) is a
// no-op at qualities 0 and 1 (and so is ProcessMetadata's
// EncodeData call, as the fast path never advances input_pos_).
//
// Mirrors ProcessMetadata in encode.c.
if this.stream_state == STREAM_PROCESSING {
this.remaining_metadata_bytes = this.metadata_length
this.stream_state = STREAM_METADATA_HEAD
}
if (this.stream_state <> STREAM_METADATA_HEAD) and
(this.stream_state <> STREAM_METADATA_BODY) {
return base."#bad call sequence"
}
while true {
if this.out_len <> 0 {
if args.dst.length() <= 0 {
yield? base."$short write"
continue
}
this.push_output!(dst: args.dst, workbuf: args.workbuf)
continue
}
if this.stream_state == STREAM_METADATA_HEAD {
this.out_len = this.write_metadata_header!(block_size: this.remaining_metadata_bytes)
this.out_in_tiny_buf = true
this.out_pos = 0
this.stream_state = STREAM_METADATA_BODY
continue
}
// Exit workflow only when there is no more input and no more
// output.
if this.remaining_metadata_bytes == 0 {
this.remaining_metadata_bytes = 0xFFFF_FFFF
this.stream_state = STREAM_PROCESSING
this.metadata_requested = false
break
}
if args.dst.length() <= 0 {
yield? base."$short write"
continue
} else if args.src.length() <= 0 {
if args.src.is_closed() {
return "#truncated input"
}
yield? base."$short read"
continue
}
// Directly copy input to output.
n32 = args.dst.limited_copy_u32_from_reader!(
up_to: this.remaining_metadata_bytes, r: args.src)
this.remaining_metadata_bytes ~mod-= n32
}
continue
}
if this.flush_requested {
op = OP_FLUSH
} else if args.src.is_closed() {
op = OP_FINISH
} else {
op = OP_PROCESS
}
if (this.stream_state == STREAM_METADATA_HEAD) or
(this.stream_state == STREAM_METADATA_BODY) {
return base."#bad call sequence"
} else if (this.stream_state <> STREAM_PROCESSING) and (args.src.length() > 0) {
return base."#bad call sequence"
}
// Mirrors BrotliEncoderCompressStreamFast in encode.c.
while true {
// Mirrors InjectFlushOrPushOutput in encode.c.
if (this.stream_state == STREAM_FLUSH_REQUESTED) and (this.last_bytes_bits <> 0) {
this.inject_byte_padding_block!(workbuf: args.workbuf)
continue
}
if (this.out_len <> 0) and (args.dst.length() > 0) {
this.push_output!(dst: args.dst, workbuf: args.workbuf)
continue
}
// Compress block only when internal output buffer is empty,
// stream is not finished, there is no pending flush request, and
// there is either additional input or pending operation.
if (this.out_len == 0) and (this.stream_state == STREAM_PROCESSING) and
((args.src.length() > 0) or (op <> OP_PROCESS)) {
avail = args.src.length()
block_size = avail.min(no_more_than: (1 as base.u64) << this.lgwin)
is_last = (avail == block_size) and (op == OP_FINISH)
force_flush = (avail == block_size) and (op == OP_FLUSH)
if force_flush and (block_size == 0) {
this.stream_state = STREAM_FLUSH_REQUESTED
continue
}
storage_len = (block_size * 2) + 503 + STORAGE_SLACK
if (storage_len > args.workbuf.length()) or (args.workbuf.length() < 2) {
return base."#bad workbuf length"
}
args.workbuf[0] = (this.last_bytes & 0xFF) as base.u8
args.workbuf[1] = (this.last_bytes >> 8) as base.u8
this.bw_ix = this.last_bytes_bits as base.u64
table_bits = this.get_hash_table!(input_size: block_size)
// The C code compresses straight from next_in. So does this
// code, from the args.src bytes that it skips over.
mark = args.src.mark()
args.src.skip?(n: block_size)
if storage_len > args.workbuf.length() {
return "#internal error: inconsistent encoder state"
}
if this.quality == 0 {
this.compress_fragment_fast!(
input: args.src.since(mark: mark),
is_last: is_last,
table_bits: table_bits,
s: args.workbuf[.. storage_len])
} else {
this.compress_fragment_two_pass!(
input: args.src.since(mark: mark),
is_last: is_last,
table_bits: table_bits,
s: args.workbuf[.. storage_len])
}
this.out_in_tiny_buf = false
this.out_pos = 0
this.out_len = this.bw_ix >> 3
ix = this.bw_ix >> 3
if ix < args.workbuf.length() {
this.last_bytes = args.workbuf[ix] as base.u32
} else {
this.last_bytes = 0
}
this.last_bytes_bits = (this.bw_ix & 7) as base.u32
if force_flush {
this.stream_state = STREAM_FLUSH_REQUESTED
}
if is_last {
this.stream_state = STREAM_FINISHED
}
continue
}
break
}
this.check_flush_complete!()
// ---- The end of a C BrotliEncoderCompressStream call.
if this.out_len <> 0 {
yield? base."$short write"
} else if op == OP_FLUSH {
this.flush_requested = false
} else if this.stream_state == STREAM_FINISHED {
return ok
} else if op == OP_PROCESS {
yield? base."$short read"
} else {
return "#internal error: inconsistent encoder state"
}
}
}
+804
View File
@@ -0,0 +1,804 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// This file holds the Huffman (prefix code) construction and storage
// functions that the quality 0 and quality 1 encoders need, from the C code's
// entropy_encode.c and brotli_bit_stream.c.
//
// The C code's HuffmanTree array (a pool of nodes) is the encoder.tree_count,
// encoder.tree_left and encoder.tree_right arrays. A node's int16_t
// index_left_ and index_right_or_value_ fields are stored as base.u16 values,
// with 0xFFFF meaning -1. The C code's pools hold at most (2 * 256) + 1
// nodes. The arrays have 1024 elements so that indexes can be masked.
// TREE_NONE is the C code's -1 for a HuffmanTree index.
pri const TREE_NONE : base.u16 = 0xFFFF
// Mirrors BrotliSetDepth in entropy_encode.c.
pri func encoder.set_depth!(p0: base.u32, depth: slice base.u8, max_depth: base.u32[..= 15]) base.bool {
var stack : array[16] base.u32
var level : base.u32[..= 15]
var p : base.u32
var v : base.u64
stack[0] = 0xFFFF_FFFF
p = args.p0
while true {
if this.tree_left[p & 1023] <> TREE_NONE {
if (level >= args.max_depth) or (level >= 15) {
return false
}
level += 1
stack[level] = this.tree_right[p & 1023] as base.u32
p = this.tree_left[p & 1023] as base.u32
continue
}
v = this.tree_right[p & 1023] as base.u64
if v < args.depth.length() {
args.depth[v] = level as base.u8
}
while stack[level] == 0xFFFF_FFFF {
if level == 0 {
return true
}
level -= 1
}
p = stack[level]
stack[level] = 0xFFFF_FFFF
}
}
// huffman_tree_less is the C code's SortHuffmanTree comparators. With
// tie_break, it mirrors the one in entropy_encode.c (ties are broken by
// index_right_or_value_, in decreasing order). Without, it mirrors the one in
// brotli_bit_stream.c (only total_count_ is compared).
pri func encoder.huffman_tree_less(c0: base.u32, r0: base.u16, c1: base.u32, r1: base.u16, tie_break: base.bool) base.bool {
if args.c0 <> args.c1 {
return args.c0 < args.c1
}
return args.tie_break and (args.r0 > args.r1)
}
// Mirrors SortHuffmanTreeItems in entropy_encode.h, sorting the first n
// nodes of the pool.
pri func encoder.sort_huffman_tree_items!(n: base.u32, tie_break: base.bool) {
var i : base.u32
var j : base.u32
var k : base.u32
var g : base.u32
var gap : base.u32
var tc : base.u32
var tl : base.u16
var tr : base.u16
if args.n < 13 {
// Insertion sort.
i = 1
while i < args.n {
tc = this.tree_count[i & 1023]
tl = this.tree_left[i & 1023]
tr = this.tree_right[i & 1023]
k = i
j = i ~mod- 1
while this.huffman_tree_less(
c0: tc,
r0: tr,
c1: this.tree_count[j & 1023],
r1: this.tree_right[j & 1023],
tie_break: args.tie_break) {
this.tree_count[k & 1023] = this.tree_count[j & 1023]
this.tree_left[k & 1023] = this.tree_left[j & 1023]
this.tree_right[k & 1023] = this.tree_right[j & 1023]
k = j
if j == 0 {
break
}
j -= 1
}
this.tree_count[k & 1023] = tc
this.tree_left[k & 1023] = tl
this.tree_right[k & 1023] = tr
i ~mod+= 1
}
return nothing
}
// Shell sort.
g = 0
if args.n < 57 {
g = 2
}
while g < 6 {
gap = SHELL_GAPS[g]
i = gap
while i < args.n {
j = i
tc = this.tree_count[i & 1023]
tl = this.tree_left[i & 1023]
tr = this.tree_right[i & 1023]
while j >= gap {
if not this.huffman_tree_less(
c0: tc,
r0: tr,
c1: this.tree_count[(j ~mod- gap) & 1023],
r1: this.tree_right[(j ~mod- gap) & 1023],
tie_break: args.tie_break) {
break
}
this.tree_count[j & 1023] = this.tree_count[(j ~mod- gap) & 1023]
this.tree_left[j & 1023] = this.tree_left[(j ~mod- gap) & 1023]
this.tree_right[j & 1023] = this.tree_right[(j ~mod- gap) & 1023]
j ~mod-= gap
}
this.tree_count[j & 1023] = tc
this.tree_left[j & 1023] = tl
this.tree_right[j & 1023] = tr
i ~mod+= 1
}
g ~mod+= 1
}
}
// set_sentinel sets the pool's i'th node to the C code's sentinel node.
pri func encoder.set_sentinel!(i: base.u32) {
this.tree_count[args.i & 1023] = 0xFFFF_FFFF
this.tree_left[args.i & 1023] = TREE_NONE
this.tree_right[args.i & 1023] = TREE_NONE
}
// merge_huffman_nodes runs the C code's "for (k = n - 1; k != 0; --k)" loop
// (shared by BrotliCreateHuffmanTree in entropy_encode.c and
// BrotliBuildAndStoreHuffmanTreeFast in brotli_bit_stream.c) that combines
// the n sorted leaf nodes into a tree, whose root is node ((2 * n) - 1).
pri func encoder.merge_huffman_nodes!(n: base.u32) {
var i : base.u32
var j : base.u32
var k : base.u32
var left : base.u32
var right : base.u32
var j_end : base.u32
this.set_sentinel!(i: args.n)
this.set_sentinel!(i: args.n ~mod+ 1)
i = 0
j = args.n ~mod+ 1
k = args.n ~mod- 1
while k <> 0 {
if this.tree_count[i & 1023] <= this.tree_count[j & 1023] {
left = i
i ~mod+= 1
} else {
left = j
j ~mod+= 1
}
if this.tree_count[i & 1023] <= this.tree_count[j & 1023] {
right = i
i ~mod+= 1
} else {
right = j
j ~mod+= 1
}
j_end = (args.n ~mod* 2) ~mod- k
this.tree_count[j_end & 1023] = this.tree_count[left & 1023] ~mod+ this.tree_count[right & 1023]
this.tree_left[j_end & 1023] = (left & 0xFFFF) as base.u16
this.tree_right[j_end & 1023] = (right & 0xFFFF) as base.u16
this.set_sentinel!(i: j_end ~mod+ 1)
k -= 1
}
}
// Mirrors BrotliCreateHuffmanTree in entropy_encode.c. Wuffs' C code
// generator does not support "slice base.u32" arguments, so the C code's data
// argument is either this.cmd_histo[args.off ..] (when args.src is 0) or
// this.cl_histo[args.off ..] (when args.src is 1).
pri func encoder.create_huffman_tree!(src: base.u32, off: base.u32, length: base.u32, tree_limit: base.u32[..= 15], depth: slice base.u8) {
var count_limit : base.u32
var n : base.u32
var i : base.u64
var count : base.u32
var v : base.u64
var depth_ok : base.bool
count_limit = 1
while true {
n = 0
i = args.length as base.u64
while i > 0 {
i -= 1
if args.src == 0 {
count = this.cmd_histo[((args.off as base.u64) ~mod+ i) & 127]
} else {
count = this.cl_histo[((args.off as base.u64) ~mod+ i) & 31]
}
if count <> 0 {
this.tree_count[n & 1023] = count.max(no_less_than: count_limit)
this.tree_left[n & 1023] = TREE_NONE
this.tree_right[n & 1023] = (i & 0xFFFF) as base.u16
n ~mod+= 1
}
}
if n == 1 {
v = this.tree_right[0] as base.u64
if v < args.depth.length() {
args.depth[v] = 1
}
break
} else if n == 0 {
// The C code never calls this with an all-zero histogram.
break
}
this.sort_huffman_tree_items!(n: n, tie_break: true)
this.merge_huffman_nodes!(n: n)
depth_ok = this.set_depth!(p0: (n ~mod* 2) ~mod- 1, depth: args.depth, max_depth: args.tree_limit)
if depth_ok {
break
}
count_limit ~mod*= 2
}
}
// Mirrors BrotliReverseBits in entropy_encode.c.
pri func encoder.reverse_bits(num_bits: base.u32, bits: base.u16) base.u16 {
var retval : base.u32
var bits : base.u32
var i : base.u32
bits = args.bits as base.u32
retval = REVERSE_BITS_LUT[bits & 15]
i = 4
while i < args.num_bits {
retval = (retval ~mod<< 4) | REVERSE_BITS_LUT[(bits >> 4) & 15]
bits >>= 4
i ~mod+= 4
}
retval >>= (0 ~mod- args.num_bits) & 3
return (retval & 0xFFFF) as base.u16
}
// Mirrors BrotliConvertBitDepthsToSymbols in entropy_encode.c. Wuffs' C code
// generator does not support "slice base.u16" arguments, so the C code's bits
// argument is this.tmp_bits, this.cmd_bits[64 ..], this.cl_bits or
// this.lit_bits when args.dst is 0, 1, 2 or 3.
pri func encoder.convert_bit_depths_to_symbols!(depth: slice base.u8, len: base.u32, dst: base.u32) {
var bl_count : array[16] base.u16
var next_code : array[16] base.u16
var i : base.u64
var code : base.u32
var d : base.u32
var r : base.u16
i = 0
while i < (args.len as base.u64) {
if i < args.depth.length() {
d = (args.depth[i] & 15) as base.u32
bl_count[d] ~mod+= 1
}
i ~mod+= 1
}
bl_count[0] = 0
next_code[0] = 0
i = 1
while i < 16 {
code = (code ~mod+ (bl_count[(i ~mod- 1) & 15] as base.u32)) ~mod<< 1
next_code[i] = (code & 0xFFFF) as base.u16
i ~mod+= 1
}
i = 0
while i < (args.len as base.u64) {
if i < args.depth.length() {
d = (args.depth[i] & 15) as base.u32
if d <> 0 {
r = this.reverse_bits(num_bits: d, bits: next_code[d])
if args.dst == 0 {
this.tmp_bits[i & 63] = r
} else if args.dst == 1 {
this.cmd_bits[(i ~mod+ 64) & 127] = r
} else if args.dst == 2 {
this.cl_bits[i & 31] = r
} else {
this.lit_bits[i & 255] = r
}
next_code[d] ~mod+= 1
}
}
i ~mod+= 1
}
}
// Mirrors Reverse in entropy_encode.c, applied to both the huffman_tree and
// huffman_tree_extra_bits arrays.
pri func encoder.reverse_huffman_tree!(start: base.u32, end: base.u32) {
var start : base.u32
var end : base.u32
var tmp : base.u8
start = args.start
end = args.end ~mod- 1
while start < end {
tmp = this.huffman_tree[start & 1023]
this.huffman_tree[start & 1023] = this.huffman_tree[end & 1023]
this.huffman_tree[end & 1023] = tmp
tmp = this.huffman_tree_extra_bits[start & 1023]
this.huffman_tree_extra_bits[start & 1023] = this.huffman_tree_extra_bits[end & 1023]
this.huffman_tree_extra_bits[end & 1023] = tmp
start ~mod+= 1
end ~mod-= 1
}
}
// push_huffman_tree appends (value, extra_bits) to the huffman_tree and
// huffman_tree_extra_bits arrays.
pri func encoder.push_huffman_tree!(value: base.u8, extra_bits: base.u8) {
this.huffman_tree[this.huffman_tree_size & 1023] = args.value
this.huffman_tree_extra_bits[this.huffman_tree_size & 1023] = args.extra_bits
this.huffman_tree_size ~mod+= 1
}
// Mirrors BrotliWriteHuffmanTreeRepetitions in entropy_encode.c.
pri func encoder.write_huffman_tree_repetitions!(previous_value: base.u8, value: base.u8, repetitions: base.u32) {
var repetitions : base.u32
var start : base.u32
repetitions = args.repetitions
if args.previous_value <> args.value {
this.push_huffman_tree!(value: args.value, extra_bits: 0)
repetitions ~mod-= 1
}
if repetitions == 7 {
this.push_huffman_tree!(value: args.value, extra_bits: 0)
repetitions ~mod-= 1
}
if repetitions < 3 {
while repetitions > 0 {
this.push_huffman_tree!(value: args.value, extra_bits: 0)
repetitions -= 1
}
} else {
start = this.huffman_tree_size
repetitions -= 3
while true {
this.push_huffman_tree!(value: 16, extra_bits: (repetitions & 3) as base.u8)
repetitions >>= 2
if repetitions == 0 {
break
}
repetitions -= 1
}
this.reverse_huffman_tree!(start: start, end: this.huffman_tree_size)
}
}
// Mirrors BrotliWriteHuffmanTreeRepetitionsZeros in entropy_encode.c.
pri func encoder.write_huffman_tree_repetitions_zeros!(repetitions: base.u32) {
var repetitions : base.u32
var start : base.u32
repetitions = args.repetitions
if repetitions == 11 {
this.push_huffman_tree!(value: 0, extra_bits: 0)
repetitions -= 1
}
if repetitions < 3 {
while repetitions > 0 {
this.push_huffman_tree!(value: 0, extra_bits: 0)
repetitions -= 1
}
} else {
start = this.huffman_tree_size
repetitions -= 3
while true {
this.push_huffman_tree!(value: 17, extra_bits: (repetitions & 7) as base.u8)
repetitions >>= 3
if repetitions == 0 {
break
}
repetitions -= 1
}
this.reverse_huffman_tree!(start: start, end: this.huffman_tree_size)
}
}
// Mirrors DecideOverRleUse in entropy_encode.c. The result's 0x01 bit is
// use_rle_for_non_zero and its 0x02 bit is use_rle_for_zero.
pri func encoder.decide_over_rle_use(depth: slice base.u8, length: base.u32) base.u32 {
var total_reps_zero : base.u32
var total_reps_non_zero : base.u32
var count_reps_zero : base.u32
var count_reps_non_zero : base.u32
var i : base.u64
var k : base.u64
var length : base.u64
var reps : base.u32
var value : base.u8
var ret : base.u32
length = args.length as base.u64
count_reps_zero = 1
count_reps_non_zero = 1
i = 0
while i < length {
if i >= args.depth.length() {
break
}
value = args.depth[i]
reps = 1
k = i ~mod+ 1
while (k < length) and (k < args.depth.length()) {
if args.depth[k] <> value {
break
}
reps ~mod+= 1
k ~mod+= 1
}
if (reps >= 3) and (value == 0) {
total_reps_zero ~mod+= reps
count_reps_zero ~mod+= 1
}
if (reps >= 4) and (value <> 0) {
total_reps_non_zero ~mod+= reps
count_reps_non_zero ~mod+= 1
}
i ~mod+= reps as base.u64
}
if total_reps_non_zero > (count_reps_non_zero ~mod* 2) {
ret |= 1
}
if total_reps_zero > (count_reps_zero ~mod* 2) {
ret |= 2
}
return ret
}
// Mirrors BrotliWriteHuffmanTree in entropy_encode.c. The output is appended
// to the huffman_tree and huffman_tree_extra_bits arrays.
pri func encoder.write_huffman_tree!(depth: slice base.u8, length: base.u32) {
var previous_value : base.u8
var i : base.u64
var k : base.u64
var length : base.u64
var reps : base.u32
var new_length : base.u64
var rle : base.u32
var value : base.u8
previous_value = 8 // BROTLI_INITIAL_REPEATED_CODE_LENGTH.
length = args.length as base.u64
// Throw away trailing zeros.
new_length = length
i = 0
while i < length {
k = (length ~mod- i) ~mod- 1
if k >= args.depth.length() {
break
}
if args.depth[k] == 0 {
new_length ~mod-= 1
} else {
break
}
i ~mod+= 1
}
// First gather statistics on if it is a good idea to do RLE.
if length > 50 {
rle = this.decide_over_rle_use(depth: args.depth, length: (new_length & 0xFFFF_FFFF) as base.u32)
}
// Actual RLE coding.
i = 0
while i < new_length {
if i >= args.depth.length() {
break
}
value = args.depth[i]
reps = 1
if ((value <> 0) and ((rle & 1) <> 0)) or ((value == 0) and ((rle & 2) <> 0)) {
k = i ~mod+ 1
while (k < new_length) and (k < args.depth.length()) {
if args.depth[k] <> value {
break
}
reps ~mod+= 1
k ~mod+= 1
}
}
if value == 0 {
this.write_huffman_tree_repetitions_zeros!(repetitions: reps)
} else {
this.write_huffman_tree_repetitions!(previous_value: previous_value, value: value, repetitions: reps)
previous_value = value
}
i ~mod+= reps as base.u64
}
}
// Mirrors BrotliStoreHuffmanTreeOfHuffmanTreeToBitMask in
// brotli_bit_stream.c, using this.cl_depth as code_length_bitdepth.
pri func encoder.store_huffman_tree_of_huffman_tree_to_bit_mask!(num_codes: base.u32, s: slice base.u8) {
var skip_some : base.u32
var codes_to_store : base.u32[..= 18]
var i : base.u32
var l : base.u32
// Throw away trailing zeros.
codes_to_store = 18
if args.num_codes > 1 {
while codes_to_store > 0 {
if this.cl_depth[STORAGE_ORDER[codes_to_store - 1]] <> 0 {
break
}
codes_to_store -= 1
}
}
if (this.cl_depth[STORAGE_ORDER[0]] == 0) and (this.cl_depth[STORAGE_ORDER[1]] == 0) {
skip_some = 2
if this.cl_depth[STORAGE_ORDER[2]] == 0 {
skip_some = 3
}
}
this.write_bits!(n_bits: 2, bits: skip_some as base.u64, s: args.s)
i = skip_some
while (i < (codes_to_store as base.u32)) and (i < 18) {
l = this.cl_depth[STORAGE_ORDER[i]] as base.u32
if l < 6 {
this.write_bits!(
n_bits: HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_BIT_LENGTHS[l],
bits: HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_SYMBOLS[l],
s: args.s)
}
i ~mod+= 1
}
}
// Mirrors BrotliStoreHuffmanTreeToBitMask in brotli_bit_stream.c, using
// this.cl_depth and this.cl_bits as code_length_bitdepth and
// code_length_bitdepth_symbols.
pri func encoder.store_huffman_tree_to_bit_mask!(s: slice base.u8) {
var i : base.u32
var ix : base.u32
i = 0
while i < this.huffman_tree_size {
ix = (this.huffman_tree[i & 1023] & 31) as base.u32
this.write_bits!(
n_bits: this.cl_depth[ix] as base.u32,
bits: this.cl_bits[ix] as base.u64,
s: args.s)
// Extra bits.
if ix == 16 { // BROTLI_REPEAT_PREVIOUS_CODE_LENGTH.
this.write_bits!(n_bits: 2, bits: this.huffman_tree_extra_bits[i & 1023] as base.u64, s: args.s)
} else if ix == 17 { // BROTLI_REPEAT_ZERO_CODE_LENGTH.
this.write_bits!(n_bits: 3, bits: this.huffman_tree_extra_bits[i & 1023] as base.u64, s: args.s)
}
i ~mod+= 1
}
}
// Mirrors BrotliStoreHuffmanTree in brotli_bit_stream.c.
pri func encoder.store_huffman_tree!(depths: slice base.u8, num: base.u32, s: slice base.u8) {
var i : base.u32
var num_codes : base.u32
var code : base.u32
this.huffman_tree_size = 0
this.write_huffman_tree!(depth: args.depths, length: args.num)
// Calculate the statistics of the Huffman tree in brotli-representation.
i = 0
while i < 32 {
this.cl_histo[i] = 0
this.cl_depth[i] = 0
i ~mod+= 1
}
i = 0
while i < this.huffman_tree_size {
this.cl_histo[this.huffman_tree[i & 1023] & 31] ~mod+= 1
i ~mod+= 1
}
i = 0
while i < 18 {
if this.cl_histo[i] <> 0 {
if num_codes == 0 {
code = i
num_codes = 1
} else if num_codes == 1 {
num_codes = 2
break
}
}
i ~mod+= 1
}
// Calculate another Huffman tree to use for compressing both the earlier
// Huffman tree with.
this.create_huffman_tree!(src: 1, off: 0, length: 18, tree_limit: 5, depth: this.cl_depth[..])
this.convert_bit_depths_to_symbols!(depth: this.cl_depth[..], len: 18, dst: 2)
// Now, we have all the data, let's start storing it.
this.store_huffman_tree_of_huffman_tree_to_bit_mask!(num_codes: num_codes, s: args.s)
if num_codes == 1 {
this.cl_depth[code & 31] = 0
}
// Store the real Huffman tree now.
this.store_huffman_tree_to_bit_mask!(s: args.s)
}
// Mirrors BrotliBuildAndStoreHuffmanTreeFast in brotli_bit_stream.c, with
// this.histogram, this.lit_depth and this.lit_bits as the histogram, depth and
// bits arguments.
pri func encoder.build_and_store_huffman_tree_fast!(histogram_total: base.u64, max_bits: base.u32, s: slice base.u8) {
var count : base.u32
var symbols : array[4] base.u32
var length : base.u32
var total : base.u64
var count_limit : base.u32
var n : base.u32
var l : base.u32
var h : base.u32
var i : base.u32
var j : base.u32
var tmp : base.u32
var depth_ok : base.bool
var value : base.u32
var prev_value : base.u32
var reps : base.u32
var k : base.u32
total = args.histogram_total
while (total <> 0) and (length < 256) {
h = this.histogram[length]
if h <> 0 {
if count < 4 {
symbols[count] = length
}
count ~mod+= 1
total ~mod-= h as base.u64
}
length ~mod+= 1
}
if count <= 1 {
this.write_bits!(n_bits: 4, bits: 1, s: args.s)
this.write_bits!(n_bits: args.max_bits, bits: symbols[0] as base.u64, s: args.s)
this.lit_depth[symbols[0] & 255] = 0
this.lit_bits[symbols[0] & 255] = 0
return nothing
}
i = 0
while i < length {
this.lit_depth[i & 255] = 0
i ~mod+= 1
}
count_limit = 1
while true {
n = 0
l = length
while l > 0 {
l -= 1
h = this.histogram[l & 255]
if h <> 0 {
if h >= count_limit {
this.tree_count[n & 1023] = h
} else {
this.tree_count[n & 1023] = count_limit
}
this.tree_left[n & 1023] = TREE_NONE
this.tree_right[n & 1023] = (l & 0xFFFF) as base.u16
n ~mod+= 1
}
}
this.sort_huffman_tree_items!(n: n, tie_break: false)
this.merge_huffman_nodes!(n: n)
depth_ok = this.set_depth!(p0: (n ~mod* 2) ~mod- 1, depth: this.lit_depth[..], max_depth: 14)
if depth_ok {
break
}
count_limit ~mod*= 2
}
this.convert_bit_depths_to_symbols!(depth: this.lit_depth[..], len: length, dst: 3)
if count <= 4 {
// Value of 1 indicates a simple Huffman code.
this.write_bits!(n_bits: 2, bits: 1, s: args.s)
this.write_bits!(n_bits: 2, bits: (count ~mod- 1) as base.u64, s: args.s) // NSYM - 1.
// Sort.
i = 0
while i < count {
j = i ~mod+ 1
while j < count {
if this.lit_depth[symbols[j & 3] & 255] < this.lit_depth[symbols[i & 3] & 255] {
tmp = symbols[j & 3]
symbols[j & 3] = symbols[i & 3]
symbols[i & 3] = tmp
}
j ~mod+= 1
}
i ~mod+= 1
}
this.write_bits!(n_bits: args.max_bits, bits: symbols[0] as base.u64, s: args.s)
this.write_bits!(n_bits: args.max_bits, bits: symbols[1] as base.u64, s: args.s)
if count >= 3 {
this.write_bits!(n_bits: args.max_bits, bits: symbols[2] as base.u64, s: args.s)
}
if count >= 4 {
this.write_bits!(n_bits: args.max_bits, bits: symbols[3] as base.u64, s: args.s)
// Tree-select.
if this.lit_depth[symbols[0] & 255] == 1 {
this.write_bits!(n_bits: 1, bits: 1, s: args.s)
} else {
this.write_bits!(n_bits: 1, bits: 0, s: args.s)
}
}
} else {
// Complex Huffman Tree.
this.write_bits!(n_bits: 40, bits: STATIC_CODE_LENGTH_CODE, s: args.s)
// Actual RLE coding.
prev_value = 8
i = 0
while i < length {
value = this.lit_depth[i & 255] as base.u32
reps = 1
k = i ~mod+ 1
while k < length {
if (this.lit_depth[k & 255] as base.u32) <> value {
break
}
reps ~mod+= 1
k ~mod+= 1
}
i ~mod+= reps
if value == 0 {
this.write_bits!(n_bits: ZERO_REPS_DEPTH[reps & 511], bits: ZERO_REPS_BITS[reps & 511], s: args.s)
} else {
if prev_value <> value {
this.write_bits!(n_bits: CODE_LENGTH_DEPTH[value & 15], bits: CODE_LENGTH_BITS[value & 15], s: args.s)
reps ~mod-= 1
}
if reps < 3 {
while reps <> 0 {
reps -= 1
this.write_bits!(n_bits: CODE_LENGTH_DEPTH[value & 15], bits: CODE_LENGTH_BITS[value & 15], s: args.s)
}
} else {
reps -= 3
this.write_bits!(n_bits: NON_ZERO_REPS_DEPTH[reps & 511], bits: NON_ZERO_REPS_BITS[reps & 511], s: args.s)
}
prev_value = value
}
}
}
}
// copy_tmp_bits_to_cmd_bits copies this.tmp_bits[args.src .. args.src + args.n]
// to this.cmd_bits[args.dst .. args.dst + args.n]. Wuffs' C code generator
// mishandles copy_from_slice on base.u16 arrays, so this is an explicit loop.
pri func encoder.copy_tmp_bits_to_cmd_bits!(dst: base.u32, src: base.u32, n: base.u32) {
var i : base.u32
i = 0
while i < args.n {
this.cmd_bits[(args.dst ~mod+ i) & 127] = this.tmp_bits[(args.src ~mod+ i) & 63]
i ~mod+= 1
}
}
+716
View File
@@ -0,0 +1,716 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// This file mirrors compress_fragment.c, the quality 0 (one-pass) encoder.
//
// The C code's input pointers (input, ip, next_emit, candidate, etc.) are
// base.u64 offsets from the start of the fragment (the C code's base_ip).
// The hash table holds the same offsets. The C code's last_distance is -1 or
// positive. Here, 0 means -1.
// FRAGMENT_MAX_DISTANCE is the C code's MAX_DISTANCE,
// BROTLI_MAX_BACKWARD_LIMIT(18).
pri const FRAGMENT_MAX_DISTANCE : base.u64 = 262128
// HASH_MUL32 is kHashMul32 in hash_base.h.
pri const HASH_MUL32 : base.u64 = 0x1E35_A7BD
// Mirrors Hash in compress_fragment.c.
pri func encoder.hash_fast(input: roslice base.u8, i: base.u64, shift: base.u32) base.u32 {
var h : base.u64
h = (this.load64le(s: args.input, i: args.i) ~mod<< 24) ~mod* HASH_MUL32
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
}
// Mirrors HashBytesAtOffset in compress_fragment.c.
pri func encoder.hash_bytes_at_offset_fast(v: base.u64, offset: base.u32, shift: base.u32) base.u32 {
var h : base.u64
h = ((args.v >> ((8 ~mod* args.offset) & 63)) ~mod<< 24) ~mod* HASH_MUL32
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
}
// Mirrors IsMatch in compress_fragment.c.
pri func encoder.is_match_fast(input: roslice base.u8, i1: base.u64, i2: base.u64) base.bool {
if (args.i1 >= args.input.length()) or (args.i2 >= args.input.length()) {
return false
}
if this.load32le(s: args.input, i: args.i1) <> this.load32le(s: args.input, i: args.i2) {
return false
}
if ((args.i1 ~mod+ 4) >= args.input.length()) or ((args.i2 ~mod+ 4) >= args.input.length()) {
return false
}
return args.input[args.i1 ~mod+ 4] == args.input[args.i2 ~mod+ 4]
}
// Mirrors BuildAndStoreLiteralPrefixCode in compress_fragment.c, for the
// input_size bytes at args.input[args.from ..].
pri func encoder.build_and_store_literal_prefix_code!(input: roslice base.u8, from: base.u64, input_size: base.u64, s: slice base.u8) base.u64 {
var histogram_total : base.u64
var i : base.u64
var j : base.u32
var adjust : base.u32
var literal_ratio : base.u64
var h : base.u32
j = 0
while j < 256 {
this.histogram[j] = 0
j ~mod+= 1
}
if args.input_size < 0x8000 {
i = 0
while i < args.input_size {
if (args.from ~mod+ i) < args.input.length() {
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
}
i ~mod+= 1
}
histogram_total = args.input_size
j = 0
while j < 256 {
// We weigh the first 11 samples with weight 3 to account for the
// balancing effect of the LZ77 phase on the histogram.
adjust = 2 * this.histogram[j].min(no_more_than: 11)
this.histogram[j] ~mod+= adjust
histogram_total ~mod+= adjust as base.u64
j ~mod+= 1
}
} else {
i = 0
while i < args.input_size {
if (args.from ~mod+ i) < args.input.length() {
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
}
i ~mod+= 29
}
histogram_total = (args.input_size ~mod+ 28) / 29
j = 0
while j < 256 {
// We add 1 to each population count to avoid 0 bit depths (since
// this is only a sample and we don't know if the symbol appears
// or not), and we weigh the first 11 samples with weight 3 to
// account for the balancing effect of the LZ77 phase on the
// histogram (more frequent symbols are more likely to be in
// backward references instead as literals).
adjust = 1 + (2 * this.histogram[j].min(no_more_than: 11))
this.histogram[j] ~mod+= adjust
histogram_total ~mod+= adjust as base.u64
j ~mod+= 1
}
}
this.build_and_store_huffman_tree_fast!(histogram_total: histogram_total, max_bits: 8, s: args.s)
j = 0
while j < 256 {
h = this.histogram[j]
if h <> 0 {
literal_ratio ~mod+= ((h ~mod* (this.lit_depth[j] as base.u32)) as base.u64)
}
j ~mod+= 1
}
// Estimated encoding ratio, millibytes per symbol.
if histogram_total == 0 {
return 0
}
return (literal_ratio ~mod* 125) / histogram_total
}
// Mirrors BuildAndStoreCommandPrefixCode in compress_fragment.c.
pri func encoder.build_and_store_command_prefix_code_fast!(s: slice base.u8) {
var i : base.u32
this.tmp_depth[.. 704].bulk_memset!(byte_value: 0)
this.create_huffman_tree!(src: 0, off: 0, length: 64, tree_limit: 15, depth: this.cmd_depth[.. 64])
this.create_huffman_tree!(src: 0, off: 64, length: 64, tree_limit: 14, depth: this.cmd_depth[64 ..])
// We have to jump through a few hoops here in order to compute the
// command bits because the symbols are in a different order than in the
// full alphabet. This looks complicated, but having the symbols in this
// order in the command bits saves a few branches in the Emit* functions.
this.tmp_depth[0 .. 24].copy_from_slice!(s: this.cmd_depth[0 .. 24])
this.tmp_depth[24 .. 32].copy_from_slice!(s: this.cmd_depth[40 .. 48])
this.tmp_depth[32 .. 40].copy_from_slice!(s: this.cmd_depth[24 .. 32])
this.tmp_depth[40 .. 48].copy_from_slice!(s: this.cmd_depth[48 .. 56])
this.tmp_depth[48 .. 56].copy_from_slice!(s: this.cmd_depth[32 .. 40])
this.tmp_depth[56 .. 64].copy_from_slice!(s: this.cmd_depth[56 .. 64])
this.convert_bit_depths_to_symbols!(depth: this.tmp_depth[..], len: 64, dst: 0)
this.copy_tmp_bits_to_cmd_bits!(dst: 0, src: 0, n: 24)
this.copy_tmp_bits_to_cmd_bits!(dst: 24, src: 32, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 32, src: 48, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 40, src: 24, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 48, src: 40, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 56, src: 56, n: 8)
this.convert_bit_depths_to_symbols!(depth: this.cmd_depth[64 ..], len: 64, dst: 1)
// Create the bit length array for the full command alphabet.
this.tmp_depth[.. 64].bulk_memset!(byte_value: 0)
this.tmp_depth[0 .. 8].copy_from_slice!(s: this.cmd_depth[0 .. 8])
this.tmp_depth[64 .. 72].copy_from_slice!(s: this.cmd_depth[8 .. 16])
this.tmp_depth[128 .. 136].copy_from_slice!(s: this.cmd_depth[16 .. 24])
this.tmp_depth[192 .. 200].copy_from_slice!(s: this.cmd_depth[24 .. 32])
this.tmp_depth[384 .. 392].copy_from_slice!(s: this.cmd_depth[32 .. 40])
i = 0
while i < 8 {
this.tmp_depth[128 + (8 * i)] = this.cmd_depth[40 + i]
this.tmp_depth[256 + (8 * i)] = this.cmd_depth[48 + i]
this.tmp_depth[448 + (8 * i)] = this.cmd_depth[56 + i]
i ~mod+= 1
}
this.store_huffman_tree!(depths: this.tmp_depth[..], num: 704, s: args.s)
this.store_huffman_tree!(depths: this.cmd_depth[64 ..], num: 64, s: args.s)
}
// emit_cmd_fast writes the command (or distance) prefix code symbol and
// updates its histogram count. It is the C code's "BrotliWriteBits(depth[code],
// bits[code], storage_ix, storage); ++histo[code];".
pri func encoder.emit_cmd_fast!(code: base.u64, s: slice base.u8) {
this.write_bits!(
n_bits: this.cmd_depth[args.code & 127] as base.u32,
bits: this.cmd_bits[args.code & 127] as base.u64,
s: args.s)
this.cmd_histo[args.code & 127] ~mod+= 1
}
// Mirrors EmitInsertLen in compress_fragment.c.
pri func encoder.emit_insert_len_fast!(insertlen: base.u64, s: slice base.u8) {
var tail : base.u64
var nbits : base.u32
var prefix : base.u64
if args.insertlen < 6 {
this.emit_cmd_fast!(code: args.insertlen + 40, s: args.s)
} else if args.insertlen < 130 {
tail = args.insertlen - 2
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
prefix = tail >> (nbits & 63)
this.emit_cmd_fast!(code: (((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 42, s: args.s)
this.write_bits!(n_bits: nbits, bits: tail ~mod- (prefix ~mod<< (nbits & 63)), s: args.s)
} else if args.insertlen < 2114 {
tail = args.insertlen - 66
nbits = this.log2_floor_non_zero(n: tail)
this.emit_cmd_fast!(code: (nbits as base.u64) + 50, s: args.s)
this.write_bits!(n_bits: nbits, bits: tail ~mod- ((1 as base.u64) << nbits), s: args.s)
} else {
this.emit_cmd_fast!(code: 61, s: args.s)
this.write_bits!(n_bits: 12, bits: args.insertlen ~mod- 2114, s: args.s)
}
}
// Mirrors EmitLongInsertLen in compress_fragment.c.
pri func encoder.emit_long_insert_len_fast!(insertlen: base.u64, s: slice base.u8) {
if args.insertlen < 22594 {
this.emit_cmd_fast!(code: 62, s: args.s)
this.write_bits!(n_bits: 14, bits: args.insertlen ~mod- 6210, s: args.s)
} else {
this.emit_cmd_fast!(code: 63, s: args.s)
this.write_bits!(n_bits: 24, bits: args.insertlen ~mod- 22594, s: args.s)
}
}
// Mirrors EmitCopyLen in compress_fragment.c.
pri func encoder.emit_copy_len_fast!(copylen: base.u64, s: slice base.u8) {
var tail : base.u64
var nbits : base.u32
var prefix : base.u64
if args.copylen < 10 {
this.emit_cmd_fast!(code: args.copylen + 14, s: args.s)
} else if args.copylen < 134 {
tail = args.copylen - 6
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
prefix = tail >> (nbits & 63)
this.emit_cmd_fast!(code: (((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 20, s: args.s)
this.write_bits!(n_bits: nbits, bits: tail ~mod- (prefix ~mod<< (nbits & 63)), s: args.s)
} else if args.copylen < 2118 {
tail = args.copylen - 70
nbits = this.log2_floor_non_zero(n: tail)
this.emit_cmd_fast!(code: (nbits as base.u64) + 28, s: args.s)
this.write_bits!(n_bits: nbits, bits: tail ~mod- ((1 as base.u64) << nbits), s: args.s)
} else {
this.emit_cmd_fast!(code: 39, s: args.s)
this.write_bits!(n_bits: 24, bits: args.copylen ~mod- 2118, s: args.s)
}
}
// Mirrors EmitCopyLenLastDistance in compress_fragment.c.
pri func encoder.emit_copy_len_last_distance_fast!(copylen: base.u64, s: slice base.u8) {
var tail : base.u64
var nbits : base.u32
var prefix : base.u64
if args.copylen < 12 {
this.emit_cmd_fast!(code: args.copylen ~mod- 4, s: args.s)
} else if args.copylen < 72 {
tail = args.copylen - 8
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
prefix = tail >> (nbits & 63)
this.emit_cmd_fast!(code: (((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 4, s: args.s)
this.write_bits!(n_bits: nbits, bits: tail ~mod- (prefix ~mod<< (nbits & 63)), s: args.s)
} else if args.copylen < 136 {
tail = args.copylen - 8
this.emit_cmd_fast!(code: (tail >> 5) + 30, s: args.s)
this.write_bits!(n_bits: 5, bits: tail & 31, s: args.s)
this.emit_cmd_fast!(code: 64, s: args.s)
} else if args.copylen < 2120 {
tail = args.copylen - 72
nbits = this.log2_floor_non_zero(n: tail)
this.emit_cmd_fast!(code: (nbits as base.u64) + 28, s: args.s)
this.write_bits!(n_bits: nbits, bits: tail ~mod- ((1 as base.u64) << nbits), s: args.s)
this.emit_cmd_fast!(code: 64, s: args.s)
} else {
this.emit_cmd_fast!(code: 39, s: args.s)
this.write_bits!(n_bits: 24, bits: args.copylen ~mod- 2120, s: args.s)
this.emit_cmd_fast!(code: 64, s: args.s)
}
}
// Mirrors EmitDistance in compress_fragment.c.
pri func encoder.emit_distance_fast!(distance: base.u64, s: slice base.u8) {
var d : base.u64
var nbits : base.u32
var prefix : base.u64
var offset : base.u64
d = args.distance ~mod+ 3
nbits = this.log2_floor_non_zero(n: d) ~mod- 1
prefix = (d >> (nbits & 63)) & 1
offset = (2 + prefix) ~mod<< (nbits & 63)
this.emit_cmd_fast!(code: (((2 * ((nbits ~mod- 1) as base.u64)) ~mod+ prefix) ~mod+ 80), s: args.s)
this.write_bits!(n_bits: nbits, bits: d ~mod- offset, s: args.s)
}
// Mirrors EmitLiterals in compress_fragment.c.
pri func encoder.emit_literals!(input: roslice base.u8, from: base.u64, len: base.u64, s: slice base.u8) {
var j : base.u64
var lit : base.u8
j = 0
while j < args.len {
if (args.from ~mod+ j) >= args.input.length() {
break
}
lit = args.input[args.from ~mod+ j]
this.write_bits!(n_bits: this.lit_depth[lit] as base.u32, bits: this.lit_bits[lit] as base.u64, s: args.s)
j ~mod+= 1
}
}
// Mirrors ShouldMergeBlock in compress_fragment.c, for the len bytes at
// args.input[args.from ..].
pri func encoder.should_merge_block!(input: roslice base.u8, from: base.u64, len: base.u64) base.bool {
var i : base.u64
var j : base.u32
var total : base.u64
var r : base.u64
var h : base.u64
j = 0
while j < 256 {
this.histogram[j] = 0
j ~mod+= 1
}
i = 0
while i < args.len {
if (args.from ~mod+ i) < args.input.length() {
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
}
i ~mod+= 43
}
total = (args.len ~mod+ 42) / 43
// r = (FastLog2(total) + 0.5) * (double)total + 200.
r = this.sf.f64_add(
a: this.sf.f64_mul(
a: this.sf.f64_add(a: this.fast_log2(v: total), b: 0x3FE0_0000_0000_0000),
b: this.sf.f64_from_u64(a: total)),
b: 0x4069_0000_0000_0000)
j = 0
while j < 256 {
// r -= (double)histo[i] * (depths[i] + FastLog2(histo[i])).
h = this.histogram[j] as base.u64
r = this.sf.f64_sub(
a: r,
b: this.sf.f64_mul(
a: this.sf.f64_from_u64(a: h),
b: this.sf.f64_add(
a: this.sf.f64_from_u32(a: this.lit_depth[j] as base.u32),
b: this.fast_log2(v: h))))
j ~mod+= 1
}
return this.sf.f64_ge(a: r, b: 0) <> 0
}
// Mirrors ShouldUseUncompressedMode in compress_fragment.c.
pri func encoder.should_use_uncompressed_mode(metablock_start: base.u64, next_emit: base.u64, insertlen: base.u64, literal_ratio: base.u64) base.bool {
var compressed : base.u64
compressed = args.next_emit ~mod- args.metablock_start
if (compressed ~mod* 50) > args.insertlen {
return false
}
// MIN_RATIO is 980.
return args.literal_ratio > 980
}
// Mirrors EmitUncompressedMetaBlock in compress_fragment.c.
pri func encoder.emit_uncompressed_meta_block_fast!(input: roslice base.u8, begin: base.u64, end: base.u64, storage_ix_start: base.u64, s: slice base.u8) {
var len : base.u64
var p : base.u64
len = args.end ~mod- args.begin
this.rewind_bit_position!(new_ix: args.storage_ix_start, s: args.s)
this.store_meta_block_header!(len: len, is_uncompressed: true, s: args.s)
this.align_to_byte!()
this.copy_bytes!(input: args.input, begin: args.begin, end: args.end, s: args.s)
this.bw_ix ~mod+= len ~mod<< 3
p = this.bw_ix >> 3
if p < args.s.length() {
args.s[p] = 0
}
}
// copy_bytes is the C code's "memcpy(&storage[*storage_ix >> 3], begin,
// len)", where bw_ix is byte-aligned.
pri func encoder.copy_bytes!(input: roslice base.u8, begin: base.u64, end: base.u64, s: slice base.u8) {
var p : base.u64
p = this.bw_ix >> 3
if (args.begin <= args.end) and (args.end <= args.input.length()) and (p <= args.s.length()) {
args.s[p ..].copy_from_slice!(s: args.input[args.begin .. args.end])
}
}
// update_table_fast updates the hash table with the hashes of some positions
// within the last copy (ending at ip), and returns the next candidate. It is
// the C code's "We could immediately start working at ip now, but to improve
// compression we first update table with the hashes..." block.
pri func encoder.update_table_fast!(input: roslice base.u8, ip: base.u64, shift: base.u32) base.u64 {
var input_bytes : base.u64
var prev_hash : base.u32
var cur_hash : base.u32
var candidate : base.u64
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 3)
prev_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 0, shift: args.shift)
cur_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 3, shift: args.shift)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 3) & 0xFFFF_FFFF) as base.u32
prev_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 1, shift: args.shift)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 2) & 0xFFFF_FFFF) as base.u32
prev_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 2, shift: args.shift)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 1) & 0xFFFF_FFFF) as base.u32
candidate = this.hash_table[cur_hash & 0x1_FFFF] as base.u64
this.hash_table[cur_hash & 0x1_FFFF] = (args.ip & 0xFFFF_FFFF) as base.u32
return candidate
}
// Mirrors BrotliCompressFragmentFast in compress_fragment.c.
pri func encoder.compress_fragment_fast!(input: roslice base.u8, is_last: base.bool, table_bits: base.u32, s: slice base.u8) {
var initial_ix : base.u64
var input_size : base.u64
initial_ix = this.bw_ix
input_size = args.input.length()
if input_size == 0 {
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // islast
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // isempty
this.align_to_byte!()
return nothing
}
this.compress_fragment_fast_impl!(input: args.input, is_last: args.is_last, table_bits: args.table_bits, s: args.s)
// If output is larger than single uncompressed block, rewrite it.
if (this.bw_ix ~mod- initial_ix) > (31 ~mod+ (input_size ~mod<< 3)) {
this.emit_uncompressed_meta_block_fast!(
input: args.input, begin: 0, end: input_size, storage_ix_start: initial_ix, s: args.s)
}
if args.is_last {
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // islast
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // isempty
this.align_to_byte!()
}
}
// Mirrors BrotliCompressFragmentFastImpl in compress_fragment.c. The C code's
// gotos (to emit_commands, trawl, emit_remainder and next_block) are labeled
// loops and the goto_next_block bool.
pri func encoder.compress_fragment_fast_impl!(input: roslice base.u8, is_last: base.bool, table_bits: base.u32, s: slice base.u8) {
var in_off : base.u64
var in_size : base.u64
var next_emit : base.u64
var metablock_start : base.u64
var block_size : base.u64
var total_block_size : base.u64
var mlen_storage_ix : base.u64
var literal_ratio : base.u64
var shift : base.u32
var i : base.u64
var ip : base.u64
var ip_end : base.u64
var ip_limit : base.u64
var len_limit : base.u64
var next_ip : base.u64
var candidate : base.u64
var base_ip : base.u64
var matched : base.u64
var distance : base.u64
var insert : base.u64
var last_distance : base.u64
var skip : base.u32
var hash : base.u32
var next_hash : base.u32
var goto_next_block : base.bool
var merge : base.bool
var saved_ix : base.u64
shift = 64 ~mod- args.table_bits
in_size = args.input.length()
block_size = in_size.min(no_more_than: 98304) // kFirstBlockSize is (3 << 15).
total_block_size = block_size
// Save the bit position of the MLEN field of the meta-block header, so
// that we can update it later if we decide to extend this meta-block.
mlen_storage_ix = this.bw_ix ~mod+ 3
this.store_meta_block_header!(len: block_size, is_uncompressed: false, s: args.s)
// No block splits, no contexts.
this.write_bits!(n_bits: 13, bits: 0, s: args.s)
literal_ratio = this.build_and_store_literal_prefix_code!(
input: args.input, from: in_off, input_size: block_size, s: args.s)
// Store the pre-compressed command and distance prefix codes.
i = 0
while (i ~mod+ 7) < this.cmd_code_numbits {
this.write_bits!(n_bits: 8, bits: this.cmd_code[(i >> 3) & 511] as base.u64, s: args.s)
i ~mod+= 8
}
this.write_bits!(
n_bits: (this.cmd_code_numbits & 7) as base.u32,
bits: this.cmd_code[(this.cmd_code_numbits >> 3) & 511] as base.u64,
s: args.s)
while.emit_commands true {
// Initialize the command and distance histograms. We will gather
// statistics of command and distance codes during the processing of
// this block and use it to update the command and distance prefix
// codes for the next block.
i = 0
while i < 128 {
this.cmd_histo[i] = CMD_HISTO_SEED[i]
i ~mod+= 1
}
ip = in_off
last_distance = 0
ip_end = in_off ~mod+ block_size
goto_next_block = false
if block_size >= 16 {
// For the last block, we need to keep a 16 bytes margin so that
// we can be sure that all distances are at most window size - 16.
// For all other blocks, we only need to keep a margin of 5 bytes
// so that we don't go over the block size with a copy.
len_limit = block_size ~mod- 5
len_limit = len_limit.min(no_more_than: in_size ~mod- 16)
ip_limit = in_off ~mod+ len_limit
ip ~mod+= 1
next_hash = this.hash_fast(input: args.input, i: ip, shift: shift)
while.outer true {
// Step 1: Scan forward in the input looking for a 5-byte-long
// match. If we get close to exhausting the input then goto
// emit_remainder.
skip = 32
next_ip = ip
while.trawl true {
hash = next_hash
ip = next_ip
next_ip = ip ~mod+ ((skip >> 5) as base.u64)
skip ~mod+= 1
if next_ip > ip_limit {
break.outer
}
next_hash = this.hash_fast(input: args.input, i: next_ip, shift: shift)
if (last_distance > 0) and (last_distance <= ip) {
candidate = ip ~mod- last_distance
if this.is_match_fast(input: args.input, i1: ip, i2: candidate) {
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
// Check copy distance. If candidate is not
// feasible, continue search.
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
continue.trawl
}
break.trawl
}
}
candidate = this.hash_table[hash & 0x1_FFFF] as base.u64
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
if this.is_match_fast(input: args.input, i1: ip, i2: candidate) {
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
continue.trawl
}
break.trawl
}
}.trawl
// Step 2: Emit the found match together with the literal bytes
// from "next_emit" to the bit stream, and then see if we can
// find a next match immediately afterwards. Repeat until we
// find no match for the input without emitting some literal
// bytes.
base_ip = ip
matched = 5 ~mod+ this.find_match_length_with_limit(
s: args.input, i1: candidate ~mod+ 5, i2: ip ~mod+ 5, limit: (ip_end ~mod- ip) ~mod- 5)
distance = base_ip ~mod- candidate
insert = base_ip ~mod- next_emit
ip ~mod+= matched
if insert < 6210 {
this.emit_insert_len_fast!(insertlen: insert, s: args.s)
} else if this.should_use_uncompressed_mode(
metablock_start: metablock_start, next_emit: next_emit,
insertlen: insert, literal_ratio: literal_ratio) {
this.emit_uncompressed_meta_block_fast!(
input: args.input, begin: metablock_start, end: base_ip,
storage_ix_start: mlen_storage_ix ~mod- 3, s: args.s)
in_size ~mod-= base_ip ~mod- in_off
in_off = base_ip
next_emit = in_off
goto_next_block = true
break.outer
} else {
this.emit_long_insert_len_fast!(insertlen: insert, s: args.s)
}
this.emit_literals!(input: args.input, from: next_emit, len: insert, s: args.s)
if distance == last_distance {
this.emit_cmd_fast!(code: 64, s: args.s)
} else {
this.emit_distance_fast!(distance: distance, s: args.s)
last_distance = distance
}
this.emit_copy_len_last_distance_fast!(copylen: matched, s: args.s)
next_emit = ip
if ip >= ip_limit {
break.outer
}
candidate = this.update_table_fast!(input: args.input, ip: ip, shift: shift)
while.inner this.is_match_fast(input: args.input, i1: ip, i2: candidate) {
// We have a 5-byte match at ip, and no need to emit any
// literal bytes prior to ip.
base_ip = ip
matched = 5 ~mod+ this.find_match_length_with_limit(
s: args.input, i1: candidate ~mod+ 5, i2: ip ~mod+ 5, limit: (ip_end ~mod- ip) ~mod- 5)
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
break.inner
}
ip ~mod+= matched
last_distance = base_ip ~mod- candidate
this.emit_copy_len_fast!(copylen: matched, s: args.s)
this.emit_distance_fast!(distance: last_distance, s: args.s)
next_emit = ip
if ip >= ip_limit {
break.outer
}
candidate = this.update_table_fast!(input: args.input, ip: ip, shift: shift)
}.inner
ip ~mod+= 1
next_hash = this.hash_fast(input: args.input, i: ip, shift: shift)
}.outer
}
if not goto_next_block {
// emit_remainder:
in_off ~mod+= block_size
in_size ~mod-= block_size
block_size = in_size.min(no_more_than: 0x1_0000) // kMergeBlockSize.
// Decide if we want to continue this meta-block instead of
// emitting the last insert-only command.
merge = false
if (in_size > 0) and ((total_block_size ~mod+ block_size) <= 0x10_0000) {
merge = this.should_merge_block!(input: args.input, from: in_off, len: block_size)
}
if merge {
// Update the size of the current meta-block and continue
// emitting commands. We can do this because the current size
// and the new size both have 5 nibbles.
total_block_size ~mod+= block_size
this.update_bits!(
n_bits: 20,
bits: ((total_block_size ~mod- 1) & 0xFFFF_FFFF) as base.u32,
pos: mlen_storage_ix,
s: args.s)
continue.emit_commands
}
// Emit the remaining bytes as literals.
if next_emit < ip_end {
insert = ip_end ~mod- next_emit
if insert < 6210 {
this.emit_insert_len_fast!(insertlen: insert, s: args.s)
this.emit_literals!(input: args.input, from: next_emit, len: insert, s: args.s)
} else if this.should_use_uncompressed_mode(
metablock_start: metablock_start, next_emit: next_emit,
insertlen: insert, literal_ratio: literal_ratio) {
this.emit_uncompressed_meta_block_fast!(
input: args.input, begin: metablock_start, end: ip_end,
storage_ix_start: mlen_storage_ix ~mod- 3, s: args.s)
} else {
this.emit_long_insert_len_fast!(insertlen: insert, s: args.s)
this.emit_literals!(input: args.input, from: next_emit, len: insert, s: args.s)
}
}
next_emit = ip_end
}
// next_block:
// If we have more data, write a new meta-block header and prefix
// codes and then continue emitting commands.
if in_size > 0 {
metablock_start = in_off
block_size = in_size.min(no_more_than: 98304)
total_block_size = block_size
mlen_storage_ix = this.bw_ix ~mod+ 3
this.store_meta_block_header!(len: block_size, is_uncompressed: false, s: args.s)
// No block splits, no contexts.
this.write_bits!(n_bits: 13, bits: 0, s: args.s)
literal_ratio = this.build_and_store_literal_prefix_code!(
input: args.input, from: in_off, input_size: block_size, s: args.s)
this.build_and_store_command_prefix_code_fast!(s: args.s)
continue.emit_commands
}
break.emit_commands
}.emit_commands
if not args.is_last {
// If this is not the last block, update the command and distance
// prefix codes for the next block and store the compressed forms.
this.cmd_code[0] = 0
saved_ix = this.bw_ix
this.bw_ix = 0
this.build_and_store_command_prefix_code_fast!(s: this.cmd_code[..])
this.cmd_code_numbits = this.bw_ix
this.bw_ix = saved_ix
}
}
+566
View File
@@ -0,0 +1,566 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// This file mirrors compress_fragment_two_pass.c, the quality 1 (two-pass)
// encoder. Like encode_fragment_fast.wuffs, the C code's pointers are
// base.u64 offsets from the start of the fragment (the C code's base_ip) and
// a last_distance of 0 means the C code's -1.
//
// The C code's commands and literals pointers (into command_buf and
// literal_buf) are the num_commands and num_literals fields.
// Mirrors Hash in compress_fragment_two_pass.c.
pri func encoder.hash_two_pass(input: roslice base.u8, i: base.u64, shift: base.u32, length: base.u32) base.u32 {
var h : base.u64
h = (this.load64le(s: args.input, i: args.i) ~mod<< ((64 ~mod- (8 ~mod* args.length)) & 63)) ~mod* HASH_MUL32
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
}
// Mirrors HashBytesAtOffset in compress_fragment_two_pass.c.
pri func encoder.hash_bytes_at_offset_two_pass(v: base.u64, offset: base.u32, shift: base.u32, length: base.u32) base.u32 {
var h : base.u64
h = ((args.v >> ((8 ~mod* args.offset) & 63)) ~mod<< ((64 ~mod- (8 ~mod* args.length)) & 63)) ~mod* HASH_MUL32
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
}
// Mirrors IsMatch in compress_fragment_two_pass.c.
pri func encoder.is_match_two_pass(input: roslice base.u8, i1: base.u64, i2: base.u64, length: base.u32) base.bool {
var q1 : roslice base.u8
var q2 : roslice base.u8
if (args.i1 >= args.input.length()) or (args.i2 >= args.input.length()) {
return false
}
if this.load32le(s: args.input, i: args.i1) <> this.load32le(s: args.input, i: args.i2) {
return false
}
if args.length == 4 {
return true
}
q1 = args.input[args.i1 ..]
q2 = args.input[args.i2 ..]
if (q1.length() >= 6) and (q2.length() >= 6) {
return (q1[4] == q2[4]) and (q1[5] == q2[5])
}
return false
}
// Mirrors BuildAndStoreCommandPrefixCode in compress_fragment_two_pass.c.
// Note that its symbol order differs from the one in compress_fragment.c.
pri func encoder.build_and_store_command_prefix_code_two_pass!(s: slice base.u8) {
var i : base.u32
this.tmp_depth[.. 704].bulk_memset!(byte_value: 0)
this.create_huffman_tree!(src: 0, off: 0, length: 64, tree_limit: 15, depth: this.cmd_depth[.. 64])
this.create_huffman_tree!(src: 0, off: 64, length: 64, tree_limit: 14, depth: this.cmd_depth[64 ..])
// We have to jump through a few hoops here in order to compute the
// command bits because the symbols are in a different order than in the
// full alphabet. This looks complicated, but having the symbols in this
// order in the command bits saves a few branches in the Emit* functions.
this.tmp_depth[0 .. 24].copy_from_slice!(s: this.cmd_depth[24 .. 48])
this.tmp_depth[24 .. 32].copy_from_slice!(s: this.cmd_depth[0 .. 8])
this.tmp_depth[32 .. 40].copy_from_slice!(s: this.cmd_depth[48 .. 56])
this.tmp_depth[40 .. 48].copy_from_slice!(s: this.cmd_depth[8 .. 16])
this.tmp_depth[48 .. 56].copy_from_slice!(s: this.cmd_depth[56 .. 64])
this.tmp_depth[56 .. 64].copy_from_slice!(s: this.cmd_depth[16 .. 24])
this.convert_bit_depths_to_symbols!(depth: this.tmp_depth[..], len: 64, dst: 0)
this.copy_tmp_bits_to_cmd_bits!(dst: 0, src: 24, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 8, src: 40, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 16, src: 56, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 24, src: 0, n: 24)
this.copy_tmp_bits_to_cmd_bits!(dst: 48, src: 32, n: 8)
this.copy_tmp_bits_to_cmd_bits!(dst: 56, src: 48, n: 8)
this.convert_bit_depths_to_symbols!(depth: this.cmd_depth[64 ..], len: 64, dst: 1)
// Create the bit length array for the full command alphabet.
this.tmp_depth[.. 64].bulk_memset!(byte_value: 0)
this.tmp_depth[0 .. 8].copy_from_slice!(s: this.cmd_depth[24 .. 32])
this.tmp_depth[64 .. 72].copy_from_slice!(s: this.cmd_depth[32 .. 40])
this.tmp_depth[128 .. 136].copy_from_slice!(s: this.cmd_depth[40 .. 48])
this.tmp_depth[192 .. 200].copy_from_slice!(s: this.cmd_depth[48 .. 56])
this.tmp_depth[384 .. 392].copy_from_slice!(s: this.cmd_depth[56 .. 64])
i = 0
while i < 8 {
this.tmp_depth[128 + (8 * i)] = this.cmd_depth[i]
this.tmp_depth[256 + (8 * i)] = this.cmd_depth[8 + i]
this.tmp_depth[448 + (8 * i)] = this.cmd_depth[16 + i]
i ~mod+= 1
}
this.store_huffman_tree!(depths: this.tmp_depth[..], num: 704, s: args.s)
this.store_huffman_tree!(depths: this.cmd_depth[64 ..], num: 64, s: args.s)
}
// push_command appends to command_buf.
pri func encoder.push_command!(c: base.u32) {
this.command_buf[this.num_commands & 0x1_FFFF] = args.c
this.num_commands ~mod+= 1
}
// Mirrors EmitInsertLen in compress_fragment_two_pass.c.
pri func encoder.emit_insert_len_two_pass!(insertlen: base.u32) {
var tail : base.u32
var nbits : base.u32
var prefix : base.u32
var extra : base.u32
if args.insertlen < 6 {
this.push_command!(c: args.insertlen)
} else if args.insertlen < 130 {
tail = args.insertlen - 2
nbits = this.log2_floor_non_zero(n: tail as base.u64) ~mod- 1
prefix = tail >> (nbits & 31)
extra = tail ~mod- (prefix ~mod<< (nbits & 31))
this.push_command!(c: ((((nbits ~mod<< 1) ~mod+ prefix) ~mod+ 2) | (extra ~mod<< 8)))
} else if args.insertlen < 2114 {
tail = args.insertlen - 66
nbits = this.log2_floor_non_zero(n: tail as base.u64)
extra = tail ~mod- ((1 as base.u32) << nbits)
this.push_command!(c: ((nbits + 10) | (extra ~mod<< 8)))
} else if args.insertlen < 6210 {
this.push_command!(c: (21 | ((args.insertlen - 2114) ~mod<< 8)))
} else if args.insertlen < 22594 {
this.push_command!(c: (22 | ((args.insertlen - 6210) ~mod<< 8)))
} else {
this.push_command!(c: (23 | ((args.insertlen - 22594) ~mod<< 8)))
}
}
// Mirrors EmitCopyLen in compress_fragment_two_pass.c.
pri func encoder.emit_copy_len_two_pass!(copylen: base.u64) {
var tail : base.u64
var nbits : base.u32
var prefix : base.u64
var extra : base.u64
if args.copylen < 10 {
this.push_command!(c: ((args.copylen + 38) & 0xFFFF_FFFF) as base.u32)
} else if args.copylen < 134 {
tail = args.copylen - 6
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
prefix = tail >> (nbits & 63)
extra = tail ~mod- (prefix ~mod<< (nbits & 63))
this.push_command!(c: ((((((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 44) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
} else if args.copylen < 2118 {
tail = args.copylen - 70
nbits = this.log2_floor_non_zero(n: tail)
extra = tail ~mod- ((1 as base.u64) << nbits)
this.push_command!(c: ((((nbits as base.u64) + 52) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
} else {
this.push_command!(c: ((63 | ((args.copylen ~mod- 2118) ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
}
}
// Mirrors EmitCopyLenLastDistance in compress_fragment_two_pass.c.
pri func encoder.emit_copy_len_last_distance_two_pass!(copylen: base.u64) {
var tail : base.u64
var nbits : base.u32
var prefix : base.u64
var extra : base.u64
if args.copylen < 12 {
this.push_command!(c: ((args.copylen + 20) & 0xFFFF_FFFF) as base.u32)
} else if args.copylen < 72 {
tail = args.copylen - 8
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
prefix = tail >> (nbits & 63)
extra = tail ~mod- (prefix ~mod<< (nbits & 63))
this.push_command!(c: ((((((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 28) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
} else if args.copylen < 136 {
tail = args.copylen - 8
this.push_command!(c: (((tail >> 5) + 54) | ((tail & 31) << 8)) as base.u32)
this.push_command!(c: 64)
} else if args.copylen < 2120 {
tail = args.copylen - 72
nbits = this.log2_floor_non_zero(n: tail)
extra = tail ~mod- ((1 as base.u64) << nbits)
this.push_command!(c: ((((nbits as base.u64) + 52) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
this.push_command!(c: 64)
} else {
this.push_command!(c: ((63 | ((args.copylen ~mod- 2120) ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
this.push_command!(c: 64)
}
}
// Mirrors EmitDistance in compress_fragment_two_pass.c.
pri func encoder.emit_distance_two_pass!(distance: base.u32) {
var d : base.u32
var nbits : base.u32
var prefix : base.u32
var offset : base.u32
var distcode : base.u32
var extra : base.u32
d = args.distance ~mod+ 3
nbits = this.log2_floor_non_zero(n: d as base.u64) ~mod- 1
prefix = (d >> (nbits & 31)) & 1
offset = (2 + prefix) ~mod<< (nbits & 31)
distcode = ((2 ~mod* (nbits ~mod- 1)) ~mod+ prefix) ~mod+ 80
extra = d ~mod- offset
this.push_command!(c: distcode | (extra ~mod<< 8))
}
// push_literals appends args.input[args.from .. args.from + args.n] to
// literal_buf, the C code's "memcpy(*literals, next_emit, insert)".
pri func encoder.push_literals!(input: roslice base.u8, from: base.u64, n: base.u64) {
var j : base.u64
j = 0
while j < args.n {
if (args.from ~mod+ j) >= args.input.length() {
break
}
this.literal_buf[this.num_literals & 0x1_FFFF] = args.input[args.from ~mod+ j]
this.num_literals ~mod+= 1
j ~mod+= 1
}
}
// update_table_two_pass updates the hash table with the hashes of some
// positions within the last copy (ending at ip), and returns the next
// candidate. The C code has two copies of this block, which differ (for
// min_match == 4) in the third hash's offset: third_offset is 0 for the first
// copy and 2 for the second copy.
pri func encoder.update_table_two_pass!(input: roslice base.u8, ip: base.u64, shift: base.u32, min_match: base.u32, third_offset: base.u32) base.u64 {
var input_bytes : base.u64
var prev_hash : base.u32
var cur_hash : base.u32
var candidate : base.u64
var mm : base.u32
mm = args.min_match
if mm == 4 {
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 3)
cur_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 3, shift: args.shift, length: mm)
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 0, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 3) & 0xFFFF_FFFF) as base.u32
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 1, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 2) & 0xFFFF_FFFF) as base.u32
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: args.third_offset, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 1) & 0xFFFF_FFFF) as base.u32
} else {
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 5)
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 0, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 5) & 0xFFFF_FFFF) as base.u32
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 1, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 4) & 0xFFFF_FFFF) as base.u32
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 2, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 3) & 0xFFFF_FFFF) as base.u32
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 2)
cur_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 2, shift: args.shift, length: mm)
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 0, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 2) & 0xFFFF_FFFF) as base.u32
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 1, shift: args.shift, length: mm)
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 1) & 0xFFFF_FFFF) as base.u32
}
candidate = this.hash_table[cur_hash & 0x1_FFFF] as base.u64
this.hash_table[cur_hash & 0x1_FFFF] = (args.ip & 0xFFFF_FFFF) as base.u32
return candidate
}
// Mirrors CreateCommands in compress_fragment_two_pass.c. The block is the
// block_size bytes at args.input[args.from ..]. input_size is the number of
// bytes remaining in the fragment, from args.from.
pri func encoder.create_commands!(input: roslice base.u8, from: base.u64, block_size: base.u64, input_size: base.u64, table_bits: base.u32, min_match: base.u32) {
var ip : base.u64
var shift : base.u32
var ip_end : base.u64
var next_emit : base.u64
var last_distance : base.u64
var len_limit : base.u64
var ip_limit : base.u64
var next_hash : base.u32
var hash : base.u32
var skip : base.u32
var next_ip : base.u64
var candidate : base.u64
var base_ip : base.u64
var matched : base.u64
var distance : base.u64
var insert : base.u64
var mm : base.u64
this.num_commands = 0
this.num_literals = 0
mm = args.min_match as base.u64
ip = args.from
shift = 64 ~mod- args.table_bits
ip_end = args.from ~mod+ args.block_size
next_emit = args.from
if args.block_size >= 16 {
// For the last block, we need to keep a 16 bytes margin so that we
// can be sure that all distances are at most window size - 16. For
// all other blocks, we only need to keep a margin of 5 bytes so that
// we don't go over the block size with a copy.
len_limit = args.block_size ~mod- mm
len_limit = len_limit.min(no_more_than: args.input_size ~mod- 16)
ip_limit = args.from ~mod+ len_limit
ip ~mod+= 1
next_hash = this.hash_two_pass(input: args.input, i: ip, shift: shift, length: args.min_match)
while.outer true {
// Step 1: Scan forward in the input looking for a 6-byte-long
// match. If we get close to exhausting the input then goto
// emit_remainder.
skip = 32
next_ip = ip
while.trawl true {
hash = next_hash
ip = next_ip
next_ip = ip ~mod+ ((skip >> 5) as base.u64)
skip ~mod+= 1
if next_ip > ip_limit {
break.outer
}
next_hash = this.hash_two_pass(input: args.input, i: next_ip, shift: shift, length: args.min_match)
if (last_distance > 0) and (last_distance <= ip) {
candidate = ip ~mod- last_distance
if this.is_match_two_pass(input: args.input, i1: ip, i2: candidate, length: args.min_match) {
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
// Check copy distance. If candidate is not feasible,
// continue search.
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
continue.trawl
}
break.trawl
}
}
candidate = this.hash_table[hash & 0x1_FFFF] as base.u64
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
if this.is_match_two_pass(input: args.input, i1: ip, i2: candidate, length: args.min_match) {
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
continue.trawl
}
break.trawl
}
}.trawl
// Step 2: Emit the found match together with the literal bytes
// from "next_emit", and then see if we can find a next match
// immediately afterwards. Repeat until we find no match for the
// input without emitting some literal bytes.
base_ip = ip
matched = mm ~mod+ this.find_match_length_with_limit(
s: args.input, i1: candidate ~mod+ mm, i2: ip ~mod+ mm, limit: (ip_end ~mod- ip) ~mod- mm)
distance = base_ip ~mod- candidate
insert = base_ip ~mod- next_emit
ip ~mod+= matched
this.emit_insert_len_two_pass!(insertlen: (insert & 0xFFFF_FFFF) as base.u32)
this.push_literals!(input: args.input, from: next_emit, n: insert)
if distance == last_distance {
this.push_command!(c: 64)
} else {
this.emit_distance_two_pass!(distance: (distance & 0xFFFF_FFFF) as base.u32)
last_distance = distance
}
this.emit_copy_len_last_distance_two_pass!(copylen: matched)
next_emit = ip
if ip >= ip_limit {
break.outer
}
candidate = this.update_table_two_pass!(
input: args.input, ip: ip, shift: shift, min_match: args.min_match, third_offset: 0)
while (ip ~mod- candidate) <= FRAGMENT_MAX_DISTANCE {
if not this.is_match_two_pass(input: args.input, i1: ip, i2: candidate, length: args.min_match) {
break
}
// We have a 6-byte match at ip, and no need to emit any
// literal bytes prior to ip.
base_ip = ip
matched = mm ~mod+ this.find_match_length_with_limit(
s: args.input, i1: candidate ~mod+ mm, i2: ip ~mod+ mm, limit: (ip_end ~mod- ip) ~mod- mm)
ip ~mod+= matched
last_distance = base_ip ~mod- candidate
this.emit_copy_len_two_pass!(copylen: matched)
this.emit_distance_two_pass!(distance: (last_distance & 0xFFFF_FFFF) as base.u32)
next_emit = ip
if ip >= ip_limit {
break.outer
}
candidate = this.update_table_two_pass!(
input: args.input, ip: ip, shift: shift, min_match: args.min_match, third_offset: 2)
}
ip ~mod+= 1
next_hash = this.hash_two_pass(input: args.input, i: ip, shift: shift, length: args.min_match)
}.outer
}
// emit_remainder:
// Emit the remaining bytes as literals.
if next_emit < ip_end {
insert = ip_end ~mod- next_emit
this.emit_insert_len_two_pass!(insertlen: (insert & 0xFFFF_FFFF) as base.u32)
this.push_literals!(input: args.input, from: next_emit, n: insert)
}
}
// Mirrors StoreCommands in compress_fragment_two_pass.c.
pri func encoder.store_commands!(s: slice base.u8) {
var i : base.u32
var j : base.u32
var cmd : base.u32
var code : base.u32
var extra : base.u32
var insert : base.u32
var lit_i : base.u32
var lit : base.u8
this.histogram[.. 256].bulk_memset!(byte_value: 0)
this.cmd_depth[.. 128].bulk_memset!(byte_value: 0)
this.cmd_bits[.. 128].bulk_memset!(byte_value: 0)
this.cmd_histo[.. 128].bulk_memset!(byte_value: 0)
i = 0
while i < this.num_literals {
this.histogram[this.literal_buf[i & 0x1_FFFF]] ~mod+= 1
i ~mod+= 1
}
this.build_and_store_huffman_tree_fast!(histogram_total: this.num_literals as base.u64, max_bits: 8, s: args.s)
i = 0
while i < this.num_commands {
code = this.command_buf[i & 0x1_FFFF] & 0xFF
this.cmd_histo[code & 127] ~mod+= 1
i ~mod+= 1
}
this.cmd_histo[1] ~mod+= 1
this.cmd_histo[2] ~mod+= 1
this.cmd_histo[64] ~mod+= 1
this.cmd_histo[84] ~mod+= 1
this.build_and_store_command_prefix_code_two_pass!(s: args.s)
i = 0
while i < this.num_commands {
cmd = this.command_buf[i & 0x1_FFFF]
code = cmd & 0x7F
extra = cmd >> 8
this.write_bits!(n_bits: this.cmd_depth[code] as base.u32, bits: this.cmd_bits[code] as base.u64, s: args.s)
this.write_bits!(n_bits: TWO_PASS_NUM_EXTRA_BITS[code], bits: extra as base.u64, s: args.s)
if code < 24 {
insert = TWO_PASS_INSERT_OFFSET[code] ~mod+ extra
j = 0
while j < insert {
lit = this.literal_buf[lit_i & 0x1_FFFF]
this.write_bits!(n_bits: this.lit_depth[lit] as base.u32, bits: this.lit_bits[lit] as base.u64, s: args.s)
lit_i ~mod+= 1
j ~mod+= 1
}
}
i ~mod+= 1
}
}
// Mirrors ShouldCompress in compress_fragment_two_pass.c, for the input_size
// bytes at args.input[args.from ..].
pri func encoder.should_compress_two_pass!(input: roslice base.u8, from: base.u64, input_size: base.u64, num_literals: base.u64) base.bool {
var corpus_size : base.u64
var max_total_bit_cost : base.u64
var i : base.u64
corpus_size = this.sf.f64_from_u64(a: args.input_size)
// MIN_RATIO is 0.98 (0x3FEF_5C28_F5C2_8F5C).
if this.sf.f64_lt(
a: this.sf.f64_from_u64(a: args.num_literals),
b: this.sf.f64_mul(a: 0x3FEF_5C28_F5C2_8F5C, b: corpus_size)) <> 0 {
return true
}
// max_total_bit_cost = corpus_size * 8 * MIN_RATIO / SAMPLE_RATE, where
// SAMPLE_RATE is 43 and 8.0 is 0x4020_0000_0000_0000.
max_total_bit_cost = this.sf.f64_div(
a: this.sf.f64_mul(
a: this.sf.f64_mul(a: corpus_size, b: 0x4020_0000_0000_0000),
b: 0x3FEF_5C28_F5C2_8F5C),
b: 0x4045_8000_0000_0000)
this.histogram[.. 256].bulk_memset!(byte_value: 0)
i = 0
while i < args.input_size {
if (args.from ~mod+ i) < args.input.length() {
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
}
i ~mod+= 43
}
return this.sf.f64_lt(a: this.bits_entropy_of_histogram(), b: max_total_bit_cost) <> 0
}
// Mirrors EmitUncompressedMetaBlock in compress_fragment_two_pass.c (which,
// unlike the one in compress_fragment.c, does not rewind).
pri func encoder.emit_uncompressed_meta_block_two_pass!(input: roslice base.u8, from: base.u64, input_size: base.u64, s: slice base.u8) {
var p : base.u64
this.store_meta_block_header!(len: args.input_size, is_uncompressed: true, s: args.s)
this.align_to_byte!()
this.copy_bytes!(input: args.input, begin: args.from, end: args.from ~mod+ args.input_size, s: args.s)
this.bw_ix ~mod+= args.input_size ~mod<< 3
p = this.bw_ix >> 3
if p < args.s.length() {
args.s[p] = 0
}
}
// Mirrors BrotliCompressFragmentTwoPass and BrotliCompressFragmentTwoPassImpl
// in compress_fragment_two_pass.c.
pri func encoder.compress_fragment_two_pass!(input: roslice base.u8, is_last: base.bool, table_bits: base.u32, s: slice base.u8) {
var initial_ix : base.u64
var min_match : base.u32
var from : base.u64
var input_size : base.u64
var block_size : base.u64
var compress : base.bool
initial_ix = this.bw_ix
min_match = 6
if args.table_bits <= 15 {
min_match = 4
}
input_size = args.input.length()
while input_size > 0 {
block_size = input_size.min(no_more_than: TWO_PASS_BLOCK_SIZE as base.u64)
this.create_commands!(
input: args.input, from: from, block_size: block_size, input_size: input_size,
table_bits: args.table_bits, min_match: min_match)
compress = this.should_compress_two_pass!(
input: args.input, from: from, input_size: block_size, num_literals: this.num_literals as base.u64)
if compress {
this.store_meta_block_header!(len: block_size, is_uncompressed: false, s: args.s)
// No block splits, no contexts.
this.write_bits!(n_bits: 13, bits: 0, s: args.s)
this.store_commands!(s: args.s)
} else {
// Since we did not find many backward references and the entropy
// of the data is close to 8 bits, we can simply emit an
// uncompressed block. This makes compression speed of
// uncompressible data about 3x faster.
this.emit_uncompressed_meta_block_two_pass!(input: args.input, from: from, input_size: block_size, s: args.s)
}
from ~mod+= block_size
input_size ~mod-= block_size
}
// If output is larger than single uncompressed block, rewrite it.
if (this.bw_ix ~mod- initial_ix) > (31 ~mod+ (args.input.length() ~mod<< 3)) {
this.rewind_bit_position!(new_ix: initial_ix, s: args.s)
this.emit_uncompressed_meta_block_two_pass!(input: args.input, from: 0, input_size: args.input.length(), s: args.s)
}
if args.is_last {
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // islast
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // isempty
this.align_to_byte!()
}
}
+81
View File
@@ -0,0 +1,81 @@
// Copyright 2026 The Wuffs Authors.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//
// SPDX-License-Identifier: Apache-2.0 OR MIT
// --------
// Quirks are discussed in (/doc/note/quirks.md). QUIRKS_BASE is defined in
// decode_quirks.wuffs.
//
// The encoder's quirks mirror the C code's BrotliEncoderParameter values, with
// the value semantics of BrotliEncoderSetParameter. The encoder also accepts
// the QUIRK_LARGE_WINDOW key (defined in decode_quirks.wuffs) for the
// BROTLI_PARAM_LARGE_WINDOW parameter: zero or non-zero (large-window brotli
// is not used at qualities 0 to 2).
//
// For all of the encoder's quirks, the (u64) value must fit in a u32 (like
// the C function's uint32_t argument), otherwise set_quirk returns
// base."#bad argument". Like the C code, the parameters can only be set
// before the first transform_io call, otherwise set_quirk returns
// base."#bad call sequence".
// --------
// When this quirk is set, it sets the encoder's BROTLI_PARAM_QUALITY
// parameter: 0 to 11. The default is 11. Like the C code, out of range values
// are clamped (and a value with its high bit set is negative, so it is
// clamped to 0). Only qualities 0 and 1 are currently supported: transform_io
// returns "#unsupported encoder quality" for the others.
//
// It is not named QUIRK_QUALITY because the base package already has a
// base.QUIRK_QUALITY key (with different value semantics).
pub const QUIRK_ENCODER_QUALITY : base.u32 = 0x2AB7_9000 | 0x01
// When this quirk is set, it sets the encoder's BROTLI_PARAM_LGWIN parameter:
// the log2 of the window size, 10 to 24 (or up to 30 for large-window brotli,
// at qualities 3 and above). The default is 22. Out of range values are
// clamped.
pub const QUIRK_LGWIN : base.u32 = 0x2AB7_9000 | 0x02
// When this quirk is set, it sets the encoder's BROTLI_PARAM_LGBLOCK
// parameter: the log2 of the input block size. 0 (the default) means to
// choose it automatically. It has no effect at qualities 0 and 1.
pub const QUIRK_LGBLOCK : base.u32 = 0x2AB7_9000 | 0x03
// When this quirk is set, it sets the encoder's BROTLI_PARAM_MODE parameter:
// 0 (BROTLI_MODE_GENERIC, the default), 1 (BROTLI_MODE_TEXT) or 2
// (BROTLI_MODE_FONT). Like the C code, other values are accepted (and have
// the same effect as BROTLI_MODE_GENERIC). It has no effect at qualities 0
// and 1.
pub const QUIRK_MODE : base.u32 = 0x2AB7_9000 | 0x04
// When this quirk is set, it sets the encoder's
// BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING parameter: 0 or 1. Other
// values are rejected with base."#bad argument". It has no effect at
// qualities 0 and 1.
pub const QUIRK_DISABLE_LITERAL_CONTEXT_MODELING : base.u32 = 0x2AB7_9000 | 0x05
// When this quirk is set, it sets the encoder's BROTLI_PARAM_SIZE_HINT
// parameter: the estimated total input size. It has no effect at qualities 0
// and 1.
pub const QUIRK_SIZE_HINT : base.u32 = 0x2AB7_9000 | 0x06
// When this quirk is set, it sets the encoder's BROTLI_PARAM_NPOSTFIX
// parameter. It has no effect at qualities 0 and 1.
pub const QUIRK_NPOSTFIX : base.u32 = 0x2AB7_9000 | 0x07
// When this quirk is set, it sets the encoder's BROTLI_PARAM_NDIRECT
// parameter. It has no effect at qualities 0 and 1.
pub const QUIRK_NDIRECT : base.u32 = 0x2AB7_9000 | 0x08
// When this quirk is set, it sets the encoder's BROTLI_PARAM_STREAM_OFFSET
// parameter: 0 to (1 << 30). Larger values are rejected with
// base."#bad argument". Like the C code, a non-zero stream offset means that
// no stream header (WBITS) is written.
pub const QUIRK_STREAM_OFFSET : base.u32 = 0x2AB7_9000 | 0x09