mirror of
https://github.com/ddidderr/wuffs.git
synced 2026-11-20 00:47:28 +01:00
std/brotli: add an encoder for quality levels 0 and 1
Adds a brotli encoder (std/brotli's encoder struct, a base.io_transformer) whose output is byte-identical to libbrotlienc v1.2.1 for quality levels 0 and 1, mirroring encode.c's BrotliEncoderCompressStream(Fast), compress_fragment.c and compress_fragment_two_pass.c, plus the entropy coding and bit stream helpers they need. Every double operation goes through std/softfloat, following the C evaluation order exactly. Parameters are quirks (encode_quirks.wuffs) that mirror BrotliEncoderSetParameter, sanitized at the first transform_io call. Each transform_io resumption corresponds to one C call with available_in equal to the readable src length; a closed src is FINISH; request_flush! and request_emit_metadata! make the next resumption a FLUSH or EMIT_METADATA call. Qualities 2 to 11 return "#unsupported encoder quality" for now. Encoder tables are generated by script/print-brotli-tables.go -encoder. Tests: round trips through the Wuffs decoder over the corpus and synthetic inputs; with -mimic, byte-exact comparisons against libbrotlienc over quality × window × mode × input chunking × flush schedule × metadata × size hint. script/manual-test-brotli-encoder-parity.c runs the full grid: 86,268 configurations, zero mismatches. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
This commit is contained in:
17 files changed
+13990
-2772
No files matched your search
@@ -152,3 +152,175 @@ invariants for memory safety but Wuffs requires explicit bounds checks.
|
||||
- Fast paths.
|
||||
- Fuzzers.
|
||||
- An encoder.
|
||||
|
||||
|
||||
## Encoder
|
||||
|
||||
This package also has an `encoder`, which implements the `base.io_transformer`
|
||||
interface. It aims for byte-exact parity with the reference implementation's
|
||||
encoder (libbrotlienc), version v1.2.1: for the same parameters and the same
|
||||
sequence of operations, it produces exactly the same bytes. Like the decoder,
|
||||
the Wuffs code mirrors the C code function by function, with comments like
|
||||
"Mirrors BrotliCompressFragmentFast in compress_fragment.c". Its floating
|
||||
point arithmetic (the C code's `double`s) uses `std/softfloat`, so that the
|
||||
results are bit-exact regardless of the compiler and CPU.
|
||||
|
||||
Only qualities 0 and 1 (the C code's `FAST_ONE_PASS_COMPRESSION_QUALITY` and
|
||||
`FAST_TWO_PASS_COMPRESSION_QUALITY`, implemented by `compress_fragment.c` and
|
||||
`compress_fragment_two_pass.c`) are implemented. For qualities 2 to 11
|
||||
(including the default quality, 11), `transform_io` returns
|
||||
`"#unsupported encoder quality"`.
|
||||
|
||||
The source files are:
|
||||
|
||||
- `encode_brotli.wuffs`: the `encoder` struct, its public methods and the
|
||||
`encode.c` streaming logic (`BrotliEncoderCompressStream`,
|
||||
`BrotliEncoderCompressStreamFast`, `ProcessMetadata`, `EnsureInitialized`,
|
||||
`InjectBytePaddingBlock`, etc).
|
||||
- `encode_quirks.wuffs`: the encoder's quirk keys.
|
||||
- `encode_bit_writer.wuffs`: `write_bits.h`, `fast_log.h`,
|
||||
`find_match_length.h` and small `brotli_bit_stream.c` helpers.
|
||||
- `encode_entropy.wuffs`: `entropy_encode.c` (Huffman tree construction) and
|
||||
the `brotli_bit_stream.c` functions that store Huffman codes.
|
||||
- `encode_fragment_fast.wuffs`: `compress_fragment.c` (quality 0).
|
||||
- `encode_fragment_two_pass.wuffs`: `compress_fragment_two_pass.c` (quality
|
||||
1).
|
||||
- `common_encoder_tables.wuffs`: tables generated by `go run
|
||||
script/print-brotli-tables.go -encoder` from the C code.
|
||||
|
||||
|
||||
### Encoder API
|
||||
|
||||
In addition to the `base.io_transformer` methods, the encoder has:
|
||||
|
||||
- `request_flush!() status`: the next `transform_io` call acts like the C
|
||||
code's `BROTLI_OPERATION_FLUSH` (instead of `PROCESS` or `FINISH`) until
|
||||
the flush is complete. It returns `"#bad call sequence"` while a metadata
|
||||
request is pending.
|
||||
- `request_emit_metadata!(length: u64) status`: the next `transform_io` call
|
||||
acts like `BROTLI_OPERATION_EMIT_METADATA`, with the next `length` bytes of
|
||||
`src` as the metadata block's contents. `length` can be at most `1 << 24`
|
||||
(otherwise `"#bad argument"`). It returns `"#bad call sequence"` while
|
||||
another request is pending.
|
||||
- `max_compressed_size(input_size: u64) u64`: mirrors
|
||||
`BrotliEncoderMaxCompressedSize` (0 means overflow).
|
||||
|
||||
`dst_history_retain_length` is 0: the encoder never re-reads `dst`.
|
||||
|
||||
|
||||
### Encoder Quirks
|
||||
|
||||
Each quirk mirrors one `BrotliEncoderParameter`, with the value semantics of
|
||||
`BrotliEncoderSetParameter`. Values are stored as is (and must fit in a u32,
|
||||
otherwise `set_quirk` returns `"#bad argument"`). Like the C code, they are
|
||||
sanitized (`SanitizeParams`, `ComputeLgBlock`, `ChooseDistanceParams`) at the
|
||||
first `transform_io` call, after which `set_quirk` returns
|
||||
`"#bad call sequence"`.
|
||||
|
||||
| Quirk | C parameter |
|
||||
| ---------------------------------------- | ---------------------------------------------- |
|
||||
| `QUIRK_ENCODER_QUALITY` | `BROTLI_PARAM_QUALITY` (default 11) |
|
||||
| `QUIRK_LGWIN` | `BROTLI_PARAM_LGWIN` (default 22) |
|
||||
| `QUIRK_LGBLOCK` | `BROTLI_PARAM_LGBLOCK` |
|
||||
| `QUIRK_MODE` | `BROTLI_PARAM_MODE` |
|
||||
| `QUIRK_DISABLE_LITERAL_CONTEXT_MODELING` | `BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING` (0 or 1) |
|
||||
| `QUIRK_SIZE_HINT` | `BROTLI_PARAM_SIZE_HINT` |
|
||||
| `QUIRK_LARGE_WINDOW` (shared with the decoder) | `BROTLI_PARAM_LARGE_WINDOW` |
|
||||
| `QUIRK_NPOSTFIX` | `BROTLI_PARAM_NPOSTFIX` |
|
||||
| `QUIRK_NDIRECT` | `BROTLI_PARAM_NDIRECT` |
|
||||
| `QUIRK_STREAM_OFFSET` | `BROTLI_PARAM_STREAM_OFFSET` (at most `1 << 30`) |
|
||||
|
||||
The quality quirk is not named `QUIRK_QUALITY` because that name would clash
|
||||
with `base.QUIRK_QUALITY`. At qualities 0 and 1, only the quality, lgwin and
|
||||
stream offset affect the output (lgblock, mode, size hint, large window,
|
||||
npostfix, ndirect and literal context modeling are stored, sanitized and then
|
||||
ignored, as in the C code).
|
||||
|
||||
|
||||
### Encoder Workbuf Layout
|
||||
|
||||
At qualities 0 and 1, the workbuf is the C code's metablock storage (its
|
||||
`GetBrotliStorage` buffer): `workbuf_len` is `(2 << lgwin) + 503 + 16` bytes
|
||||
(the storage for a `1 << lgwin` byte block plus 16 bytes of slack for 8-byte
|
||||
bit writes), where lgwin is the sanitized value. A shorter workbuf still works
|
||||
if every input block is shorter: a block of `n` bytes needs `(2 * n) + 519`
|
||||
bytes, otherwise `transform_io` returns `"#bad workbuf length"`. At other
|
||||
qualities, `workbuf_len` is 0.
|
||||
|
||||
The hash table (up to `1 << 17` entries), the quality 1 command and literal
|
||||
buffers (`1 << 17` entries each) and the Huffman scratch space are arrays in
|
||||
the `encoder` struct (which is therefore more than 1 MiB). They do not need
|
||||
zero-initializing: `WUFFS_INITIALIZE__LEAVE_INTERNAL_BUFFERS_UNINITIALIZED` is
|
||||
fine. Unlike the C code, the input block is not copied: the encoder compresses
|
||||
directly from `src`.
|
||||
|
||||
|
||||
### Encoder Operation Mapping
|
||||
|
||||
Each `transform_io` call (initial or resumed) corresponds to one or more
|
||||
`BrotliEncoderCompressStream` calls, with `src`'s readable bytes as the C
|
||||
code's `available_in` and `dst`'s writable bytes as `available_out`. The
|
||||
operation is:
|
||||
|
||||
- `EMIT_METADATA` if `request_emit_metadata` is pending (until the metadata
|
||||
block is complete),
|
||||
- otherwise `FLUSH` if `request_flush` is pending (until the flush is
|
||||
complete),
|
||||
- otherwise `FINISH` if `src` is closed,
|
||||
- otherwise `PROCESS`.
|
||||
|
||||
`transform_io` suspends with `"$short write"` when there is pending output
|
||||
(the C code's `BrotliEncoderHasMoreOutput`) but `dst` is full, suspends with
|
||||
`"$short read"` when all of `src` is consumed (and, for a flush, the flush is
|
||||
complete) and returns ok once the stream is finished (the C code's
|
||||
`BrotliEncoderIsFinished`). So these two loops produce the same bytes:
|
||||
|
||||
- C: for each chunk, call `BrotliEncoderCompressStream(op)` until all of the
|
||||
chunk is consumed and there is no more output (or, for `FINISH`, until
|
||||
finished).
|
||||
- Wuffs: for each chunk, call `request_flush` (for `FLUSH`) or
|
||||
`request_emit_metadata(len)` (for `EMIT_METADATA`), make `src` hold exactly
|
||||
that chunk (closed for `FINISH`) and call `transform_io` until it returns
|
||||
`"$short read"` (or ok for `FINISH`), giving it more `dst` room after each
|
||||
`"$short write"`.
|
||||
|
||||
At qualities 0 and 1 the C code compresses each call's available input
|
||||
immediately (in blocks of at most `1 << lgwin` bytes), so the output depends
|
||||
on how the input is split into chunks. The test programs' encode drivers
|
||||
(`wuffs_brotli_encode_plan` in `test/c/std/brotli.c` and
|
||||
`mimic_brotli_encode` in `test/c/mimiclib/brotli.c`) run the same "plan" of
|
||||
(operation, length) steps with each library.
|
||||
|
||||
|
||||
### Encoder Testing
|
||||
|
||||
`wuffs test -mimic std/brotli` checks byte-exact parity with libbrotlienc
|
||||
over a grid of qualities, lgwins, modes, input chunkings, flushes, metadata
|
||||
blocks, size hints and other parameters, as well as round trips through the
|
||||
decoder. `script/manual-test-brotli-encoder-parity.c` runs a larger grid
|
||||
(about 86 thousand configurations over every file in `test/data/brotli` and
|
||||
some synthetic inputs).
|
||||
|
||||
|
||||
### Encoder Differences from the C Code
|
||||
|
||||
- The input block is not copied (see the workbuf layout above), and output is
|
||||
always staged in the workbuf (or a small internal buffer) and then copied to
|
||||
`dst`. The C code sometimes writes directly to `next_out`. The bytes are the
|
||||
same.
|
||||
- Wuffs' C code generator does not support `slice base.u16` or
|
||||
`slice base.u32` arguments (and mishandles `copy_from_slice` on `base.u16`
|
||||
arrays), so some functions (e.g. `create_huffman_tree`,
|
||||
`convert_bit_depths_to_symbols`) take a selector naming one of the
|
||||
encoder's arrays instead of a slice.
|
||||
- Like the C code, there is no fallback to an uncompressed stream (that is
|
||||
only in the C code's one-shot `BrotliEncoderCompress`).
|
||||
|
||||
|
||||
### Encoder TODO
|
||||
|
||||
- Qualities 2 to 11 (the C code's `EncodeData`, hashers, block splitting,
|
||||
context modeling, Zopfli, etc).
|
||||
- Large-window encoding (only used at qualities 3 and above).
|
||||
- Custom and shared dictionaries (`BrotliEncoderAttachPreparedDictionary`).
|
||||
- Changing parameters mid-stream.
|
||||
@@ -0,0 +1,558 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// The tables in this file were generated by script/print-brotli-tables.go
|
||||
// (with the -encoder flag) from the brotli (C) source code. Do not edit by
|
||||
// hand. These tables are only used by the encoder.
|
||||
|
||||
// From enc/fast_log.c kBrotliLog2Table, as the bit patterns of IEEE 754
|
||||
// binary64 values. The C literals are float literals widened to double.
|
||||
pri const LOG2_TABLE : roarray[256] base.u64 = [
|
||||
0x0000_0000_0000_0000, 0x0000_0000_0000_0000, 0x3FF0_0000_0000_0000, 0x3FF9_5C01_A000_0000,
|
||||
0x4000_0000_0000_0000, 0x4002_934F_0000_0000, 0x4004_AE00_E000_0000, 0x4006_7576_8000_0000,
|
||||
0x4008_0000_0000_0000, 0x4009_5C01_A000_0000, 0x400A_934F_0000_0000, 0x400B_ACEA_8000_0000,
|
||||
0x400C_AE00_E000_0000, 0x400D_9A80_2000_0000, 0x400E_7576_8000_0000, 0x400F_414F_E000_0000,
|
||||
0x4010_0000_0000_0000, 0x4010_598F_E000_0000, 0x4010_AE00_E000_0000, 0x4010_FDE0_C000_0000,
|
||||
0x4011_49A7_8000_0000, 0x4011_91BB_A000_0000, 0x4011_D675_4000_0000, 0x4012_1820_A000_0000,
|
||||
0x4012_5700_6000_0000, 0x4012_934F_0000_0000, 0x4012_CD40_2000_0000, 0x4013_0501_4000_0000,
|
||||
0x4013_3ABB_4000_0000, 0x4013_6E92_A000_0000, 0x4013_A0A7_E000_0000, 0x4013_D118_E000_0000,
|
||||
0x4014_0000_0000_0000, 0x4014_2D75_A000_0000, 0x4014_598F_E000_0000, 0x4014_8462_C000_0000,
|
||||
0x4014_AE00_E000_0000, 0x4014_D67B_0000_0000, 0x4014_FDE0_C000_0000, 0x4015_2440_8000_0000,
|
||||
0x4015_49A7_8000_0000, 0x4015_6E22_2000_0000, 0x4015_91BB_A000_0000, 0x4015_B47E_C000_0000,
|
||||
0x4015_D675_4000_0000, 0x4015_F7A8_6000_0000, 0x4016_1820_A000_0000, 0x4016_37E6_2000_0000,
|
||||
0x4016_5700_6000_0000, 0x4016_7576_8000_0000, 0x4016_934F_0000_0000, 0x4016_B090_4000_0000,
|
||||
0x4016_CD40_2000_0000, 0x4016_E964_0000_0000, 0x4017_0501_4000_0000, 0x4017_201C_C000_0000,
|
||||
0x4017_3ABB_4000_0000, 0x4017_54E1_2000_0000, 0x4017_6E92_A000_0000, 0x4017_87D3_A000_0000,
|
||||
0x4017_A0A7_E000_0000, 0x4017_B913_4000_0000, 0x4017_D118_E000_0000, 0x4017_E8BC_2000_0000,
|
||||
0x4018_0000_0000_0000, 0x4018_16E7_A000_0000, 0x4018_2D75_A000_0000, 0x4018_43AC_E000_0000,
|
||||
0x4018_598F_E000_0000, 0x4018_6F21_0000_0000, 0x4018_8462_C000_0000, 0x4018_9957_4000_0000,
|
||||
0x4018_AE00_E000_0000, 0x4018_C261_6000_0000, 0x4018_D67B_0000_0000, 0x4018_EA4F_8000_0000,
|
||||
0x4018_FDE0_C000_0000, 0x4019_1130_8000_0000, 0x4019_2440_8000_0000, 0x4019_3712_4000_0000,
|
||||
0x4019_49A7_8000_0000, 0x4019_5C01_A000_0000, 0x4019_6E22_2000_0000, 0x4019_800A_6000_0000,
|
||||
0x4019_91BB_A000_0000, 0x4019_A337_6000_0000, 0x4019_B47E_C000_0000, 0x4019_C593_0000_0000,
|
||||
0x4019_D675_4000_0000, 0x4019_E726_A000_0000, 0x4019_F7A8_6000_0000, 0x401A_07FB_6000_0000,
|
||||
0x401A_1820_A000_0000, 0x401A_2819_4000_0000, 0x401A_37E6_2000_0000, 0x401A_4788_4000_0000,
|
||||
0x401A_5700_6000_0000, 0x401A_664F_8000_0000, 0x401A_7576_8000_0000, 0x401A_8476_0000_0000,
|
||||
0x401A_934F_0000_0000, 0x401A_A202_4000_0000, 0x401A_B090_4000_0000, 0x401A_BEFA_0000_0000,
|
||||
0x401A_CD40_2000_0000, 0x401A_DB63_2000_0000, 0x401A_E964_0000_0000, 0x401A_F743_2000_0000,
|
||||
0x401B_0501_4000_0000, 0x401B_129E_E000_0000, 0x401B_201C_C000_0000, 0x401B_2D7B_6000_0000,
|
||||
0x401B_3ABB_4000_0000, 0x401B_47DD_0000_0000, 0x401B_54E1_2000_0000, 0x401B_61C8_2000_0000,
|
||||
0x401B_6E92_A000_0000, 0x401B_7B40_E000_0000, 0x401B_87D3_A000_0000, 0x401B_944B_2000_0000,
|
||||
0x401B_A0A7_E000_0000, 0x401B_ACEA_8000_0000, 0x401B_B913_4000_0000, 0x401B_C522_8000_0000,
|
||||
0x401B_D118_E000_0000, 0x401B_DCF6_8000_0000, 0x401B_E8BC_2000_0000, 0x401B_F469_C000_0000,
|
||||
0x401C_0000_0000_0000, 0x401C_0B7F_2000_0000, 0x401C_16E7_A000_0000, 0x401C_2239_A000_0000,
|
||||
0x401C_2D75_A000_0000, 0x401C_389C_0000_0000, 0x401C_43AC_E000_0000, 0x401C_4EA8_C000_0000,
|
||||
0x401C_598F_E000_0000, 0x401C_6462_8000_0000, 0x401C_6F21_0000_0000, 0x401C_79CB_C000_0000,
|
||||
0x401C_8462_C000_0000, 0x401C_8EE6_8000_0000, 0x401C_9957_4000_0000, 0x401C_A3B5_4000_0000,
|
||||
0x401C_AE00_E000_0000, 0x401C_B83A_2000_0000, 0x401C_C261_6000_0000, 0x401C_CC76_E000_0000,
|
||||
0x401C_D67B_0000_0000, 0x401C_E06D_C000_0000, 0x401C_EA4F_8000_0000, 0x401C_F420_6000_0000,
|
||||
0x401C_FDE0_C000_0000, 0x401D_0790_A000_0000, 0x401D_1130_8000_0000, 0x401D_1AC0_6000_0000,
|
||||
0x401D_2440_8000_0000, 0x401D_2DB1_0000_0000, 0x401D_3712_4000_0000, 0x401D_4064_6000_0000,
|
||||
0x401D_49A7_8000_0000, 0x401D_52DB_E000_0000, 0x401D_5C01_A000_0000, 0x401D_6519_0000_0000,
|
||||
0x401D_6E22_2000_0000, 0x401D_771D_2000_0000, 0x401D_800A_6000_0000, 0x401D_88E9_C000_0000,
|
||||
0x401D_91BB_A000_0000, 0x401D_9A80_2000_0000, 0x401D_A337_6000_0000, 0x401D_ABE1_8000_0000,
|
||||
0x401D_B47E_C000_0000, 0x401D_BD0F_2000_0000, 0x401D_C593_0000_0000, 0x401D_CE0A_4000_0000,
|
||||
0x401D_D675_4000_0000, 0x401D_DED4_0000_0000, 0x401D_E726_A000_0000, 0x401D_EF6D_6000_0000,
|
||||
0x401D_F7A8_6000_0000, 0x401D_FFD7_A000_0000, 0x401E_07FB_6000_0000, 0x401E_1013_A000_0000,
|
||||
0x401E_1820_A000_0000, 0x401E_2022_8000_0000, 0x401E_2819_4000_0000, 0x401E_3005_2000_0000,
|
||||
0x401E_37E6_2000_0000, 0x401E_3FBC_8000_0000, 0x401E_4788_4000_0000, 0x401E_4F49_8000_0000,
|
||||
0x401E_5700_6000_0000, 0x401E_5EAD_0000_0000, 0x401E_664F_8000_0000, 0x401E_6DE8_0000_0000,
|
||||
0x401E_7576_8000_0000, 0x401E_7CFB_2000_0000, 0x401E_8476_0000_0000, 0x401E_8BE7_6000_0000,
|
||||
0x401E_934F_0000_0000, 0x401E_9AAD_4000_0000, 0x401E_A202_4000_0000, 0x401E_A94D_E000_0000,
|
||||
0x401E_B090_4000_0000, 0x401E_B7C9_A000_0000, 0x401E_BEFA_0000_0000, 0x401E_C621_8000_0000,
|
||||
0x401E_CD40_2000_0000, 0x401E_D456_0000_0000, 0x401E_DB63_2000_0000, 0x401E_E267_E000_0000,
|
||||
0x401E_E964_0000_0000, 0x401E_F057_C000_0000, 0x401E_F743_2000_0000, 0x401E_FE26_4000_0000,
|
||||
0x401F_0501_4000_0000, 0x401F_0BD4_2000_0000, 0x401F_129E_E000_0000, 0x401F_1961_C000_0000,
|
||||
0x401F_201C_C000_0000, 0x401F_26CF_E000_0000, 0x401F_2D7B_6000_0000, 0x401F_341F_2000_0000,
|
||||
0x401F_3ABB_4000_0000, 0x401F_414F_E000_0000, 0x401F_47DD_0000_0000, 0x401F_4E62_C000_0000,
|
||||
0x401F_54E1_2000_0000, 0x401F_5B58_4000_0000, 0x401F_61C8_2000_0000, 0x401F_6830_E000_0000,
|
||||
0x401F_6E92_A000_0000, 0x401F_74ED_4000_0000, 0x401F_7B40_E000_0000, 0x401F_818D_A000_0000,
|
||||
0x401F_87D3_A000_0000, 0x401F_8E12_C000_0000, 0x401F_944B_2000_0000, 0x401F_9A7C_E000_0000,
|
||||
0x401F_A0A7_E000_0000, 0x401F_A6CC_8000_0000, 0x401F_ACEA_8000_0000, 0x401F_B302_0000_0000,
|
||||
0x401F_B913_4000_0000, 0x401F_BF1E_0000_0000, 0x401F_C522_8000_0000, 0x401F_CB20_C000_0000,
|
||||
0x401F_D118_E000_0000, 0x401F_D70A_C000_0000, 0x401F_DCF6_8000_0000, 0x401F_E2DC_6000_0000,
|
||||
0x401F_E8BC_2000_0000, 0x401F_EE95_E000_0000, 0x401F_F469_C000_0000, 0x401F_FA37_C000_0000,
|
||||
]
|
||||
|
||||
// From enc/encode.c kDefaultCommandCodeNumBits.
|
||||
pri const DEFAULT_COMMAND_CODE_NUM_BITS : base.u32 = 448
|
||||
|
||||
// From enc/encode.c kDefaultCommandDepths.
|
||||
pri const DEFAULT_COMMAND_DEPTHS : roarray[128] base.u8[..= 15] = [
|
||||
0, 4, 4, 5, 6, 6, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8,
|
||||
0, 0, 0, 4, 4, 4, 4, 4, 5, 5, 6, 6, 6, 6, 7, 7,
|
||||
7, 7, 10, 10, 10, 10, 10, 10, 0, 4, 4, 5, 5, 5, 6, 6,
|
||||
7, 8, 8, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
|
||||
5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
6, 6, 6, 6, 6, 6, 5, 5, 5, 5, 5, 5, 4, 4, 4, 4,
|
||||
4, 4, 4, 5, 5, 5, 5, 5, 5, 6, 6, 7, 7, 7, 8, 10,
|
||||
12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 0, 0, 0, 0,
|
||||
]
|
||||
|
||||
// From enc/encode.c kDefaultCommandBits.
|
||||
pri const DEFAULT_COMMAND_BITS : roarray[128] base.u16 = [
|
||||
0, 0, 8, 9, 3, 35, 7, 71, 39, 103, 23, 47,
|
||||
175, 111, 239, 31, 0, 0, 0, 4, 12, 2, 10, 6,
|
||||
13, 29, 11, 43, 27, 59, 87, 55, 15, 79, 319, 831,
|
||||
191, 703, 447, 959, 0, 14, 1, 25, 5, 21, 19, 51,
|
||||
119, 159, 95, 223, 479, 991, 63, 575, 127, 639, 383, 895,
|
||||
255, 767, 511, 1023, 14, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 27, 59, 7, 39,
|
||||
23, 55, 30, 1, 17, 9, 25, 5, 0, 8, 4, 12,
|
||||
2, 10, 6, 21, 13, 29, 3, 19, 11, 15, 47, 31,
|
||||
95, 63, 127, 255, 767, 2815, 1791, 3839, 511, 2559, 1535, 3583,
|
||||
1023, 3071, 2047, 4095, 0, 0, 0, 0,
|
||||
]
|
||||
|
||||
// From enc/encode.c kDefaultCommandCode.
|
||||
pri const DEFAULT_COMMAND_CODE : roarray[57] base.u8 = [
|
||||
0xFF, 0x77, 0xD5, 0xBF, 0xE7, 0xDE, 0xEA, 0x9E, 0x51, 0x5D, 0xDE, 0xC6,
|
||||
0x70, 0x57, 0xBC, 0x58, 0x58, 0x58, 0xD8, 0xD8, 0x58, 0xD5, 0xCB, 0x8C,
|
||||
0xEA, 0xE0, 0xC3, 0x87, 0x1F, 0x83, 0xC1, 0x60, 0x1C, 0x67, 0xB2, 0xAA,
|
||||
0x06, 0x83, 0xC1, 0x60, 0x30, 0x18, 0xCC, 0xA1, 0xCE, 0x88, 0x54, 0x94,
|
||||
0x46, 0xE1, 0xB0, 0xD0, 0x4E, 0xB2, 0xF7, 0x04, 0x00,
|
||||
]
|
||||
|
||||
// From enc/compress_fragment.c kCmdHistoSeed.
|
||||
pri const CMD_HISTO_SEED : roarray[128] base.u32[..= 1] = [
|
||||
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
|
||||
0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
|
||||
1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1,
|
||||
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
|
||||
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
|
||||
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
|
||||
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0,
|
||||
]
|
||||
|
||||
// From enc/compress_fragment_two_pass.c kNumExtraBits.
|
||||
pri const TWO_PASS_NUM_EXTRA_BITS : roarray[128] base.u32[..= 24] = [
|
||||
0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5,
|
||||
6, 7, 8, 9, 10, 12, 14, 24, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
1, 1, 2, 2, 3, 3, 4, 4, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 7, 8, 9, 10, 24,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8,
|
||||
9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16,
|
||||
17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 24,
|
||||
]
|
||||
|
||||
// From enc/compress_fragment_two_pass.c kInsertOffset.
|
||||
pri const TWO_PASS_INSERT_OFFSET : roarray[24] base.u32[..= 22594] = [
|
||||
0, 1, 2, 3, 4, 5, 6, 8, 10, 14, 18, 26,
|
||||
34, 50, 66, 98, 130, 194, 322, 578, 1090, 2114, 6210, 22594,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode.c kBrotliShellGaps.
|
||||
pri const SHELL_GAPS : roarray[6] base.u32[..= 132] = [
|
||||
132, 57, 23, 10, 4, 1,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode.c kLut (in BrotliReverseBits).
|
||||
pri const REVERSE_BITS_LUT : roarray[16] base.u32[..= 15] = [
|
||||
0x00, 0x08, 0x04, 0x0C, 0x02, 0x0A, 0x06, 0x0E, 0x01, 0x09, 0x05, 0x0D,
|
||||
0x03, 0x0B, 0x07, 0x0F,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode_static.h StoreStaticCodeLengthCode: 40 bits.
|
||||
pri const STATIC_CODE_LENGTH_CODE : base.u64 = 0xFF_5555_5554
|
||||
|
||||
// From enc/entropy_encode_static.h kCodeLengthDepth.
|
||||
pri const CODE_LENGTH_DEPTH : roarray[18] base.u32[..= 56] = [
|
||||
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 0, 4, 4,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode_static.h kCodeLengthBits.
|
||||
pri const CODE_LENGTH_BITS : roarray[18] base.u64 = [
|
||||
0, 8, 4, 12, 2, 10, 6, 14, 1, 9, 5, 13, 3, 15, 31, 0, 11, 7,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode_static.h kZeroRepsBits.
|
||||
pri const ZERO_REPS_BITS : roarray[704] base.u64 = [
|
||||
0x0000, 0x0000, 0x0000, 0x0007, 0x0017, 0x0027, 0x0037, 0x0047,
|
||||
0x0057, 0x0067, 0x0077, 0x0770, 0x0B87, 0x1387, 0x1B87, 0x2387,
|
||||
0x2B87, 0x3387, 0x3B87, 0x0397, 0x0B97, 0x1397, 0x1B97, 0x2397,
|
||||
0x2B97, 0x3397, 0x3B97, 0x03A7, 0x0BA7, 0x13A7, 0x1BA7, 0x23A7,
|
||||
0x2BA7, 0x33A7, 0x3BA7, 0x03B7, 0x0BB7, 0x13B7, 0x1BB7, 0x23B7,
|
||||
0x2BB7, 0x33B7, 0x3BB7, 0x03C7, 0x0BC7, 0x13C7, 0x1BC7, 0x23C7,
|
||||
0x2BC7, 0x33C7, 0x3BC7, 0x03D7, 0x0BD7, 0x13D7, 0x1BD7, 0x23D7,
|
||||
0x2BD7, 0x33D7, 0x3BD7, 0x03E7, 0x0BE7, 0x13E7, 0x1BE7, 0x23E7,
|
||||
0x2BE7, 0x33E7, 0x3BE7, 0x03F7, 0x0BF7, 0x13F7, 0x1BF7, 0x23F7,
|
||||
0x2BF7, 0x33F7, 0x3BF7, 0x1_C387, 0x5_C387, 0x9_C387, 0xD_C387, 0x11_C387,
|
||||
0x15_C387, 0x19_C387, 0x1D_C387, 0x1_CB87, 0x5_CB87, 0x9_CB87, 0xD_CB87, 0x11_CB87,
|
||||
0x15_CB87, 0x19_CB87, 0x1D_CB87, 0x1_D387, 0x5_D387, 0x9_D387, 0xD_D387, 0x11_D387,
|
||||
0x15_D387, 0x19_D387, 0x1D_D387, 0x1_DB87, 0x5_DB87, 0x9_DB87, 0xD_DB87, 0x11_DB87,
|
||||
0x15_DB87, 0x19_DB87, 0x1D_DB87, 0x1_E387, 0x5_E387, 0x9_E387, 0xD_E387, 0x11_E387,
|
||||
0x15_E387, 0x19_E387, 0x1D_E387, 0x1_EB87, 0x5_EB87, 0x9_EB87, 0xD_EB87, 0x11_EB87,
|
||||
0x15_EB87, 0x19_EB87, 0x1D_EB87, 0x1_F387, 0x5_F387, 0x9_F387, 0xD_F387, 0x11_F387,
|
||||
0x15_F387, 0x19_F387, 0x1D_F387, 0x1_FB87, 0x5_FB87, 0x9_FB87, 0xD_FB87, 0x11_FB87,
|
||||
0x15_FB87, 0x19_FB87, 0x1D_FB87, 0x1_C397, 0x5_C397, 0x9_C397, 0xD_C397, 0x11_C397,
|
||||
0x15_C397, 0x19_C397, 0x1D_C397, 0x1_CB97, 0x5_CB97, 0x9_CB97, 0xD_CB97, 0x11_CB97,
|
||||
0x15_CB97, 0x19_CB97, 0x1D_CB97, 0x1_D397, 0x5_D397, 0x9_D397, 0xD_D397, 0x11_D397,
|
||||
0x15_D397, 0x19_D397, 0x1D_D397, 0x1_DB97, 0x5_DB97, 0x9_DB97, 0xD_DB97, 0x11_DB97,
|
||||
0x15_DB97, 0x19_DB97, 0x1D_DB97, 0x1_E397, 0x5_E397, 0x9_E397, 0xD_E397, 0x11_E397,
|
||||
0x15_E397, 0x19_E397, 0x1D_E397, 0x1_EB97, 0x5_EB97, 0x9_EB97, 0xD_EB97, 0x11_EB97,
|
||||
0x15_EB97, 0x19_EB97, 0x1D_EB97, 0x1_F397, 0x5_F397, 0x9_F397, 0xD_F397, 0x11_F397,
|
||||
0x15_F397, 0x19_F397, 0x1D_F397, 0x1_FB97, 0x5_FB97, 0x9_FB97, 0xD_FB97, 0x11_FB97,
|
||||
0x15_FB97, 0x19_FB97, 0x1D_FB97, 0x1_C3A7, 0x5_C3A7, 0x9_C3A7, 0xD_C3A7, 0x11_C3A7,
|
||||
0x15_C3A7, 0x19_C3A7, 0x1D_C3A7, 0x1_CBA7, 0x5_CBA7, 0x9_CBA7, 0xD_CBA7, 0x11_CBA7,
|
||||
0x15_CBA7, 0x19_CBA7, 0x1D_CBA7, 0x1_D3A7, 0x5_D3A7, 0x9_D3A7, 0xD_D3A7, 0x11_D3A7,
|
||||
0x15_D3A7, 0x19_D3A7, 0x1D_D3A7, 0x1_DBA7, 0x5_DBA7, 0x9_DBA7, 0xD_DBA7, 0x11_DBA7,
|
||||
0x15_DBA7, 0x19_DBA7, 0x1D_DBA7, 0x1_E3A7, 0x5_E3A7, 0x9_E3A7, 0xD_E3A7, 0x11_E3A7,
|
||||
0x15_E3A7, 0x19_E3A7, 0x1D_E3A7, 0x1_EBA7, 0x5_EBA7, 0x9_EBA7, 0xD_EBA7, 0x11_EBA7,
|
||||
0x15_EBA7, 0x19_EBA7, 0x1D_EBA7, 0x1_F3A7, 0x5_F3A7, 0x9_F3A7, 0xD_F3A7, 0x11_F3A7,
|
||||
0x15_F3A7, 0x19_F3A7, 0x1D_F3A7, 0x1_FBA7, 0x5_FBA7, 0x9_FBA7, 0xD_FBA7, 0x11_FBA7,
|
||||
0x15_FBA7, 0x19_FBA7, 0x1D_FBA7, 0x1_C3B7, 0x5_C3B7, 0x9_C3B7, 0xD_C3B7, 0x11_C3B7,
|
||||
0x15_C3B7, 0x19_C3B7, 0x1D_C3B7, 0x1_CBB7, 0x5_CBB7, 0x9_CBB7, 0xD_CBB7, 0x11_CBB7,
|
||||
0x15_CBB7, 0x19_CBB7, 0x1D_CBB7, 0x1_D3B7, 0x5_D3B7, 0x9_D3B7, 0xD_D3B7, 0x11_D3B7,
|
||||
0x15_D3B7, 0x19_D3B7, 0x1D_D3B7, 0x1_DBB7, 0x5_DBB7, 0x9_DBB7, 0xD_DBB7, 0x11_DBB7,
|
||||
0x15_DBB7, 0x19_DBB7, 0x1D_DBB7, 0x1_E3B7, 0x5_E3B7, 0x9_E3B7, 0xD_E3B7, 0x11_E3B7,
|
||||
0x15_E3B7, 0x19_E3B7, 0x1D_E3B7, 0x1_EBB7, 0x5_EBB7, 0x9_EBB7, 0xD_EBB7, 0x11_EBB7,
|
||||
0x15_EBB7, 0x19_EBB7, 0x1D_EBB7, 0x1_F3B7, 0x5_F3B7, 0x9_F3B7, 0xD_F3B7, 0x11_F3B7,
|
||||
0x15_F3B7, 0x19_F3B7, 0x1D_F3B7, 0x1_FBB7, 0x5_FBB7, 0x9_FBB7, 0xD_FBB7, 0x11_FBB7,
|
||||
0x15_FBB7, 0x19_FBB7, 0x1D_FBB7, 0x1_C3C7, 0x5_C3C7, 0x9_C3C7, 0xD_C3C7, 0x11_C3C7,
|
||||
0x15_C3C7, 0x19_C3C7, 0x1D_C3C7, 0x1_CBC7, 0x5_CBC7, 0x9_CBC7, 0xD_CBC7, 0x11_CBC7,
|
||||
0x15_CBC7, 0x19_CBC7, 0x1D_CBC7, 0x1_D3C7, 0x5_D3C7, 0x9_D3C7, 0xD_D3C7, 0x11_D3C7,
|
||||
0x15_D3C7, 0x19_D3C7, 0x1D_D3C7, 0x1_DBC7, 0x5_DBC7, 0x9_DBC7, 0xD_DBC7, 0x11_DBC7,
|
||||
0x15_DBC7, 0x19_DBC7, 0x1D_DBC7, 0x1_E3C7, 0x5_E3C7, 0x9_E3C7, 0xD_E3C7, 0x11_E3C7,
|
||||
0x15_E3C7, 0x19_E3C7, 0x1D_E3C7, 0x1_EBC7, 0x5_EBC7, 0x9_EBC7, 0xD_EBC7, 0x11_EBC7,
|
||||
0x15_EBC7, 0x19_EBC7, 0x1D_EBC7, 0x1_F3C7, 0x5_F3C7, 0x9_F3C7, 0xD_F3C7, 0x11_F3C7,
|
||||
0x15_F3C7, 0x19_F3C7, 0x1D_F3C7, 0x1_FBC7, 0x5_FBC7, 0x9_FBC7, 0xD_FBC7, 0x11_FBC7,
|
||||
0x15_FBC7, 0x19_FBC7, 0x1D_FBC7, 0x1_C3D7, 0x5_C3D7, 0x9_C3D7, 0xD_C3D7, 0x11_C3D7,
|
||||
0x15_C3D7, 0x19_C3D7, 0x1D_C3D7, 0x1_CBD7, 0x5_CBD7, 0x9_CBD7, 0xD_CBD7, 0x11_CBD7,
|
||||
0x15_CBD7, 0x19_CBD7, 0x1D_CBD7, 0x1_D3D7, 0x5_D3D7, 0x9_D3D7, 0xD_D3D7, 0x11_D3D7,
|
||||
0x15_D3D7, 0x19_D3D7, 0x1D_D3D7, 0x1_DBD7, 0x5_DBD7, 0x9_DBD7, 0xD_DBD7, 0x11_DBD7,
|
||||
0x15_DBD7, 0x19_DBD7, 0x1D_DBD7, 0x1_E3D7, 0x5_E3D7, 0x9_E3D7, 0xD_E3D7, 0x11_E3D7,
|
||||
0x15_E3D7, 0x19_E3D7, 0x1D_E3D7, 0x1_EBD7, 0x5_EBD7, 0x9_EBD7, 0xD_EBD7, 0x11_EBD7,
|
||||
0x15_EBD7, 0x19_EBD7, 0x1D_EBD7, 0x1_F3D7, 0x5_F3D7, 0x9_F3D7, 0xD_F3D7, 0x11_F3D7,
|
||||
0x15_F3D7, 0x19_F3D7, 0x1D_F3D7, 0x1_FBD7, 0x5_FBD7, 0x9_FBD7, 0xD_FBD7, 0x11_FBD7,
|
||||
0x15_FBD7, 0x19_FBD7, 0x1D_FBD7, 0x1_C3E7, 0x5_C3E7, 0x9_C3E7, 0xD_C3E7, 0x11_C3E7,
|
||||
0x15_C3E7, 0x19_C3E7, 0x1D_C3E7, 0x1_CBE7, 0x5_CBE7, 0x9_CBE7, 0xD_CBE7, 0x11_CBE7,
|
||||
0x15_CBE7, 0x19_CBE7, 0x1D_CBE7, 0x1_D3E7, 0x5_D3E7, 0x9_D3E7, 0xD_D3E7, 0x11_D3E7,
|
||||
0x15_D3E7, 0x19_D3E7, 0x1D_D3E7, 0x1_DBE7, 0x5_DBE7, 0x9_DBE7, 0xD_DBE7, 0x11_DBE7,
|
||||
0x15_DBE7, 0x19_DBE7, 0x1D_DBE7, 0x1_E3E7, 0x5_E3E7, 0x9_E3E7, 0xD_E3E7, 0x11_E3E7,
|
||||
0x15_E3E7, 0x19_E3E7, 0x1D_E3E7, 0x1_EBE7, 0x5_EBE7, 0x9_EBE7, 0xD_EBE7, 0x11_EBE7,
|
||||
0x15_EBE7, 0x19_EBE7, 0x1D_EBE7, 0x1_F3E7, 0x5_F3E7, 0x9_F3E7, 0xD_F3E7, 0x11_F3E7,
|
||||
0x15_F3E7, 0x19_F3E7, 0x1D_F3E7, 0x1_FBE7, 0x5_FBE7, 0x9_FBE7, 0xD_FBE7, 0x11_FBE7,
|
||||
0x15_FBE7, 0x19_FBE7, 0x1D_FBE7, 0x1_C3F7, 0x5_C3F7, 0x9_C3F7, 0xD_C3F7, 0x11_C3F7,
|
||||
0x15_C3F7, 0x19_C3F7, 0x1D_C3F7, 0x1_CBF7, 0x5_CBF7, 0x9_CBF7, 0xD_CBF7, 0x11_CBF7,
|
||||
0x15_CBF7, 0x19_CBF7, 0x1D_CBF7, 0x1_D3F7, 0x5_D3F7, 0x9_D3F7, 0xD_D3F7, 0x11_D3F7,
|
||||
0x15_D3F7, 0x19_D3F7, 0x1D_D3F7, 0x1_DBF7, 0x5_DBF7, 0x9_DBF7, 0xD_DBF7, 0x11_DBF7,
|
||||
0x15_DBF7, 0x19_DBF7, 0x1D_DBF7, 0x1_E3F7, 0x5_E3F7, 0x9_E3F7, 0xD_E3F7, 0x11_E3F7,
|
||||
0x15_E3F7, 0x19_E3F7, 0x1D_E3F7, 0x1_EBF7, 0x5_EBF7, 0x9_EBF7, 0xD_EBF7, 0x11_EBF7,
|
||||
0x15_EBF7, 0x19_EBF7, 0x1D_EBF7, 0x1_F3F7, 0x5_F3F7, 0x9_F3F7, 0xD_F3F7, 0x11_F3F7,
|
||||
0x15_F3F7, 0x19_F3F7, 0x1D_F3F7, 0x1_FBF7, 0x5_FBF7, 0x9_FBF7, 0xD_FBF7, 0x11_FBF7,
|
||||
0x15_FBF7, 0x19_FBF7, 0x1D_FBF7, 0xE1_C387, 0x2E1_C387, 0x4E1_C387, 0x6E1_C387, 0x8E1_C387,
|
||||
0xAE1_C387, 0xCE1_C387, 0xEE1_C387, 0xE5_C387, 0x2E5_C387, 0x4E5_C387, 0x6E5_C387, 0x8E5_C387,
|
||||
0xAE5_C387, 0xCE5_C387, 0xEE5_C387, 0xE9_C387, 0x2E9_C387, 0x4E9_C387, 0x6E9_C387, 0x8E9_C387,
|
||||
0xAE9_C387, 0xCE9_C387, 0xEE9_C387, 0xED_C387, 0x2ED_C387, 0x4ED_C387, 0x6ED_C387, 0x8ED_C387,
|
||||
0xAED_C387, 0xCED_C387, 0xEED_C387, 0xF1_C387, 0x2F1_C387, 0x4F1_C387, 0x6F1_C387, 0x8F1_C387,
|
||||
0xAF1_C387, 0xCF1_C387, 0xEF1_C387, 0xF5_C387, 0x2F5_C387, 0x4F5_C387, 0x6F5_C387, 0x8F5_C387,
|
||||
0xAF5_C387, 0xCF5_C387, 0xEF5_C387, 0xF9_C387, 0x2F9_C387, 0x4F9_C387, 0x6F9_C387, 0x8F9_C387,
|
||||
0xAF9_C387, 0xCF9_C387, 0xEF9_C387, 0xFD_C387, 0x2FD_C387, 0x4FD_C387, 0x6FD_C387, 0x8FD_C387,
|
||||
0xAFD_C387, 0xCFD_C387, 0xEFD_C387, 0xE1_CB87, 0x2E1_CB87, 0x4E1_CB87, 0x6E1_CB87, 0x8E1_CB87,
|
||||
0xAE1_CB87, 0xCE1_CB87, 0xEE1_CB87, 0xE5_CB87, 0x2E5_CB87, 0x4E5_CB87, 0x6E5_CB87, 0x8E5_CB87,
|
||||
0xAE5_CB87, 0xCE5_CB87, 0xEE5_CB87, 0xE9_CB87, 0x2E9_CB87, 0x4E9_CB87, 0x6E9_CB87, 0x8E9_CB87,
|
||||
0xAE9_CB87, 0xCE9_CB87, 0xEE9_CB87, 0xED_CB87, 0x2ED_CB87, 0x4ED_CB87, 0x6ED_CB87, 0x8ED_CB87,
|
||||
0xAED_CB87, 0xCED_CB87, 0xEED_CB87, 0xF1_CB87, 0x2F1_CB87, 0x4F1_CB87, 0x6F1_CB87, 0x8F1_CB87,
|
||||
0xAF1_CB87, 0xCF1_CB87, 0xEF1_CB87, 0xF5_CB87, 0x2F5_CB87, 0x4F5_CB87, 0x6F5_CB87, 0x8F5_CB87,
|
||||
0xAF5_CB87, 0xCF5_CB87, 0xEF5_CB87, 0xF9_CB87, 0x2F9_CB87, 0x4F9_CB87, 0x6F9_CB87, 0x8F9_CB87,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode_static.h kZeroRepsDepth.
|
||||
pri const ZERO_REPS_DEPTH : roarray[704] base.u32[..= 56] = [
|
||||
0, 4, 8, 7, 7, 7, 7, 7, 7, 7, 7, 11, 14, 14, 14, 14,
|
||||
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
|
||||
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
|
||||
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
|
||||
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
|
||||
21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 28, 28, 28, 28, 28,
|
||||
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
|
||||
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
|
||||
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
|
||||
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
|
||||
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
|
||||
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
|
||||
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode_static.h kNonZeroRepsBits.
|
||||
pri const NON_ZERO_REPS_BITS : roarray[704] base.u64 = [
|
||||
0x000B, 0x001B, 0x002B, 0x003B,
|
||||
0x02CB, 0x06CB, 0x0ACB, 0x0ECB,
|
||||
0x02DB, 0x06DB, 0x0ADB, 0x0EDB,
|
||||
0x02EB, 0x06EB, 0x0AEB, 0x0EEB,
|
||||
0x02FB, 0x06FB, 0x0AFB, 0x0EFB,
|
||||
0xB2CB, 0x1_B2CB, 0x2_B2CB, 0x3_B2CB,
|
||||
0xB6CB, 0x1_B6CB, 0x2_B6CB, 0x3_B6CB,
|
||||
0xBACB, 0x1_BACB, 0x2_BACB, 0x3_BACB,
|
||||
0xBECB, 0x1_BECB, 0x2_BECB, 0x3_BECB,
|
||||
0xB2DB, 0x1_B2DB, 0x2_B2DB, 0x3_B2DB,
|
||||
0xB6DB, 0x1_B6DB, 0x2_B6DB, 0x3_B6DB,
|
||||
0xBADB, 0x1_BADB, 0x2_BADB, 0x3_BADB,
|
||||
0xBEDB, 0x1_BEDB, 0x2_BEDB, 0x3_BEDB,
|
||||
0xB2EB, 0x1_B2EB, 0x2_B2EB, 0x3_B2EB,
|
||||
0xB6EB, 0x1_B6EB, 0x2_B6EB, 0x3_B6EB,
|
||||
0xBAEB, 0x1_BAEB, 0x2_BAEB, 0x3_BAEB,
|
||||
0xBEEB, 0x1_BEEB, 0x2_BEEB, 0x3_BEEB,
|
||||
0xB2FB, 0x1_B2FB, 0x2_B2FB, 0x3_B2FB,
|
||||
0xB6FB, 0x1_B6FB, 0x2_B6FB, 0x3_B6FB,
|
||||
0xBAFB, 0x1_BAFB, 0x2_BAFB, 0x3_BAFB,
|
||||
0xBEFB, 0x1_BEFB, 0x2_BEFB, 0x3_BEFB,
|
||||
0x2C_B2CB, 0x6C_B2CB, 0xAC_B2CB, 0xEC_B2CB,
|
||||
0x2D_B2CB, 0x6D_B2CB, 0xAD_B2CB, 0xED_B2CB,
|
||||
0x2E_B2CB, 0x6E_B2CB, 0xAE_B2CB, 0xEE_B2CB,
|
||||
0x2F_B2CB, 0x6F_B2CB, 0xAF_B2CB, 0xEF_B2CB,
|
||||
0x2C_B6CB, 0x6C_B6CB, 0xAC_B6CB, 0xEC_B6CB,
|
||||
0x2D_B6CB, 0x6D_B6CB, 0xAD_B6CB, 0xED_B6CB,
|
||||
0x2E_B6CB, 0x6E_B6CB, 0xAE_B6CB, 0xEE_B6CB,
|
||||
0x2F_B6CB, 0x6F_B6CB, 0xAF_B6CB, 0xEF_B6CB,
|
||||
0x2C_BACB, 0x6C_BACB, 0xAC_BACB, 0xEC_BACB,
|
||||
0x2D_BACB, 0x6D_BACB, 0xAD_BACB, 0xED_BACB,
|
||||
0x2E_BACB, 0x6E_BACB, 0xAE_BACB, 0xEE_BACB,
|
||||
0x2F_BACB, 0x6F_BACB, 0xAF_BACB, 0xEF_BACB,
|
||||
0x2C_BECB, 0x6C_BECB, 0xAC_BECB, 0xEC_BECB,
|
||||
0x2D_BECB, 0x6D_BECB, 0xAD_BECB, 0xED_BECB,
|
||||
0x2E_BECB, 0x6E_BECB, 0xAE_BECB, 0xEE_BECB,
|
||||
0x2F_BECB, 0x6F_BECB, 0xAF_BECB, 0xEF_BECB,
|
||||
0x2C_B2DB, 0x6C_B2DB, 0xAC_B2DB, 0xEC_B2DB,
|
||||
0x2D_B2DB, 0x6D_B2DB, 0xAD_B2DB, 0xED_B2DB,
|
||||
0x2E_B2DB, 0x6E_B2DB, 0xAE_B2DB, 0xEE_B2DB,
|
||||
0x2F_B2DB, 0x6F_B2DB, 0xAF_B2DB, 0xEF_B2DB,
|
||||
0x2C_B6DB, 0x6C_B6DB, 0xAC_B6DB, 0xEC_B6DB,
|
||||
0x2D_B6DB, 0x6D_B6DB, 0xAD_B6DB, 0xED_B6DB,
|
||||
0x2E_B6DB, 0x6E_B6DB, 0xAE_B6DB, 0xEE_B6DB,
|
||||
0x2F_B6DB, 0x6F_B6DB, 0xAF_B6DB, 0xEF_B6DB,
|
||||
0x2C_BADB, 0x6C_BADB, 0xAC_BADB, 0xEC_BADB,
|
||||
0x2D_BADB, 0x6D_BADB, 0xAD_BADB, 0xED_BADB,
|
||||
0x2E_BADB, 0x6E_BADB, 0xAE_BADB, 0xEE_BADB,
|
||||
0x2F_BADB, 0x6F_BADB, 0xAF_BADB, 0xEF_BADB,
|
||||
0x2C_BEDB, 0x6C_BEDB, 0xAC_BEDB, 0xEC_BEDB,
|
||||
0x2D_BEDB, 0x6D_BEDB, 0xAD_BEDB, 0xED_BEDB,
|
||||
0x2E_BEDB, 0x6E_BEDB, 0xAE_BEDB, 0xEE_BEDB,
|
||||
0x2F_BEDB, 0x6F_BEDB, 0xAF_BEDB, 0xEF_BEDB,
|
||||
0x2C_B2EB, 0x6C_B2EB, 0xAC_B2EB, 0xEC_B2EB,
|
||||
0x2D_B2EB, 0x6D_B2EB, 0xAD_B2EB, 0xED_B2EB,
|
||||
0x2E_B2EB, 0x6E_B2EB, 0xAE_B2EB, 0xEE_B2EB,
|
||||
0x2F_B2EB, 0x6F_B2EB, 0xAF_B2EB, 0xEF_B2EB,
|
||||
0x2C_B6EB, 0x6C_B6EB, 0xAC_B6EB, 0xEC_B6EB,
|
||||
0x2D_B6EB, 0x6D_B6EB, 0xAD_B6EB, 0xED_B6EB,
|
||||
0x2E_B6EB, 0x6E_B6EB, 0xAE_B6EB, 0xEE_B6EB,
|
||||
0x2F_B6EB, 0x6F_B6EB, 0xAF_B6EB, 0xEF_B6EB,
|
||||
0x2C_BAEB, 0x6C_BAEB, 0xAC_BAEB, 0xEC_BAEB,
|
||||
0x2D_BAEB, 0x6D_BAEB, 0xAD_BAEB, 0xED_BAEB,
|
||||
0x2E_BAEB, 0x6E_BAEB, 0xAE_BAEB, 0xEE_BAEB,
|
||||
0x2F_BAEB, 0x6F_BAEB, 0xAF_BAEB, 0xEF_BAEB,
|
||||
0x2C_BEEB, 0x6C_BEEB, 0xAC_BEEB, 0xEC_BEEB,
|
||||
0x2D_BEEB, 0x6D_BEEB, 0xAD_BEEB, 0xED_BEEB,
|
||||
0x2E_BEEB, 0x6E_BEEB, 0xAE_BEEB, 0xEE_BEEB,
|
||||
0x2F_BEEB, 0x6F_BEEB, 0xAF_BEEB, 0xEF_BEEB,
|
||||
0x2C_B2FB, 0x6C_B2FB, 0xAC_B2FB, 0xEC_B2FB,
|
||||
0x2D_B2FB, 0x6D_B2FB, 0xAD_B2FB, 0xED_B2FB,
|
||||
0x2E_B2FB, 0x6E_B2FB, 0xAE_B2FB, 0xEE_B2FB,
|
||||
0x2F_B2FB, 0x6F_B2FB, 0xAF_B2FB, 0xEF_B2FB,
|
||||
0x2C_B6FB, 0x6C_B6FB, 0xAC_B6FB, 0xEC_B6FB,
|
||||
0x2D_B6FB, 0x6D_B6FB, 0xAD_B6FB, 0xED_B6FB,
|
||||
0x2E_B6FB, 0x6E_B6FB, 0xAE_B6FB, 0xEE_B6FB,
|
||||
0x2F_B6FB, 0x6F_B6FB, 0xAF_B6FB, 0xEF_B6FB,
|
||||
0x2C_BAFB, 0x6C_BAFB, 0xAC_BAFB, 0xEC_BAFB,
|
||||
0x2D_BAFB, 0x6D_BAFB, 0xAD_BAFB, 0xED_BAFB,
|
||||
0x2E_BAFB, 0x6E_BAFB, 0xAE_BAFB, 0xEE_BAFB,
|
||||
0x2F_BAFB, 0x6F_BAFB, 0xAF_BAFB, 0xEF_BAFB,
|
||||
0x2C_BEFB, 0x6C_BEFB, 0xAC_BEFB, 0xEC_BEFB,
|
||||
0x2D_BEFB, 0x6D_BEFB, 0xAD_BEFB, 0xED_BEFB,
|
||||
0x2E_BEFB, 0x6E_BEFB, 0xAE_BEFB, 0xEE_BEFB,
|
||||
0x2F_BEFB, 0x6F_BEFB, 0xAF_BEFB, 0xEF_BEFB,
|
||||
0xB2C_B2CB, 0x1B2C_B2CB, 0x2B2C_B2CB, 0x3B2C_B2CB,
|
||||
0xB6C_B2CB, 0x1B6C_B2CB, 0x2B6C_B2CB, 0x3B6C_B2CB,
|
||||
0xBAC_B2CB, 0x1BAC_B2CB, 0x2BAC_B2CB, 0x3BAC_B2CB,
|
||||
0xBEC_B2CB, 0x1BEC_B2CB, 0x2BEC_B2CB, 0x3BEC_B2CB,
|
||||
0xB2D_B2CB, 0x1B2D_B2CB, 0x2B2D_B2CB, 0x3B2D_B2CB,
|
||||
0xB6D_B2CB, 0x1B6D_B2CB, 0x2B6D_B2CB, 0x3B6D_B2CB,
|
||||
0xBAD_B2CB, 0x1BAD_B2CB, 0x2BAD_B2CB, 0x3BAD_B2CB,
|
||||
0xBED_B2CB, 0x1BED_B2CB, 0x2BED_B2CB, 0x3BED_B2CB,
|
||||
0xB2E_B2CB, 0x1B2E_B2CB, 0x2B2E_B2CB, 0x3B2E_B2CB,
|
||||
0xB6E_B2CB, 0x1B6E_B2CB, 0x2B6E_B2CB, 0x3B6E_B2CB,
|
||||
0xBAE_B2CB, 0x1BAE_B2CB, 0x2BAE_B2CB, 0x3BAE_B2CB,
|
||||
0xBEE_B2CB, 0x1BEE_B2CB, 0x2BEE_B2CB, 0x3BEE_B2CB,
|
||||
0xB2F_B2CB, 0x1B2F_B2CB, 0x2B2F_B2CB, 0x3B2F_B2CB,
|
||||
0xB6F_B2CB, 0x1B6F_B2CB, 0x2B6F_B2CB, 0x3B6F_B2CB,
|
||||
0xBAF_B2CB, 0x1BAF_B2CB, 0x2BAF_B2CB, 0x3BAF_B2CB,
|
||||
0xBEF_B2CB, 0x1BEF_B2CB, 0x2BEF_B2CB, 0x3BEF_B2CB,
|
||||
0xB2C_B6CB, 0x1B2C_B6CB, 0x2B2C_B6CB, 0x3B2C_B6CB,
|
||||
0xB6C_B6CB, 0x1B6C_B6CB, 0x2B6C_B6CB, 0x3B6C_B6CB,
|
||||
0xBAC_B6CB, 0x1BAC_B6CB, 0x2BAC_B6CB, 0x3BAC_B6CB,
|
||||
0xBEC_B6CB, 0x1BEC_B6CB, 0x2BEC_B6CB, 0x3BEC_B6CB,
|
||||
0xB2D_B6CB, 0x1B2D_B6CB, 0x2B2D_B6CB, 0x3B2D_B6CB,
|
||||
0xB6D_B6CB, 0x1B6D_B6CB, 0x2B6D_B6CB, 0x3B6D_B6CB,
|
||||
0xBAD_B6CB, 0x1BAD_B6CB, 0x2BAD_B6CB, 0x3BAD_B6CB,
|
||||
0xBED_B6CB, 0x1BED_B6CB, 0x2BED_B6CB, 0x3BED_B6CB,
|
||||
0xB2E_B6CB, 0x1B2E_B6CB, 0x2B2E_B6CB, 0x3B2E_B6CB,
|
||||
0xB6E_B6CB, 0x1B6E_B6CB, 0x2B6E_B6CB, 0x3B6E_B6CB,
|
||||
0xBAE_B6CB, 0x1BAE_B6CB, 0x2BAE_B6CB, 0x3BAE_B6CB,
|
||||
0xBEE_B6CB, 0x1BEE_B6CB, 0x2BEE_B6CB, 0x3BEE_B6CB,
|
||||
0xB2F_B6CB, 0x1B2F_B6CB, 0x2B2F_B6CB, 0x3B2F_B6CB,
|
||||
0xB6F_B6CB, 0x1B6F_B6CB, 0x2B6F_B6CB, 0x3B6F_B6CB,
|
||||
0xBAF_B6CB, 0x1BAF_B6CB, 0x2BAF_B6CB, 0x3BAF_B6CB,
|
||||
0xBEF_B6CB, 0x1BEF_B6CB, 0x2BEF_B6CB, 0x3BEF_B6CB,
|
||||
0xB2C_BACB, 0x1B2C_BACB, 0x2B2C_BACB, 0x3B2C_BACB,
|
||||
0xB6C_BACB, 0x1B6C_BACB, 0x2B6C_BACB, 0x3B6C_BACB,
|
||||
0xBAC_BACB, 0x1BAC_BACB, 0x2BAC_BACB, 0x3BAC_BACB,
|
||||
0xBEC_BACB, 0x1BEC_BACB, 0x2BEC_BACB, 0x3BEC_BACB,
|
||||
0xB2D_BACB, 0x1B2D_BACB, 0x2B2D_BACB, 0x3B2D_BACB,
|
||||
0xB6D_BACB, 0x1B6D_BACB, 0x2B6D_BACB, 0x3B6D_BACB,
|
||||
0xBAD_BACB, 0x1BAD_BACB, 0x2BAD_BACB, 0x3BAD_BACB,
|
||||
0xBED_BACB, 0x1BED_BACB, 0x2BED_BACB, 0x3BED_BACB,
|
||||
0xB2E_BACB, 0x1B2E_BACB, 0x2B2E_BACB, 0x3B2E_BACB,
|
||||
0xB6E_BACB, 0x1B6E_BACB, 0x2B6E_BACB, 0x3B6E_BACB,
|
||||
0xBAE_BACB, 0x1BAE_BACB, 0x2BAE_BACB, 0x3BAE_BACB,
|
||||
0xBEE_BACB, 0x1BEE_BACB, 0x2BEE_BACB, 0x3BEE_BACB,
|
||||
0xB2F_BACB, 0x1B2F_BACB, 0x2B2F_BACB, 0x3B2F_BACB,
|
||||
0xB6F_BACB, 0x1B6F_BACB, 0x2B6F_BACB, 0x3B6F_BACB,
|
||||
0xBAF_BACB, 0x1BAF_BACB, 0x2BAF_BACB, 0x3BAF_BACB,
|
||||
0xBEF_BACB, 0x1BEF_BACB, 0x2BEF_BACB, 0x3BEF_BACB,
|
||||
0xB2C_BECB, 0x1B2C_BECB, 0x2B2C_BECB, 0x3B2C_BECB,
|
||||
0xB6C_BECB, 0x1B6C_BECB, 0x2B6C_BECB, 0x3B6C_BECB,
|
||||
0xBAC_BECB, 0x1BAC_BECB, 0x2BAC_BECB, 0x3BAC_BECB,
|
||||
0xBEC_BECB, 0x1BEC_BECB, 0x2BEC_BECB, 0x3BEC_BECB,
|
||||
0xB2D_BECB, 0x1B2D_BECB, 0x2B2D_BECB, 0x3B2D_BECB,
|
||||
0xB6D_BECB, 0x1B6D_BECB, 0x2B6D_BECB, 0x3B6D_BECB,
|
||||
0xBAD_BECB, 0x1BAD_BECB, 0x2BAD_BECB, 0x3BAD_BECB,
|
||||
0xBED_BECB, 0x1BED_BECB, 0x2BED_BECB, 0x3BED_BECB,
|
||||
0xB2E_BECB, 0x1B2E_BECB, 0x2B2E_BECB, 0x3B2E_BECB,
|
||||
0xB6E_BECB, 0x1B6E_BECB, 0x2B6E_BECB, 0x3B6E_BECB,
|
||||
0xBAE_BECB, 0x1BAE_BECB, 0x2BAE_BECB, 0x3BAE_BECB,
|
||||
0xBEE_BECB, 0x1BEE_BECB, 0x2BEE_BECB, 0x3BEE_BECB,
|
||||
0xB2F_BECB, 0x1B2F_BECB, 0x2B2F_BECB, 0x3B2F_BECB,
|
||||
0xB6F_BECB, 0x1B6F_BECB, 0x2B6F_BECB, 0x3B6F_BECB,
|
||||
0xBAF_BECB, 0x1BAF_BECB, 0x2BAF_BECB, 0x3BAF_BECB,
|
||||
0xBEF_BECB, 0x1BEF_BECB, 0x2BEF_BECB, 0x3BEF_BECB,
|
||||
0xB2C_B2DB, 0x1B2C_B2DB, 0x2B2C_B2DB, 0x3B2C_B2DB,
|
||||
0xB6C_B2DB, 0x1B6C_B2DB, 0x2B6C_B2DB, 0x3B6C_B2DB,
|
||||
0xBAC_B2DB, 0x1BAC_B2DB, 0x2BAC_B2DB, 0x3BAC_B2DB,
|
||||
0xBEC_B2DB, 0x1BEC_B2DB, 0x2BEC_B2DB, 0x3BEC_B2DB,
|
||||
0xB2D_B2DB, 0x1B2D_B2DB, 0x2B2D_B2DB, 0x3B2D_B2DB,
|
||||
0xB6D_B2DB, 0x1B6D_B2DB, 0x2B6D_B2DB, 0x3B6D_B2DB,
|
||||
0xBAD_B2DB, 0x1BAD_B2DB, 0x2BAD_B2DB, 0x3BAD_B2DB,
|
||||
0xBED_B2DB, 0x1BED_B2DB, 0x2BED_B2DB, 0x3BED_B2DB,
|
||||
0xB2E_B2DB, 0x1B2E_B2DB, 0x2B2E_B2DB, 0x3B2E_B2DB,
|
||||
0xB6E_B2DB, 0x1B6E_B2DB, 0x2B6E_B2DB, 0x3B6E_B2DB,
|
||||
0xBAE_B2DB, 0x1BAE_B2DB, 0x2BAE_B2DB, 0x3BAE_B2DB,
|
||||
0xBEE_B2DB, 0x1BEE_B2DB, 0x2BEE_B2DB, 0x3BEE_B2DB,
|
||||
0xB2F_B2DB, 0x1B2F_B2DB, 0x2B2F_B2DB, 0x3B2F_B2DB,
|
||||
0xB6F_B2DB, 0x1B6F_B2DB, 0x2B6F_B2DB, 0x3B6F_B2DB,
|
||||
0xBAF_B2DB, 0x1BAF_B2DB, 0x2BAF_B2DB, 0x3BAF_B2DB,
|
||||
0xBEF_B2DB, 0x1BEF_B2DB, 0x2BEF_B2DB, 0x3BEF_B2DB,
|
||||
0xB2C_B6DB, 0x1B2C_B6DB, 0x2B2C_B6DB, 0x3B2C_B6DB,
|
||||
0xB6C_B6DB, 0x1B6C_B6DB, 0x2B6C_B6DB, 0x3B6C_B6DB,
|
||||
0xBAC_B6DB, 0x1BAC_B6DB, 0x2BAC_B6DB, 0x3BAC_B6DB,
|
||||
0xBEC_B6DB, 0x1BEC_B6DB, 0x2BEC_B6DB, 0x3BEC_B6DB,
|
||||
0xB2D_B6DB, 0x1B2D_B6DB, 0x2B2D_B6DB, 0x3B2D_B6DB,
|
||||
0xB6D_B6DB, 0x1B6D_B6DB, 0x2B6D_B6DB, 0x3B6D_B6DB,
|
||||
0xBAD_B6DB, 0x1BAD_B6DB, 0x2BAD_B6DB, 0x3BAD_B6DB,
|
||||
0xBED_B6DB, 0x1BED_B6DB, 0x2BED_B6DB, 0x3BED_B6DB,
|
||||
0xB2E_B6DB, 0x1B2E_B6DB, 0x2B2E_B6DB, 0x3B2E_B6DB,
|
||||
0xB6E_B6DB, 0x1B6E_B6DB, 0x2B6E_B6DB, 0x3B6E_B6DB,
|
||||
0xBAE_B6DB, 0x1BAE_B6DB, 0x2BAE_B6DB, 0x3BAE_B6DB,
|
||||
]
|
||||
|
||||
// From enc/entropy_encode_static.h kNonZeroRepsDepth.
|
||||
pri const NON_ZERO_REPS_DEPTH : roarray[704] base.u32[..= 56] = [
|
||||
6, 6, 6, 6, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
|
||||
12, 12, 12, 12, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
|
||||
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
|
||||
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
|
||||
18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
|
||||
18, 18, 18, 18, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
|
||||
24, 24, 24, 24, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30,
|
||||
]
|
||||
|
||||
// From enc/brotli_bit_stream.c kStorageOrder.
|
||||
pri const STORAGE_ORDER : roarray[18] base.u8[..= 17] = [
|
||||
1, 2, 3, 4, 0, 5, 17, 6, 16, 7, 8, 9, 10, 11, 12, 13, 14, 15,
|
||||
]
|
||||
|
||||
// From enc/brotli_bit_stream.c kHuffmanBitLengthHuffmanCodeSymbols.
|
||||
pri const HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_SYMBOLS : roarray[6] base.u64[..= 15] = [
|
||||
0, 7, 3, 2, 1, 15,
|
||||
]
|
||||
|
||||
// From enc/brotli_bit_stream.c kHuffmanBitLengthHuffmanCodeBitLengths.
|
||||
pri const HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_BIT_LENGTHS : roarray[6] base.u32[..= 4] = [
|
||||
2, 4, 3, 2, 2, 4,
|
||||
]
|
||||
@@ -0,0 +1,32 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// Quirks are discussed in (/doc/note/quirks.md).
|
||||
//
|
||||
// The base38 encoding of "brot" is 0x0A_ADE4. Left shifting by 10 gives
|
||||
// 0x2AB7_9000.
|
||||
pri const QUIRKS_BASE : base.u32 = 0x2AB7_9000
|
||||
|
||||
// --------
|
||||
|
||||
// When this quirk is set, a positive value enables large-window brotli
|
||||
// (https://datatracker.ietf.org/doc/draft-vandevenne-shared-brotli-format/),
|
||||
// with window sizes up to (1 << 30) bytes. Zero means standard (RFC 7932)
|
||||
// brotli. It mirrors the C code's BROTLI_DECODER_PARAM_LARGE_WINDOW.
|
||||
//
|
||||
// It must be set before decoding starts. Setting it also increases the
|
||||
// workbuf_len.
|
||||
//
|
||||
// Without this quirk, a stream that starts with the large-window WBITS escape
|
||||
// is rejected with "#unsupported large window". With this quirk, a stream that
|
||||
// does not start with that escape is decoded as standard brotli.
|
||||
pub const QUIRK_LARGE_WINDOW : base.u32 = 0x2AB7_9000 | 0x00
|
||||
@@ -0,0 +1,259 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// This file holds the encoder's bit writer and other small helpers shared by
|
||||
// the quality 0 and quality 1 code paths. The bit position (the C code's
|
||||
// *storage_ix) is the encoder.bw_ix field. The storage (the C code's uint8_t*
|
||||
// storage) is a slice argument.
|
||||
|
||||
// Mirrors BrotliWriteBits in write_bits.h (its BROTLI_LITTLE_ENDIAN branch).
|
||||
//
|
||||
// Like the C code, this ORs the bits into the byte at (bw_ix >> 3) and then
|
||||
// stores 8 bytes, little-endian, zeroing any bits above the newly written
|
||||
// ones. The byte-exact output depends on that zeroing, as later code reads
|
||||
// back (and ORs into) those bytes. The storage slice always has enough slack
|
||||
// that the 8-byte store fits. If it ever did not, the bits are written byte
|
||||
// by byte, as far as they fit.
|
||||
pri func encoder.write_bits!(n_bits: base.u32, bits: base.u64, s: slice base.u8) {
|
||||
var p : base.u64
|
||||
var v : base.u64
|
||||
var q : slice base.u8
|
||||
|
||||
p = this.bw_ix >> 3
|
||||
if p < args.s.length() {
|
||||
q = args.s[p ..]
|
||||
v = (args.s[p] as base.u64) | (args.bits ~mod<< (this.bw_ix & 7))
|
||||
if q.length() >= 8 {
|
||||
q.poke_u64le!(a: v)
|
||||
} else {
|
||||
while q.length() > 0 {
|
||||
q[0] = (v & 0xFF) as base.u8
|
||||
v >>= 8
|
||||
q = q[1 ..]
|
||||
}
|
||||
}
|
||||
}
|
||||
this.bw_ix ~mod+= args.n_bits as base.u64
|
||||
}
|
||||
|
||||
// Mirrors JumpToByteBoundary in brotli_bit_stream.c, without the zeroing
|
||||
// (which the C code's callers in this file rely on the 8-byte stores for).
|
||||
// The "(*storage_ix + 7u) & ~7u" expressions in compress_fragment.c and
|
||||
// compress_fragment_two_pass.c.
|
||||
pri func encoder.align_to_byte!() {
|
||||
this.bw_ix = (this.bw_ix ~mod+ 7) & 0xFFFF_FFFF_FFFF_FFF8
|
||||
}
|
||||
|
||||
// Mirrors RewindBitPosition in compress_fragment.c and
|
||||
// compress_fragment_two_pass.c.
|
||||
pri func encoder.rewind_bit_position!(new_ix: base.u64, s: slice base.u8) {
|
||||
var p : base.u64
|
||||
var mask : base.u8
|
||||
|
||||
p = args.new_ix >> 3
|
||||
mask = (((1 as base.u32) << (args.new_ix & 7)) - 1) as base.u8
|
||||
if p < args.s.length() {
|
||||
args.s[p] &= mask
|
||||
}
|
||||
this.bw_ix = args.new_ix
|
||||
}
|
||||
|
||||
// Mirrors UpdateBits in compress_fragment.c.
|
||||
pri func encoder.update_bits!(n_bits: base.u32, bits: base.u32, pos: base.u64, s: slice base.u8) {
|
||||
var n_bits : base.u32
|
||||
var bits : base.u32
|
||||
var pos : base.u64
|
||||
var byte_pos : base.u64
|
||||
var n_unchanged_bits : base.u32[..= 7]
|
||||
var n_changed_bits : base.u32[..= 8]
|
||||
var total_bits : base.u32[..= 15]
|
||||
var mask : base.u32
|
||||
var unchanged_bits : base.u32
|
||||
var changed_bits : base.u32
|
||||
|
||||
n_bits = args.n_bits
|
||||
bits = args.bits
|
||||
pos = args.pos
|
||||
while n_bits > 0 {
|
||||
byte_pos = pos >> 3
|
||||
n_unchanged_bits = (pos & 7) as base.u32
|
||||
n_changed_bits = n_bits.min(no_more_than: 8 - n_unchanged_bits)
|
||||
total_bits = n_unchanged_bits + n_changed_bits
|
||||
mask = (0xFFFF_FFFF ^ (((1 as base.u32) << total_bits) - 1)) |
|
||||
(((1 as base.u32) << n_unchanged_bits) - 1)
|
||||
if byte_pos >= args.s.length() {
|
||||
return nothing
|
||||
}
|
||||
unchanged_bits = (args.s[byte_pos] as base.u32) & mask
|
||||
changed_bits = bits & (((1 as base.u32) << n_changed_bits) - 1)
|
||||
args.s[byte_pos] = (((changed_bits << n_unchanged_bits) | unchanged_bits) & 0xFF) as base.u8
|
||||
n_bits ~mod-= n_changed_bits
|
||||
bits = bits >> n_changed_bits
|
||||
pos ~mod+= n_changed_bits as base.u64
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliStoreMetaBlockHeader in compress_fragment.c (and the identical
|
||||
// function in compress_fragment_two_pass.c).
|
||||
pri func encoder.store_meta_block_header!(len: base.u64, is_uncompressed: base.bool, s: slice base.u8) {
|
||||
var nibbles_minus_4 : base.u32[..= 2]
|
||||
|
||||
nibbles_minus_4 = 2
|
||||
// ISLAST.
|
||||
this.write_bits!(n_bits: 1, bits: 0, s: args.s)
|
||||
if args.len <= 0x1_0000 {
|
||||
nibbles_minus_4 = 0
|
||||
} else if args.len <= 0x10_0000 {
|
||||
nibbles_minus_4 = 1
|
||||
}
|
||||
this.write_bits!(n_bits: 2, bits: nibbles_minus_4 as base.u64, s: args.s)
|
||||
this.write_bits!(n_bits: (nibbles_minus_4 + 4) * 4, bits: args.len ~mod- 1, s: args.s)
|
||||
// ISUNCOMPRESSED.
|
||||
if args.is_uncompressed {
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s)
|
||||
} else {
|
||||
this.write_bits!(n_bits: 1, bits: 0, s: args.s)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors Log2FloorNonZero in fast_log.h, which (with BROTLI_BSR32) truncates
|
||||
// its argument to 32 bits. It returns 0 for an argument of 0.
|
||||
pri func encoder.log2_floor_non_zero(n: base.u64) base.u32[..= 31] {
|
||||
var x : base.u32
|
||||
var r : base.u32
|
||||
|
||||
x = (args.n & 0xFFFF_FFFF) as base.u32
|
||||
if x >= 0x1_0000 {
|
||||
x >>= 16
|
||||
r = 16
|
||||
}
|
||||
if x >= 0x100 {
|
||||
x >>= 8
|
||||
r ~mod+= 8
|
||||
}
|
||||
if x >= 0x10 {
|
||||
x >>= 4
|
||||
r ~mod+= 4
|
||||
}
|
||||
if x >= 4 {
|
||||
x >>= 2
|
||||
r ~mod+= 2
|
||||
}
|
||||
if x >= 2 {
|
||||
r ~mod+= 1
|
||||
}
|
||||
return r & 31
|
||||
}
|
||||
|
||||
// Mirrors FastLog2 in fast_log.h. The result is the bit pattern of an IEEE
|
||||
// 754 binary64 value.
|
||||
pri func encoder.fast_log2(v: base.u64) base.u64 {
|
||||
if args.v < 256 {
|
||||
return LOG2_TABLE[args.v]
|
||||
}
|
||||
return this.sf.log2_u64(v: args.v)
|
||||
}
|
||||
|
||||
// Mirrors BrotliBitsEntropy in bit_cost.c, over this.histogram (with size
|
||||
// 256). The result is the bit pattern of an IEEE 754 binary64 value.
|
||||
pri func encoder.bits_entropy_of_histogram() base.u64 {
|
||||
var sum : base.u64
|
||||
var retval : base.u64
|
||||
var i : base.u32
|
||||
var p : base.u64
|
||||
|
||||
i = 0
|
||||
while i < 256 {
|
||||
p = this.histogram[i] as base.u64
|
||||
sum ~mod+= p
|
||||
retval = this.sf.f64_sub(a: retval, b: this.sf.f64_mul(
|
||||
a: this.sf.f64_from_u64(a: p), b: this.fast_log2(v: p)))
|
||||
i += 1
|
||||
}
|
||||
if sum <> 0 {
|
||||
retval = this.sf.f64_add(a: retval, b: this.sf.f64_mul(
|
||||
a: this.sf.f64_from_u64(a: sum), b: this.fast_log2(v: sum)))
|
||||
}
|
||||
if this.sf.f64_lt(a: retval, b: this.sf.f64_from_u64(a: sum)) <> 0 {
|
||||
retval = this.sf.f64_from_u64(a: sum)
|
||||
}
|
||||
return retval
|
||||
}
|
||||
|
||||
// load64le returns the 8 bytes (little-endian) of s starting at i, like the C
|
||||
// code's BROTLI_UNALIGNED_LOAD64LE. The C code never reads out of bounds, so
|
||||
// the zero result for an out of bounds read should never be used.
|
||||
pri func encoder.load64le(s: roslice base.u8, i: base.u64) base.u64 {
|
||||
var q : roslice base.u8
|
||||
|
||||
if args.i < args.s.length() {
|
||||
q = args.s[args.i ..]
|
||||
if q.length() >= 8 {
|
||||
return q.peek_u64le()
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// load32le is like load64le but for 4 bytes.
|
||||
pri func encoder.load32le(s: roslice base.u8, i: base.u64) base.u32 {
|
||||
var q : roslice base.u8
|
||||
|
||||
if args.i < args.s.length() {
|
||||
q = args.s[args.i ..]
|
||||
if q.length() >= 4 {
|
||||
return q.peek_u32le()
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// Mirrors FindMatchLengthWithLimit in find_match_length.h: the length of the
|
||||
// common prefix of s[i1 ..] and s[i2 ..], up to limit bytes.
|
||||
pri func encoder.find_match_length_with_limit(s: roslice base.u8, i1: base.u64, i2: base.u64, limit: base.u64) base.u64 {
|
||||
var matched : base.u64
|
||||
var n : base.u64
|
||||
var a : roslice base.u8
|
||||
var b : roslice base.u8
|
||||
var x : base.u64
|
||||
|
||||
if (args.i1 > args.s.length()) or (args.i2 > args.s.length()) {
|
||||
return 0
|
||||
}
|
||||
a = args.s[args.i1 ..]
|
||||
b = args.s[args.i2 ..]
|
||||
n = args.limit
|
||||
while (n >= 8) and (a.length() >= 8) and (b.length() >= 8) {
|
||||
x = a.peek_u64le() ^ b.peek_u64le()
|
||||
if x <> 0 {
|
||||
while (x & 0xFF) == 0 {
|
||||
matched ~mod+= 1
|
||||
x >>= 8
|
||||
}
|
||||
return matched
|
||||
}
|
||||
matched ~mod+= 8
|
||||
n -= 8
|
||||
a = a[8 ..]
|
||||
b = b[8 ..]
|
||||
}
|
||||
while (n > 0) and (a.length() > 0) and (b.length() > 0) {
|
||||
if a[0] <> b[0] {
|
||||
break
|
||||
}
|
||||
matched ~mod+= 1
|
||||
n -= 1
|
||||
a = a[1 ..]
|
||||
b = b[1 ..]
|
||||
}
|
||||
return matched
|
||||
}
|
||||
@@ -0,0 +1,833 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// The encoder aims for parity with the reference C implementation's encoder
|
||||
// (libbrotlienc, tag v1.2.1): for the same parameters and the same sequence of
|
||||
// BrotliEncoderCompressStream operations (with the same input chunking), it
|
||||
// produces byte-for-byte identical output. Comments like "Mirrors Foo in
|
||||
// encode.c" name the C function whose logic a Wuffs function mirrors.
|
||||
//
|
||||
// The C code's floating point (double) arithmetic is done with the
|
||||
// std/softfloat package, which is bit-exact. FastLog2's log2 matches glibc's.
|
||||
//
|
||||
// Only qualities 0 and 1 (the C code's BrotliCompressFragmentFast and
|
||||
// BrotliCompressFragmentTwoPass) are implemented so far. See README.md for
|
||||
// the API, including how the C code's FLUSH and EMIT_METADATA operations map
|
||||
// to the request_flush and request_emit_metadata methods.
|
||||
//
|
||||
// TODO: qualities 2 to 11 (the C code's EncodeData, WriteMetaBlockInternal,
|
||||
// hashers, block splitting, context modeling and Zopfli), large-window
|
||||
// brotli at those qualities and custom (prepared or shared) dictionaries.
|
||||
|
||||
use "std/softfloat"
|
||||
|
||||
pub status "#unsupported encoder quality"
|
||||
|
||||
pri status "#internal error: inconsistent encoder state"
|
||||
|
||||
pub const ENCODER_DST_HISTORY_RETAIN_LENGTH_MAX_INCL_WORST_CASE : base.u64 = 0
|
||||
|
||||
// The C code's BrotliEncoderStreamState values.
|
||||
pri const STREAM_PROCESSING : base.u32 = 0
|
||||
pri const STREAM_FLUSH_REQUESTED : base.u32 = 1
|
||||
pri const STREAM_FINISHED : base.u32 = 2
|
||||
pri const STREAM_METADATA_HEAD : base.u32 = 3
|
||||
pri const STREAM_METADATA_BODY : base.u32 = 4
|
||||
|
||||
// The C code's BrotliEncoderOperation values.
|
||||
pri const OP_PROCESS : base.u32 = 0
|
||||
pri const OP_FLUSH : base.u32 = 1
|
||||
pri const OP_FINISH : base.u32 = 2
|
||||
pri const OP_EMIT_METADATA : base.u32 = 3
|
||||
|
||||
// The C code's BrotliEncoderFlintState values, as two's complement u32s.
|
||||
pri const FLINT_NEEDS_2_BYTES : base.u32 = 2
|
||||
pri const FLINT_DONE : base.u32 = 0xFFFF_FFFF
|
||||
|
||||
// STORAGE_SLACK is how many bytes the storage (the workbuf) needs on top of
|
||||
// the C code's (2 * block_size + 503) bytes for a metablock: the 8-byte
|
||||
// stores of write_bits and up to 3 bytes for InjectBytePaddingBlock.
|
||||
pri const STORAGE_SLACK : base.u64 = 16
|
||||
|
||||
// Mirrors kCompressFragmentTwoPassBlockSize in compress_fragment_two_pass.h.
|
||||
pri const TWO_PASS_BLOCK_SIZE : base.u32 = 0x2_0000
|
||||
|
||||
pub struct encoder? implements base.io_transformer(
|
||||
// The raw parameters, as set by set_quirk (the C code's
|
||||
// BrotliEncoderSetParameter). The quality and lgwin defaults (11 and
|
||||
// 22) apply unless the p_quality_set and p_lgwin_set bools are set.
|
||||
p_mode : base.u32,
|
||||
p_quality : base.u32,
|
||||
p_quality_set : base.bool,
|
||||
p_lgwin : base.u32,
|
||||
p_lgwin_set : base.bool,
|
||||
p_lgblock : base.u32,
|
||||
p_disable_lcm : base.u32,
|
||||
p_size_hint : base.u32,
|
||||
p_large_window : base.u32,
|
||||
p_npostfix : base.u32,
|
||||
p_ndirect : base.u32,
|
||||
p_stream_offset : base.u32,
|
||||
|
||||
// The parameters after EnsureInitialized (SanitizeParams,
|
||||
// ComputeLgBlock and ChooseDistanceParams).
|
||||
is_initialized : base.bool,
|
||||
quality : base.u32[..= 11],
|
||||
lgwin : base.u32[..= 30],
|
||||
lgblock : base.u32[..= 30],
|
||||
large_window : base.bool,
|
||||
size_hint : base.u32,
|
||||
stream_offset : base.u32,
|
||||
distance_postfix_bits : base.u32,
|
||||
num_direct_distance_codes : base.u32,
|
||||
|
||||
// The C code's BrotliEncoderState fields of the same name.
|
||||
dist_cache : array[4] base.u32,
|
||||
saved_dist_cache : array[4] base.u32,
|
||||
flint : base.u32,
|
||||
last_bytes : base.u32[..= 0xFFFF],
|
||||
last_bytes_bits : base.u32[..= 15],
|
||||
stream_state : base.u32,
|
||||
remaining_metadata_bytes : base.u32,
|
||||
|
||||
// The pending output: out_len bytes starting at out_pos, in either
|
||||
// the tiny_buf array (if out_in_tiny_buf) or in the storage (the
|
||||
// workbuf). These are the C code's next_out_ and available_out_.
|
||||
out_in_tiny_buf : base.bool,
|
||||
out_pos : base.u64,
|
||||
out_len : base.u64,
|
||||
|
||||
// The pending requests. See request_flush and request_emit_metadata.
|
||||
flush_requested : base.bool,
|
||||
metadata_requested : base.bool,
|
||||
metadata_length : base.u32,
|
||||
|
||||
// bw_ix is the bit writer's position, the C code's *storage_ix.
|
||||
bw_ix : base.u64,
|
||||
|
||||
// cmd_code_numbits is the C code's BrotliOnePassArena.cmd_code_numbits.
|
||||
cmd_code_numbits : base.u64,
|
||||
|
||||
// huffman_tree_size is BrotliStoreHuffmanTree's huffman_tree_size.
|
||||
huffman_tree_size : base.u32,
|
||||
|
||||
// num_commands and num_literals are the lengths of command_buf and
|
||||
// literal_buf, as filled by CreateCommands.
|
||||
num_commands : base.u32,
|
||||
num_literals : base.u32,
|
||||
|
||||
sf : softfloat.arith,
|
||||
util : base.utility,
|
||||
) + (
|
||||
// tiny_buf is the C code's tiny_buf_ (plus slack for the 8-byte
|
||||
// stores of write_bits).
|
||||
tiny_buf : array[32] base.u8,
|
||||
|
||||
// These are the fields of the C code's BrotliOnePassArena and
|
||||
// BrotliTwoPassArena (an encoder uses only one of the two). The
|
||||
// histogram field is the C code's BrotliOnePassArena.histogram and
|
||||
// BrotliTwoPassArena.lit_histo.
|
||||
lit_depth : array[256] base.u8,
|
||||
lit_bits : array[256] base.u16,
|
||||
cmd_depth : array[128] base.u8,
|
||||
cmd_bits : array[128] base.u16,
|
||||
cmd_histo : array[128] base.u32,
|
||||
cmd_code : array[520] base.u8,
|
||||
histogram : array[256] base.u32,
|
||||
tmp_depth : array[704] base.u8,
|
||||
tmp_bits : array[64] base.u16,
|
||||
|
||||
// The C code's HuffmanTree pool. See encode_entropy.wuffs.
|
||||
tree_count : array[1024] base.u32,
|
||||
tree_left : array[1024] base.u16,
|
||||
tree_right : array[1024] base.u16,
|
||||
|
||||
// BrotliStoreHuffmanTree's local variables: huffman_tree,
|
||||
// huffman_tree_extra_bits, huffman_tree_histogram,
|
||||
// code_length_bitdepth and code_length_bitdepth_symbols.
|
||||
huffman_tree : array[1024] base.u8,
|
||||
huffman_tree_extra_bits : array[1024] base.u8,
|
||||
cl_histo : array[32] base.u32,
|
||||
cl_depth : array[32] base.u8,
|
||||
cl_bits : array[32] base.u16,
|
||||
|
||||
// hash_table is the C code's small_table_ or large_table_ (the
|
||||
// GetHashTable result), with up to (1 << 17) elements.
|
||||
hash_table : array[131072] base.u32,
|
||||
|
||||
// command_buf and literal_buf are the C code's command_buf_ and
|
||||
// literal_buf_ (quality 1 only).
|
||||
command_buf : array[131072] base.u32,
|
||||
literal_buf : array[131072] base.u8,
|
||||
)
|
||||
|
||||
pub func encoder.get_quirk(key: base.u32) base.u64 {
|
||||
if args.key == QUIRK_MODE {
|
||||
return this.p_mode as base.u64
|
||||
} else if args.key == QUIRK_ENCODER_QUALITY {
|
||||
if not this.p_quality_set {
|
||||
return 11
|
||||
}
|
||||
return this.p_quality as base.u64
|
||||
} else if args.key == QUIRK_LGWIN {
|
||||
if not this.p_lgwin_set {
|
||||
return 22
|
||||
}
|
||||
return this.p_lgwin as base.u64
|
||||
} else if args.key == QUIRK_LGBLOCK {
|
||||
return this.p_lgblock as base.u64
|
||||
} else if args.key == QUIRK_DISABLE_LITERAL_CONTEXT_MODELING {
|
||||
return this.p_disable_lcm as base.u64
|
||||
} else if args.key == QUIRK_SIZE_HINT {
|
||||
return this.p_size_hint as base.u64
|
||||
} else if args.key == QUIRK_LARGE_WINDOW {
|
||||
return this.p_large_window as base.u64
|
||||
} else if args.key == QUIRK_NPOSTFIX {
|
||||
return this.p_npostfix as base.u64
|
||||
} else if args.key == QUIRK_NDIRECT {
|
||||
return this.p_ndirect as base.u64
|
||||
} else if args.key == QUIRK_STREAM_OFFSET {
|
||||
return this.p_stream_offset as base.u64
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// Mirrors BrotliEncoderSetParameter in encode.c.
|
||||
pub func encoder.set_quirk!(key: base.u32, value: base.u64) base.status {
|
||||
var v : base.u32
|
||||
|
||||
// Changing parameters on the fly is not implemented (in the C code).
|
||||
if this.is_initialized {
|
||||
return base."#bad call sequence"
|
||||
} else if args.value > 0xFFFF_FFFF {
|
||||
return base."#bad argument"
|
||||
}
|
||||
v = (args.value & 0xFFFF_FFFF) as base.u32
|
||||
|
||||
if args.key == QUIRK_MODE {
|
||||
this.p_mode = v
|
||||
} else if args.key == QUIRK_ENCODER_QUALITY {
|
||||
this.p_quality = v
|
||||
this.p_quality_set = true
|
||||
} else if args.key == QUIRK_LGWIN {
|
||||
this.p_lgwin = v
|
||||
this.p_lgwin_set = true
|
||||
} else if args.key == QUIRK_LGBLOCK {
|
||||
this.p_lgblock = v
|
||||
} else if args.key == QUIRK_DISABLE_LITERAL_CONTEXT_MODELING {
|
||||
if v > 1 {
|
||||
return base."#bad argument"
|
||||
}
|
||||
this.p_disable_lcm = v
|
||||
} else if args.key == QUIRK_SIZE_HINT {
|
||||
this.p_size_hint = v
|
||||
} else if args.key == QUIRK_LARGE_WINDOW {
|
||||
if v <> 0 {
|
||||
this.p_large_window = 1
|
||||
} else {
|
||||
this.p_large_window = 0
|
||||
}
|
||||
} else if args.key == QUIRK_NPOSTFIX {
|
||||
this.p_npostfix = v
|
||||
} else if args.key == QUIRK_NDIRECT {
|
||||
this.p_ndirect = v
|
||||
} else if args.key == QUIRK_STREAM_OFFSET {
|
||||
if v > 0x4000_0000 {
|
||||
return base."#bad argument"
|
||||
}
|
||||
this.p_stream_offset = v
|
||||
} else {
|
||||
return base."#unsupported option"
|
||||
}
|
||||
return ok
|
||||
}
|
||||
|
||||
pub func encoder.dst_history_retain_length() base.optional_u63 {
|
||||
return this.util.make_optional_u63(has_value: true, value: 0)
|
||||
}
|
||||
|
||||
// workbuf_len returns the workbuf length needed for the current parameters
|
||||
// (after the C code's SanitizeParams). At qualities 0 and 1 the workbuf is
|
||||
// the storage for one metablock's output: ((2 << lgwin) + 503) bytes (the C
|
||||
// code's GetBrotliStorage size for a (1 << lgwin) byte block) plus 16 bytes
|
||||
// of slack. A shorter workbuf still works if every transform_io input block
|
||||
// is short enough: a block of n bytes needs ((2 * n) + 519) bytes.
|
||||
pub func encoder.workbuf_len() base.range_ii_u64 {
|
||||
var q : base.u32[..= 11]
|
||||
var n : base.u64
|
||||
|
||||
q = this.sanitized_quality()
|
||||
if q <= 1 {
|
||||
n = ((2 as base.u64) << this.sanitized_lgwin(quality: q)) + 503 + STORAGE_SLACK
|
||||
}
|
||||
return this.util.make_range_ii_u64(min_incl: n, max_incl: n)
|
||||
}
|
||||
|
||||
// max_compressed_size mirrors BrotliEncoderMaxCompressedSize in encode.c:
|
||||
// an upper bound on the compressed size of input_size bytes (with an
|
||||
// uncompressed fallback, which this encoder does not do itself), or 0 on
|
||||
// overflow.
|
||||
pub func encoder.max_compressed_size(input_size: base.u64) base.u64 {
|
||||
var num_large_blocks : base.u64
|
||||
var overhead : base.u64
|
||||
var result : base.u64
|
||||
|
||||
num_large_blocks = args.input_size >> 14
|
||||
overhead = 2 + (4 * num_large_blocks) + 3 + 1
|
||||
result = args.input_size ~mod+ overhead
|
||||
if args.input_size == 0 {
|
||||
return 2
|
||||
} else if result < args.input_size {
|
||||
return 0
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// request_flush makes the next transform_io call act like the C code's
|
||||
// BrotliEncoderCompressStream with BROTLI_OPERATION_FLUSH (instead of
|
||||
// PROCESS or FINISH), until the flush is complete. It is complete once a
|
||||
// transform_io call returns "$short read" (or ok).
|
||||
pub func encoder.request_flush!() base.status {
|
||||
if this.metadata_requested {
|
||||
return base."#bad call sequence"
|
||||
}
|
||||
this.flush_requested = true
|
||||
return ok
|
||||
}
|
||||
|
||||
// request_emit_metadata makes the next transform_io call act like the C
|
||||
// code's BrotliEncoderCompressStream with BROTLI_OPERATION_EMIT_METADATA,
|
||||
// with the next length bytes of src as the metadata. Once those length bytes
|
||||
// have been read (transform_io returns "$short read" if src has fewer of
|
||||
// them), and the metadata block written, transform_io continues as usual.
|
||||
// Like the C code, length can be at most (1 << 24).
|
||||
pub func encoder.request_emit_metadata!(length: base.u64) base.status {
|
||||
if args.length > 0x100_0000 {
|
||||
return base."#bad argument"
|
||||
} else if this.flush_requested or this.metadata_requested {
|
||||
return base."#bad call sequence"
|
||||
}
|
||||
this.metadata_requested = true
|
||||
this.metadata_length = (args.length & 0xFFFF_FFFF) as base.u32
|
||||
return ok
|
||||
}
|
||||
|
||||
// sanitized_quality mirrors SanitizeParams in quality.h, for the quality.
|
||||
pri func encoder.sanitized_quality() base.u32[..= 11] {
|
||||
if not this.p_quality_set {
|
||||
return 11
|
||||
} else if this.p_quality >= 0x8000_0000 {
|
||||
// A negative (int) value.
|
||||
return 0
|
||||
}
|
||||
return this.p_quality.min(no_more_than: 11)
|
||||
}
|
||||
|
||||
// sanitized_lgwin mirrors SanitizeParams in quality.h, for the lgwin.
|
||||
pri func encoder.sanitized_lgwin(quality: base.u32[..= 11]) base.u32[..= 30] {
|
||||
var lgwin : base.u32
|
||||
|
||||
lgwin = 22
|
||||
if this.p_lgwin_set {
|
||||
lgwin = this.p_lgwin
|
||||
}
|
||||
if (lgwin >= 0x8000_0000) or (lgwin < 10) {
|
||||
return 10
|
||||
} else if (this.p_large_window <> 0) and (args.quality > 2) {
|
||||
return lgwin.min(no_more_than: 30)
|
||||
}
|
||||
return lgwin.min(no_more_than: 24)
|
||||
}
|
||||
|
||||
// Mirrors EnsureInitialized (and BrotliEncoderInitState's distance cache
|
||||
// initialization) in encode.c.
|
||||
pri func encoder.ensure_initialized!() {
|
||||
var lgblock : base.u32
|
||||
var lgwin : base.u32[..= 30]
|
||||
var limit : base.u32
|
||||
var npostfix : base.u32
|
||||
var ndirect : base.u32
|
||||
var ndirect_msb : base.u32
|
||||
var i : base.u32
|
||||
|
||||
if this.is_initialized {
|
||||
return nothing
|
||||
}
|
||||
|
||||
// BrotliEncoderInitState.
|
||||
this.dist_cache[0] = 4
|
||||
this.dist_cache[1] = 11
|
||||
this.dist_cache[2] = 15
|
||||
this.dist_cache[3] = 16
|
||||
this.saved_dist_cache[0] = 4
|
||||
this.saved_dist_cache[1] = 11
|
||||
this.saved_dist_cache[2] = 15
|
||||
this.saved_dist_cache[3] = 16
|
||||
this.stream_state = STREAM_PROCESSING
|
||||
|
||||
this.last_bytes_bits = 0
|
||||
this.last_bytes = 0
|
||||
this.flint = FLINT_DONE
|
||||
this.remaining_metadata_bytes = 0xFFFF_FFFF
|
||||
|
||||
// SanitizeParams.
|
||||
this.quality = this.sanitized_quality()
|
||||
this.large_window = (this.p_large_window <> 0) and (this.quality > 2)
|
||||
this.lgwin = this.sanitized_lgwin(quality: this.quality)
|
||||
this.size_hint = this.p_size_hint
|
||||
this.stream_offset = this.p_stream_offset
|
||||
|
||||
// ComputeLgBlock.
|
||||
if this.quality <= 1 {
|
||||
this.lgblock = this.lgwin
|
||||
} else if this.quality < 4 {
|
||||
this.lgblock = 14
|
||||
} else if this.p_lgblock == 0 {
|
||||
this.lgblock = 16
|
||||
if (this.quality >= 9) and (this.lgwin > 16) {
|
||||
this.lgblock = this.lgwin.min(no_more_than: 18)
|
||||
}
|
||||
} else {
|
||||
lgblock = this.p_lgblock
|
||||
if lgblock >= 0x8000_0000 {
|
||||
// A negative (int) value.
|
||||
lgblock = 16
|
||||
}
|
||||
this.lgblock = lgblock.max(no_less_than: 16).min(no_more_than: 24)
|
||||
}
|
||||
|
||||
// ChooseDistanceParams. BrotliInitDistanceParams' other fields are not
|
||||
// needed at qualities 0 and 1.
|
||||
if this.quality >= 4 {
|
||||
if this.p_mode == 2 {
|
||||
npostfix = 1
|
||||
ndirect = 12
|
||||
} else {
|
||||
npostfix = this.p_npostfix
|
||||
ndirect = this.p_ndirect
|
||||
}
|
||||
if npostfix > 3 {
|
||||
npostfix = 0
|
||||
ndirect = 0
|
||||
} else {
|
||||
ndirect_msb = (ndirect >> npostfix) & 0x0F
|
||||
if (ndirect > 120) or ((ndirect_msb << npostfix) <> ndirect) {
|
||||
npostfix = 0
|
||||
ndirect = 0
|
||||
}
|
||||
}
|
||||
}
|
||||
this.distance_postfix_bits = npostfix
|
||||
this.num_direct_distance_codes = ndirect
|
||||
|
||||
if this.stream_offset <> 0 {
|
||||
this.flint = FLINT_NEEDS_2_BYTES
|
||||
// Poison the distance cache. -16 +- 3 is still less than zero
|
||||
// (invalid).
|
||||
i = 0
|
||||
while i < 4 {
|
||||
this.dist_cache[i] = 0xFFFF_FFF0
|
||||
this.saved_dist_cache[i] = 0xFFFF_FFF0
|
||||
i += 1
|
||||
}
|
||||
}
|
||||
|
||||
// Initialize last byte with stream header.
|
||||
lgwin = this.lgwin
|
||||
if this.quality <= 1 {
|
||||
lgwin = lgwin.max(no_less_than: 18)
|
||||
}
|
||||
if this.stream_offset == 0 {
|
||||
this.encode_window_bits!(lgwin: lgwin)
|
||||
} else {
|
||||
// Bigger values have the same effect, but could cause overflows.
|
||||
limit = ((1 as base.u32) << lgwin) ~mod- 16
|
||||
this.stream_offset = this.stream_offset.min(no_more_than: limit)
|
||||
}
|
||||
|
||||
if this.quality == 0 {
|
||||
this.init_command_prefix_codes!()
|
||||
}
|
||||
|
||||
this.is_initialized = true
|
||||
}
|
||||
|
||||
// Mirrors EncodeWindowBits in encode.c.
|
||||
pri func encoder.encode_window_bits!(lgwin: base.u32[..= 30]) {
|
||||
if this.large_window {
|
||||
this.last_bytes = ((args.lgwin & 0x3F) << 8) | 0x11
|
||||
this.last_bytes_bits = 14
|
||||
} else if args.lgwin == 16 {
|
||||
this.last_bytes = 0
|
||||
this.last_bytes_bits = 1
|
||||
} else if args.lgwin == 17 {
|
||||
this.last_bytes = 1
|
||||
this.last_bytes_bits = 7
|
||||
} else if args.lgwin > 17 {
|
||||
this.last_bytes = ((args.lgwin - 17) << 1) | 0x01
|
||||
this.last_bytes_bits = 4
|
||||
} else if args.lgwin >= 8 {
|
||||
this.last_bytes = ((args.lgwin - 8) << 4) | 0x01
|
||||
this.last_bytes_bits = 7
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors InitCommandPrefixCodes in encode.c.
|
||||
pri func encoder.init_command_prefix_codes!() {
|
||||
var i : base.u32
|
||||
|
||||
i = 0
|
||||
while i < 128 {
|
||||
this.cmd_depth[i] = DEFAULT_COMMAND_DEPTHS[i]
|
||||
this.cmd_bits[i] = DEFAULT_COMMAND_BITS[i]
|
||||
i += 1
|
||||
}
|
||||
i = 0
|
||||
while i < 57 {
|
||||
this.cmd_code[i] = DEFAULT_COMMAND_CODE[i]
|
||||
i += 1
|
||||
}
|
||||
this.cmd_code_numbits = DEFAULT_COMMAND_CODE_NUM_BITS as base.u64
|
||||
}
|
||||
|
||||
// Mirrors HashTableSize and GetHashTable in encode.c. It zeroes the first
|
||||
// htsize elements of this.hash_table and returns log2(htsize), the C code's
|
||||
// table_bits (Log2FloorNonZero(table_size) in BrotliCompressFragmentFast and
|
||||
// BrotliCompressFragmentTwoPass).
|
||||
pri func encoder.get_hash_table!(input_size: base.u64) base.u32 {
|
||||
var max_table_size : base.u64
|
||||
var htsize : base.u64
|
||||
var table_bits : base.u32
|
||||
|
||||
// MaxHashTableSize in quality.h.
|
||||
max_table_size = 0x2_0000
|
||||
if this.quality == 0 {
|
||||
max_table_size = 0x8000
|
||||
}
|
||||
htsize = 256
|
||||
table_bits = 8
|
||||
while (htsize < max_table_size) and (htsize < args.input_size) {
|
||||
htsize ~mod<<= 1
|
||||
table_bits ~mod+= 1
|
||||
}
|
||||
if this.quality == 0 {
|
||||
// Only odd shifts are supported by fast-one-pass.
|
||||
if (htsize & 0xA_AAAA) == 0 {
|
||||
htsize ~mod<<= 1
|
||||
table_bits ~mod+= 1
|
||||
}
|
||||
}
|
||||
if htsize <= 0x2_0000 {
|
||||
this.hash_table[.. htsize].bulk_memset!(byte_value: 0)
|
||||
}
|
||||
return table_bits
|
||||
}
|
||||
|
||||
// Mirrors InjectBytePaddingBlock in encode.c. Its destination is right after
|
||||
// the pending output (in the storage or in tiny_buf).
|
||||
pri func encoder.inject_byte_padding_block!(workbuf: slice base.u8) {
|
||||
var seal : base.u32
|
||||
var seal_bits : base.u32
|
||||
var p : base.u64
|
||||
var n : base.u32
|
||||
|
||||
seal = this.last_bytes
|
||||
seal_bits = this.last_bytes_bits
|
||||
this.last_bytes = 0
|
||||
this.last_bytes_bits = 0
|
||||
// is_last = 0, data_nibbles = 11, reserved = 0, meta_nibbles = 00.
|
||||
seal |= (6 as base.u32) << seal_bits
|
||||
seal_bits += 6
|
||||
n = (seal_bits + 7) >> 3
|
||||
|
||||
if this.out_len == 0 {
|
||||
this.out_in_tiny_buf = true
|
||||
this.out_pos = 0
|
||||
}
|
||||
p = this.out_pos ~mod+ this.out_len
|
||||
if this.out_in_tiny_buf {
|
||||
if p <= 24 {
|
||||
this.tiny_buf[p] = (seal & 0xFF) as base.u8
|
||||
this.tiny_buf[p + 1] = ((seal >> 8) & 0xFF) as base.u8
|
||||
this.tiny_buf[p + 2] = ((seal >> 16) & 0xFF) as base.u8
|
||||
}
|
||||
} else if p < args.workbuf.length() {
|
||||
args.workbuf[p] = (seal & 0xFF) as base.u8
|
||||
if seal_bits > 8 {
|
||||
p ~mod+= 1
|
||||
if p < args.workbuf.length() {
|
||||
args.workbuf[p] = ((seal >> 8) & 0xFF) as base.u8
|
||||
}
|
||||
}
|
||||
if seal_bits > 16 {
|
||||
p ~mod+= 1
|
||||
if p < args.workbuf.length() {
|
||||
args.workbuf[p] = ((seal >> 16) & 0xFF) as base.u8
|
||||
}
|
||||
}
|
||||
}
|
||||
this.out_len ~mod+= n as base.u64
|
||||
}
|
||||
|
||||
// push_output copies as much pending output to args.dst as fits, like the
|
||||
// second half of InjectFlushOrPushOutput in encode.c.
|
||||
pri func encoder.push_output!(dst: base.io_writer, workbuf: slice base.u8) {
|
||||
var n : base.u64
|
||||
var end : base.u64
|
||||
|
||||
end = this.out_pos ~mod+ this.out_len
|
||||
if this.out_in_tiny_buf {
|
||||
if (this.out_pos <= end) and (end <= 32) {
|
||||
n = args.dst.copy_from_slice!(s: this.tiny_buf[this.out_pos .. end])
|
||||
}
|
||||
} else if (this.out_pos <= end) and (end <= args.workbuf.length()) {
|
||||
n = args.dst.copy_from_slice!(s: args.workbuf[this.out_pos .. end])
|
||||
}
|
||||
if n == 0 {
|
||||
// Should be unreachable, since the caller checks that args.dst is
|
||||
// not full. Discard the output so that the caller makes progress.
|
||||
this.out_len = 0
|
||||
return nothing
|
||||
}
|
||||
n = n.min(no_more_than: this.out_len)
|
||||
this.out_pos ~mod+= n
|
||||
this.out_len ~mod-= n
|
||||
}
|
||||
|
||||
// Mirrors CheckFlushComplete in encode.c.
|
||||
pri func encoder.check_flush_complete!() {
|
||||
if (this.stream_state == STREAM_FLUSH_REQUESTED) and (this.out_len == 0) {
|
||||
this.stream_state = STREAM_PROCESSING
|
||||
this.out_in_tiny_buf = false
|
||||
this.out_pos = 0
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors WriteMetadataHeader in encode.c, writing to tiny_buf.
|
||||
pri func encoder.write_metadata_header!(block_size: base.u32) base.u64 {
|
||||
var nbits : base.u32[..= 32]
|
||||
var nbytes : base.u32[..= 4]
|
||||
|
||||
this.bw_ix = this.last_bytes_bits as base.u64
|
||||
this.tiny_buf[0] = (this.last_bytes & 0xFF) as base.u8
|
||||
this.tiny_buf[1] = (this.last_bytes >> 8) as base.u8
|
||||
this.last_bytes = 0
|
||||
this.last_bytes_bits = 0
|
||||
|
||||
this.write_bits!(n_bits: 1, bits: 0, s: this.tiny_buf[..])
|
||||
this.write_bits!(n_bits: 2, bits: 3, s: this.tiny_buf[..])
|
||||
this.write_bits!(n_bits: 1, bits: 0, s: this.tiny_buf[..])
|
||||
if args.block_size == 0 {
|
||||
this.write_bits!(n_bits: 2, bits: 0, s: this.tiny_buf[..])
|
||||
} else {
|
||||
nbits = 1
|
||||
if args.block_size > 1 {
|
||||
nbits = this.log2_floor_non_zero(n: (args.block_size - 1) as base.u64) + 1
|
||||
}
|
||||
nbytes = (nbits + 7) / 8
|
||||
this.write_bits!(n_bits: 2, bits: nbytes as base.u64, s: this.tiny_buf[..])
|
||||
this.write_bits!(n_bits: 8 * nbytes, bits: (args.block_size - 1) as base.u64, s: this.tiny_buf[..])
|
||||
}
|
||||
return (this.bw_ix ~mod+ 7) >> 3
|
||||
}
|
||||
|
||||
// transform_io mirrors BrotliEncoderCompressStream in encode.c (and, for
|
||||
// qualities 0 and 1, BrotliEncoderCompressStreamFast). Each (initial or
|
||||
// resumed) transform_io call corresponds to one or more C calls. See
|
||||
// README.md for the details.
|
||||
pub func encoder.transform_io?(dst: base.io_writer, src: base.io_reader, workbuf: slice base.u8) {
|
||||
var op : base.u32
|
||||
var avail : base.u64
|
||||
var block_size : base.u64
|
||||
var is_last : base.bool
|
||||
var force_flush : base.bool
|
||||
var storage_len : base.u64
|
||||
var table_bits : base.u32
|
||||
var mark : base.u64
|
||||
var n32 : base.u32
|
||||
var ix : base.u64
|
||||
|
||||
this.ensure_initialized!()
|
||||
if this.quality > 1 {
|
||||
// TODO: implement the C code's EncodeData (and its callees) for
|
||||
// qualities 2 to 11.
|
||||
return "#unsupported encoder quality"
|
||||
}
|
||||
|
||||
while true {
|
||||
// ---- The start of a C BrotliEncoderCompressStream call.
|
||||
|
||||
if this.metadata_requested {
|
||||
// op == BROTLI_OPERATION_EMIT_METADATA. UpdateSizeHint(s, 0) is a
|
||||
// no-op at qualities 0 and 1 (and so is ProcessMetadata's
|
||||
// EncodeData call, as the fast path never advances input_pos_).
|
||||
//
|
||||
// Mirrors ProcessMetadata in encode.c.
|
||||
if this.stream_state == STREAM_PROCESSING {
|
||||
this.remaining_metadata_bytes = this.metadata_length
|
||||
this.stream_state = STREAM_METADATA_HEAD
|
||||
}
|
||||
if (this.stream_state <> STREAM_METADATA_HEAD) and
|
||||
(this.stream_state <> STREAM_METADATA_BODY) {
|
||||
return base."#bad call sequence"
|
||||
}
|
||||
while true {
|
||||
if this.out_len <> 0 {
|
||||
if args.dst.length() <= 0 {
|
||||
yield? base."$short write"
|
||||
continue
|
||||
}
|
||||
this.push_output!(dst: args.dst, workbuf: args.workbuf)
|
||||
continue
|
||||
}
|
||||
if this.stream_state == STREAM_METADATA_HEAD {
|
||||
this.out_len = this.write_metadata_header!(block_size: this.remaining_metadata_bytes)
|
||||
this.out_in_tiny_buf = true
|
||||
this.out_pos = 0
|
||||
this.stream_state = STREAM_METADATA_BODY
|
||||
continue
|
||||
}
|
||||
// Exit workflow only when there is no more input and no more
|
||||
// output.
|
||||
if this.remaining_metadata_bytes == 0 {
|
||||
this.remaining_metadata_bytes = 0xFFFF_FFFF
|
||||
this.stream_state = STREAM_PROCESSING
|
||||
this.metadata_requested = false
|
||||
break
|
||||
}
|
||||
if args.dst.length() <= 0 {
|
||||
yield? base."$short write"
|
||||
continue
|
||||
} else if args.src.length() <= 0 {
|
||||
if args.src.is_closed() {
|
||||
return "#truncated input"
|
||||
}
|
||||
yield? base."$short read"
|
||||
continue
|
||||
}
|
||||
// Directly copy input to output.
|
||||
n32 = args.dst.limited_copy_u32_from_reader!(
|
||||
up_to: this.remaining_metadata_bytes, r: args.src)
|
||||
this.remaining_metadata_bytes ~mod-= n32
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if this.flush_requested {
|
||||
op = OP_FLUSH
|
||||
} else if args.src.is_closed() {
|
||||
op = OP_FINISH
|
||||
} else {
|
||||
op = OP_PROCESS
|
||||
}
|
||||
|
||||
if (this.stream_state == STREAM_METADATA_HEAD) or
|
||||
(this.stream_state == STREAM_METADATA_BODY) {
|
||||
return base."#bad call sequence"
|
||||
} else if (this.stream_state <> STREAM_PROCESSING) and (args.src.length() > 0) {
|
||||
return base."#bad call sequence"
|
||||
}
|
||||
|
||||
// Mirrors BrotliEncoderCompressStreamFast in encode.c.
|
||||
while true {
|
||||
// Mirrors InjectFlushOrPushOutput in encode.c.
|
||||
if (this.stream_state == STREAM_FLUSH_REQUESTED) and (this.last_bytes_bits <> 0) {
|
||||
this.inject_byte_padding_block!(workbuf: args.workbuf)
|
||||
continue
|
||||
}
|
||||
if (this.out_len <> 0) and (args.dst.length() > 0) {
|
||||
this.push_output!(dst: args.dst, workbuf: args.workbuf)
|
||||
continue
|
||||
}
|
||||
|
||||
// Compress block only when internal output buffer is empty,
|
||||
// stream is not finished, there is no pending flush request, and
|
||||
// there is either additional input or pending operation.
|
||||
if (this.out_len == 0) and (this.stream_state == STREAM_PROCESSING) and
|
||||
((args.src.length() > 0) or (op <> OP_PROCESS)) {
|
||||
avail = args.src.length()
|
||||
block_size = avail.min(no_more_than: (1 as base.u64) << this.lgwin)
|
||||
is_last = (avail == block_size) and (op == OP_FINISH)
|
||||
force_flush = (avail == block_size) and (op == OP_FLUSH)
|
||||
if force_flush and (block_size == 0) {
|
||||
this.stream_state = STREAM_FLUSH_REQUESTED
|
||||
continue
|
||||
}
|
||||
|
||||
storage_len = (block_size * 2) + 503 + STORAGE_SLACK
|
||||
if (storage_len > args.workbuf.length()) or (args.workbuf.length() < 2) {
|
||||
return base."#bad workbuf length"
|
||||
}
|
||||
args.workbuf[0] = (this.last_bytes & 0xFF) as base.u8
|
||||
args.workbuf[1] = (this.last_bytes >> 8) as base.u8
|
||||
this.bw_ix = this.last_bytes_bits as base.u64
|
||||
table_bits = this.get_hash_table!(input_size: block_size)
|
||||
|
||||
// The C code compresses straight from next_in. So does this
|
||||
// code, from the args.src bytes that it skips over.
|
||||
mark = args.src.mark()
|
||||
args.src.skip?(n: block_size)
|
||||
if storage_len > args.workbuf.length() {
|
||||
return "#internal error: inconsistent encoder state"
|
||||
}
|
||||
if this.quality == 0 {
|
||||
this.compress_fragment_fast!(
|
||||
input: args.src.since(mark: mark),
|
||||
is_last: is_last,
|
||||
table_bits: table_bits,
|
||||
s: args.workbuf[.. storage_len])
|
||||
} else {
|
||||
this.compress_fragment_two_pass!(
|
||||
input: args.src.since(mark: mark),
|
||||
is_last: is_last,
|
||||
table_bits: table_bits,
|
||||
s: args.workbuf[.. storage_len])
|
||||
}
|
||||
|
||||
this.out_in_tiny_buf = false
|
||||
this.out_pos = 0
|
||||
this.out_len = this.bw_ix >> 3
|
||||
ix = this.bw_ix >> 3
|
||||
if ix < args.workbuf.length() {
|
||||
this.last_bytes = args.workbuf[ix] as base.u32
|
||||
} else {
|
||||
this.last_bytes = 0
|
||||
}
|
||||
this.last_bytes_bits = (this.bw_ix & 7) as base.u32
|
||||
|
||||
if force_flush {
|
||||
this.stream_state = STREAM_FLUSH_REQUESTED
|
||||
}
|
||||
if is_last {
|
||||
this.stream_state = STREAM_FINISHED
|
||||
}
|
||||
continue
|
||||
}
|
||||
break
|
||||
}
|
||||
this.check_flush_complete!()
|
||||
|
||||
// ---- The end of a C BrotliEncoderCompressStream call.
|
||||
|
||||
if this.out_len <> 0 {
|
||||
yield? base."$short write"
|
||||
} else if op == OP_FLUSH {
|
||||
this.flush_requested = false
|
||||
} else if this.stream_state == STREAM_FINISHED {
|
||||
return ok
|
||||
} else if op == OP_PROCESS {
|
||||
yield? base."$short read"
|
||||
} else {
|
||||
return "#internal error: inconsistent encoder state"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,804 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// This file holds the Huffman (prefix code) construction and storage
|
||||
// functions that the quality 0 and quality 1 encoders need, from the C code's
|
||||
// entropy_encode.c and brotli_bit_stream.c.
|
||||
//
|
||||
// The C code's HuffmanTree array (a pool of nodes) is the encoder.tree_count,
|
||||
// encoder.tree_left and encoder.tree_right arrays. A node's int16_t
|
||||
// index_left_ and index_right_or_value_ fields are stored as base.u16 values,
|
||||
// with 0xFFFF meaning -1. The C code's pools hold at most (2 * 256) + 1
|
||||
// nodes. The arrays have 1024 elements so that indexes can be masked.
|
||||
|
||||
// TREE_NONE is the C code's -1 for a HuffmanTree index.
|
||||
pri const TREE_NONE : base.u16 = 0xFFFF
|
||||
|
||||
// Mirrors BrotliSetDepth in entropy_encode.c.
|
||||
pri func encoder.set_depth!(p0: base.u32, depth: slice base.u8, max_depth: base.u32[..= 15]) base.bool {
|
||||
var stack : array[16] base.u32
|
||||
var level : base.u32[..= 15]
|
||||
var p : base.u32
|
||||
var v : base.u64
|
||||
|
||||
stack[0] = 0xFFFF_FFFF
|
||||
p = args.p0
|
||||
while true {
|
||||
if this.tree_left[p & 1023] <> TREE_NONE {
|
||||
if (level >= args.max_depth) or (level >= 15) {
|
||||
return false
|
||||
}
|
||||
level += 1
|
||||
stack[level] = this.tree_right[p & 1023] as base.u32
|
||||
p = this.tree_left[p & 1023] as base.u32
|
||||
continue
|
||||
}
|
||||
v = this.tree_right[p & 1023] as base.u64
|
||||
if v < args.depth.length() {
|
||||
args.depth[v] = level as base.u8
|
||||
}
|
||||
while stack[level] == 0xFFFF_FFFF {
|
||||
if level == 0 {
|
||||
return true
|
||||
}
|
||||
level -= 1
|
||||
}
|
||||
p = stack[level]
|
||||
stack[level] = 0xFFFF_FFFF
|
||||
}
|
||||
}
|
||||
|
||||
// huffman_tree_less is the C code's SortHuffmanTree comparators. With
|
||||
// tie_break, it mirrors the one in entropy_encode.c (ties are broken by
|
||||
// index_right_or_value_, in decreasing order). Without, it mirrors the one in
|
||||
// brotli_bit_stream.c (only total_count_ is compared).
|
||||
pri func encoder.huffman_tree_less(c0: base.u32, r0: base.u16, c1: base.u32, r1: base.u16, tie_break: base.bool) base.bool {
|
||||
if args.c0 <> args.c1 {
|
||||
return args.c0 < args.c1
|
||||
}
|
||||
return args.tie_break and (args.r0 > args.r1)
|
||||
}
|
||||
|
||||
// Mirrors SortHuffmanTreeItems in entropy_encode.h, sorting the first n
|
||||
// nodes of the pool.
|
||||
pri func encoder.sort_huffman_tree_items!(n: base.u32, tie_break: base.bool) {
|
||||
var i : base.u32
|
||||
var j : base.u32
|
||||
var k : base.u32
|
||||
var g : base.u32
|
||||
var gap : base.u32
|
||||
var tc : base.u32
|
||||
var tl : base.u16
|
||||
var tr : base.u16
|
||||
|
||||
if args.n < 13 {
|
||||
// Insertion sort.
|
||||
i = 1
|
||||
while i < args.n {
|
||||
tc = this.tree_count[i & 1023]
|
||||
tl = this.tree_left[i & 1023]
|
||||
tr = this.tree_right[i & 1023]
|
||||
k = i
|
||||
j = i ~mod- 1
|
||||
while this.huffman_tree_less(
|
||||
c0: tc,
|
||||
r0: tr,
|
||||
c1: this.tree_count[j & 1023],
|
||||
r1: this.tree_right[j & 1023],
|
||||
tie_break: args.tie_break) {
|
||||
this.tree_count[k & 1023] = this.tree_count[j & 1023]
|
||||
this.tree_left[k & 1023] = this.tree_left[j & 1023]
|
||||
this.tree_right[k & 1023] = this.tree_right[j & 1023]
|
||||
k = j
|
||||
if j == 0 {
|
||||
break
|
||||
}
|
||||
j -= 1
|
||||
}
|
||||
this.tree_count[k & 1023] = tc
|
||||
this.tree_left[k & 1023] = tl
|
||||
this.tree_right[k & 1023] = tr
|
||||
i ~mod+= 1
|
||||
}
|
||||
return nothing
|
||||
}
|
||||
|
||||
// Shell sort.
|
||||
g = 0
|
||||
if args.n < 57 {
|
||||
g = 2
|
||||
}
|
||||
while g < 6 {
|
||||
gap = SHELL_GAPS[g]
|
||||
i = gap
|
||||
while i < args.n {
|
||||
j = i
|
||||
tc = this.tree_count[i & 1023]
|
||||
tl = this.tree_left[i & 1023]
|
||||
tr = this.tree_right[i & 1023]
|
||||
while j >= gap {
|
||||
if not this.huffman_tree_less(
|
||||
c0: tc,
|
||||
r0: tr,
|
||||
c1: this.tree_count[(j ~mod- gap) & 1023],
|
||||
r1: this.tree_right[(j ~mod- gap) & 1023],
|
||||
tie_break: args.tie_break) {
|
||||
break
|
||||
}
|
||||
this.tree_count[j & 1023] = this.tree_count[(j ~mod- gap) & 1023]
|
||||
this.tree_left[j & 1023] = this.tree_left[(j ~mod- gap) & 1023]
|
||||
this.tree_right[j & 1023] = this.tree_right[(j ~mod- gap) & 1023]
|
||||
j ~mod-= gap
|
||||
}
|
||||
this.tree_count[j & 1023] = tc
|
||||
this.tree_left[j & 1023] = tl
|
||||
this.tree_right[j & 1023] = tr
|
||||
i ~mod+= 1
|
||||
}
|
||||
g ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
// set_sentinel sets the pool's i'th node to the C code's sentinel node.
|
||||
pri func encoder.set_sentinel!(i: base.u32) {
|
||||
this.tree_count[args.i & 1023] = 0xFFFF_FFFF
|
||||
this.tree_left[args.i & 1023] = TREE_NONE
|
||||
this.tree_right[args.i & 1023] = TREE_NONE
|
||||
}
|
||||
|
||||
// merge_huffman_nodes runs the C code's "for (k = n - 1; k != 0; --k)" loop
|
||||
// (shared by BrotliCreateHuffmanTree in entropy_encode.c and
|
||||
// BrotliBuildAndStoreHuffmanTreeFast in brotli_bit_stream.c) that combines
|
||||
// the n sorted leaf nodes into a tree, whose root is node ((2 * n) - 1).
|
||||
pri func encoder.merge_huffman_nodes!(n: base.u32) {
|
||||
var i : base.u32
|
||||
var j : base.u32
|
||||
var k : base.u32
|
||||
var left : base.u32
|
||||
var right : base.u32
|
||||
var j_end : base.u32
|
||||
|
||||
this.set_sentinel!(i: args.n)
|
||||
this.set_sentinel!(i: args.n ~mod+ 1)
|
||||
i = 0
|
||||
j = args.n ~mod+ 1
|
||||
k = args.n ~mod- 1
|
||||
while k <> 0 {
|
||||
if this.tree_count[i & 1023] <= this.tree_count[j & 1023] {
|
||||
left = i
|
||||
i ~mod+= 1
|
||||
} else {
|
||||
left = j
|
||||
j ~mod+= 1
|
||||
}
|
||||
if this.tree_count[i & 1023] <= this.tree_count[j & 1023] {
|
||||
right = i
|
||||
i ~mod+= 1
|
||||
} else {
|
||||
right = j
|
||||
j ~mod+= 1
|
||||
}
|
||||
j_end = (args.n ~mod* 2) ~mod- k
|
||||
this.tree_count[j_end & 1023] = this.tree_count[left & 1023] ~mod+ this.tree_count[right & 1023]
|
||||
this.tree_left[j_end & 1023] = (left & 0xFFFF) as base.u16
|
||||
this.tree_right[j_end & 1023] = (right & 0xFFFF) as base.u16
|
||||
this.set_sentinel!(i: j_end ~mod+ 1)
|
||||
k -= 1
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliCreateHuffmanTree in entropy_encode.c. Wuffs' C code
|
||||
// generator does not support "slice base.u32" arguments, so the C code's data
|
||||
// argument is either this.cmd_histo[args.off ..] (when args.src is 0) or
|
||||
// this.cl_histo[args.off ..] (when args.src is 1).
|
||||
pri func encoder.create_huffman_tree!(src: base.u32, off: base.u32, length: base.u32, tree_limit: base.u32[..= 15], depth: slice base.u8) {
|
||||
var count_limit : base.u32
|
||||
var n : base.u32
|
||||
var i : base.u64
|
||||
var count : base.u32
|
||||
var v : base.u64
|
||||
var depth_ok : base.bool
|
||||
|
||||
count_limit = 1
|
||||
while true {
|
||||
n = 0
|
||||
i = args.length as base.u64
|
||||
while i > 0 {
|
||||
i -= 1
|
||||
if args.src == 0 {
|
||||
count = this.cmd_histo[((args.off as base.u64) ~mod+ i) & 127]
|
||||
} else {
|
||||
count = this.cl_histo[((args.off as base.u64) ~mod+ i) & 31]
|
||||
}
|
||||
if count <> 0 {
|
||||
this.tree_count[n & 1023] = count.max(no_less_than: count_limit)
|
||||
this.tree_left[n & 1023] = TREE_NONE
|
||||
this.tree_right[n & 1023] = (i & 0xFFFF) as base.u16
|
||||
n ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
if n == 1 {
|
||||
v = this.tree_right[0] as base.u64
|
||||
if v < args.depth.length() {
|
||||
args.depth[v] = 1
|
||||
}
|
||||
break
|
||||
} else if n == 0 {
|
||||
// The C code never calls this with an all-zero histogram.
|
||||
break
|
||||
}
|
||||
|
||||
this.sort_huffman_tree_items!(n: n, tie_break: true)
|
||||
this.merge_huffman_nodes!(n: n)
|
||||
depth_ok = this.set_depth!(p0: (n ~mod* 2) ~mod- 1, depth: args.depth, max_depth: args.tree_limit)
|
||||
if depth_ok {
|
||||
break
|
||||
}
|
||||
count_limit ~mod*= 2
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliReverseBits in entropy_encode.c.
|
||||
pri func encoder.reverse_bits(num_bits: base.u32, bits: base.u16) base.u16 {
|
||||
var retval : base.u32
|
||||
var bits : base.u32
|
||||
var i : base.u32
|
||||
|
||||
bits = args.bits as base.u32
|
||||
retval = REVERSE_BITS_LUT[bits & 15]
|
||||
i = 4
|
||||
while i < args.num_bits {
|
||||
retval = (retval ~mod<< 4) | REVERSE_BITS_LUT[(bits >> 4) & 15]
|
||||
bits >>= 4
|
||||
i ~mod+= 4
|
||||
}
|
||||
retval >>= (0 ~mod- args.num_bits) & 3
|
||||
return (retval & 0xFFFF) as base.u16
|
||||
}
|
||||
|
||||
// Mirrors BrotliConvertBitDepthsToSymbols in entropy_encode.c. Wuffs' C code
|
||||
// generator does not support "slice base.u16" arguments, so the C code's bits
|
||||
// argument is this.tmp_bits, this.cmd_bits[64 ..], this.cl_bits or
|
||||
// this.lit_bits when args.dst is 0, 1, 2 or 3.
|
||||
pri func encoder.convert_bit_depths_to_symbols!(depth: slice base.u8, len: base.u32, dst: base.u32) {
|
||||
var bl_count : array[16] base.u16
|
||||
var next_code : array[16] base.u16
|
||||
var i : base.u64
|
||||
var code : base.u32
|
||||
var d : base.u32
|
||||
var r : base.u16
|
||||
|
||||
i = 0
|
||||
while i < (args.len as base.u64) {
|
||||
if i < args.depth.length() {
|
||||
d = (args.depth[i] & 15) as base.u32
|
||||
bl_count[d] ~mod+= 1
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
bl_count[0] = 0
|
||||
next_code[0] = 0
|
||||
i = 1
|
||||
while i < 16 {
|
||||
code = (code ~mod+ (bl_count[(i ~mod- 1) & 15] as base.u32)) ~mod<< 1
|
||||
next_code[i] = (code & 0xFFFF) as base.u16
|
||||
i ~mod+= 1
|
||||
}
|
||||
i = 0
|
||||
while i < (args.len as base.u64) {
|
||||
if i < args.depth.length() {
|
||||
d = (args.depth[i] & 15) as base.u32
|
||||
if d <> 0 {
|
||||
r = this.reverse_bits(num_bits: d, bits: next_code[d])
|
||||
if args.dst == 0 {
|
||||
this.tmp_bits[i & 63] = r
|
||||
} else if args.dst == 1 {
|
||||
this.cmd_bits[(i ~mod+ 64) & 127] = r
|
||||
} else if args.dst == 2 {
|
||||
this.cl_bits[i & 31] = r
|
||||
} else {
|
||||
this.lit_bits[i & 255] = r
|
||||
}
|
||||
next_code[d] ~mod+= 1
|
||||
}
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors Reverse in entropy_encode.c, applied to both the huffman_tree and
|
||||
// huffman_tree_extra_bits arrays.
|
||||
pri func encoder.reverse_huffman_tree!(start: base.u32, end: base.u32) {
|
||||
var start : base.u32
|
||||
var end : base.u32
|
||||
var tmp : base.u8
|
||||
|
||||
start = args.start
|
||||
end = args.end ~mod- 1
|
||||
while start < end {
|
||||
tmp = this.huffman_tree[start & 1023]
|
||||
this.huffman_tree[start & 1023] = this.huffman_tree[end & 1023]
|
||||
this.huffman_tree[end & 1023] = tmp
|
||||
tmp = this.huffman_tree_extra_bits[start & 1023]
|
||||
this.huffman_tree_extra_bits[start & 1023] = this.huffman_tree_extra_bits[end & 1023]
|
||||
this.huffman_tree_extra_bits[end & 1023] = tmp
|
||||
start ~mod+= 1
|
||||
end ~mod-= 1
|
||||
}
|
||||
}
|
||||
|
||||
// push_huffman_tree appends (value, extra_bits) to the huffman_tree and
|
||||
// huffman_tree_extra_bits arrays.
|
||||
pri func encoder.push_huffman_tree!(value: base.u8, extra_bits: base.u8) {
|
||||
this.huffman_tree[this.huffman_tree_size & 1023] = args.value
|
||||
this.huffman_tree_extra_bits[this.huffman_tree_size & 1023] = args.extra_bits
|
||||
this.huffman_tree_size ~mod+= 1
|
||||
}
|
||||
|
||||
// Mirrors BrotliWriteHuffmanTreeRepetitions in entropy_encode.c.
|
||||
pri func encoder.write_huffman_tree_repetitions!(previous_value: base.u8, value: base.u8, repetitions: base.u32) {
|
||||
var repetitions : base.u32
|
||||
var start : base.u32
|
||||
|
||||
repetitions = args.repetitions
|
||||
if args.previous_value <> args.value {
|
||||
this.push_huffman_tree!(value: args.value, extra_bits: 0)
|
||||
repetitions ~mod-= 1
|
||||
}
|
||||
if repetitions == 7 {
|
||||
this.push_huffman_tree!(value: args.value, extra_bits: 0)
|
||||
repetitions ~mod-= 1
|
||||
}
|
||||
if repetitions < 3 {
|
||||
while repetitions > 0 {
|
||||
this.push_huffman_tree!(value: args.value, extra_bits: 0)
|
||||
repetitions -= 1
|
||||
}
|
||||
} else {
|
||||
start = this.huffman_tree_size
|
||||
repetitions -= 3
|
||||
while true {
|
||||
this.push_huffman_tree!(value: 16, extra_bits: (repetitions & 3) as base.u8)
|
||||
repetitions >>= 2
|
||||
if repetitions == 0 {
|
||||
break
|
||||
}
|
||||
repetitions -= 1
|
||||
}
|
||||
this.reverse_huffman_tree!(start: start, end: this.huffman_tree_size)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliWriteHuffmanTreeRepetitionsZeros in entropy_encode.c.
|
||||
pri func encoder.write_huffman_tree_repetitions_zeros!(repetitions: base.u32) {
|
||||
var repetitions : base.u32
|
||||
var start : base.u32
|
||||
|
||||
repetitions = args.repetitions
|
||||
if repetitions == 11 {
|
||||
this.push_huffman_tree!(value: 0, extra_bits: 0)
|
||||
repetitions -= 1
|
||||
}
|
||||
if repetitions < 3 {
|
||||
while repetitions > 0 {
|
||||
this.push_huffman_tree!(value: 0, extra_bits: 0)
|
||||
repetitions -= 1
|
||||
}
|
||||
} else {
|
||||
start = this.huffman_tree_size
|
||||
repetitions -= 3
|
||||
while true {
|
||||
this.push_huffman_tree!(value: 17, extra_bits: (repetitions & 7) as base.u8)
|
||||
repetitions >>= 3
|
||||
if repetitions == 0 {
|
||||
break
|
||||
}
|
||||
repetitions -= 1
|
||||
}
|
||||
this.reverse_huffman_tree!(start: start, end: this.huffman_tree_size)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors DecideOverRleUse in entropy_encode.c. The result's 0x01 bit is
|
||||
// use_rle_for_non_zero and its 0x02 bit is use_rle_for_zero.
|
||||
pri func encoder.decide_over_rle_use(depth: slice base.u8, length: base.u32) base.u32 {
|
||||
var total_reps_zero : base.u32
|
||||
var total_reps_non_zero : base.u32
|
||||
var count_reps_zero : base.u32
|
||||
var count_reps_non_zero : base.u32
|
||||
var i : base.u64
|
||||
var k : base.u64
|
||||
var length : base.u64
|
||||
var reps : base.u32
|
||||
var value : base.u8
|
||||
var ret : base.u32
|
||||
|
||||
length = args.length as base.u64
|
||||
count_reps_zero = 1
|
||||
count_reps_non_zero = 1
|
||||
i = 0
|
||||
while i < length {
|
||||
if i >= args.depth.length() {
|
||||
break
|
||||
}
|
||||
value = args.depth[i]
|
||||
reps = 1
|
||||
k = i ~mod+ 1
|
||||
while (k < length) and (k < args.depth.length()) {
|
||||
if args.depth[k] <> value {
|
||||
break
|
||||
}
|
||||
reps ~mod+= 1
|
||||
k ~mod+= 1
|
||||
}
|
||||
if (reps >= 3) and (value == 0) {
|
||||
total_reps_zero ~mod+= reps
|
||||
count_reps_zero ~mod+= 1
|
||||
}
|
||||
if (reps >= 4) and (value <> 0) {
|
||||
total_reps_non_zero ~mod+= reps
|
||||
count_reps_non_zero ~mod+= 1
|
||||
}
|
||||
i ~mod+= reps as base.u64
|
||||
}
|
||||
if total_reps_non_zero > (count_reps_non_zero ~mod* 2) {
|
||||
ret |= 1
|
||||
}
|
||||
if total_reps_zero > (count_reps_zero ~mod* 2) {
|
||||
ret |= 2
|
||||
}
|
||||
return ret
|
||||
}
|
||||
|
||||
// Mirrors BrotliWriteHuffmanTree in entropy_encode.c. The output is appended
|
||||
// to the huffman_tree and huffman_tree_extra_bits arrays.
|
||||
pri func encoder.write_huffman_tree!(depth: slice base.u8, length: base.u32) {
|
||||
var previous_value : base.u8
|
||||
var i : base.u64
|
||||
var k : base.u64
|
||||
var length : base.u64
|
||||
var reps : base.u32
|
||||
var new_length : base.u64
|
||||
var rle : base.u32
|
||||
var value : base.u8
|
||||
|
||||
previous_value = 8 // BROTLI_INITIAL_REPEATED_CODE_LENGTH.
|
||||
length = args.length as base.u64
|
||||
|
||||
// Throw away trailing zeros.
|
||||
new_length = length
|
||||
i = 0
|
||||
while i < length {
|
||||
k = (length ~mod- i) ~mod- 1
|
||||
if k >= args.depth.length() {
|
||||
break
|
||||
}
|
||||
if args.depth[k] == 0 {
|
||||
new_length ~mod-= 1
|
||||
} else {
|
||||
break
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
|
||||
// First gather statistics on if it is a good idea to do RLE.
|
||||
if length > 50 {
|
||||
rle = this.decide_over_rle_use(depth: args.depth, length: (new_length & 0xFFFF_FFFF) as base.u32)
|
||||
}
|
||||
|
||||
// Actual RLE coding.
|
||||
i = 0
|
||||
while i < new_length {
|
||||
if i >= args.depth.length() {
|
||||
break
|
||||
}
|
||||
value = args.depth[i]
|
||||
reps = 1
|
||||
if ((value <> 0) and ((rle & 1) <> 0)) or ((value == 0) and ((rle & 2) <> 0)) {
|
||||
k = i ~mod+ 1
|
||||
while (k < new_length) and (k < args.depth.length()) {
|
||||
if args.depth[k] <> value {
|
||||
break
|
||||
}
|
||||
reps ~mod+= 1
|
||||
k ~mod+= 1
|
||||
}
|
||||
}
|
||||
if value == 0 {
|
||||
this.write_huffman_tree_repetitions_zeros!(repetitions: reps)
|
||||
} else {
|
||||
this.write_huffman_tree_repetitions!(previous_value: previous_value, value: value, repetitions: reps)
|
||||
previous_value = value
|
||||
}
|
||||
i ~mod+= reps as base.u64
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliStoreHuffmanTreeOfHuffmanTreeToBitMask in
|
||||
// brotli_bit_stream.c, using this.cl_depth as code_length_bitdepth.
|
||||
pri func encoder.store_huffman_tree_of_huffman_tree_to_bit_mask!(num_codes: base.u32, s: slice base.u8) {
|
||||
var skip_some : base.u32
|
||||
var codes_to_store : base.u32[..= 18]
|
||||
var i : base.u32
|
||||
var l : base.u32
|
||||
|
||||
// Throw away trailing zeros.
|
||||
codes_to_store = 18
|
||||
if args.num_codes > 1 {
|
||||
while codes_to_store > 0 {
|
||||
if this.cl_depth[STORAGE_ORDER[codes_to_store - 1]] <> 0 {
|
||||
break
|
||||
}
|
||||
codes_to_store -= 1
|
||||
}
|
||||
}
|
||||
if (this.cl_depth[STORAGE_ORDER[0]] == 0) and (this.cl_depth[STORAGE_ORDER[1]] == 0) {
|
||||
skip_some = 2
|
||||
if this.cl_depth[STORAGE_ORDER[2]] == 0 {
|
||||
skip_some = 3
|
||||
}
|
||||
}
|
||||
this.write_bits!(n_bits: 2, bits: skip_some as base.u64, s: args.s)
|
||||
i = skip_some
|
||||
while (i < (codes_to_store as base.u32)) and (i < 18) {
|
||||
l = this.cl_depth[STORAGE_ORDER[i]] as base.u32
|
||||
if l < 6 {
|
||||
this.write_bits!(
|
||||
n_bits: HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_BIT_LENGTHS[l],
|
||||
bits: HUFFMAN_BIT_LENGTH_HUFFMAN_CODE_SYMBOLS[l],
|
||||
s: args.s)
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliStoreHuffmanTreeToBitMask in brotli_bit_stream.c, using
|
||||
// this.cl_depth and this.cl_bits as code_length_bitdepth and
|
||||
// code_length_bitdepth_symbols.
|
||||
pri func encoder.store_huffman_tree_to_bit_mask!(s: slice base.u8) {
|
||||
var i : base.u32
|
||||
var ix : base.u32
|
||||
|
||||
i = 0
|
||||
while i < this.huffman_tree_size {
|
||||
ix = (this.huffman_tree[i & 1023] & 31) as base.u32
|
||||
this.write_bits!(
|
||||
n_bits: this.cl_depth[ix] as base.u32,
|
||||
bits: this.cl_bits[ix] as base.u64,
|
||||
s: args.s)
|
||||
// Extra bits.
|
||||
if ix == 16 { // BROTLI_REPEAT_PREVIOUS_CODE_LENGTH.
|
||||
this.write_bits!(n_bits: 2, bits: this.huffman_tree_extra_bits[i & 1023] as base.u64, s: args.s)
|
||||
} else if ix == 17 { // BROTLI_REPEAT_ZERO_CODE_LENGTH.
|
||||
this.write_bits!(n_bits: 3, bits: this.huffman_tree_extra_bits[i & 1023] as base.u64, s: args.s)
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliStoreHuffmanTree in brotli_bit_stream.c.
|
||||
pri func encoder.store_huffman_tree!(depths: slice base.u8, num: base.u32, s: slice base.u8) {
|
||||
var i : base.u32
|
||||
var num_codes : base.u32
|
||||
var code : base.u32
|
||||
|
||||
this.huffman_tree_size = 0
|
||||
this.write_huffman_tree!(depth: args.depths, length: args.num)
|
||||
|
||||
// Calculate the statistics of the Huffman tree in brotli-representation.
|
||||
i = 0
|
||||
while i < 32 {
|
||||
this.cl_histo[i] = 0
|
||||
this.cl_depth[i] = 0
|
||||
i ~mod+= 1
|
||||
}
|
||||
i = 0
|
||||
while i < this.huffman_tree_size {
|
||||
this.cl_histo[this.huffman_tree[i & 1023] & 31] ~mod+= 1
|
||||
i ~mod+= 1
|
||||
}
|
||||
|
||||
i = 0
|
||||
while i < 18 {
|
||||
if this.cl_histo[i] <> 0 {
|
||||
if num_codes == 0 {
|
||||
code = i
|
||||
num_codes = 1
|
||||
} else if num_codes == 1 {
|
||||
num_codes = 2
|
||||
break
|
||||
}
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
|
||||
// Calculate another Huffman tree to use for compressing both the earlier
|
||||
// Huffman tree with.
|
||||
this.create_huffman_tree!(src: 1, off: 0, length: 18, tree_limit: 5, depth: this.cl_depth[..])
|
||||
this.convert_bit_depths_to_symbols!(depth: this.cl_depth[..], len: 18, dst: 2)
|
||||
|
||||
// Now, we have all the data, let's start storing it.
|
||||
this.store_huffman_tree_of_huffman_tree_to_bit_mask!(num_codes: num_codes, s: args.s)
|
||||
|
||||
if num_codes == 1 {
|
||||
this.cl_depth[code & 31] = 0
|
||||
}
|
||||
|
||||
// Store the real Huffman tree now.
|
||||
this.store_huffman_tree_to_bit_mask!(s: args.s)
|
||||
}
|
||||
|
||||
// Mirrors BrotliBuildAndStoreHuffmanTreeFast in brotli_bit_stream.c, with
|
||||
// this.histogram, this.lit_depth and this.lit_bits as the histogram, depth and
|
||||
// bits arguments.
|
||||
pri func encoder.build_and_store_huffman_tree_fast!(histogram_total: base.u64, max_bits: base.u32, s: slice base.u8) {
|
||||
var count : base.u32
|
||||
var symbols : array[4] base.u32
|
||||
var length : base.u32
|
||||
var total : base.u64
|
||||
var count_limit : base.u32
|
||||
var n : base.u32
|
||||
var l : base.u32
|
||||
var h : base.u32
|
||||
var i : base.u32
|
||||
var j : base.u32
|
||||
var tmp : base.u32
|
||||
var depth_ok : base.bool
|
||||
var value : base.u32
|
||||
var prev_value : base.u32
|
||||
var reps : base.u32
|
||||
var k : base.u32
|
||||
|
||||
total = args.histogram_total
|
||||
while (total <> 0) and (length < 256) {
|
||||
h = this.histogram[length]
|
||||
if h <> 0 {
|
||||
if count < 4 {
|
||||
symbols[count] = length
|
||||
}
|
||||
count ~mod+= 1
|
||||
total ~mod-= h as base.u64
|
||||
}
|
||||
length ~mod+= 1
|
||||
}
|
||||
|
||||
if count <= 1 {
|
||||
this.write_bits!(n_bits: 4, bits: 1, s: args.s)
|
||||
this.write_bits!(n_bits: args.max_bits, bits: symbols[0] as base.u64, s: args.s)
|
||||
this.lit_depth[symbols[0] & 255] = 0
|
||||
this.lit_bits[symbols[0] & 255] = 0
|
||||
return nothing
|
||||
}
|
||||
|
||||
i = 0
|
||||
while i < length {
|
||||
this.lit_depth[i & 255] = 0
|
||||
i ~mod+= 1
|
||||
}
|
||||
|
||||
count_limit = 1
|
||||
while true {
|
||||
n = 0
|
||||
l = length
|
||||
while l > 0 {
|
||||
l -= 1
|
||||
h = this.histogram[l & 255]
|
||||
if h <> 0 {
|
||||
if h >= count_limit {
|
||||
this.tree_count[n & 1023] = h
|
||||
} else {
|
||||
this.tree_count[n & 1023] = count_limit
|
||||
}
|
||||
this.tree_left[n & 1023] = TREE_NONE
|
||||
this.tree_right[n & 1023] = (l & 0xFFFF) as base.u16
|
||||
n ~mod+= 1
|
||||
}
|
||||
}
|
||||
this.sort_huffman_tree_items!(n: n, tie_break: false)
|
||||
this.merge_huffman_nodes!(n: n)
|
||||
depth_ok = this.set_depth!(p0: (n ~mod* 2) ~mod- 1, depth: this.lit_depth[..], max_depth: 14)
|
||||
if depth_ok {
|
||||
break
|
||||
}
|
||||
count_limit ~mod*= 2
|
||||
}
|
||||
this.convert_bit_depths_to_symbols!(depth: this.lit_depth[..], len: length, dst: 3)
|
||||
|
||||
if count <= 4 {
|
||||
// Value of 1 indicates a simple Huffman code.
|
||||
this.write_bits!(n_bits: 2, bits: 1, s: args.s)
|
||||
this.write_bits!(n_bits: 2, bits: (count ~mod- 1) as base.u64, s: args.s) // NSYM - 1.
|
||||
|
||||
// Sort.
|
||||
i = 0
|
||||
while i < count {
|
||||
j = i ~mod+ 1
|
||||
while j < count {
|
||||
if this.lit_depth[symbols[j & 3] & 255] < this.lit_depth[symbols[i & 3] & 255] {
|
||||
tmp = symbols[j & 3]
|
||||
symbols[j & 3] = symbols[i & 3]
|
||||
symbols[i & 3] = tmp
|
||||
}
|
||||
j ~mod+= 1
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
|
||||
this.write_bits!(n_bits: args.max_bits, bits: symbols[0] as base.u64, s: args.s)
|
||||
this.write_bits!(n_bits: args.max_bits, bits: symbols[1] as base.u64, s: args.s)
|
||||
if count >= 3 {
|
||||
this.write_bits!(n_bits: args.max_bits, bits: symbols[2] as base.u64, s: args.s)
|
||||
}
|
||||
if count >= 4 {
|
||||
this.write_bits!(n_bits: args.max_bits, bits: symbols[3] as base.u64, s: args.s)
|
||||
// Tree-select.
|
||||
if this.lit_depth[symbols[0] & 255] == 1 {
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s)
|
||||
} else {
|
||||
this.write_bits!(n_bits: 1, bits: 0, s: args.s)
|
||||
}
|
||||
}
|
||||
|
||||
} else {
|
||||
// Complex Huffman Tree.
|
||||
this.write_bits!(n_bits: 40, bits: STATIC_CODE_LENGTH_CODE, s: args.s)
|
||||
|
||||
// Actual RLE coding.
|
||||
prev_value = 8
|
||||
i = 0
|
||||
while i < length {
|
||||
value = this.lit_depth[i & 255] as base.u32
|
||||
reps = 1
|
||||
k = i ~mod+ 1
|
||||
while k < length {
|
||||
if (this.lit_depth[k & 255] as base.u32) <> value {
|
||||
break
|
||||
}
|
||||
reps ~mod+= 1
|
||||
k ~mod+= 1
|
||||
}
|
||||
i ~mod+= reps
|
||||
if value == 0 {
|
||||
this.write_bits!(n_bits: ZERO_REPS_DEPTH[reps & 511], bits: ZERO_REPS_BITS[reps & 511], s: args.s)
|
||||
} else {
|
||||
if prev_value <> value {
|
||||
this.write_bits!(n_bits: CODE_LENGTH_DEPTH[value & 15], bits: CODE_LENGTH_BITS[value & 15], s: args.s)
|
||||
reps ~mod-= 1
|
||||
}
|
||||
if reps < 3 {
|
||||
while reps <> 0 {
|
||||
reps -= 1
|
||||
this.write_bits!(n_bits: CODE_LENGTH_DEPTH[value & 15], bits: CODE_LENGTH_BITS[value & 15], s: args.s)
|
||||
}
|
||||
} else {
|
||||
reps -= 3
|
||||
this.write_bits!(n_bits: NON_ZERO_REPS_DEPTH[reps & 511], bits: NON_ZERO_REPS_BITS[reps & 511], s: args.s)
|
||||
}
|
||||
prev_value = value
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// copy_tmp_bits_to_cmd_bits copies this.tmp_bits[args.src .. args.src + args.n]
|
||||
// to this.cmd_bits[args.dst .. args.dst + args.n]. Wuffs' C code generator
|
||||
// mishandles copy_from_slice on base.u16 arrays, so this is an explicit loop.
|
||||
pri func encoder.copy_tmp_bits_to_cmd_bits!(dst: base.u32, src: base.u32, n: base.u32) {
|
||||
var i : base.u32
|
||||
|
||||
i = 0
|
||||
while i < args.n {
|
||||
this.cmd_bits[(args.dst ~mod+ i) & 127] = this.tmp_bits[(args.src ~mod+ i) & 63]
|
||||
i ~mod+= 1
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,716 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// This file mirrors compress_fragment.c, the quality 0 (one-pass) encoder.
|
||||
//
|
||||
// The C code's input pointers (input, ip, next_emit, candidate, etc.) are
|
||||
// base.u64 offsets from the start of the fragment (the C code's base_ip).
|
||||
// The hash table holds the same offsets. The C code's last_distance is -1 or
|
||||
// positive. Here, 0 means -1.
|
||||
|
||||
// FRAGMENT_MAX_DISTANCE is the C code's MAX_DISTANCE,
|
||||
// BROTLI_MAX_BACKWARD_LIMIT(18).
|
||||
pri const FRAGMENT_MAX_DISTANCE : base.u64 = 262128
|
||||
|
||||
// HASH_MUL32 is kHashMul32 in hash_base.h.
|
||||
pri const HASH_MUL32 : base.u64 = 0x1E35_A7BD
|
||||
|
||||
// Mirrors Hash in compress_fragment.c.
|
||||
pri func encoder.hash_fast(input: roslice base.u8, i: base.u64, shift: base.u32) base.u32 {
|
||||
var h : base.u64
|
||||
|
||||
h = (this.load64le(s: args.input, i: args.i) ~mod<< 24) ~mod* HASH_MUL32
|
||||
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
|
||||
}
|
||||
|
||||
// Mirrors HashBytesAtOffset in compress_fragment.c.
|
||||
pri func encoder.hash_bytes_at_offset_fast(v: base.u64, offset: base.u32, shift: base.u32) base.u32 {
|
||||
var h : base.u64
|
||||
|
||||
h = ((args.v >> ((8 ~mod* args.offset) & 63)) ~mod<< 24) ~mod* HASH_MUL32
|
||||
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
|
||||
}
|
||||
|
||||
// Mirrors IsMatch in compress_fragment.c.
|
||||
pri func encoder.is_match_fast(input: roslice base.u8, i1: base.u64, i2: base.u64) base.bool {
|
||||
if (args.i1 >= args.input.length()) or (args.i2 >= args.input.length()) {
|
||||
return false
|
||||
}
|
||||
if this.load32le(s: args.input, i: args.i1) <> this.load32le(s: args.input, i: args.i2) {
|
||||
return false
|
||||
}
|
||||
if ((args.i1 ~mod+ 4) >= args.input.length()) or ((args.i2 ~mod+ 4) >= args.input.length()) {
|
||||
return false
|
||||
}
|
||||
return args.input[args.i1 ~mod+ 4] == args.input[args.i2 ~mod+ 4]
|
||||
}
|
||||
|
||||
// Mirrors BuildAndStoreLiteralPrefixCode in compress_fragment.c, for the
|
||||
// input_size bytes at args.input[args.from ..].
|
||||
pri func encoder.build_and_store_literal_prefix_code!(input: roslice base.u8, from: base.u64, input_size: base.u64, s: slice base.u8) base.u64 {
|
||||
var histogram_total : base.u64
|
||||
var i : base.u64
|
||||
var j : base.u32
|
||||
var adjust : base.u32
|
||||
var literal_ratio : base.u64
|
||||
var h : base.u32
|
||||
|
||||
j = 0
|
||||
while j < 256 {
|
||||
this.histogram[j] = 0
|
||||
j ~mod+= 1
|
||||
}
|
||||
|
||||
if args.input_size < 0x8000 {
|
||||
i = 0
|
||||
while i < args.input_size {
|
||||
if (args.from ~mod+ i) < args.input.length() {
|
||||
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
histogram_total = args.input_size
|
||||
j = 0
|
||||
while j < 256 {
|
||||
// We weigh the first 11 samples with weight 3 to account for the
|
||||
// balancing effect of the LZ77 phase on the histogram.
|
||||
adjust = 2 * this.histogram[j].min(no_more_than: 11)
|
||||
this.histogram[j] ~mod+= adjust
|
||||
histogram_total ~mod+= adjust as base.u64
|
||||
j ~mod+= 1
|
||||
}
|
||||
} else {
|
||||
i = 0
|
||||
while i < args.input_size {
|
||||
if (args.from ~mod+ i) < args.input.length() {
|
||||
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
|
||||
}
|
||||
i ~mod+= 29
|
||||
}
|
||||
histogram_total = (args.input_size ~mod+ 28) / 29
|
||||
j = 0
|
||||
while j < 256 {
|
||||
// We add 1 to each population count to avoid 0 bit depths (since
|
||||
// this is only a sample and we don't know if the symbol appears
|
||||
// or not), and we weigh the first 11 samples with weight 3 to
|
||||
// account for the balancing effect of the LZ77 phase on the
|
||||
// histogram (more frequent symbols are more likely to be in
|
||||
// backward references instead as literals).
|
||||
adjust = 1 + (2 * this.histogram[j].min(no_more_than: 11))
|
||||
this.histogram[j] ~mod+= adjust
|
||||
histogram_total ~mod+= adjust as base.u64
|
||||
j ~mod+= 1
|
||||
}
|
||||
}
|
||||
this.build_and_store_huffman_tree_fast!(histogram_total: histogram_total, max_bits: 8, s: args.s)
|
||||
|
||||
j = 0
|
||||
while j < 256 {
|
||||
h = this.histogram[j]
|
||||
if h <> 0 {
|
||||
literal_ratio ~mod+= ((h ~mod* (this.lit_depth[j] as base.u32)) as base.u64)
|
||||
}
|
||||
j ~mod+= 1
|
||||
}
|
||||
// Estimated encoding ratio, millibytes per symbol.
|
||||
if histogram_total == 0 {
|
||||
return 0
|
||||
}
|
||||
return (literal_ratio ~mod* 125) / histogram_total
|
||||
}
|
||||
|
||||
// Mirrors BuildAndStoreCommandPrefixCode in compress_fragment.c.
|
||||
pri func encoder.build_and_store_command_prefix_code_fast!(s: slice base.u8) {
|
||||
var i : base.u32
|
||||
|
||||
this.tmp_depth[.. 704].bulk_memset!(byte_value: 0)
|
||||
|
||||
this.create_huffman_tree!(src: 0, off: 0, length: 64, tree_limit: 15, depth: this.cmd_depth[.. 64])
|
||||
this.create_huffman_tree!(src: 0, off: 64, length: 64, tree_limit: 14, depth: this.cmd_depth[64 ..])
|
||||
// We have to jump through a few hoops here in order to compute the
|
||||
// command bits because the symbols are in a different order than in the
|
||||
// full alphabet. This looks complicated, but having the symbols in this
|
||||
// order in the command bits saves a few branches in the Emit* functions.
|
||||
this.tmp_depth[0 .. 24].copy_from_slice!(s: this.cmd_depth[0 .. 24])
|
||||
this.tmp_depth[24 .. 32].copy_from_slice!(s: this.cmd_depth[40 .. 48])
|
||||
this.tmp_depth[32 .. 40].copy_from_slice!(s: this.cmd_depth[24 .. 32])
|
||||
this.tmp_depth[40 .. 48].copy_from_slice!(s: this.cmd_depth[48 .. 56])
|
||||
this.tmp_depth[48 .. 56].copy_from_slice!(s: this.cmd_depth[32 .. 40])
|
||||
this.tmp_depth[56 .. 64].copy_from_slice!(s: this.cmd_depth[56 .. 64])
|
||||
this.convert_bit_depths_to_symbols!(depth: this.tmp_depth[..], len: 64, dst: 0)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 0, src: 0, n: 24)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 24, src: 32, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 32, src: 48, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 40, src: 24, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 48, src: 40, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 56, src: 56, n: 8)
|
||||
this.convert_bit_depths_to_symbols!(depth: this.cmd_depth[64 ..], len: 64, dst: 1)
|
||||
|
||||
// Create the bit length array for the full command alphabet.
|
||||
this.tmp_depth[.. 64].bulk_memset!(byte_value: 0)
|
||||
this.tmp_depth[0 .. 8].copy_from_slice!(s: this.cmd_depth[0 .. 8])
|
||||
this.tmp_depth[64 .. 72].copy_from_slice!(s: this.cmd_depth[8 .. 16])
|
||||
this.tmp_depth[128 .. 136].copy_from_slice!(s: this.cmd_depth[16 .. 24])
|
||||
this.tmp_depth[192 .. 200].copy_from_slice!(s: this.cmd_depth[24 .. 32])
|
||||
this.tmp_depth[384 .. 392].copy_from_slice!(s: this.cmd_depth[32 .. 40])
|
||||
i = 0
|
||||
while i < 8 {
|
||||
this.tmp_depth[128 + (8 * i)] = this.cmd_depth[40 + i]
|
||||
this.tmp_depth[256 + (8 * i)] = this.cmd_depth[48 + i]
|
||||
this.tmp_depth[448 + (8 * i)] = this.cmd_depth[56 + i]
|
||||
i ~mod+= 1
|
||||
}
|
||||
this.store_huffman_tree!(depths: this.tmp_depth[..], num: 704, s: args.s)
|
||||
this.store_huffman_tree!(depths: this.cmd_depth[64 ..], num: 64, s: args.s)
|
||||
}
|
||||
|
||||
// emit_cmd_fast writes the command (or distance) prefix code symbol and
|
||||
// updates its histogram count. It is the C code's "BrotliWriteBits(depth[code],
|
||||
// bits[code], storage_ix, storage); ++histo[code];".
|
||||
pri func encoder.emit_cmd_fast!(code: base.u64, s: slice base.u8) {
|
||||
this.write_bits!(
|
||||
n_bits: this.cmd_depth[args.code & 127] as base.u32,
|
||||
bits: this.cmd_bits[args.code & 127] as base.u64,
|
||||
s: args.s)
|
||||
this.cmd_histo[args.code & 127] ~mod+= 1
|
||||
}
|
||||
|
||||
// Mirrors EmitInsertLen in compress_fragment.c.
|
||||
pri func encoder.emit_insert_len_fast!(insertlen: base.u64, s: slice base.u8) {
|
||||
var tail : base.u64
|
||||
var nbits : base.u32
|
||||
var prefix : base.u64
|
||||
|
||||
if args.insertlen < 6 {
|
||||
this.emit_cmd_fast!(code: args.insertlen + 40, s: args.s)
|
||||
} else if args.insertlen < 130 {
|
||||
tail = args.insertlen - 2
|
||||
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
|
||||
prefix = tail >> (nbits & 63)
|
||||
this.emit_cmd_fast!(code: (((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 42, s: args.s)
|
||||
this.write_bits!(n_bits: nbits, bits: tail ~mod- (prefix ~mod<< (nbits & 63)), s: args.s)
|
||||
} else if args.insertlen < 2114 {
|
||||
tail = args.insertlen - 66
|
||||
nbits = this.log2_floor_non_zero(n: tail)
|
||||
this.emit_cmd_fast!(code: (nbits as base.u64) + 50, s: args.s)
|
||||
this.write_bits!(n_bits: nbits, bits: tail ~mod- ((1 as base.u64) << nbits), s: args.s)
|
||||
} else {
|
||||
this.emit_cmd_fast!(code: 61, s: args.s)
|
||||
this.write_bits!(n_bits: 12, bits: args.insertlen ~mod- 2114, s: args.s)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors EmitLongInsertLen in compress_fragment.c.
|
||||
pri func encoder.emit_long_insert_len_fast!(insertlen: base.u64, s: slice base.u8) {
|
||||
if args.insertlen < 22594 {
|
||||
this.emit_cmd_fast!(code: 62, s: args.s)
|
||||
this.write_bits!(n_bits: 14, bits: args.insertlen ~mod- 6210, s: args.s)
|
||||
} else {
|
||||
this.emit_cmd_fast!(code: 63, s: args.s)
|
||||
this.write_bits!(n_bits: 24, bits: args.insertlen ~mod- 22594, s: args.s)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors EmitCopyLen in compress_fragment.c.
|
||||
pri func encoder.emit_copy_len_fast!(copylen: base.u64, s: slice base.u8) {
|
||||
var tail : base.u64
|
||||
var nbits : base.u32
|
||||
var prefix : base.u64
|
||||
|
||||
if args.copylen < 10 {
|
||||
this.emit_cmd_fast!(code: args.copylen + 14, s: args.s)
|
||||
} else if args.copylen < 134 {
|
||||
tail = args.copylen - 6
|
||||
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
|
||||
prefix = tail >> (nbits & 63)
|
||||
this.emit_cmd_fast!(code: (((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 20, s: args.s)
|
||||
this.write_bits!(n_bits: nbits, bits: tail ~mod- (prefix ~mod<< (nbits & 63)), s: args.s)
|
||||
} else if args.copylen < 2118 {
|
||||
tail = args.copylen - 70
|
||||
nbits = this.log2_floor_non_zero(n: tail)
|
||||
this.emit_cmd_fast!(code: (nbits as base.u64) + 28, s: args.s)
|
||||
this.write_bits!(n_bits: nbits, bits: tail ~mod- ((1 as base.u64) << nbits), s: args.s)
|
||||
} else {
|
||||
this.emit_cmd_fast!(code: 39, s: args.s)
|
||||
this.write_bits!(n_bits: 24, bits: args.copylen ~mod- 2118, s: args.s)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors EmitCopyLenLastDistance in compress_fragment.c.
|
||||
pri func encoder.emit_copy_len_last_distance_fast!(copylen: base.u64, s: slice base.u8) {
|
||||
var tail : base.u64
|
||||
var nbits : base.u32
|
||||
var prefix : base.u64
|
||||
|
||||
if args.copylen < 12 {
|
||||
this.emit_cmd_fast!(code: args.copylen ~mod- 4, s: args.s)
|
||||
} else if args.copylen < 72 {
|
||||
tail = args.copylen - 8
|
||||
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
|
||||
prefix = tail >> (nbits & 63)
|
||||
this.emit_cmd_fast!(code: (((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 4, s: args.s)
|
||||
this.write_bits!(n_bits: nbits, bits: tail ~mod- (prefix ~mod<< (nbits & 63)), s: args.s)
|
||||
} else if args.copylen < 136 {
|
||||
tail = args.copylen - 8
|
||||
this.emit_cmd_fast!(code: (tail >> 5) + 30, s: args.s)
|
||||
this.write_bits!(n_bits: 5, bits: tail & 31, s: args.s)
|
||||
this.emit_cmd_fast!(code: 64, s: args.s)
|
||||
} else if args.copylen < 2120 {
|
||||
tail = args.copylen - 72
|
||||
nbits = this.log2_floor_non_zero(n: tail)
|
||||
this.emit_cmd_fast!(code: (nbits as base.u64) + 28, s: args.s)
|
||||
this.write_bits!(n_bits: nbits, bits: tail ~mod- ((1 as base.u64) << nbits), s: args.s)
|
||||
this.emit_cmd_fast!(code: 64, s: args.s)
|
||||
} else {
|
||||
this.emit_cmd_fast!(code: 39, s: args.s)
|
||||
this.write_bits!(n_bits: 24, bits: args.copylen ~mod- 2120, s: args.s)
|
||||
this.emit_cmd_fast!(code: 64, s: args.s)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors EmitDistance in compress_fragment.c.
|
||||
pri func encoder.emit_distance_fast!(distance: base.u64, s: slice base.u8) {
|
||||
var d : base.u64
|
||||
var nbits : base.u32
|
||||
var prefix : base.u64
|
||||
var offset : base.u64
|
||||
|
||||
d = args.distance ~mod+ 3
|
||||
nbits = this.log2_floor_non_zero(n: d) ~mod- 1
|
||||
prefix = (d >> (nbits & 63)) & 1
|
||||
offset = (2 + prefix) ~mod<< (nbits & 63)
|
||||
this.emit_cmd_fast!(code: (((2 * ((nbits ~mod- 1) as base.u64)) ~mod+ prefix) ~mod+ 80), s: args.s)
|
||||
this.write_bits!(n_bits: nbits, bits: d ~mod- offset, s: args.s)
|
||||
}
|
||||
|
||||
// Mirrors EmitLiterals in compress_fragment.c.
|
||||
pri func encoder.emit_literals!(input: roslice base.u8, from: base.u64, len: base.u64, s: slice base.u8) {
|
||||
var j : base.u64
|
||||
var lit : base.u8
|
||||
|
||||
j = 0
|
||||
while j < args.len {
|
||||
if (args.from ~mod+ j) >= args.input.length() {
|
||||
break
|
||||
}
|
||||
lit = args.input[args.from ~mod+ j]
|
||||
this.write_bits!(n_bits: this.lit_depth[lit] as base.u32, bits: this.lit_bits[lit] as base.u64, s: args.s)
|
||||
j ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors ShouldMergeBlock in compress_fragment.c, for the len bytes at
|
||||
// args.input[args.from ..].
|
||||
pri func encoder.should_merge_block!(input: roslice base.u8, from: base.u64, len: base.u64) base.bool {
|
||||
var i : base.u64
|
||||
var j : base.u32
|
||||
var total : base.u64
|
||||
var r : base.u64
|
||||
var h : base.u64
|
||||
|
||||
j = 0
|
||||
while j < 256 {
|
||||
this.histogram[j] = 0
|
||||
j ~mod+= 1
|
||||
}
|
||||
i = 0
|
||||
while i < args.len {
|
||||
if (args.from ~mod+ i) < args.input.length() {
|
||||
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
|
||||
}
|
||||
i ~mod+= 43
|
||||
}
|
||||
|
||||
total = (args.len ~mod+ 42) / 43
|
||||
// r = (FastLog2(total) + 0.5) * (double)total + 200.
|
||||
r = this.sf.f64_add(
|
||||
a: this.sf.f64_mul(
|
||||
a: this.sf.f64_add(a: this.fast_log2(v: total), b: 0x3FE0_0000_0000_0000),
|
||||
b: this.sf.f64_from_u64(a: total)),
|
||||
b: 0x4069_0000_0000_0000)
|
||||
j = 0
|
||||
while j < 256 {
|
||||
// r -= (double)histo[i] * (depths[i] + FastLog2(histo[i])).
|
||||
h = this.histogram[j] as base.u64
|
||||
r = this.sf.f64_sub(
|
||||
a: r,
|
||||
b: this.sf.f64_mul(
|
||||
a: this.sf.f64_from_u64(a: h),
|
||||
b: this.sf.f64_add(
|
||||
a: this.sf.f64_from_u32(a: this.lit_depth[j] as base.u32),
|
||||
b: this.fast_log2(v: h))))
|
||||
j ~mod+= 1
|
||||
}
|
||||
return this.sf.f64_ge(a: r, b: 0) <> 0
|
||||
}
|
||||
|
||||
// Mirrors ShouldUseUncompressedMode in compress_fragment.c.
|
||||
pri func encoder.should_use_uncompressed_mode(metablock_start: base.u64, next_emit: base.u64, insertlen: base.u64, literal_ratio: base.u64) base.bool {
|
||||
var compressed : base.u64
|
||||
|
||||
compressed = args.next_emit ~mod- args.metablock_start
|
||||
if (compressed ~mod* 50) > args.insertlen {
|
||||
return false
|
||||
}
|
||||
// MIN_RATIO is 980.
|
||||
return args.literal_ratio > 980
|
||||
}
|
||||
|
||||
// Mirrors EmitUncompressedMetaBlock in compress_fragment.c.
|
||||
pri func encoder.emit_uncompressed_meta_block_fast!(input: roslice base.u8, begin: base.u64, end: base.u64, storage_ix_start: base.u64, s: slice base.u8) {
|
||||
var len : base.u64
|
||||
var p : base.u64
|
||||
|
||||
len = args.end ~mod- args.begin
|
||||
this.rewind_bit_position!(new_ix: args.storage_ix_start, s: args.s)
|
||||
this.store_meta_block_header!(len: len, is_uncompressed: true, s: args.s)
|
||||
this.align_to_byte!()
|
||||
this.copy_bytes!(input: args.input, begin: args.begin, end: args.end, s: args.s)
|
||||
this.bw_ix ~mod+= len ~mod<< 3
|
||||
p = this.bw_ix >> 3
|
||||
if p < args.s.length() {
|
||||
args.s[p] = 0
|
||||
}
|
||||
}
|
||||
|
||||
// copy_bytes is the C code's "memcpy(&storage[*storage_ix >> 3], begin,
|
||||
// len)", where bw_ix is byte-aligned.
|
||||
pri func encoder.copy_bytes!(input: roslice base.u8, begin: base.u64, end: base.u64, s: slice base.u8) {
|
||||
var p : base.u64
|
||||
|
||||
p = this.bw_ix >> 3
|
||||
if (args.begin <= args.end) and (args.end <= args.input.length()) and (p <= args.s.length()) {
|
||||
args.s[p ..].copy_from_slice!(s: args.input[args.begin .. args.end])
|
||||
}
|
||||
}
|
||||
|
||||
// update_table_fast updates the hash table with the hashes of some positions
|
||||
// within the last copy (ending at ip), and returns the next candidate. It is
|
||||
// the C code's "We could immediately start working at ip now, but to improve
|
||||
// compression we first update table with the hashes..." block.
|
||||
pri func encoder.update_table_fast!(input: roslice base.u8, ip: base.u64, shift: base.u32) base.u64 {
|
||||
var input_bytes : base.u64
|
||||
var prev_hash : base.u32
|
||||
var cur_hash : base.u32
|
||||
var candidate : base.u64
|
||||
|
||||
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 3)
|
||||
prev_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 0, shift: args.shift)
|
||||
cur_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 3, shift: args.shift)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 3) & 0xFFFF_FFFF) as base.u32
|
||||
prev_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 1, shift: args.shift)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 2) & 0xFFFF_FFFF) as base.u32
|
||||
prev_hash = this.hash_bytes_at_offset_fast(v: input_bytes, offset: 2, shift: args.shift)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 1) & 0xFFFF_FFFF) as base.u32
|
||||
|
||||
candidate = this.hash_table[cur_hash & 0x1_FFFF] as base.u64
|
||||
this.hash_table[cur_hash & 0x1_FFFF] = (args.ip & 0xFFFF_FFFF) as base.u32
|
||||
return candidate
|
||||
}
|
||||
|
||||
// Mirrors BrotliCompressFragmentFast in compress_fragment.c.
|
||||
pri func encoder.compress_fragment_fast!(input: roslice base.u8, is_last: base.bool, table_bits: base.u32, s: slice base.u8) {
|
||||
var initial_ix : base.u64
|
||||
var input_size : base.u64
|
||||
|
||||
initial_ix = this.bw_ix
|
||||
input_size = args.input.length()
|
||||
|
||||
if input_size == 0 {
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // islast
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // isempty
|
||||
this.align_to_byte!()
|
||||
return nothing
|
||||
}
|
||||
|
||||
this.compress_fragment_fast_impl!(input: args.input, is_last: args.is_last, table_bits: args.table_bits, s: args.s)
|
||||
|
||||
// If output is larger than single uncompressed block, rewrite it.
|
||||
if (this.bw_ix ~mod- initial_ix) > (31 ~mod+ (input_size ~mod<< 3)) {
|
||||
this.emit_uncompressed_meta_block_fast!(
|
||||
input: args.input, begin: 0, end: input_size, storage_ix_start: initial_ix, s: args.s)
|
||||
}
|
||||
|
||||
if args.is_last {
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // islast
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // isempty
|
||||
this.align_to_byte!()
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliCompressFragmentFastImpl in compress_fragment.c. The C code's
|
||||
// gotos (to emit_commands, trawl, emit_remainder and next_block) are labeled
|
||||
// loops and the goto_next_block bool.
|
||||
pri func encoder.compress_fragment_fast_impl!(input: roslice base.u8, is_last: base.bool, table_bits: base.u32, s: slice base.u8) {
|
||||
var in_off : base.u64
|
||||
var in_size : base.u64
|
||||
var next_emit : base.u64
|
||||
var metablock_start : base.u64
|
||||
var block_size : base.u64
|
||||
var total_block_size : base.u64
|
||||
var mlen_storage_ix : base.u64
|
||||
var literal_ratio : base.u64
|
||||
var shift : base.u32
|
||||
var i : base.u64
|
||||
var ip : base.u64
|
||||
var ip_end : base.u64
|
||||
var ip_limit : base.u64
|
||||
var len_limit : base.u64
|
||||
var next_ip : base.u64
|
||||
var candidate : base.u64
|
||||
var base_ip : base.u64
|
||||
var matched : base.u64
|
||||
var distance : base.u64
|
||||
var insert : base.u64
|
||||
var last_distance : base.u64
|
||||
var skip : base.u32
|
||||
var hash : base.u32
|
||||
var next_hash : base.u32
|
||||
var goto_next_block : base.bool
|
||||
var merge : base.bool
|
||||
var saved_ix : base.u64
|
||||
|
||||
shift = 64 ~mod- args.table_bits
|
||||
|
||||
in_size = args.input.length()
|
||||
block_size = in_size.min(no_more_than: 98304) // kFirstBlockSize is (3 << 15).
|
||||
total_block_size = block_size
|
||||
// Save the bit position of the MLEN field of the meta-block header, so
|
||||
// that we can update it later if we decide to extend this meta-block.
|
||||
mlen_storage_ix = this.bw_ix ~mod+ 3
|
||||
|
||||
this.store_meta_block_header!(len: block_size, is_uncompressed: false, s: args.s)
|
||||
// No block splits, no contexts.
|
||||
this.write_bits!(n_bits: 13, bits: 0, s: args.s)
|
||||
|
||||
literal_ratio = this.build_and_store_literal_prefix_code!(
|
||||
input: args.input, from: in_off, input_size: block_size, s: args.s)
|
||||
|
||||
// Store the pre-compressed command and distance prefix codes.
|
||||
i = 0
|
||||
while (i ~mod+ 7) < this.cmd_code_numbits {
|
||||
this.write_bits!(n_bits: 8, bits: this.cmd_code[(i >> 3) & 511] as base.u64, s: args.s)
|
||||
i ~mod+= 8
|
||||
}
|
||||
this.write_bits!(
|
||||
n_bits: (this.cmd_code_numbits & 7) as base.u32,
|
||||
bits: this.cmd_code[(this.cmd_code_numbits >> 3) & 511] as base.u64,
|
||||
s: args.s)
|
||||
|
||||
while.emit_commands true {
|
||||
// Initialize the command and distance histograms. We will gather
|
||||
// statistics of command and distance codes during the processing of
|
||||
// this block and use it to update the command and distance prefix
|
||||
// codes for the next block.
|
||||
i = 0
|
||||
while i < 128 {
|
||||
this.cmd_histo[i] = CMD_HISTO_SEED[i]
|
||||
i ~mod+= 1
|
||||
}
|
||||
|
||||
ip = in_off
|
||||
last_distance = 0
|
||||
ip_end = in_off ~mod+ block_size
|
||||
goto_next_block = false
|
||||
|
||||
if block_size >= 16 {
|
||||
// For the last block, we need to keep a 16 bytes margin so that
|
||||
// we can be sure that all distances are at most window size - 16.
|
||||
// For all other blocks, we only need to keep a margin of 5 bytes
|
||||
// so that we don't go over the block size with a copy.
|
||||
len_limit = block_size ~mod- 5
|
||||
len_limit = len_limit.min(no_more_than: in_size ~mod- 16)
|
||||
ip_limit = in_off ~mod+ len_limit
|
||||
|
||||
ip ~mod+= 1
|
||||
next_hash = this.hash_fast(input: args.input, i: ip, shift: shift)
|
||||
while.outer true {
|
||||
// Step 1: Scan forward in the input looking for a 5-byte-long
|
||||
// match. If we get close to exhausting the input then goto
|
||||
// emit_remainder.
|
||||
skip = 32
|
||||
next_ip = ip
|
||||
|
||||
while.trawl true {
|
||||
hash = next_hash
|
||||
ip = next_ip
|
||||
next_ip = ip ~mod+ ((skip >> 5) as base.u64)
|
||||
skip ~mod+= 1
|
||||
if next_ip > ip_limit {
|
||||
break.outer
|
||||
}
|
||||
next_hash = this.hash_fast(input: args.input, i: next_ip, shift: shift)
|
||||
if (last_distance > 0) and (last_distance <= ip) {
|
||||
candidate = ip ~mod- last_distance
|
||||
if this.is_match_fast(input: args.input, i1: ip, i2: candidate) {
|
||||
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
|
||||
// Check copy distance. If candidate is not
|
||||
// feasible, continue search.
|
||||
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
|
||||
continue.trawl
|
||||
}
|
||||
break.trawl
|
||||
}
|
||||
}
|
||||
candidate = this.hash_table[hash & 0x1_FFFF] as base.u64
|
||||
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
|
||||
if this.is_match_fast(input: args.input, i1: ip, i2: candidate) {
|
||||
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
|
||||
continue.trawl
|
||||
}
|
||||
break.trawl
|
||||
}
|
||||
}.trawl
|
||||
|
||||
// Step 2: Emit the found match together with the literal bytes
|
||||
// from "next_emit" to the bit stream, and then see if we can
|
||||
// find a next match immediately afterwards. Repeat until we
|
||||
// find no match for the input without emitting some literal
|
||||
// bytes.
|
||||
base_ip = ip
|
||||
matched = 5 ~mod+ this.find_match_length_with_limit(
|
||||
s: args.input, i1: candidate ~mod+ 5, i2: ip ~mod+ 5, limit: (ip_end ~mod- ip) ~mod- 5)
|
||||
distance = base_ip ~mod- candidate
|
||||
insert = base_ip ~mod- next_emit
|
||||
ip ~mod+= matched
|
||||
if insert < 6210 {
|
||||
this.emit_insert_len_fast!(insertlen: insert, s: args.s)
|
||||
} else if this.should_use_uncompressed_mode(
|
||||
metablock_start: metablock_start, next_emit: next_emit,
|
||||
insertlen: insert, literal_ratio: literal_ratio) {
|
||||
this.emit_uncompressed_meta_block_fast!(
|
||||
input: args.input, begin: metablock_start, end: base_ip,
|
||||
storage_ix_start: mlen_storage_ix ~mod- 3, s: args.s)
|
||||
in_size ~mod-= base_ip ~mod- in_off
|
||||
in_off = base_ip
|
||||
next_emit = in_off
|
||||
goto_next_block = true
|
||||
break.outer
|
||||
} else {
|
||||
this.emit_long_insert_len_fast!(insertlen: insert, s: args.s)
|
||||
}
|
||||
this.emit_literals!(input: args.input, from: next_emit, len: insert, s: args.s)
|
||||
if distance == last_distance {
|
||||
this.emit_cmd_fast!(code: 64, s: args.s)
|
||||
} else {
|
||||
this.emit_distance_fast!(distance: distance, s: args.s)
|
||||
last_distance = distance
|
||||
}
|
||||
this.emit_copy_len_last_distance_fast!(copylen: matched, s: args.s)
|
||||
|
||||
next_emit = ip
|
||||
if ip >= ip_limit {
|
||||
break.outer
|
||||
}
|
||||
candidate = this.update_table_fast!(input: args.input, ip: ip, shift: shift)
|
||||
|
||||
while.inner this.is_match_fast(input: args.input, i1: ip, i2: candidate) {
|
||||
// We have a 5-byte match at ip, and no need to emit any
|
||||
// literal bytes prior to ip.
|
||||
base_ip = ip
|
||||
matched = 5 ~mod+ this.find_match_length_with_limit(
|
||||
s: args.input, i1: candidate ~mod+ 5, i2: ip ~mod+ 5, limit: (ip_end ~mod- ip) ~mod- 5)
|
||||
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
|
||||
break.inner
|
||||
}
|
||||
ip ~mod+= matched
|
||||
last_distance = base_ip ~mod- candidate
|
||||
this.emit_copy_len_fast!(copylen: matched, s: args.s)
|
||||
this.emit_distance_fast!(distance: last_distance, s: args.s)
|
||||
|
||||
next_emit = ip
|
||||
if ip >= ip_limit {
|
||||
break.outer
|
||||
}
|
||||
candidate = this.update_table_fast!(input: args.input, ip: ip, shift: shift)
|
||||
}.inner
|
||||
|
||||
ip ~mod+= 1
|
||||
next_hash = this.hash_fast(input: args.input, i: ip, shift: shift)
|
||||
}.outer
|
||||
}
|
||||
|
||||
if not goto_next_block {
|
||||
// emit_remainder:
|
||||
in_off ~mod+= block_size
|
||||
in_size ~mod-= block_size
|
||||
block_size = in_size.min(no_more_than: 0x1_0000) // kMergeBlockSize.
|
||||
|
||||
// Decide if we want to continue this meta-block instead of
|
||||
// emitting the last insert-only command.
|
||||
merge = false
|
||||
if (in_size > 0) and ((total_block_size ~mod+ block_size) <= 0x10_0000) {
|
||||
merge = this.should_merge_block!(input: args.input, from: in_off, len: block_size)
|
||||
}
|
||||
if merge {
|
||||
// Update the size of the current meta-block and continue
|
||||
// emitting commands. We can do this because the current size
|
||||
// and the new size both have 5 nibbles.
|
||||
total_block_size ~mod+= block_size
|
||||
this.update_bits!(
|
||||
n_bits: 20,
|
||||
bits: ((total_block_size ~mod- 1) & 0xFFFF_FFFF) as base.u32,
|
||||
pos: mlen_storage_ix,
|
||||
s: args.s)
|
||||
continue.emit_commands
|
||||
}
|
||||
|
||||
// Emit the remaining bytes as literals.
|
||||
if next_emit < ip_end {
|
||||
insert = ip_end ~mod- next_emit
|
||||
if insert < 6210 {
|
||||
this.emit_insert_len_fast!(insertlen: insert, s: args.s)
|
||||
this.emit_literals!(input: args.input, from: next_emit, len: insert, s: args.s)
|
||||
} else if this.should_use_uncompressed_mode(
|
||||
metablock_start: metablock_start, next_emit: next_emit,
|
||||
insertlen: insert, literal_ratio: literal_ratio) {
|
||||
this.emit_uncompressed_meta_block_fast!(
|
||||
input: args.input, begin: metablock_start, end: ip_end,
|
||||
storage_ix_start: mlen_storage_ix ~mod- 3, s: args.s)
|
||||
} else {
|
||||
this.emit_long_insert_len_fast!(insertlen: insert, s: args.s)
|
||||
this.emit_literals!(input: args.input, from: next_emit, len: insert, s: args.s)
|
||||
}
|
||||
}
|
||||
next_emit = ip_end
|
||||
}
|
||||
|
||||
// next_block:
|
||||
// If we have more data, write a new meta-block header and prefix
|
||||
// codes and then continue emitting commands.
|
||||
if in_size > 0 {
|
||||
metablock_start = in_off
|
||||
block_size = in_size.min(no_more_than: 98304)
|
||||
total_block_size = block_size
|
||||
mlen_storage_ix = this.bw_ix ~mod+ 3
|
||||
this.store_meta_block_header!(len: block_size, is_uncompressed: false, s: args.s)
|
||||
// No block splits, no contexts.
|
||||
this.write_bits!(n_bits: 13, bits: 0, s: args.s)
|
||||
literal_ratio = this.build_and_store_literal_prefix_code!(
|
||||
input: args.input, from: in_off, input_size: block_size, s: args.s)
|
||||
this.build_and_store_command_prefix_code_fast!(s: args.s)
|
||||
continue.emit_commands
|
||||
}
|
||||
break.emit_commands
|
||||
}.emit_commands
|
||||
|
||||
if not args.is_last {
|
||||
// If this is not the last block, update the command and distance
|
||||
// prefix codes for the next block and store the compressed forms.
|
||||
this.cmd_code[0] = 0
|
||||
saved_ix = this.bw_ix
|
||||
this.bw_ix = 0
|
||||
this.build_and_store_command_prefix_code_fast!(s: this.cmd_code[..])
|
||||
this.cmd_code_numbits = this.bw_ix
|
||||
this.bw_ix = saved_ix
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,566 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// This file mirrors compress_fragment_two_pass.c, the quality 1 (two-pass)
|
||||
// encoder. Like encode_fragment_fast.wuffs, the C code's pointers are
|
||||
// base.u64 offsets from the start of the fragment (the C code's base_ip) and
|
||||
// a last_distance of 0 means the C code's -1.
|
||||
//
|
||||
// The C code's commands and literals pointers (into command_buf and
|
||||
// literal_buf) are the num_commands and num_literals fields.
|
||||
|
||||
// Mirrors Hash in compress_fragment_two_pass.c.
|
||||
pri func encoder.hash_two_pass(input: roslice base.u8, i: base.u64, shift: base.u32, length: base.u32) base.u32 {
|
||||
var h : base.u64
|
||||
|
||||
h = (this.load64le(s: args.input, i: args.i) ~mod<< ((64 ~mod- (8 ~mod* args.length)) & 63)) ~mod* HASH_MUL32
|
||||
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
|
||||
}
|
||||
|
||||
// Mirrors HashBytesAtOffset in compress_fragment_two_pass.c.
|
||||
pri func encoder.hash_bytes_at_offset_two_pass(v: base.u64, offset: base.u32, shift: base.u32, length: base.u32) base.u32 {
|
||||
var h : base.u64
|
||||
|
||||
h = ((args.v >> ((8 ~mod* args.offset) & 63)) ~mod<< ((64 ~mod- (8 ~mod* args.length)) & 63)) ~mod* HASH_MUL32
|
||||
return ((h >> (args.shift & 63)) & 0xFFFF_FFFF) as base.u32
|
||||
}
|
||||
|
||||
// Mirrors IsMatch in compress_fragment_two_pass.c.
|
||||
pri func encoder.is_match_two_pass(input: roslice base.u8, i1: base.u64, i2: base.u64, length: base.u32) base.bool {
|
||||
var q1 : roslice base.u8
|
||||
var q2 : roslice base.u8
|
||||
|
||||
if (args.i1 >= args.input.length()) or (args.i2 >= args.input.length()) {
|
||||
return false
|
||||
}
|
||||
if this.load32le(s: args.input, i: args.i1) <> this.load32le(s: args.input, i: args.i2) {
|
||||
return false
|
||||
}
|
||||
if args.length == 4 {
|
||||
return true
|
||||
}
|
||||
q1 = args.input[args.i1 ..]
|
||||
q2 = args.input[args.i2 ..]
|
||||
if (q1.length() >= 6) and (q2.length() >= 6) {
|
||||
return (q1[4] == q2[4]) and (q1[5] == q2[5])
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Mirrors BuildAndStoreCommandPrefixCode in compress_fragment_two_pass.c.
|
||||
// Note that its symbol order differs from the one in compress_fragment.c.
|
||||
pri func encoder.build_and_store_command_prefix_code_two_pass!(s: slice base.u8) {
|
||||
var i : base.u32
|
||||
|
||||
this.tmp_depth[.. 704].bulk_memset!(byte_value: 0)
|
||||
this.create_huffman_tree!(src: 0, off: 0, length: 64, tree_limit: 15, depth: this.cmd_depth[.. 64])
|
||||
this.create_huffman_tree!(src: 0, off: 64, length: 64, tree_limit: 14, depth: this.cmd_depth[64 ..])
|
||||
// We have to jump through a few hoops here in order to compute the
|
||||
// command bits because the symbols are in a different order than in the
|
||||
// full alphabet. This looks complicated, but having the symbols in this
|
||||
// order in the command bits saves a few branches in the Emit* functions.
|
||||
this.tmp_depth[0 .. 24].copy_from_slice!(s: this.cmd_depth[24 .. 48])
|
||||
this.tmp_depth[24 .. 32].copy_from_slice!(s: this.cmd_depth[0 .. 8])
|
||||
this.tmp_depth[32 .. 40].copy_from_slice!(s: this.cmd_depth[48 .. 56])
|
||||
this.tmp_depth[40 .. 48].copy_from_slice!(s: this.cmd_depth[8 .. 16])
|
||||
this.tmp_depth[48 .. 56].copy_from_slice!(s: this.cmd_depth[56 .. 64])
|
||||
this.tmp_depth[56 .. 64].copy_from_slice!(s: this.cmd_depth[16 .. 24])
|
||||
this.convert_bit_depths_to_symbols!(depth: this.tmp_depth[..], len: 64, dst: 0)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 0, src: 24, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 8, src: 40, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 16, src: 56, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 24, src: 0, n: 24)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 48, src: 32, n: 8)
|
||||
this.copy_tmp_bits_to_cmd_bits!(dst: 56, src: 48, n: 8)
|
||||
this.convert_bit_depths_to_symbols!(depth: this.cmd_depth[64 ..], len: 64, dst: 1)
|
||||
|
||||
// Create the bit length array for the full command alphabet.
|
||||
this.tmp_depth[.. 64].bulk_memset!(byte_value: 0)
|
||||
this.tmp_depth[0 .. 8].copy_from_slice!(s: this.cmd_depth[24 .. 32])
|
||||
this.tmp_depth[64 .. 72].copy_from_slice!(s: this.cmd_depth[32 .. 40])
|
||||
this.tmp_depth[128 .. 136].copy_from_slice!(s: this.cmd_depth[40 .. 48])
|
||||
this.tmp_depth[192 .. 200].copy_from_slice!(s: this.cmd_depth[48 .. 56])
|
||||
this.tmp_depth[384 .. 392].copy_from_slice!(s: this.cmd_depth[56 .. 64])
|
||||
i = 0
|
||||
while i < 8 {
|
||||
this.tmp_depth[128 + (8 * i)] = this.cmd_depth[i]
|
||||
this.tmp_depth[256 + (8 * i)] = this.cmd_depth[8 + i]
|
||||
this.tmp_depth[448 + (8 * i)] = this.cmd_depth[16 + i]
|
||||
i ~mod+= 1
|
||||
}
|
||||
this.store_huffman_tree!(depths: this.tmp_depth[..], num: 704, s: args.s)
|
||||
this.store_huffman_tree!(depths: this.cmd_depth[64 ..], num: 64, s: args.s)
|
||||
}
|
||||
|
||||
// push_command appends to command_buf.
|
||||
pri func encoder.push_command!(c: base.u32) {
|
||||
this.command_buf[this.num_commands & 0x1_FFFF] = args.c
|
||||
this.num_commands ~mod+= 1
|
||||
}
|
||||
|
||||
// Mirrors EmitInsertLen in compress_fragment_two_pass.c.
|
||||
pri func encoder.emit_insert_len_two_pass!(insertlen: base.u32) {
|
||||
var tail : base.u32
|
||||
var nbits : base.u32
|
||||
var prefix : base.u32
|
||||
var extra : base.u32
|
||||
|
||||
if args.insertlen < 6 {
|
||||
this.push_command!(c: args.insertlen)
|
||||
} else if args.insertlen < 130 {
|
||||
tail = args.insertlen - 2
|
||||
nbits = this.log2_floor_non_zero(n: tail as base.u64) ~mod- 1
|
||||
prefix = tail >> (nbits & 31)
|
||||
extra = tail ~mod- (prefix ~mod<< (nbits & 31))
|
||||
this.push_command!(c: ((((nbits ~mod<< 1) ~mod+ prefix) ~mod+ 2) | (extra ~mod<< 8)))
|
||||
} else if args.insertlen < 2114 {
|
||||
tail = args.insertlen - 66
|
||||
nbits = this.log2_floor_non_zero(n: tail as base.u64)
|
||||
extra = tail ~mod- ((1 as base.u32) << nbits)
|
||||
this.push_command!(c: ((nbits + 10) | (extra ~mod<< 8)))
|
||||
} else if args.insertlen < 6210 {
|
||||
this.push_command!(c: (21 | ((args.insertlen - 2114) ~mod<< 8)))
|
||||
} else if args.insertlen < 22594 {
|
||||
this.push_command!(c: (22 | ((args.insertlen - 6210) ~mod<< 8)))
|
||||
} else {
|
||||
this.push_command!(c: (23 | ((args.insertlen - 22594) ~mod<< 8)))
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors EmitCopyLen in compress_fragment_two_pass.c.
|
||||
pri func encoder.emit_copy_len_two_pass!(copylen: base.u64) {
|
||||
var tail : base.u64
|
||||
var nbits : base.u32
|
||||
var prefix : base.u64
|
||||
var extra : base.u64
|
||||
|
||||
if args.copylen < 10 {
|
||||
this.push_command!(c: ((args.copylen + 38) & 0xFFFF_FFFF) as base.u32)
|
||||
} else if args.copylen < 134 {
|
||||
tail = args.copylen - 6
|
||||
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
|
||||
prefix = tail >> (nbits & 63)
|
||||
extra = tail ~mod- (prefix ~mod<< (nbits & 63))
|
||||
this.push_command!(c: ((((((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 44) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
|
||||
} else if args.copylen < 2118 {
|
||||
tail = args.copylen - 70
|
||||
nbits = this.log2_floor_non_zero(n: tail)
|
||||
extra = tail ~mod- ((1 as base.u64) << nbits)
|
||||
this.push_command!(c: ((((nbits as base.u64) + 52) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
|
||||
} else {
|
||||
this.push_command!(c: ((63 | ((args.copylen ~mod- 2118) ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors EmitCopyLenLastDistance in compress_fragment_two_pass.c.
|
||||
pri func encoder.emit_copy_len_last_distance_two_pass!(copylen: base.u64) {
|
||||
var tail : base.u64
|
||||
var nbits : base.u32
|
||||
var prefix : base.u64
|
||||
var extra : base.u64
|
||||
|
||||
if args.copylen < 12 {
|
||||
this.push_command!(c: ((args.copylen + 20) & 0xFFFF_FFFF) as base.u32)
|
||||
} else if args.copylen < 72 {
|
||||
tail = args.copylen - 8
|
||||
nbits = this.log2_floor_non_zero(n: tail) ~mod- 1
|
||||
prefix = tail >> (nbits & 63)
|
||||
extra = tail ~mod- (prefix ~mod<< (nbits & 63))
|
||||
this.push_command!(c: ((((((nbits as base.u64) ~mod<< 1) ~mod+ prefix) ~mod+ 28) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
|
||||
} else if args.copylen < 136 {
|
||||
tail = args.copylen - 8
|
||||
this.push_command!(c: (((tail >> 5) + 54) | ((tail & 31) << 8)) as base.u32)
|
||||
this.push_command!(c: 64)
|
||||
} else if args.copylen < 2120 {
|
||||
tail = args.copylen - 72
|
||||
nbits = this.log2_floor_non_zero(n: tail)
|
||||
extra = tail ~mod- ((1 as base.u64) << nbits)
|
||||
this.push_command!(c: ((((nbits as base.u64) + 52) | (extra ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
|
||||
this.push_command!(c: 64)
|
||||
} else {
|
||||
this.push_command!(c: ((63 | ((args.copylen ~mod- 2120) ~mod<< 8)) & 0xFFFF_FFFF) as base.u32)
|
||||
this.push_command!(c: 64)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors EmitDistance in compress_fragment_two_pass.c.
|
||||
pri func encoder.emit_distance_two_pass!(distance: base.u32) {
|
||||
var d : base.u32
|
||||
var nbits : base.u32
|
||||
var prefix : base.u32
|
||||
var offset : base.u32
|
||||
var distcode : base.u32
|
||||
var extra : base.u32
|
||||
|
||||
d = args.distance ~mod+ 3
|
||||
nbits = this.log2_floor_non_zero(n: d as base.u64) ~mod- 1
|
||||
prefix = (d >> (nbits & 31)) & 1
|
||||
offset = (2 + prefix) ~mod<< (nbits & 31)
|
||||
distcode = ((2 ~mod* (nbits ~mod- 1)) ~mod+ prefix) ~mod+ 80
|
||||
extra = d ~mod- offset
|
||||
this.push_command!(c: distcode | (extra ~mod<< 8))
|
||||
}
|
||||
|
||||
// push_literals appends args.input[args.from .. args.from + args.n] to
|
||||
// literal_buf, the C code's "memcpy(*literals, next_emit, insert)".
|
||||
pri func encoder.push_literals!(input: roslice base.u8, from: base.u64, n: base.u64) {
|
||||
var j : base.u64
|
||||
|
||||
j = 0
|
||||
while j < args.n {
|
||||
if (args.from ~mod+ j) >= args.input.length() {
|
||||
break
|
||||
}
|
||||
this.literal_buf[this.num_literals & 0x1_FFFF] = args.input[args.from ~mod+ j]
|
||||
this.num_literals ~mod+= 1
|
||||
j ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
// update_table_two_pass updates the hash table with the hashes of some
|
||||
// positions within the last copy (ending at ip), and returns the next
|
||||
// candidate. The C code has two copies of this block, which differ (for
|
||||
// min_match == 4) in the third hash's offset: third_offset is 0 for the first
|
||||
// copy and 2 for the second copy.
|
||||
pri func encoder.update_table_two_pass!(input: roslice base.u8, ip: base.u64, shift: base.u32, min_match: base.u32, third_offset: base.u32) base.u64 {
|
||||
var input_bytes : base.u64
|
||||
var prev_hash : base.u32
|
||||
var cur_hash : base.u32
|
||||
var candidate : base.u64
|
||||
var mm : base.u32
|
||||
|
||||
mm = args.min_match
|
||||
if mm == 4 {
|
||||
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 3)
|
||||
cur_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 3, shift: args.shift, length: mm)
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 0, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 3) & 0xFFFF_FFFF) as base.u32
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 1, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 2) & 0xFFFF_FFFF) as base.u32
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: args.third_offset, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 1) & 0xFFFF_FFFF) as base.u32
|
||||
} else {
|
||||
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 5)
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 0, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 5) & 0xFFFF_FFFF) as base.u32
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 1, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 4) & 0xFFFF_FFFF) as base.u32
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 2, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 3) & 0xFFFF_FFFF) as base.u32
|
||||
input_bytes = this.load64le(s: args.input, i: args.ip ~mod- 2)
|
||||
cur_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 2, shift: args.shift, length: mm)
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 0, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 2) & 0xFFFF_FFFF) as base.u32
|
||||
prev_hash = this.hash_bytes_at_offset_two_pass(v: input_bytes, offset: 1, shift: args.shift, length: mm)
|
||||
this.hash_table[prev_hash & 0x1_FFFF] = ((args.ip ~mod- 1) & 0xFFFF_FFFF) as base.u32
|
||||
}
|
||||
candidate = this.hash_table[cur_hash & 0x1_FFFF] as base.u64
|
||||
this.hash_table[cur_hash & 0x1_FFFF] = (args.ip & 0xFFFF_FFFF) as base.u32
|
||||
return candidate
|
||||
}
|
||||
|
||||
// Mirrors CreateCommands in compress_fragment_two_pass.c. The block is the
|
||||
// block_size bytes at args.input[args.from ..]. input_size is the number of
|
||||
// bytes remaining in the fragment, from args.from.
|
||||
pri func encoder.create_commands!(input: roslice base.u8, from: base.u64, block_size: base.u64, input_size: base.u64, table_bits: base.u32, min_match: base.u32) {
|
||||
var ip : base.u64
|
||||
var shift : base.u32
|
||||
var ip_end : base.u64
|
||||
var next_emit : base.u64
|
||||
var last_distance : base.u64
|
||||
var len_limit : base.u64
|
||||
var ip_limit : base.u64
|
||||
var next_hash : base.u32
|
||||
var hash : base.u32
|
||||
var skip : base.u32
|
||||
var next_ip : base.u64
|
||||
var candidate : base.u64
|
||||
var base_ip : base.u64
|
||||
var matched : base.u64
|
||||
var distance : base.u64
|
||||
var insert : base.u64
|
||||
var mm : base.u64
|
||||
|
||||
this.num_commands = 0
|
||||
this.num_literals = 0
|
||||
mm = args.min_match as base.u64
|
||||
ip = args.from
|
||||
shift = 64 ~mod- args.table_bits
|
||||
ip_end = args.from ~mod+ args.block_size
|
||||
next_emit = args.from
|
||||
|
||||
if args.block_size >= 16 {
|
||||
// For the last block, we need to keep a 16 bytes margin so that we
|
||||
// can be sure that all distances are at most window size - 16. For
|
||||
// all other blocks, we only need to keep a margin of 5 bytes so that
|
||||
// we don't go over the block size with a copy.
|
||||
len_limit = args.block_size ~mod- mm
|
||||
len_limit = len_limit.min(no_more_than: args.input_size ~mod- 16)
|
||||
ip_limit = args.from ~mod+ len_limit
|
||||
|
||||
ip ~mod+= 1
|
||||
next_hash = this.hash_two_pass(input: args.input, i: ip, shift: shift, length: args.min_match)
|
||||
while.outer true {
|
||||
// Step 1: Scan forward in the input looking for a 6-byte-long
|
||||
// match. If we get close to exhausting the input then goto
|
||||
// emit_remainder.
|
||||
skip = 32
|
||||
next_ip = ip
|
||||
|
||||
while.trawl true {
|
||||
hash = next_hash
|
||||
ip = next_ip
|
||||
next_ip = ip ~mod+ ((skip >> 5) as base.u64)
|
||||
skip ~mod+= 1
|
||||
if next_ip > ip_limit {
|
||||
break.outer
|
||||
}
|
||||
next_hash = this.hash_two_pass(input: args.input, i: next_ip, shift: shift, length: args.min_match)
|
||||
if (last_distance > 0) and (last_distance <= ip) {
|
||||
candidate = ip ~mod- last_distance
|
||||
if this.is_match_two_pass(input: args.input, i1: ip, i2: candidate, length: args.min_match) {
|
||||
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
|
||||
// Check copy distance. If candidate is not feasible,
|
||||
// continue search.
|
||||
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
|
||||
continue.trawl
|
||||
}
|
||||
break.trawl
|
||||
}
|
||||
}
|
||||
candidate = this.hash_table[hash & 0x1_FFFF] as base.u64
|
||||
this.hash_table[hash & 0x1_FFFF] = (ip & 0xFFFF_FFFF) as base.u32
|
||||
if this.is_match_two_pass(input: args.input, i1: ip, i2: candidate, length: args.min_match) {
|
||||
if (ip ~mod- candidate) > FRAGMENT_MAX_DISTANCE {
|
||||
continue.trawl
|
||||
}
|
||||
break.trawl
|
||||
}
|
||||
}.trawl
|
||||
|
||||
// Step 2: Emit the found match together with the literal bytes
|
||||
// from "next_emit", and then see if we can find a next match
|
||||
// immediately afterwards. Repeat until we find no match for the
|
||||
// input without emitting some literal bytes.
|
||||
base_ip = ip
|
||||
matched = mm ~mod+ this.find_match_length_with_limit(
|
||||
s: args.input, i1: candidate ~mod+ mm, i2: ip ~mod+ mm, limit: (ip_end ~mod- ip) ~mod- mm)
|
||||
distance = base_ip ~mod- candidate
|
||||
insert = base_ip ~mod- next_emit
|
||||
ip ~mod+= matched
|
||||
this.emit_insert_len_two_pass!(insertlen: (insert & 0xFFFF_FFFF) as base.u32)
|
||||
this.push_literals!(input: args.input, from: next_emit, n: insert)
|
||||
if distance == last_distance {
|
||||
this.push_command!(c: 64)
|
||||
} else {
|
||||
this.emit_distance_two_pass!(distance: (distance & 0xFFFF_FFFF) as base.u32)
|
||||
last_distance = distance
|
||||
}
|
||||
this.emit_copy_len_last_distance_two_pass!(copylen: matched)
|
||||
|
||||
next_emit = ip
|
||||
if ip >= ip_limit {
|
||||
break.outer
|
||||
}
|
||||
candidate = this.update_table_two_pass!(
|
||||
input: args.input, ip: ip, shift: shift, min_match: args.min_match, third_offset: 0)
|
||||
|
||||
while (ip ~mod- candidate) <= FRAGMENT_MAX_DISTANCE {
|
||||
if not this.is_match_two_pass(input: args.input, i1: ip, i2: candidate, length: args.min_match) {
|
||||
break
|
||||
}
|
||||
// We have a 6-byte match at ip, and no need to emit any
|
||||
// literal bytes prior to ip.
|
||||
base_ip = ip
|
||||
matched = mm ~mod+ this.find_match_length_with_limit(
|
||||
s: args.input, i1: candidate ~mod+ mm, i2: ip ~mod+ mm, limit: (ip_end ~mod- ip) ~mod- mm)
|
||||
ip ~mod+= matched
|
||||
last_distance = base_ip ~mod- candidate
|
||||
this.emit_copy_len_two_pass!(copylen: matched)
|
||||
this.emit_distance_two_pass!(distance: (last_distance & 0xFFFF_FFFF) as base.u32)
|
||||
|
||||
next_emit = ip
|
||||
if ip >= ip_limit {
|
||||
break.outer
|
||||
}
|
||||
candidate = this.update_table_two_pass!(
|
||||
input: args.input, ip: ip, shift: shift, min_match: args.min_match, third_offset: 2)
|
||||
}
|
||||
|
||||
ip ~mod+= 1
|
||||
next_hash = this.hash_two_pass(input: args.input, i: ip, shift: shift, length: args.min_match)
|
||||
}.outer
|
||||
}
|
||||
|
||||
// emit_remainder:
|
||||
// Emit the remaining bytes as literals.
|
||||
if next_emit < ip_end {
|
||||
insert = ip_end ~mod- next_emit
|
||||
this.emit_insert_len_two_pass!(insertlen: (insert & 0xFFFF_FFFF) as base.u32)
|
||||
this.push_literals!(input: args.input, from: next_emit, n: insert)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors StoreCommands in compress_fragment_two_pass.c.
|
||||
pri func encoder.store_commands!(s: slice base.u8) {
|
||||
var i : base.u32
|
||||
var j : base.u32
|
||||
var cmd : base.u32
|
||||
var code : base.u32
|
||||
var extra : base.u32
|
||||
var insert : base.u32
|
||||
var lit_i : base.u32
|
||||
var lit : base.u8
|
||||
|
||||
this.histogram[.. 256].bulk_memset!(byte_value: 0)
|
||||
this.cmd_depth[.. 128].bulk_memset!(byte_value: 0)
|
||||
this.cmd_bits[.. 128].bulk_memset!(byte_value: 0)
|
||||
this.cmd_histo[.. 128].bulk_memset!(byte_value: 0)
|
||||
i = 0
|
||||
while i < this.num_literals {
|
||||
this.histogram[this.literal_buf[i & 0x1_FFFF]] ~mod+= 1
|
||||
i ~mod+= 1
|
||||
}
|
||||
this.build_and_store_huffman_tree_fast!(histogram_total: this.num_literals as base.u64, max_bits: 8, s: args.s)
|
||||
|
||||
i = 0
|
||||
while i < this.num_commands {
|
||||
code = this.command_buf[i & 0x1_FFFF] & 0xFF
|
||||
this.cmd_histo[code & 127] ~mod+= 1
|
||||
i ~mod+= 1
|
||||
}
|
||||
this.cmd_histo[1] ~mod+= 1
|
||||
this.cmd_histo[2] ~mod+= 1
|
||||
this.cmd_histo[64] ~mod+= 1
|
||||
this.cmd_histo[84] ~mod+= 1
|
||||
this.build_and_store_command_prefix_code_two_pass!(s: args.s)
|
||||
|
||||
i = 0
|
||||
while i < this.num_commands {
|
||||
cmd = this.command_buf[i & 0x1_FFFF]
|
||||
code = cmd & 0x7F
|
||||
extra = cmd >> 8
|
||||
this.write_bits!(n_bits: this.cmd_depth[code] as base.u32, bits: this.cmd_bits[code] as base.u64, s: args.s)
|
||||
this.write_bits!(n_bits: TWO_PASS_NUM_EXTRA_BITS[code], bits: extra as base.u64, s: args.s)
|
||||
if code < 24 {
|
||||
insert = TWO_PASS_INSERT_OFFSET[code] ~mod+ extra
|
||||
j = 0
|
||||
while j < insert {
|
||||
lit = this.literal_buf[lit_i & 0x1_FFFF]
|
||||
this.write_bits!(n_bits: this.lit_depth[lit] as base.u32, bits: this.lit_bits[lit] as base.u64, s: args.s)
|
||||
lit_i ~mod+= 1
|
||||
j ~mod+= 1
|
||||
}
|
||||
}
|
||||
i ~mod+= 1
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors ShouldCompress in compress_fragment_two_pass.c, for the input_size
|
||||
// bytes at args.input[args.from ..].
|
||||
pri func encoder.should_compress_two_pass!(input: roslice base.u8, from: base.u64, input_size: base.u64, num_literals: base.u64) base.bool {
|
||||
var corpus_size : base.u64
|
||||
var max_total_bit_cost : base.u64
|
||||
var i : base.u64
|
||||
|
||||
corpus_size = this.sf.f64_from_u64(a: args.input_size)
|
||||
// MIN_RATIO is 0.98 (0x3FEF_5C28_F5C2_8F5C).
|
||||
if this.sf.f64_lt(
|
||||
a: this.sf.f64_from_u64(a: args.num_literals),
|
||||
b: this.sf.f64_mul(a: 0x3FEF_5C28_F5C2_8F5C, b: corpus_size)) <> 0 {
|
||||
return true
|
||||
}
|
||||
// max_total_bit_cost = corpus_size * 8 * MIN_RATIO / SAMPLE_RATE, where
|
||||
// SAMPLE_RATE is 43 and 8.0 is 0x4020_0000_0000_0000.
|
||||
max_total_bit_cost = this.sf.f64_div(
|
||||
a: this.sf.f64_mul(
|
||||
a: this.sf.f64_mul(a: corpus_size, b: 0x4020_0000_0000_0000),
|
||||
b: 0x3FEF_5C28_F5C2_8F5C),
|
||||
b: 0x4045_8000_0000_0000)
|
||||
this.histogram[.. 256].bulk_memset!(byte_value: 0)
|
||||
i = 0
|
||||
while i < args.input_size {
|
||||
if (args.from ~mod+ i) < args.input.length() {
|
||||
this.histogram[args.input[args.from ~mod+ i]] ~mod+= 1
|
||||
}
|
||||
i ~mod+= 43
|
||||
}
|
||||
return this.sf.f64_lt(a: this.bits_entropy_of_histogram(), b: max_total_bit_cost) <> 0
|
||||
}
|
||||
|
||||
// Mirrors EmitUncompressedMetaBlock in compress_fragment_two_pass.c (which,
|
||||
// unlike the one in compress_fragment.c, does not rewind).
|
||||
pri func encoder.emit_uncompressed_meta_block_two_pass!(input: roslice base.u8, from: base.u64, input_size: base.u64, s: slice base.u8) {
|
||||
var p : base.u64
|
||||
|
||||
this.store_meta_block_header!(len: args.input_size, is_uncompressed: true, s: args.s)
|
||||
this.align_to_byte!()
|
||||
this.copy_bytes!(input: args.input, begin: args.from, end: args.from ~mod+ args.input_size, s: args.s)
|
||||
this.bw_ix ~mod+= args.input_size ~mod<< 3
|
||||
p = this.bw_ix >> 3
|
||||
if p < args.s.length() {
|
||||
args.s[p] = 0
|
||||
}
|
||||
}
|
||||
|
||||
// Mirrors BrotliCompressFragmentTwoPass and BrotliCompressFragmentTwoPassImpl
|
||||
// in compress_fragment_two_pass.c.
|
||||
pri func encoder.compress_fragment_two_pass!(input: roslice base.u8, is_last: base.bool, table_bits: base.u32, s: slice base.u8) {
|
||||
var initial_ix : base.u64
|
||||
var min_match : base.u32
|
||||
var from : base.u64
|
||||
var input_size : base.u64
|
||||
var block_size : base.u64
|
||||
var compress : base.bool
|
||||
|
||||
initial_ix = this.bw_ix
|
||||
min_match = 6
|
||||
if args.table_bits <= 15 {
|
||||
min_match = 4
|
||||
}
|
||||
|
||||
input_size = args.input.length()
|
||||
while input_size > 0 {
|
||||
block_size = input_size.min(no_more_than: TWO_PASS_BLOCK_SIZE as base.u64)
|
||||
this.create_commands!(
|
||||
input: args.input, from: from, block_size: block_size, input_size: input_size,
|
||||
table_bits: args.table_bits, min_match: min_match)
|
||||
compress = this.should_compress_two_pass!(
|
||||
input: args.input, from: from, input_size: block_size, num_literals: this.num_literals as base.u64)
|
||||
if compress {
|
||||
this.store_meta_block_header!(len: block_size, is_uncompressed: false, s: args.s)
|
||||
// No block splits, no contexts.
|
||||
this.write_bits!(n_bits: 13, bits: 0, s: args.s)
|
||||
this.store_commands!(s: args.s)
|
||||
} else {
|
||||
// Since we did not find many backward references and the entropy
|
||||
// of the data is close to 8 bits, we can simply emit an
|
||||
// uncompressed block. This makes compression speed of
|
||||
// uncompressible data about 3x faster.
|
||||
this.emit_uncompressed_meta_block_two_pass!(input: args.input, from: from, input_size: block_size, s: args.s)
|
||||
}
|
||||
from ~mod+= block_size
|
||||
input_size ~mod-= block_size
|
||||
}
|
||||
|
||||
// If output is larger than single uncompressed block, rewrite it.
|
||||
if (this.bw_ix ~mod- initial_ix) > (31 ~mod+ (args.input.length() ~mod<< 3)) {
|
||||
this.rewind_bit_position!(new_ix: initial_ix, s: args.s)
|
||||
this.emit_uncompressed_meta_block_two_pass!(input: args.input, from: 0, input_size: args.input.length(), s: args.s)
|
||||
}
|
||||
|
||||
if args.is_last {
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // islast
|
||||
this.write_bits!(n_bits: 1, bits: 1, s: args.s) // isempty
|
||||
this.align_to_byte!()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// --------
|
||||
|
||||
// Quirks are discussed in (/doc/note/quirks.md). QUIRKS_BASE is defined in
|
||||
// decode_quirks.wuffs.
|
||||
//
|
||||
// The encoder's quirks mirror the C code's BrotliEncoderParameter values, with
|
||||
// the value semantics of BrotliEncoderSetParameter. The encoder also accepts
|
||||
// the QUIRK_LARGE_WINDOW key (defined in decode_quirks.wuffs) for the
|
||||
// BROTLI_PARAM_LARGE_WINDOW parameter: zero or non-zero (large-window brotli
|
||||
// is not used at qualities 0 to 2).
|
||||
//
|
||||
// For all of the encoder's quirks, the (u64) value must fit in a u32 (like
|
||||
// the C function's uint32_t argument), otherwise set_quirk returns
|
||||
// base."#bad argument". Like the C code, the parameters can only be set
|
||||
// before the first transform_io call, otherwise set_quirk returns
|
||||
// base."#bad call sequence".
|
||||
|
||||
// --------
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_QUALITY
|
||||
// parameter: 0 to 11. The default is 11. Like the C code, out of range values
|
||||
// are clamped (and a value with its high bit set is negative, so it is
|
||||
// clamped to 0). Only qualities 0 and 1 are currently supported: transform_io
|
||||
// returns "#unsupported encoder quality" for the others.
|
||||
//
|
||||
// It is not named QUIRK_QUALITY because the base package already has a
|
||||
// base.QUIRK_QUALITY key (with different value semantics).
|
||||
pub const QUIRK_ENCODER_QUALITY : base.u32 = 0x2AB7_9000 | 0x01
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_LGWIN parameter:
|
||||
// the log2 of the window size, 10 to 24 (or up to 30 for large-window brotli,
|
||||
// at qualities 3 and above). The default is 22. Out of range values are
|
||||
// clamped.
|
||||
pub const QUIRK_LGWIN : base.u32 = 0x2AB7_9000 | 0x02
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_LGBLOCK
|
||||
// parameter: the log2 of the input block size. 0 (the default) means to
|
||||
// choose it automatically. It has no effect at qualities 0 and 1.
|
||||
pub const QUIRK_LGBLOCK : base.u32 = 0x2AB7_9000 | 0x03
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_MODE parameter:
|
||||
// 0 (BROTLI_MODE_GENERIC, the default), 1 (BROTLI_MODE_TEXT) or 2
|
||||
// (BROTLI_MODE_FONT). Like the C code, other values are accepted (and have
|
||||
// the same effect as BROTLI_MODE_GENERIC). It has no effect at qualities 0
|
||||
// and 1.
|
||||
pub const QUIRK_MODE : base.u32 = 0x2AB7_9000 | 0x04
|
||||
|
||||
// When this quirk is set, it sets the encoder's
|
||||
// BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING parameter: 0 or 1. Other
|
||||
// values are rejected with base."#bad argument". It has no effect at
|
||||
// qualities 0 and 1.
|
||||
pub const QUIRK_DISABLE_LITERAL_CONTEXT_MODELING : base.u32 = 0x2AB7_9000 | 0x05
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_SIZE_HINT
|
||||
// parameter: the estimated total input size. It has no effect at qualities 0
|
||||
// and 1.
|
||||
pub const QUIRK_SIZE_HINT : base.u32 = 0x2AB7_9000 | 0x06
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_NPOSTFIX
|
||||
// parameter. It has no effect at qualities 0 and 1.
|
||||
pub const QUIRK_NPOSTFIX : base.u32 = 0x2AB7_9000 | 0x07
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_NDIRECT
|
||||
// parameter. It has no effect at qualities 0 and 1.
|
||||
pub const QUIRK_NDIRECT : base.u32 = 0x2AB7_9000 | 0x08
|
||||
|
||||
// When this quirk is set, it sets the encoder's BROTLI_PARAM_STREAM_OFFSET
|
||||
// parameter: 0 to (1 << 30). Larger values are rejected with
|
||||
// base."#bad argument". Like the C code, a non-zero stream offset means that
|
||||
// no stream header (WBITS) is written.
|
||||
pub const QUIRK_STREAM_OFFSET : base.u32 = 0x2AB7_9000 | 0x09
|
||||
Reference in new issue
Block a user