Commit Graph
5 Commits
Author SHA1 Message Date
Claude 49356e06e4 std/brotli: fill the FastLog2 caches lazily
Each bit_cost (one in the encoder, one in its metablock_builder) has a
64Ki-entry FastLog2 cache. The metablock_builder filled all of it before
building each quality 10 or 11 metablock (about 20 ms per encoder, since
each entry is an exact software log2), and the encoder's own cache was
never filled, so the quality 10 and 11 literal cost estimates computed
one software log2 per input byte.

Instead, a bitmap (zeroed when the struct is initialized) records which
cache entries are valid, and the new fast_log2_memo fills an entry the
first time it is needed. Typically only a few thousand entries are used.
bits_entropy, population_cost and their callers become impure (!) to
call it. The values are computed by the same functions, so the results
are unchanged.

On alice29 at quality 10, the encoder executes about 20% fewer
instructions, and a 1000-byte input takes 6 ms instead of 21 ms.
2026-10-27 09:57:01 +00:00
Claude 7875e0bd41 std/brotli: size the q10/q11 workbuf with a practical metablock bound
workbuf_len's minimum now assumes at most (n / 16) + 1 blocks per
category and (n / 64) + 64 first-phase clusters (measured maxima are
about 1/83 and 1/258 of n), and 16 BackwardMatches per byte at
quality 11 (measured maximum about 7.4). The maximum stays the C
worst case. Add q10/q11 mimic tests and workbuf_len checks, and use
the minimum in the tests' encode plan driver.
2026-10-27 03:07:53 +00:00
Claude 5393d92234 std/brotli: document the metablock builder and add golden tests
Bound the number of block types by the number of symbols in
workbuf_len_worst_case, document the metablock_builder interface and its
workbuf layout in README.md, and add deterministic bit cost and
build_meta_block tests (with golden values from libbrotlienc v1.2.1) to
test/c/std/brotli.c.
2026-10-27 01:12:51 +00:00
Claude 9ad7ae6d69 std/brotli: add metablock_builder.workbuf_len_worst_case
Give the metablock builder an upper bound on the workbuf length that
build_meta_block needs, expose it through the metablock_tester and check
it against the high water mark in the parity harness. Add shard options
(-largeoff, -edge, -manytypes) and a stress phase that reaches 256 block
types to the harness.
2026-10-27 01:03:11 +00:00
Claude 202b406248 std/brotli: port block_splitter.c, cluster.c and metablock.c
Add the metablock_builder struct, mirroring BrotliSplitBlock (with its
MyRand sampling, InitialEntropyCodes, RefineEntropyCodes, FindBlocks,
RemapBlockIds, BuildBlockHistograms and ClusterBlocks), the histogram
clustering functions (BrotliCompareAndPushToQueue,
BrotliHistogramCombine, BrotliHistogramBitCostDistance,
BrotliHistogramRemap, BrotliHistogramReindex, BrotliClusterHistograms),
BrotliInitDistanceParams, BrotliBuildMetaBlock (including the distance
parameter search) and BrotliOptimizeHistograms (with
BrotliOptimizeHuffmanCountsForRle). All memory that the C code
allocates lives in a caller-supplied workbuf, carved out by a bump
allocator.

bit_cost gets an optional 64Ki-entry FastLog2 cache (filled by the
same softfloat log2), which makes the clustering several times faster.

Extend the metablock_tester and the parity harness to compare
SplitBlock, ClusterHistograms*, BuildMetaBlock and OptimizeHistograms
results with libbrotlienc v1.2.1.
2026-10-27 00:52:42 +00:00