Commit Graph
51 Commits
Author SHA1 Message Date
Claude eb5ae76130 std/brotli: document large-window encoding and short workbufs 2026-10-27 02:40:28 +00:00
Claude b9f9d6ce65 std/brotli: document the encoder at qualities 2 to 9 2026-10-27 01:20:26 +00:00
Claude ca1ba9d134 std/brotli: add an encoder for quality levels 0 and 1
Adds a brotli encoder (std/brotli's encoder struct, a base.io_transformer)
whose output is byte-identical to libbrotlienc v1.2.1 for quality levels 0
and 1, mirroring encode.c's BrotliEncoderCompressStream(Fast),
compress_fragment.c and compress_fragment_two_pass.c, plus the entropy
coding and bit stream helpers they need. Every double operation goes
through std/softfloat, following the C evaluation order exactly.

Parameters are quirks (encode_quirks.wuffs) that mirror
BrotliEncoderSetParameter, sanitized at the first transform_io call.
Each transform_io resumption corresponds to one C call with available_in
equal to the readable src length; a closed src is FINISH; request_flush!
and request_emit_metadata! make the next resumption a FLUSH or
EMIT_METADATA call. Qualities 2 to 11 return "#unsupported encoder
quality" for now. Encoder tables are generated by
script/print-brotli-tables.go -encoder.

Tests: round trips through the Wuffs decoder over the corpus and synthetic
inputs; with -mimic, byte-exact comparisons against libbrotlienc over
quality × window × mode × input chunking × flush schedule × metadata ×
size hint. script/manual-test-brotli-encoder-parity.c runs the full
grid: 86,268 configurations, zero mismatches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 23:55:56 +00:00
Claude 4978207531 std/brotli: add a brotli decoder with parity to libbrotlidec v1.2.1
This is the first milestone of a byte-exact rewrite of brotli (reference
implementation tag v1.2.1) in Wuffs. The decoder mirrors c/dec/decode.c
function by function, including its validation checks: the same output for
valid input and the same verdict, with each BROTLI_DECODER_ERROR_FORMAT_*
code mapped to a Wuffs status (see std/brotli/README.md). The C state
machine is replaced by Wuffs coroutines, so decoding works with arbitrarily
small src and dst buffers.

The history window lives in the caller's workbuf. Huffman table groups,
context maps and the static dictionary scratch space live in the decoder
struct. The static dictionary, transforms, context lookup tables and prefix
code tables in common_tables.wuffs are generated from the brotli source by
script/print-brotli-tables.go.

Tests: golden tests on the upstream tests/testdata vectors and on streams
produced by the v1.2.1 CLI at several qualities and window sizes, 1-byte
I/O limits, truncation at every prefix, malformed inputs per error code,
trailing data and workbuf boundaries. With -mimic, every vector and every
malformed input is also compared against libbrotlidec, including the exact
error code. A differential run over about a million mutated inputs found
no divergence.

Not yet supported (left as TODOs): large-window brotli, custom/shared
dictionaries, metadata callbacks and fast paths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 22:46:34 +00:00
Claude f33e9f3377 std/softfloat: add bit-exact IEEE 754 binary64/binary32 arithmetic
Wuffs the Language has no floating point types, but some reference
implementations (notably the brotli encoder) make decisions with double and
float arithmetic and libm's log2. Porting them with byte-identical output
needs bit-exact floating point results.

This package implements f64 and f32 add, sub, mul, div, fma, comparisons,
min/max, integer conversions and f64<->f32 conversions on bit patterns,
with round-to-nearest-even, subnormals, infinities and NaNs, following the
structure of Berkeley SoftFloat 3. It also ports glibc 2.39's x86-64
__log2_fma log2 implementation (the variant selected on CPUs with FMA and
AVX2), following the shipped machine code's fused operations.

Verification: test/c/std/softfloat.c checks every operation against the C
compiler's native arithmetic on edge cases and random operands. The
script/manual-test-softfloat-*.c programs run 10^8 random operations per
operation group and compare log2_u64 against libm for every integer in
[1, 2^32) plus 10^7 larger values; all with zero mismatches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 22:46:32 +00:00
Nigel Tao 6e9e2e00c2 std/vp8: accept VP8X (but ignore any ALPH chunk) 2026-07-20 21:24:35 +10:00
Nigel Tao 43a6814dfe std/handsum: add new package 2025-02-24 11:18:24 +11:00
Nigel Tao 43b6fef8b1 std/thumbhash: add QUIRK_JUST_RAW_THUMBHASH 2024-09-06 13:23:38 +10:00
Nigel Tao b8740d8881 std/thumbhash: add new package 2024-09-03 20:07:37 +10:00
Nigel Tao 43df015669 std/etc2: add new package 2024-08-17 15:31:29 +10:00
Nigel Tao d06700f00b std/qoi: add new package
Fixes #63
2024-08-08 23:49:17 +10:00
Nigel Tao 09f4500e19 std/webp: add new package (and vp8 placeholder) 2024-05-10 13:52:07 +10:00
Nigel Tao 793ab36569 std/lzip: add new package 2024-03-01 09:52:54 +11:00
Nigel Tao ce0cee444d std/xz: add new package 2023-12-25 13:24:24 +11:00
Nigel Tao cc01be3b17 doc/std: update README.md 2023-12-17 21:48:21 +11:00
Nigel Tao eed1af7bee base: rename history_retain_length
It was renamed to dst_history_retain_length.

It also was removed from image_decoders and token_decoders. It only
applies to io_transformers, as retaining history applies to the
destination (not source) IO buffer.

Also adjust example/bzcat and example/zcat to apply the history retain
length to the dst (not src) buffer.
2023-12-17 12:26:08 +11:00
Nigel Tao 57f01ec3f6 std/lzma: add new package 2023-12-12 23:41:26 +11:00
Nigel Tao b2ab3ba008 std/crc64: add new package 2023-10-14 23:56:41 +11:00
Nigel Tao 71dca5adce base: add hasher_bitvec256 type 2023-10-12 21:35:47 +11:00
Nigel Tao 1304ee6265 std/xxhash64: add new package 2023-10-06 13:10:47 +11:00
Nigel Tao 88efa8360f base: add hasher_u64 2023-10-06 10:33:53 +11:00
Nigel Tao 80b1988237 std: add hasher_u32 update!, checksum_u32 methods 2023-10-05 19:27:02 +11:00
Nigel Tao b5eee6ac4f doc/std: update lists of base interface methods 2023-10-05 19:08:34 +11:00
Nigel Tao bec02d7c31 std/xxhash32: add new package 2023-10-05 19:08:22 +11:00
Nigel Tao 8e42e01d3c std/gif: remove the std/lzw dependency
name                                         old speed      new speed      delta

wuffs_gif_decode_1k_bw/clang11                683MB/s ± 0%   682MB/s ± 0%     ~     (p=0.690 n=5+5)
wuffs_gif_decode_1k_color_full_init/clang11   168MB/s ± 0%   184MB/s ± 0%  +10.01%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/clang11   204MB/s ± 0%   230MB/s ± 0%  +12.77%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang11             746MB/s ± 0%   819MB/s ± 0%   +9.90%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang11          200MB/s ± 0%   218MB/s ± 0%   +8.97%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang11                  246MB/s ± 0%   257MB/s ± 0%   +4.33%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang11      543MB/s ± 0%   562MB/s ± 0%   +3.44%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang11       210MB/s ± 0%   220MB/s ± 0%   +4.90%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/clang11      214MB/s ± 0%   224MB/s ± 0%   +4.96%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/clang11      214MB/s ± 0%   224MB/s ± 0%   +4.98%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang11      1.24GB/s ± 0%  1.29GB/s ± 0%   +3.85%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc10                  649MB/s ± 0%   588MB/s ± 0%   -9.42%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/gcc10     167MB/s ± 0%   198MB/s ± 0%  +18.57%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/gcc10     195MB/s ± 0%   251MB/s ± 0%  +28.64%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc10               753MB/s ± 0%   871MB/s ± 0%  +15.64%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc10            199MB/s ± 0%   233MB/s ± 0%  +16.86%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc10                    258MB/s ± 0%   265MB/s ± 0%   +2.93%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc10        560MB/s ± 0%   561MB/s ± 0%   +0.19%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc10         221MB/s ± 0%   227MB/s ± 0%   +2.65%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/gcc10        227MB/s ± 0%   232MB/s ± 0%   +2.29%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/gcc10        227MB/s ± 0%   232MB/s ± 0%   +2.28%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc10        1.29GB/s ± 0%  1.28GB/s ± 0%   -0.22%  (p=0.008 n=5+5)
2023-10-04 20:19:25 +11:00
Nigel Tao 0904751e2d example: add toy-aux-image 2023-06-04 10:47:22 +10:00
Nigel Tao 85b244aa49 std/netpbm: add new package 2023-05-16 16:29:58 +10:00
Nigel Tao 0bbc4c0083 std/jpeg: add new package
This is just a placeholder decoder implementation (and all of the
required plumbing) that produces a 1x1 reddish-purple image. Follow-up
commits will add flesh to the metaphorical bones.

Updates #42
2023-04-21 18:33:00 +10:00
Nigel Tao e126c998e0 std: limit supported image dimensions to 0xFF_FFFF 2023-04-16 08:57:55 +10:00
Nigel Tao 6e84987d33 std: change set_quirk_enabled to set_quirk 2023-04-14 21:42:15 +10:00
Nigel Tao 3ff4bf4674 Add script/print-image-metadata.cc
Updates #58
2022-06-03 14:15:13 +10:00
Nigel Tao 8d03a67e6e Re-do image decoders' call sequence state machine 2022-05-28 15:06:56 +10:00
Nigel Tao a2e3a5a272 Update doc/std/compression-decoders.md for bzcat 2022-05-28 14:23:25 +10:00
Nigel Tao f700e00df8 Update doc/std/compression-decoders.md for bzip2 2022-05-28 10:11:34 +10:00
Nigel Tao 23c9de18c6 Update doc/std/image-decoders.md Metadata for v0.3
Fixes #60
2022-02-06 21:38:41 +11:00
Nigel Tao 94736e2eae Add "API Listing" sections to doc/std 2022-02-06 13:38:21 +11:00
Nigel Tao e4d237df89 Add std/tga to doc/std
Updates #67
2022-02-06 13:30:46 +11:00
Nigel Tao a04fda42d6 Update doc/std/image-decoders.md 2021-03-11 23:04:53 +11:00
Nigel Tao 2e0f2a68be Add doc/std/README.md "Modules and Dependencies" 2021-03-11 12:38:58 +11:00
Nigel Tao aae62e407b Fix doc/std/README.md wuffs_base__status typo 2021-03-04 23:42:34 +11:00
Nigel Tao cf8884d44c Rename example/library to example/toy-genlib 2020-08-05 11:55:51 +10:00
Nigel Tao 5f8d2da74f Rename decode_io_writer methods to transform_io 2020-01-11 10:19:25 +11:00
Nigel Tao 0fa32bd09f Rename warnings to notes 2020-01-05 13:15:00 +11:00
Nigel Tao d098582cc6 Add image-decoders.md link to Skia example 2019-12-17 10:24:46 +11:00
Nigel Tao d0e393e0ce Update doc/std/hashers.md for timing attacks 2019-12-11 11:59:56 +11:00
Nigel Tao 5362ca7573 Add doc/note/pixel-formats.md 2019-12-02 20:34:55 +11:00
Nigel Tao f4da538291 Fix broken link to decode_quirks.wuffs 2019-11-29 20:44:58 +11:00
Nigel Tao fd38af62dc Add doc/std/compression-decoders.md 2019-11-26 20:57:31 +11:00
Nigel Tao e32d680ca4 Add doc/std/README.md 2019-11-25 09:43:29 +11:00
Nigel Tao 0201b85455 Add doc/std/image-decoders.md 2019-11-25 08:45:17 +11:00