Commit Graph
609 Commits
Author SHA1 Message Date
Claude 361091fee6 std/brotli: add quality 10 and 11 (Zopfli) encoder, untested
Port H10 (hash_to_binary_tree_inc.h), literal_cost.c,
backward_references_hq.c, the static dictionary match finder
(static_dict.c) and the quality 10+ parts of the metablock writer,
using the metablock_builder. The variable-size Zopfli and
metablock_builder scratch lives in a new workbuf layout region.
2026-10-27 02:36:05 +00:00
Claude b10d57f20f Merge branch 'm4b-metablock' into m4a-encoder-core
# Conflicts:
#	std/brotli/README.md
2026-10-27 01:26:59 +00:00
Claude 5393d92234 std/brotli: document the metablock builder and add golden tests
Bound the number of block types by the number of symbols in
workbuf_len_worst_case, document the metablock_builder interface and its
workbuf layout in README.md, and add deterministic bit cost and
build_meta_block tests (with golden values from libbrotlienc v1.2.1) to
test/c/std/brotli.c.
2026-10-27 01:12:51 +00:00
Claude d8f3e150fb test/c/std/brotli: add encoder tests for qualities 2 to 9 2026-10-27 01:09:19 +00:00
Claude 09d4b23d09 std/brotli: WIP encoder qualities 2 to 9 (EncodeData, hashers, greedy metablocks) 2026-10-27 00:58:17 +00:00
Claude d74d28bef9 Merge branch 'm1c-decoder-extensions' into claude/project-thread-0izcgq
Combines the encoder (quality 0 and 1) with the decoder's large-window
mode and custom dictionaries. The test and mimic files take both sides'
new sections; the fuzzer gains the std/softfloat module that the
encoder now pulls into std/brotli.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 23:59:14 +00:00
Claude ca1ba9d134 std/brotli: add an encoder for quality levels 0 and 1
Adds a brotli encoder (std/brotli's encoder struct, a base.io_transformer)
whose output is byte-identical to libbrotlienc v1.2.1 for quality levels 0
and 1, mirroring encode.c's BrotliEncoderCompressStream(Fast),
compress_fragment.c and compress_fragment_two_pass.c, plus the entropy
coding and bit stream helpers they need. Every double operation goes
through std/softfloat, following the C evaluation order exactly.

Parameters are quirks (encode_quirks.wuffs) that mirror
BrotliEncoderSetParameter, sanitized at the first transform_io call.
Each transform_io resumption corresponds to one C call with available_in
equal to the readable src length; a closed src is FINISH; request_flush!
and request_emit_metadata! make the next resumption a FLUSH or
EMIT_METADATA call. Qualities 2 to 11 return "#unsupported encoder
quality" for now. Encoder tables are generated by
script/print-brotli-tables.go -encoder.

Tests: round trips through the Wuffs decoder over the corpus and synthetic
inputs; with -mimic, byte-exact comparisons against libbrotlienc over
quality × window × mode × input chunking × flush schedule × metadata ×
size hint. script/manual-test-brotli-encoder-parity.c runs the full
grid: 86,268 configurations, zero mismatches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 23:55:56 +00:00
Claude b9be8feaee std/brotli: large-window mode, custom dictionaries and a fuzzer
Completes the decoder's feature set relative to libbrotlidec v1.2.1:

- QUIRK_LARGE_WINDOW mirrors BROTLI_DECODER_PARAM_LARGE_WINDOW: the
  large-window WBITS escape, window bits up to 30, the 544-symbol distance
  alphabet and 64-bit distance arithmetic, with workbuf_len growing to
  (1 << 30) + 542 when the quirk is set.
- decoder.add_dictionary! mirrors BrotliDecoderAttachDictionary for raw
  (LZ77 prefix) and serialized shared dictionaries: compound-dictionary
  distance mapping, the shared-dictionary parser (custom word lists,
  transform lists including the SHIFT transforms, context-dependent
  dictionary selection, prefix dictionaries) and all of its validation.
  Wuffs structs cannot hold slices, so the dictionary bytes are copied
  into the front of the caller's workbuf; the layout is documented in
  the README. Serialized dictionaries are accepted as in a
  BROTLI_EXPERIMENTAL build of libbrotli.
- fuzz/c/std/brotli_fuzzer.c, exercising the quirk, dictionaries and
  random I/O limits.

Tests: new golden vectors for large windows (including a 17 MiB
far-distance stream generated in the test), raw and serialized
dictionaries, nine more malformed inputs, and mimic comparisons that
attach the same dictionaries to libbrotlidec. Differential runs over
2.7 million mutated large-window and dictionary streams found no
divergence from libbrotlidec.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 23:45:06 +00:00
Claude 4978207531 std/brotli: add a brotli decoder with parity to libbrotlidec v1.2.1
This is the first milestone of a byte-exact rewrite of brotli (reference
implementation tag v1.2.1) in Wuffs. The decoder mirrors c/dec/decode.c
function by function, including its validation checks: the same output for
valid input and the same verdict, with each BROTLI_DECODER_ERROR_FORMAT_*
code mapped to a Wuffs status (see std/brotli/README.md). The C state
machine is replaced by Wuffs coroutines, so decoding works with arbitrarily
small src and dst buffers.

The history window lives in the caller's workbuf. Huffman table groups,
context maps and the static dictionary scratch space live in the decoder
struct. The static dictionary, transforms, context lookup tables and prefix
code tables in common_tables.wuffs are generated from the brotli source by
script/print-brotli-tables.go.

Tests: golden tests on the upstream tests/testdata vectors and on streams
produced by the v1.2.1 CLI at several qualities and window sizes, 1-byte
I/O limits, truncation at every prefix, malformed inputs per error code,
trailing data and workbuf boundaries. With -mimic, every vector and every
malformed input is also compared against libbrotlidec, including the exact
error code. A differential run over about a million mutated inputs found
no divergence.

Not yet supported (left as TODOs): large-window brotli, custom/shared
dictionaries, metadata callbacks and fast paths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 22:46:34 +00:00
Claude f33e9f3377 std/softfloat: add bit-exact IEEE 754 binary64/binary32 arithmetic
Wuffs the Language has no floating point types, but some reference
implementations (notably the brotli encoder) make decisions with double and
float arithmetic and libm's log2. Porting them with byte-identical output
needs bit-exact floating point results.

This package implements f64 and f32 add, sub, mul, div, fma, comparisons,
min/max, integer conversions and f64<->f32 conversions on bit patterns,
with round-to-nearest-even, subnormals, infinities and NaNs, following the
structure of Berkeley SoftFloat 3. It also ports glibc 2.39's x86-64
__log2_fma log2 implementation (the variant selected on CPUs with FMA and
AVX2), following the shipped machine code's fused operations.

Verification: test/c/std/softfloat.c checks every operation against the C
compiler's native arithmetic on edge cases and random operands. The
script/manual-test-softfloat-*.c programs run 10^8 random operations per
operation group and compare log2_u64 against libm for every integer in
[1, 2^32) plus 10^7 larger values; all with zero mismatches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
2026-10-26 22:46:32 +00:00
Nigel Tao 4be865da25 base: fix parse_number_f64("0.000...0001e30001")
Updates PR #184
2026-10-21 12:24:09 +11:00
Nigel Tao c99f5af9fe base: fix parse_number_f64, 801 digits before '.'
Fixes #183
2026-10-20 14:09:09 +11:00
sabith saheb 9d41525a2f base: don't read past the end of the slice in parse_number_{u64,f64} 2026-10-19 12:59:04 +05:30
Nigel Tao 466cf6f15d std/webp: tweak workbuf_len determination 2026-08-10 09:23:05 +10:00
Nigel Tao 7b71d6840a base: rename color_u32_at color_u32_argb_premul_at 2026-08-06 11:53:49 +10:00
Nigel Tao d9cf0751ca std/vp8: blend VP8X ALPH chunk 2026-08-06 11:05:26 +10:00
Nigel Tao e7f6c79752 std/vp8: decode VP8X ALPH chunk 2026-07-29 15:58:52 +10:00
Nigel Tao 6e9e2e00c2 std/vp8: accept VP8X (but ignore any ALPH chunk) 2026-07-20 21:24:35 +10:00
Nigel Tao e9ebad85a7 std/vp8: fix skip for all-zero DCT coefficients
"wuffs test -mimic std/webp" now passes.

This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications.

Updates PR #168
2026-06-22 15:23:11 +10:00
Nigel Tao b4c8b9c238 test/c/mimiclib/webp: use no_fancy_upsampling
This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications.

Updates PR #168
2026-06-22 11:03:14 +10:00
Nigel Tao 9e274bee24 std/vp8: use YCC_MODEL__BT_601_STUDIO_RANGE
This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications.

Updates PR #168
2026-06-22 10:42:05 +10:00
Nigel Tao cae59b4ce8 std/vp8: fix luma subblock prediction modes
This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications.

Updates PR #168
2026-06-19 19:04:31 +10:00
Nigel Tao 24fae427c1 std/vp8: swizzle to the dst pixel buffer
This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications.

Updates PR #168
2026-06-19 08:45:56 +10:00
Nigel Tao 68d13b8740 std/vp8: implement IDCT/IWHT
This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications.

Updates PR #168
2026-06-17 16:11:36 +10:00
Nigel Tao bb3c85e2ef std/vp8: calculate the predicted values
This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications.

Updates PR #168
2026-06-16 14:00:28 +10:00
Nigel Tao b51fcb811a std/vp8: parse the first partition
This commit, part of a series, is inspired by pull request PR #168 by
Damian Kaczmarek (with AI assistance), with some modifications. There
were many stylistic changes but also:

- decoder.workbuf_len was fixed to account for all of the partitions,
  not just the first two (the first partition and the first 'other'
  partition). The PR's vp8 code, after an update to handle multiple
  other partitions (which then implied random access to the partitioned
  source bytes), assumed that enough workbuf was available (which it
  often is, test/c/std/webp.c tests pass the entire g_work_slice_u8).
  But strictly speaking, this isn't guaranteed unless workbuf_len
  reports (after decode_image_config is called) what's needed.
- Calculating that workbuf_len (at decode_image_config time) lead to
  introducing a new base.QUIRK_SOURCE_LENGTH quirk, to pass that
  contextual information from std/webp to std/vp8. This calculation
  belongs in std/vp8, not std/webp (or a future std/webm).
- Added support (and a test) for the case when the source bytes are
  streamed and some (but not yet all) of the partition data in args.src
  can be copied to args.workbuf.

Updates PR #168
2026-05-29 14:36:04 +10:00
Nigel Tao 979a9d898c lib/handsum: renumber quality from 0..=3 to 1..=4 2026-05-06 21:39:21 +10:00
Nigel Tao acbf9d2ecf test/data: rename some *.handsum files 2026-05-01 20:36:12 +10:00
Nigel Tao f8c6b35ff5 std/handsum: redo Handsum image file format 2026-04-28 21:12:28 +10:00
Nigel Tao dd8db5dbb8 std/handsum: add Potato, Gray variants 2026-04-13 00:20:06 +10:00
Nigel Tao 50b62c7704 test/data: rename some *.handsum files 2026-04-12 15:55:39 +10:00
Nigel Tao eec7d22ea3 pixconv: swizzle PIXEL_FORMAT__BGRA_PREMUL_4X16LE
Binary size, before:
145472 gen/lib/c/clang-dynamic/wuffs-base-pixconv.lo
145472 gen/lib/c/clang-static/wuffs-base-pixconv.o
247720 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
247688 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
155120 gen/lib/c/clang-dynamic/wuffs-base-pixconv.lo
155120 gen/lib/c/clang-static/wuffs-base-pixconv.o
266992 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
266896 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2025-07-15 15:34:48 +10:00
Nigel Tao 1222a6886d std/nie: support NIA (animated NIE) 2025-06-30 22:03:55 +10:00
Nigel Tao 5ec14737f3 pixconv: allow swizzle from PIXEL_FORMAT__Y_16LE
Binary size, before:
142648 gen/lib/c/clang-dynamic/wuffs-base-pixconv.lo
142648 gen/lib/c/clang-static/wuffs-base-pixconv.o
243368 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
243240 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
145472 gen/lib/c/clang-dynamic/wuffs-base-pixconv.lo
145472 gen/lib/c/clang-static/wuffs-base-pixconv.o
247720 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
247688 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2025-06-10 11:33:08 +10:00
Nigel Tao b6be81a83b std/handsum: implement do_decode_frame 2025-02-24 22:16:08 +11:00
Nigel Tao 43a6814dfe std/handsum: add new package 2025-02-24 11:18:24 +11:00
Nigel Tao 0b2d8d15f5 std/vp8: decode to a placeholder gradient 2024-09-15 11:35:13 +10:00
Nigel Tao 42ba720d4e pixconv: allow swizzling to Y (gray) more often
Binary size, before:
124816 gen/lib/c/clang-dynamic/wuffs-base-pixconv.lo
124816 gen/lib/c/clang-static/wuffs-base-pixconv.o
196736 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
196672 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
142032 gen/lib/c/clang-dynamic/wuffs-base-pixconv.lo
142032 gen/lib/c/clang-static/wuffs-base-pixconv.o
242712 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
242584 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2024-09-08 22:54:35 +10:00
Nigel Tao 85af5a4bed std/tga: rename to std/targa 2024-09-08 09:00:10 +10:00
Nigel Tao 43b6fef8b1 std/thumbhash: add QUIRK_JUST_RAW_THUMBHASH 2024-09-06 13:23:38 +10:00
Nigel Tao 8c854ae702 std/thumbhash: implement do_decode_frame 2024-09-06 12:35:53 +10:00
Nigel Tao b8740d8881 std/thumbhash: add new package 2024-09-03 20:07:37 +10:00
Nigel Tao 96eda2ec0d std/etc2: support transparency 2024-08-18 15:09:51 +10:00
Nigel Tao 43df015669 std/etc2: add new package 2024-08-17 15:31:29 +10:00
Nigel Tao d06700f00b std/qoi: add new package
Fixes #63
2024-08-08 23:49:17 +10:00
Nigel Tao c9fff5b207 std/jpeg: add and use WUFFS_BASE__QUIRK_QUALITY 2024-08-04 23:15:58 +10:00
Nigel Tao affa1e02d4 base: add WUFFS_PRIVATE_IMPL__CPU_ARCH__X86_64_V2
Updates #148
2024-07-09 09:17:09 +10:00
Nigel Tao 37a3c3ce9b base: rename WUFFS_BASE__CPU_ARCH__ETC
See
https://github.com/google/wuffs/issues/148#issuecomment-2130535812

Updates #148
2024-07-08 18:14:59 +10:00
Nigel Tao 6c8a43390b test: add webp mimic tests and benches 2024-07-08 15:54:40 +10:00
Nigel Tao 5654506497 base: allow swizzling WebP Lossless to gray 2024-07-08 12:35:40 +10:00