Commit Graph
56 Commits
Author SHA1 Message Date
Nigel Tao d0603f3b94 test/3pdata: add webpsuite 2024-07-09 21:43:24 +10:00
Nigel Tao 00e6e71672 std/webp: implement VP8 Lossless 2024-07-07 22:13:50 +10:00
Nigel Tao 09f4500e19 std/webp: add new package (and vp8 placeholder) 2024-05-10 13:52:07 +10:00
Nigel Tao fdd08200e3 test/3pdata: add note re XZ backdoor test files
This mirrors the nigeltao/xz-tests-files repo's
https://github.com/nigeltao/xz-tests-files/commit/d163360fd24b54562de56f012b2b05cc183736df
2024-04-05 17:33:18 +11:00
Nigel Tao 13c72db350 std/lzma: add QUIRK_ALLOW_NON_ZERO_INITIAL_BYTE 2024-03-01 21:11:28 +11:00
Nigel Tao 27407b5332 test/3pdata: add xzsuite/lzip-testsuite/*.lz 2024-03-01 21:03:56 +11:00
Nigel Tao a9739a8921 std/lzip: actually implement the format 2024-03-01 20:55:24 +11:00
Nigel Tao 793ab36569 std/lzip: add new package 2024-03-01 09:52:54 +11:00
Nigel Tao 1c0b07f1aa std/lzma: accept optional End of Stream code 2024-02-29 22:01:04 +11:00
Nigel Tao 22b4ff4842 std/xz: decode concatenated streams 2024-02-26 23:02:40 +11:00
Nigel Tao 5ec31fc291 std/xz: verify index blocks' comp/uncomp sizes 2024-02-16 16:03:49 +11:00
Nigel Tao c0264e4496 std/xz: verify index and footer 2024-02-15 21:46:14 +11:00
Nigel Tao 06599b8e9b std/xz: tighten VLI (variable length int) decode 2024-02-15 19:19:46 +11:00
Nigel Tao 6017c37232 std/lzma: stash LZMA state across multiple chunks 2024-02-14 20:32:07 +11:00
Nigel Tao 749e3559f2 std/xz: implement apply_filter_0b_riscv 2024-02-13 21:11:06 +11:00
Nigel Tao 94e32e8e56 test/3pdata: add xzsuite/*-riscv-*.xz 2024-02-13 20:22:02 +11:00
Nigel Tao 782e78a689 std/xz: support BCJ start option 2024-02-12 21:01:27 +11:00
Nigel Tao 30133ad4b2 std/xz: implement apply_filter_05_powerpc 2024-02-11 19:10:55 +11:00
Nigel Tao d2a63a7e34 std/xz: implement apply_filter_04_x86 2024-02-11 13:04:29 +11:00
Nigel Tao 76a17ce742 std/xz: implement apply_filter_0a_arm64 2024-02-11 11:42:40 +11:00
Nigel Tao 03bb04d80c std/xz: implement apply_filter_09_sparc 2024-02-11 10:54:44 +11:00
Nigel Tao d4ececf1d4 std/xz: add delta filter 2024-01-07 09:14:11 +11:00
Nigel Tao 1e5cf45090 std/lzma: fix uncompressed prior decoded length 2023-12-27 23:40:10 +11:00
Nigel Tao 2afec50f62 std/lzma: track prior decoded length 2023-12-26 23:48:47 +11:00
Nigel Tao 5a7299b5c2 script: have print-*-checksums.sh show exit codes 2023-12-26 22:21:13 +11:00
Nigel Tao cf927b20bf std/xz: verify block {un,}compressed size 2023-12-26 17:04:16 +11:00
Nigel Tao a440f6dd5b std/lzma: support LZMA2 uncompressed chunks 2023-12-26 17:04:14 +11:00
Nigel Tao 13d9d259e8 test/3pdata: add xzsuite 2023-12-26 12:41:01 +11:00
Patrick Georgi f563f32f82 Add SPDX identifier to license headers
SPDX is an ISO standard for describing licenses in a computer readable
format, which is useful when applying license compliance tools and the
like to source code.

In addition to ensuring that the Wuffs C distribution is properly
annotated to the benefit of projects using Wuffs and SPDX-supporting
tools, add it everywhere else where there's a license in this repo for
consistency, too.
2023-10-06 20:52:36 +02:00
Nigel Tao 74a6ea2fcd all: relicense from Apache2 to dual Apache2/MIT
The LICENSE has changed from a single license (Apache2) to a dual
license (Apache2 or MIT, at your option).

If you've previously used Wuffs under the Apache2 license, you are able
to keep doing so. No further action required by you.

The two files listed below (previous releases) are exceptions, as they
are older snapshots, frozen in time. Their license has not changed:
release/c/wuffs-v0.2.c
release/c/wuffs-v0.3.c

Every name in the AUTHORS file, other than Google Inc., has signed
Google's CLA (Contributor License Agreement).
2023-10-06 10:00:59 +11:00
Nigel Tao 6184059c05 test/3pdata: tweak blinksuite JPEG commentary 2023-09-27 22:31:37 +10:00
Nigel Tao ca50faaa92 test/3pdata: add more commentary about djpeg diffs 2023-09-26 22:09:25 +10:00
Nigel Tao 70c1a78cd8 test/3pdata: add etc-blinksuite.commentary.md 2023-09-24 11:50:55 +10:00
Nigel Tao 186bdc5fe2 Roll back 'decode truncated... on MCU boundaries'
See the previous commit for the rationale.
2023-09-16 20:53:15 +10:00
Nigel Tao a3cecef69a std/jpeg: decode truncated JPEGs on MCU boundaries
This only affects truncated (malformed) JPEGs. It moves Wuffs' std/jpeg
decoder closer to libjpeg-turbo, as seen by the "hash of the decoded
output" for test/data/peacock.default.truncated.jpeg now matching
test/data/peacock.default.truncated.decoded-by-djpeg.png instead of
test/data/peacock.default.truncated.decoded-by-wuffs.png

The decoding of test/data/peacock.progressive.truncated.jpeg is not yet
affected, requiring further investigation, but a future commit could
similarly make that match djpeg.

The two libraries don't use exactly the same algorithm because, for
performance, Wuffs' decoder reads from the bitstream 8 bytes at a time
where libjpeg-turbo reads only 1 byte at a time.

While this change moves Wuffs' output closer to libjpeg-turbo (to
repeat, only for truncated JPEGs; well-formed JPEGs are unaffected), the
benchmarks below show a negative performance impact, and visual
inspection of test/data/peacock.*.truncated.decoded-by-*.png shows that
matching libjpeg-turbo *exactly* on truncated JPEGs doesn't actually
always look better.

On balance, it doesn't seem worth it to keep this change (and spend more
time on that further investigation).

This commit will soon be followed by a rollback, but it is committed
anyway so that we can refer to these numbers in the git log.

name                                             old speed      new speed      delta

wuffs_jpeg_decode_19k_8bpp/clang11                378MB/s ± 0%   364MB/s ± 0%  -3.53%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/clang11   325MB/s ± 0%   340MB/s ± 0%  +4.62%  (p=0.016 n=5+4)
wuffs_jpeg_decode_30k_24bpp_sequential/clang11   1.02GB/s ± 0%  0.99GB/s ± 0%  -2.64%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/clang11               842MB/s ± 0%   815MB/s ± 0%  -3.16%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/clang11          916MB/s ± 0%   887MB/s ± 0%  -3.08%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/clang11         1.10GB/s ± 0%  1.07GB/s ± 2%  -3.39%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/clang11             919MB/s ± 0%   892MB/s ± 0%  -2.95%  (p=0.008 n=5+5)

wuffs_jpeg_decode_19k_8bpp/gcc10                  360MB/s ± 0%   348MB/s ± 0%  -3.34%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/gcc10     290MB/s ± 0%   283MB/s ± 0%  -2.46%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/gcc10      925MB/s ± 0%   909MB/s ± 0%  -1.70%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/gcc10                 758MB/s ± 0%   732MB/s ± 0%  -3.43%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/gcc10            829MB/s ± 0%   799MB/s ± 0%  -3.54%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/gcc10            981MB/s ± 0%   964MB/s ± 0%  -1.72%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/gcc10               835MB/s ± 0%   811MB/s ± 0%  -2.89%  (p=0.008 n=5+5)
2023-09-16 20:49:46 +10:00
Nigel Tao d89ec2d40b std/jpeg: drain bitstream for truncated input
name                                             old speed      new speed      delta

wuffs_jpeg_decode_19k_8bpp/clang11                367MB/s ± 0%   378MB/s ± 0%  +2.91%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/clang11   333MB/s ± 0%   325MB/s ± 0%  -2.46%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/clang11   1.00GB/s ± 0%  1.02GB/s ± 0%  +1.37%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/clang11               822MB/s ± 0%   842MB/s ± 0%  +2.39%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/clang11          898MB/s ± 0%   916MB/s ± 0%  +1.98%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/clang11         1.09GB/s ± 0%  1.10GB/s ± 0%  +1.21%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/clang11             898MB/s ± 0%   919MB/s ± 0%  +2.29%  (p=0.008 n=5+5)

wuffs_jpeg_decode_19k_8bpp/gcc10                  362MB/s ± 0%   360MB/s ± 0%  -0.55%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/gcc10     292MB/s ± 0%   290MB/s ± 0%  -0.57%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/gcc10      934MB/s ± 0%   925MB/s ± 0%  -1.03%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/gcc10                 759MB/s ± 0%   758MB/s ± 0%  -0.25%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/gcc10            828MB/s ± 0%   829MB/s ± 0%    ~     (p=0.222 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/gcc10           1.00GB/s ± 0%  0.98GB/s ± 0%  -1.44%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/gcc10               837MB/s ± 0%   835MB/s ± 0%  -0.29%  (p=0.008 n=5+5)
2023-08-28 19:35:08 +10:00
Nigel Tao dfe5bcd36e std/jpeg: decode something for truncated input 2023-08-27 10:15:40 +10:00
Nigel Tao 80e847dceb std/jpeg: stash mcu_blocks during progressive IDCT 2023-08-25 23:03:06 +10:00
Nigel Tao 4417dcea4e std/jpeg: use default Huffman tables if missing 2023-08-24 16:18:11 +10:00
Nigel Tao 6489915ed3 std/jpeg: decode CMYK and YCbCrK JPEGs
Once again, the benchmarks don't exercise the CMYK / YCbCrK code paths
per se, but here are the numbers:

name                                             old speed      new speed      delta

wuffs_jpeg_decode_19k_8bpp/clang11                376MB/s ± 0%   376MB/s ± 0%    ~     (p=0.310 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/clang11   341MB/s ± 0%   340MB/s ± 0%  -0.42%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/clang11   1.01GB/s ± 0%  1.01GB/s ± 0%  +0.27%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/clang11               836MB/s ± 1%   839MB/s ± 0%  +0.37%  (p=0.016 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/clang11          912MB/s ± 0%   914MB/s ± 0%  +0.28%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/clang11         1.10GB/s ± 0%  1.11GB/s ± 0%  +0.82%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/clang11             916MB/s ± 0%   916MB/s ± 0%    ~     (p=0.841 n=5+5)

wuffs_jpeg_decode_19k_8bpp/gcc10                  368MB/s ± 0%   361MB/s ± 0%  -1.87%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/gcc10     279MB/s ± 0%   292MB/s ± 0%  +4.68%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/gcc10      947MB/s ± 1%   930MB/s ± 0%  -1.74%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/gcc10                 770MB/s ± 0%   760MB/s ± 0%  -1.32%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/gcc10            838MB/s ± 0%   826MB/s ± 0%  -1.36%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/gcc10           1.00GB/s ± 0%  1.01GB/s ± 0%  +1.04%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/gcc10               847MB/s ± 0%   833MB/s ± 0%  -1.67%  (p=0.008 n=5+5)
2023-08-21 22:13:39 +10:00
Nigel Tao 72b9c23120 std/jpeg: decode RGB (not just YCbCr) JPEGs
The benchmarks don't exercise the RGB code paths per se, but it does
affect the numbers. Especially for gcc, this essentially undoes the
performance impact of the previous (trivial) commit. This is all
presumably another code alignment coincidence per
https://news.ycombinator.com/item?id=23633583

name                                             old speed      new speed      delta

wuffs_jpeg_decode_19k_8bpp/clang11                376MB/s ± 0%   376MB/s ± 0%     ~     (p=1.000 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/clang11   338MB/s ± 0%   341MB/s ± 0%   +0.98%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/clang11   1.01GB/s ± 0%  1.01GB/s ± 0%   +0.35%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/clang11               841MB/s ± 0%   836MB/s ± 1%   -0.65%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/clang11          913MB/s ± 0%   912MB/s ± 0%   -0.18%  (p=0.016 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/clang11         1.11GB/s ± 0%  1.10GB/s ± 0%   -0.34%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/clang11             920MB/s ± 0%   916MB/s ± 0%   -0.42%  (p=0.008 n=5+5)

wuffs_jpeg_decode_19k_8bpp/gcc10                  349MB/s ± 0%   368MB/s ± 0%   +5.60%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/gcc10     328MB/s ± 0%   279MB/s ± 0%  -14.82%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/gcc10      925MB/s ± 0%   947MB/s ± 1%   +2.33%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/gcc10                 732MB/s ± 0%   770MB/s ± 0%   +5.23%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/gcc10            798MB/s ± 0%   838MB/s ± 0%   +4.99%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/gcc10            962MB/s ± 0%   996MB/s ± 0%   +3.54%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/gcc10               806MB/s ± 0%   847MB/s ± 0%   +5.19%  (p=0.008 n=5+5)
2023-08-21 16:12:21 +10:00
Nigel Tao 14a8bd0feb std/jpeg: allow fewer-components sequential scans
name                                             old speed      new speed      delta

wuffs_jpeg_decode_19k_8bpp/clang11                366MB/s ± 0%   369MB/s ± 0%  +0.81%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/clang11   302MB/s ± 0%   309MB/s ± 0%  +2.28%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/clang11   1.00GB/s ± 0%  1.00GB/s ± 0%  -0.58%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/clang11               820MB/s ± 0%   827MB/s ± 0%  +0.80%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/clang11          895MB/s ± 0%   897MB/s ± 0%  +0.24%  (p=0.008 n=5+5)
wuffs_jpeg_decode_552k_24bpp_444/clang11         1.11GB/s ± 0%  1.10GB/s ± 0%  -0.30%  (p=0.008 n=5+5)
wuffs_jpeg_decode_4002k_24bpp/clang11             897MB/s ± 0%   900MB/s ± 0%  +0.30%  (p=0.008 n=5+5)

wuffs_jpeg_decode_19k_8bpp/gcc10                  370MB/s ± 0%   369MB/s ± 0%  -0.26%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_progressive/gcc10     309MB/s ± 0%   283MB/s ± 0%  -8.35%  (p=0.008 n=5+5)
wuffs_jpeg_decode_30k_24bpp_sequential/gcc10      948MB/s ± 0%   945MB/s ± 0%  -0.27%  (p=0.008 n=5+5)
wuffs_jpeg_decode_77k_24bpp/gcc10                 769MB/s ± 0%   769MB/s ± 0%    ~     (p=0.310 n=5+5)
wuffs_jpeg_decode_552k_24bpp_420/gcc10            841MB/s ± 0%   842MB/s ± 0%  +0.13%  (p=0.016 n=4+5)
wuffs_jpeg_decode_552k_24bpp_444/gcc10           1.00GB/s ± 0%  1.01GB/s ± 0%  +0.75%  (p=0.016 n=4+5)
wuffs_jpeg_decode_4002k_24bpp/gcc10               846MB/s ± 0%   848MB/s ± 0%  +0.22%  (p=0.008 n=5+5)
2023-08-19 17:22:47 +10:00
Nigel Tao 2f5f203d1b test/3pdata: regenerate nia-checksums-of-*.txt 2023-08-19 17:18:11 +10:00
Nigel Tao d248ade0e6 test/3pdata: add blinksuite, bmpsuite, golangsuite 2023-06-09 22:10:07 +10:00
Nigel Tao f4002c5b21 Have std/tga decode RLE images
Updates #67
2022-01-15 14:10:05 +11:00
Nigel Tao fc77dc35ff Have std/tga decode BGRX5551 images
Updates #67
2022-01-15 12:53:24 +11:00
Nigel Tao c997066a2a Have std/tga decode color-mapped images
Updates #67
2022-01-15 12:38:21 +11:00
Nigel Tao 2c10bc32b0 Have std/tga decode non-color-mapped images
Updates #67
2022-01-15 11:37:45 +11:00
Nigel Tao 5aa7a9339f Have std/tga decode width and height
Updates #67
2022-01-15 08:28:53 +11:00
Nigel Tao 4c09c77bb9 Add std/tga package
This is just a placeholder decoder implementation  (and all of the
required plumbing) that produces a 1x1 reddish-purple image. Follow-up
commits will add flesh to the metaphorical bones.

TGA has no magic identifier, so MIME sniffers have to use heuristics.
The test/data/cbor-rfc-7049-examples.cbor file (which is CBOR binary
data, not a TGA image) is mistakenly sniffed as TGA. The /usr/bin/file
program makes the same classification (a false positive).

$ file test/data/cbor-rfc-7049-examples.cbor
test/data/cbor-rfc-7049-examples.cbor: Targa image data - Map 59395 x 26
x 15 +6169 +6500 - 2-bit alpha - interleave "@\033"

Some test/3pdata/tgasuite/*.tga test files, which are invalid TGA
images, are also false positives for now.

Updates #67
2022-01-14 22:53:20 +11:00