Commit Graph
52 Commits
Author SHA1 Message Date
Nigel Tao 34cfc3f863 Have std/gif's palettes be BGRA, not RGB 2018-05-24 10:33:02 +10:00
Nigel Tao 981bf19ad8 Add test/data/artificial GIF data 2018-05-23 12:20:45 +10:00
Nigel Tao f1d3a6f521 Add script/build-all.sh 2018-05-11 15:18:59 +10:00
Nigel Tao a106e2a857 Fill in multiple std/crc32 tables 2018-05-03 13:14:12 +10:00
Nigel Tao 8827e222b1 Add wuffs_base__io_buffer__{reader,writer} funcs 2018-04-27 20:54:44 +10:00
Nigel Tao 035b138cb1 Spin out a std/adler32 package 2018-04-27 17:20:10 +10:00
Nigel Tao d2075ce913 Add script/bench-c-deflate-fragmentation.c 2018-04-25 15:26:29 +10:00
Nigel Tao 97aab37462 Rename buf1, reader1, etc to io_buffer, io_reader
Also remove the unused Buf1, Buf2 and Usize tokens.
2018-04-05 20:49:48 +10:00
Nigel Tao a2949fe28d Add Adler32 and CRC32/IEEE tests for pi's digits 2018-03-25 21:54:07 +11:00
Nigel Tao 3a7ccfe9b4 Implement crc32's ieee.update 2018-03-25 18:09:51 +11:00
Nigel Tao 0ebfb75699 Add crc32 ieee test cases 2018-03-25 17:20:31 +11:00
Nigel Tao 4675efdcc5 Fix deflate decode when the distance is 32768 2018-03-23 12:49:42 +11:00
Nigel Tao f11e1f97c6 Add deflate-distance-32768 test case 2018-03-21 12:32:51 +11:00
Nigel Tao d1399fac19 Rename test/testdata to test/data 2018-03-16 10:48:33 +11:00
Nigel Tao 39bc8688f6 Mask Huffman bits faster, for hand-written C
Benchmark numbers are better for gcc and worse for clang, but clang 3.8
is also relatively old. Given gcc's speeds, it's arguably a clang
codegen bug, or at least it should be clang's aspirational target.

name                             old speed     new speed     delta
wuffs_deflate_decode_1k/clang    142MB/s ± 1%  135MB/s ± 1%  -4.54%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/clang   295MB/s ± 0%  291MB/s ± 0%  -1.09%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k/clang  354MB/s ± 1%  342MB/s ± 0%  -3.48%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k/gcc      153MB/s ± 1%  156MB/s ± 1%  +1.51%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/gcc     354MB/s ± 1%  371MB/s ± 0%  +4.92%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k/gcc    468MB/s ± 2%  499MB/s ± 1%  +6.64%  (p=0.008 n=5+5)

Note that these are benchmark numbers for an 'ideal', hand-written C
implementation, not for Wuffs' generated C code.
2018-02-22 15:53:42 +11:00
Nigel Tao 68e972c551 Load Huffman bits faster, for hand-written C
Benchmark numbers are better for gcc and worse for clang, but clang 3.8
is also relatively old. Given gcc's speeds, it's arguably a clang
codegen bug, or at least it should be clang's aspirational target.

name                             old speed     new speed     delta
wuffs_deflate_decode_1k/clang    157MB/s ± 1%  142MB/s ± 1%   -9.47%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/clang   313MB/s ± 0%  295MB/s ± 0%   -5.98%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k/clang  403MB/s ± 1%  354MB/s ± 1%  -12.12%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k/gcc      165MB/s ± 1%  153MB/s ± 1%   -6.93%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/gcc     337MB/s ± 0%  354MB/s ± 1%   +4.90%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k/gcc    440MB/s ± 0%  468MB/s ± 2%   +6.24%  (p=0.008 n=5+5)

Note that these are benchmark numbers for an 'ideal', hand-written C
implementation, not for Wuffs' generated C code.
2018-02-22 15:12:08 +11:00
Nigel Tao 9a463d4675 Add some more Go and Rust GIF benchmarks
On my machine (Debian Testing as of November 2017, Intel x86_64
Broadwell):

Rust    28.0  rustc 1.14  github.com/Geal/gif.rs
Rust    29.1  rustc 1.14  github.com/PistonDevelopers/image-gif
Go      62.7  go 1.9      standard library's image/gif
C       97.3  giflib 5.1  from Debian Testing's /usr/lib
Wuffs  123    clang 5.0
Wuffs  136    gcc 7.2

Numbers are megabytes per second, equivalent to megapixels per second
for GIF images.
2018-02-02 12:09:22 +11:00
Nigel Tao 5ebc62e7da Rename bench-gif-dot-rs to bench-rust-gif-dot-rs
There may be future bench-go-xxx programs, or of other languages.

On my machine, the throughput number that the program output also
changes from 24.7 (as reported in a prior commit message) to 28.0
megapixels per second. This is not due to any code changes, but because
in the meantime, I have also used the "cpupower" command line tool to
change policy from "powersave" to "performance".
2018-02-02 09:46:43 +11:00
Nigel Tao bcc4450624 Rust style fixes 2018-01-31 16:50:38 +11:00
Nigel Tao d74b3a3e29 Add a benchmark for gif.rs, a Rust GIF library
On my machine:

gif.rs        24.7 MB/s  (rustc 1.21)
giflib        97.2 MB/s  (giflib 5.1)
wuffs/clang  118   MB/s  (clang 3.8)
wuffs/gcc    141   MB/s  (gcc 6.3)
2018-01-27 16:12:30 +11:00
Nigel Tao 44f5b2d384 Allow varying BUFFER_SIZE in adler32-standalone.c 2017-12-08 11:14:30 +11:00
Nigel Tao 36051062cb Update a script for the std/flate rename 2017-12-07 16:23:18 +11:00
Nigel Tao cb17fddfc2 script/adler32-standalone.c 2017-12-07 12:59:03 +11:00
Nigel Tao 802b9d8bfd Rename deflate-related scripts 2017-12-04 10:43:32 +11:00
Nigel Tao 024dc7f97a Rename PUFFS to WUFFS, part 11 of N 2017-11-30 18:18:40 +11:00
Nigel Tao 79a94558be Rename PUFFS to WUFFS, part 5 of N 2017-11-30 16:37:20 +11:00
Nigel Tao 456e620d02 Rename PUFFS to WUFFS, part 4 of N 2017-11-30 16:28:28 +11:00
Nigel Tao becd1a6b25 Fix a comment typo. 2017-11-08 16:56:22 +11:00
Nigel Tao 7ccb9234a4 Fix hand-written decode_huffman_fast C impl. 2017-11-08 10:12:47 +11:00
Nigel Tao e3643574d7 Fix script/inline-c-relative-includes.go. 2017-11-03 15:33:23 +11:00
Nigel Tao e57c8d77db Add script/inline-c-relative-includes.go. 2017-11-03 15:05:19 +11:00
Nigel Tao 60f77e8b23 Fix script/puffs-flate-decoder-decode-huffman.c. 2017-10-21 16:20:52 +11:00
Nigel Tao 6815375de5 Generate double '_'s for pkg__type__method names. 2017-10-21 15:10:25 +11:00
Nigel Tao 2639f8f46a Rename std/flate's decoder to flate_decoder. 2017-10-21 13:05:19 +11:00
Nigel Tao f9cea5c17c Fix the adler.update implementation. 2017-10-19 20:45:43 +11:00
Nigel Tao d4372cb5b8 Change license to Apache 2.
Also remove explicit PATENTS file. The new LICENSE file already
discusses patents.

Note that the Puffs project has not yet been publicly released.
2017-10-12 11:17:41 +11:00
Nigel Tao cf6b1be5b5 Re-organize flate Huffman decoder table bits.
Benchmarks are mixed, but it seems conceptually cleaner to arrange the
bits in the order that they're examined.

name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    125MB/s ± 0%  123MB/s ± 1%  -1.91%  (p=0.008 n=5+5)
puffs_flate_decode_10k/clang   163MB/s ± 1%  157MB/s ± 0%  -3.55%  (p=0.008 n=5+5)
puffs_flate_decode_100k/clang  175MB/s ± 0%  173MB/s ± 0%  -0.75%  (p=0.016 n=4+5)
puffs_flate_decode_1k/gcc      127MB/s ± 0%  130MB/s ± 0%  +2.14%  (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc     142MB/s ± 1%  149MB/s ± 0%  +4.99%  (p=0.016 n=5+4)
puffs_flate_decode_100k/gcc    162MB/s ± 2%  168MB/s ± 0%  +3.66%  (p=0.016 n=5+4)
2017-10-06 13:07:49 +11:00
Nigel Tao 2e6b364dcc Do 8- instead of 1-byte back-copies.
name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    191MB/s ± 1%  171MB/s ±12%     ~     (p=0.095 n=5+5)
puffs_flate_decode_10k/clang   277MB/s ± 1%  325MB/s ± 7%  +17.47%  (p=0.008 n=5+5)
puffs_flate_decode_100k/clang  315MB/s ± 1%  416MB/s ± 1%  +31.88%  (p=0.008 n=5+5)
puffs_flate_decode_1k/gcc      194MB/s ± 2%  197MB/s ± 0%   +1.98%  (p=0.016 n=5+4)
puffs_flate_decode_10k/gcc     279MB/s ± 1%  361MB/s ± 1%  +29.19%  (p=0.016 n=4+5)
puffs_flate_decode_100k/gcc    315MB/s ± 0%  437MB/s ± 0%  +38.77%  (p=0.008 n=5+5)

Note that these are benchmark numbers for an 'ideal', hand-written C
implementation, not for Puffs' generated C code.
2017-08-03 17:45:47 +10:00
Nigel Tao c3aa366c66 Clean up #ifdef style; remove unnecessary ()s. 2017-08-03 11:05:33 +10:00
Nigel Tao 6af3852fa8 Change the back-copy unroll count from 8 to 3.
We're possibly overfitting now on the very small benchmark corpus (3
files), but we can test on larger corpuses later. Algorithmically, it
does match what zlib does.

name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    188MB/s ± 0%  191MB/s ± 1%   +1.55%  (p=0.008 n=5+5)
puffs_flate_decode_10k/clang   242MB/s ± 2%  277MB/s ± 1%  +14.72%  (p=0.008 n=5+5)
puffs_flate_decode_100k/clang  273MB/s ± 1%  315MB/s ± 1%  +15.63%  (p=0.008 n=5+5)
puffs_flate_decode_1k/gcc      187MB/s ± 0%  194MB/s ± 2%   +3.56%  (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc     242MB/s ± 0%  279MB/s ± 1%  +15.29%  (p=0.016 n=5+4)
puffs_flate_decode_100k/gcc    288MB/s ± 1%  315MB/s ± 0%   +9.29%  (p=0.008 n=5+5)

Note that these are benchmark numbers for an 'ideal', hand-written C
implementation, not for Puffs' generated C code.
2017-08-03 10:53:41 +10:00
Nigel Tao fa89c397b0 Do one 8-byte load instead of six 1-byte loads.
name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    178MB/s ± 2%  188MB/s ± 0%  +5.98%  (p=0.008 n=5+5)
puffs_flate_decode_10k/clang   235MB/s ± 1%  242MB/s ± 2%  +2.69%  (p=0.032 n=5+5)
puffs_flate_decode_100k/clang  274MB/s ± 0%  273MB/s ± 1%    ~     (p=0.690 n=5+5)
puffs_flate_decode_1k/gcc      177MB/s ± 0%  187MB/s ± 0%  +5.44%  (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc     234MB/s ± 0%  242MB/s ± 0%  +3.64%  (p=0.008 n=5+5)
puffs_flate_decode_100k/gcc    275MB/s ± 0%  288MB/s ± 1%  +4.68%  (p=0.008 n=5+5)

Note that these are benchmark numbers for an 'ideal', hand-written C
implementation, not for Puffs' generated C code.
2017-08-03 10:47:44 +10:00
Nigel Tao bc92bf4ff8 Fix C decode_huffman's exit with excess high bits. 2017-08-03 10:34:37 +10:00
Nigel Tao 265d3a5a1a Add a hand-written C impl of decode_huffman.
This gives a rough upper bound on how fast Puffs' generated C code can
be, with a sufficiently smart Puffs compiler.

A comparison of the 'old', safe, generated C code and the 'new', unsafe,
hand-written C code:

name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    127MB/s ± 2%  181MB/s ± 2%  +42.27%  (p=0.000 n=83+5)
puffs_flate_decode_10k/clang   151MB/s ± 1%  235MB/s ± 2%  +56.04%  (p=0.000 n=82+5)
puffs_flate_decode_100k/clang  165MB/s ± 1%  275MB/s ± 3%  +66.69%  (p=0.000 n=89+5)
puffs_flate_decode_1k/gcc      155MB/s ± 2%  181MB/s ± 1%  +16.96%  (p=0.000 n=91+5)
puffs_flate_decode_10k/gcc     146MB/s ± 2%  241MB/s ± 2%  +64.78%  (p=0.000 n=86+5)
puffs_flate_decode_100k/gcc    166MB/s ± 1%  277MB/s ± 3%  +66.78%  (p=0.000 n=88+5)

While we're discussing benchmark numbers, below is an overall snapshot.

std/flate

puffs_flate_decode_1k/clang       127MB/s ± 2%
puffs_flate_decode_10k/clang      151MB/s ± 1%
puffs_flate_decode_100k/clang     165MB/s ± 1%

puffs_flate_decode_1k/gcc         155MB/s ± 2%
puffs_flate_decode_10k/gcc        146MB/s ± 2%
puffs_flate_decode_100k/gcc       166MB/s ± 1%

mimic_adler32_10k                2.98GB/s ± 1%
mimic_adler32_100k               2.93GB/s ± 1%
mimic_crc32_10k                  1.38GB/s ± 1%
mimic_crc32_100k                 1.37GB/s ± 1%

mimic_flate_decode_1k             220MB/s ± 2%
mimic_flate_decode_10k            262MB/s ± 1%
mimic_flate_decode_100k           281MB/s ± 1%
mimic_gzip_decode_10k             221MB/s ± 1%
mimic_gzip_decode_100k            259MB/s ± 1%
mimic_zlib_decode_10k             242MB/s ± 2%
mimic_zlib_decode_100k            289MB/s ± 1%

std/gif

puffs_gif_lzw_decode_20k/clang    125MB/s ± 1%
puffs_gif_lzw_decode_100k/clang   173MB/s ± 1%
puffs_gif_decode_1k/clang         376MB/s ± 3%
puffs_gif_decode_10k/clang        128MB/s ± 2%
puffs_gif_decode_100k/clang       120MB/s ± 2%
puffs_gif_decode_1000k/clang      121MB/s ± 3%

puffs_gif_lzw_decode_20k/gcc      153MB/s ± 1%
puffs_gif_lzw_decode_100k/gcc     208MB/s ± 1%
puffs_gif_decode_1k/gcc           429MB/s ± 1%
puffs_gif_decode_10k/gcc          161MB/s ± 1%
puffs_gif_decode_100k/gcc         140MB/s ± 1%
puffs_gif_decode_1000k/gcc        143MB/s ± 1%

mimic_gif_decode_1k               158MB/s ± 1%
mimic_gif_decode_10k             93.9MB/s ± 1%
mimic_gif_decode_100k             100MB/s ± 1%
mimic_gif_decode_1000k            102MB/s ± 1%
2017-07-30 11:13:40 +10:00
Nigel Tao 89d4f68036 Fill array-of-u32 Huffman table for base+extra. 2017-07-27 17:20:31 +10:00
Nigel Tao adadb0fd5a Add a worked example of the flate file format. 2017-07-20 15:36:59 +10:00
Nigel Tao c463f4956f Add script/print-bits.go. 2017-07-17 17:28:19 +10:00
Nigel Tao 547351cfec Add testdata/romeo.txt.{flate,zlib}. 2017-07-17 16:53:45 +10:00
Nigel Tao 03572374dd Add zlib (not just raw flate) benchmark. 2017-07-14 12:00:00 +10:00
Nigel Tao 4efce30bfc Add more testdata/*.gz files. 2017-07-06 16:35:18 +10:00
Nigel Tao 30ca6d42b2 Rename some testdata files for consistency. 2017-06-30 10:47:00 +10:00