Commit Graph
429 Commits
Author SHA1 Message Date
Nigel Tao c3aa366c66 Clean up #ifdef style; remove unnecessary ()s. 2017-08-03 11:05:33 +10:00
Nigel Tao 6af3852fa8 Change the back-copy unroll count from 8 to 3.
We're possibly overfitting now on the very small benchmark corpus (3
files), but we can test on larger corpuses later. Algorithmically, it
does match what zlib does.

name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    188MB/s ± 0%  191MB/s ± 1%   +1.55%  (p=0.008 n=5+5)
puffs_flate_decode_10k/clang   242MB/s ± 2%  277MB/s ± 1%  +14.72%  (p=0.008 n=5+5)
puffs_flate_decode_100k/clang  273MB/s ± 1%  315MB/s ± 1%  +15.63%  (p=0.008 n=5+5)
puffs_flate_decode_1k/gcc      187MB/s ± 0%  194MB/s ± 2%   +3.56%  (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc     242MB/s ± 0%  279MB/s ± 1%  +15.29%  (p=0.016 n=5+4)
puffs_flate_decode_100k/gcc    288MB/s ± 1%  315MB/s ± 0%   +9.29%  (p=0.008 n=5+5)

Note that these are benchmark numbers for an 'ideal', hand-written C
implementation, not for Puffs' generated C code.
2017-08-03 10:53:41 +10:00
Nigel Tao fa89c397b0 Do one 8-byte load instead of six 1-byte loads.
name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    178MB/s ± 2%  188MB/s ± 0%  +5.98%  (p=0.008 n=5+5)
puffs_flate_decode_10k/clang   235MB/s ± 1%  242MB/s ± 2%  +2.69%  (p=0.032 n=5+5)
puffs_flate_decode_100k/clang  274MB/s ± 0%  273MB/s ± 1%    ~     (p=0.690 n=5+5)
puffs_flate_decode_1k/gcc      177MB/s ± 0%  187MB/s ± 0%  +5.44%  (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc     234MB/s ± 0%  242MB/s ± 0%  +3.64%  (p=0.008 n=5+5)
puffs_flate_decode_100k/gcc    275MB/s ± 0%  288MB/s ± 1%  +4.68%  (p=0.008 n=5+5)

Note that these are benchmark numbers for an 'ideal', hand-written C
implementation, not for Puffs' generated C code.
2017-08-03 10:47:44 +10:00
Nigel Tao bc92bf4ff8 Fix C decode_huffman's exit with excess high bits. 2017-08-03 10:34:37 +10:00
Nigel Tao 61c5b40bf6 Unroll the copy_history32 loop.
name                             old speed      new speed      delta
puffs_flate_decode_1k/clang       127MB/s ± 2%   140MB/s ± 0%  +10.13%  (p=0.000 n=83+95)
puffs_flate_decode_10k/clang      151MB/s ± 1%   166MB/s ± 0%  +10.38%  (p=0.000 n=82+97)
puffs_flate_decode_100k/clang     165MB/s ± 1%   177MB/s ± 0%   +7.42%  (p=0.000 n=89+94)
puffs_flate_decode_1k/gcc         155MB/s ± 2%   153MB/s ± 1%   -1.67%  (p=0.000 n=91+89)
puffs_flate_decode_10k/gcc        146MB/s ± 2%   147MB/s ± 0%   +0.41%  (p=0.000 n=86+83)
puffs_flate_decode_100k/gcc       166MB/s ± 1%   170MB/s ± 0%   +2.34%  (p=0.000 n=88+84)
2017-07-30 13:36:32 +10:00
Nigel Tao df8772dadf Remove hand-written edit to generated C code. 2017-07-30 11:30:32 +10:00
Nigel Tao 265d3a5a1a Add a hand-written C impl of decode_huffman.
This gives a rough upper bound on how fast Puffs' generated C code can
be, with a sufficiently smart Puffs compiler.

A comparison of the 'old', safe, generated C code and the 'new', unsafe,
hand-written C code:

name                           old speed     new speed     delta
puffs_flate_decode_1k/clang    127MB/s ± 2%  181MB/s ± 2%  +42.27%  (p=0.000 n=83+5)
puffs_flate_decode_10k/clang   151MB/s ± 1%  235MB/s ± 2%  +56.04%  (p=0.000 n=82+5)
puffs_flate_decode_100k/clang  165MB/s ± 1%  275MB/s ± 3%  +66.69%  (p=0.000 n=89+5)
puffs_flate_decode_1k/gcc      155MB/s ± 2%  181MB/s ± 1%  +16.96%  (p=0.000 n=91+5)
puffs_flate_decode_10k/gcc     146MB/s ± 2%  241MB/s ± 2%  +64.78%  (p=0.000 n=86+5)
puffs_flate_decode_100k/gcc    166MB/s ± 1%  277MB/s ± 3%  +66.78%  (p=0.000 n=88+5)

While we're discussing benchmark numbers, below is an overall snapshot.

std/flate

puffs_flate_decode_1k/clang       127MB/s ± 2%
puffs_flate_decode_10k/clang      151MB/s ± 1%
puffs_flate_decode_100k/clang     165MB/s ± 1%

puffs_flate_decode_1k/gcc         155MB/s ± 2%
puffs_flate_decode_10k/gcc        146MB/s ± 2%
puffs_flate_decode_100k/gcc       166MB/s ± 1%

mimic_adler32_10k                2.98GB/s ± 1%
mimic_adler32_100k               2.93GB/s ± 1%
mimic_crc32_10k                  1.38GB/s ± 1%
mimic_crc32_100k                 1.37GB/s ± 1%

mimic_flate_decode_1k             220MB/s ± 2%
mimic_flate_decode_10k            262MB/s ± 1%
mimic_flate_decode_100k           281MB/s ± 1%
mimic_gzip_decode_10k             221MB/s ± 1%
mimic_gzip_decode_100k            259MB/s ± 1%
mimic_zlib_decode_10k             242MB/s ± 2%
mimic_zlib_decode_100k            289MB/s ± 1%

std/gif

puffs_gif_lzw_decode_20k/clang    125MB/s ± 1%
puffs_gif_lzw_decode_100k/clang   173MB/s ± 1%
puffs_gif_decode_1k/clang         376MB/s ± 3%
puffs_gif_decode_10k/clang        128MB/s ± 2%
puffs_gif_decode_100k/clang       120MB/s ± 2%
puffs_gif_decode_1000k/clang      121MB/s ± 3%

puffs_gif_lzw_decode_20k/gcc      153MB/s ± 1%
puffs_gif_lzw_decode_100k/gcc     208MB/s ± 1%
puffs_gif_decode_1k/gcc           429MB/s ± 1%
puffs_gif_decode_10k/gcc          161MB/s ± 1%
puffs_gif_decode_100k/gcc         140MB/s ± 1%
puffs_gif_decode_1000k/gcc        143MB/s ± 1%

mimic_gif_decode_1k               158MB/s ± 1%
mimic_gif_decode_10k             93.9MB/s ± 1%
mimic_gif_decode_100k             100MB/s ± 1%
mimic_gif_decode_1000k            102MB/s ± 1%
2017-07-30 11:13:40 +10:00
Nigel Tao e0f87fb2a7 Add a -reps=N flag to "puffs bench". 2017-07-30 08:27:52 +10:00
Nigel Tao 0a45eee9d0 Fix some comment grammar. 2017-07-29 17:24:01 +10:00
Nigel Tao 7cf7829c9e Implement std/flate Huffman table redirects. 2017-07-29 16:25:00 +10:00
Nigel Tao 7fe783d785 Call memset instead of using an explicit loop. 2017-07-28 16:24:47 +10:00
Nigel Tao 23a4aeb7ce Turn some struct fields into local variables. 2017-07-28 16:12:45 +10:00
Nigel Tao 3a8106daa5 Refactor std/flate comments. 2017-07-28 16:03:26 +10:00
Nigel Tao 6b019831ad Check over- and under-subscribed Huffman codes. 2017-07-28 15:49:05 +10:00
Nigel Tao 3b5e858fea Add the printf prototype to generated C files. 2017-07-28 14:15:26 +10:00
Nigel Tao bfbcae4365 Decode std/flate fixed Huffman tables. 2017-07-28 13:28:31 +10:00
Nigel Tao cbcf1b6025 Remove test_puffs_flate_work_in_progress. 2017-07-28 11:33:48 +10:00
Nigel Tao d2aca16f3c Add a copy_history32 method. 2017-07-28 11:25:03 +10:00
Nigel Tao d7cb39b766 Decode base+extra length and distance pairs. 2017-07-28 09:57:39 +10:00
Nigel Tao 9e03aeabe0 Refactor std/flate's decoder.decode. 2017-07-28 09:04:34 +10:00
Nigel Tao 89d4f68036 Fill array-of-u32 Huffman table for base+extra. 2017-07-27 17:20:31 +10:00
Nigel Tao 6387834cfb Consistently use 0x89AB style for hex constants. 2017-07-27 16:16:45 +10:00
Nigel Tao 67b95c01c3 Decode literals from the lcode Huffman table. 2017-07-27 15:32:05 +10:00
Nigel Tao be3b9b81e7 Generate the lcode and dcode Huffman tables. 2017-07-27 14:56:23 +10:00
Nigel Tao bf51551eda Have init_huff take a slice of the code_lengths. 2017-07-27 14:42:26 +10:00
Nigel Tao ea935e1f8c Use array-of-u32 Huffman decoder. 2017-07-27 14:18:32 +10:00
Nigel Tao e1be6d9582 Type and bounds check "x | y | z". 2017-07-27 13:07:10 +10:00
Nigel Tao eee67e083d Add array-of-u32 Huffman decoder. 2017-07-27 11:30:43 +10:00
Nigel Tao d4ab9caa33 Add table to reverse the bits in a byte. 2017-07-26 09:05:45 +10:00
Nigel Tao 69a3b5804f Remove the unnecessary huffman_decoder struct. 2017-07-26 09:04:14 +10:00
Nigel Tao 7a955406f7 Add instructions on how to generate a hex dump. 2017-07-22 18:00:24 +10:00
Nigel Tao 59320a07d0 Rename "go generate"d files to be data.go. 2017-07-22 17:57:41 +10:00
Nigel Tao e772352036 Decode the lcode and dcode code lengths. 2017-07-22 16:45:54 +10:00
Nigel Tao 9933c971a2 Factor out a evalConstValueBinaryOp function. 2017-07-22 13:55:39 +10:00
Nigel Tao dec234779e Generate lang/check/reason.go. 2017-07-22 12:49:00 +10:00
Nigel Tao 61b771a418 Refactor the bounds checker's list of reasons. 2017-07-22 10:24:48 +10:00
Nigel Tao 389d9bc8bf Remove the placeholder code limit count. 2017-07-22 10:14:50 +10:00
Nigel Tao fd07fa1f27 Decode flate's clcode symbols. 2017-07-21 17:47:58 +10:00
Nigel Tao 74400247f2 Refactor std/flate's decoder.decode_dynamic. 2017-07-21 16:16:09 +10:00
Nigel Tao f7fb991619 Implement init_huffman. 2017-07-21 15:55:49 +10:00
Nigel Tao 02cd837d9d Add a huffman_decoder.consts field. 2017-07-21 11:48:15 +10:00
Nigel Tao 34198f3de5 Start the init_huffman implementation. 2017-07-21 11:00:56 +10:00
Nigel Tao 65c4efbe8b Add a placeholder huffman_decoder struct. 2017-07-21 10:14:59 +10:00
Nigel Tao 4b7a350a68 Rename the code_length_code_lengths field. 2017-07-21 10:09:44 +10:00
Nigel Tao fc74e7e83a Add brief description of the .zip file format. 2017-07-21 10:07:02 +10:00
Nigel Tao 62811d440d Fix topological sort. 2017-07-20 16:01:41 +10:00
Nigel Tao adadb0fd5a Add a worked example of the flate file format. 2017-07-20 15:36:59 +10:00
Nigel Tao 5a1903fdfb Add TODO to describe more keywords. 2017-07-17 17:31:47 +10:00
Nigel Tao c463f4956f Add script/print-bits.go. 2017-07-17 17:28:19 +10:00
Nigel Tao 547351cfec Add testdata/romeo.txt.{flate,zlib}. 2017-07-17 16:53:45 +10:00