Commit Graph
9 Commits
Author SHA1 Message Date
Nigel Tao 29c4122360 lang: have hasher_u32.update_u32 take a roslice 2023-04-10 21:16:53 +10:00
Nigel Tao ea3fbc37f5 lang: change const array to const roarray 2023-04-10 16:06:10 +10:00
Nigel Tao f5eb38d76e wuffsfmt: indent with 4 spaces (not a tab) 2023-04-09 15:28:07 +10:00
Nigel Tao 9c03b8c71d wuffsfmt: double-indents hanging lines 2023-04-09 15:22:25 +10:00
Nigel Tao b24e046670 Replace SIMD CRC-32 k6' magic constant with Px'
This is just a change of name (and a change of comment), not a change of
value.

From my colleague, Eric Biggers:

The number is not k6' = (x^32 mod P(x) << 32)' << 1 = 0x1DB710640, but
rather P(x)' = 0x1DB710641. This value represents the generator
polynomial of CRC-32 as a 33-bit value, using the CRC-32 bit ordering
convention (lowest order bit = coefficient of highest order polynomial
term). The lowest order bit is the coefficient of x^32, which by
definition is 1 for CRC-32. P(x) is used as a multiplicand in Barrett
reduction, which is the step which reduces a 64-bit value to the final
32-bit remainder. See "Algorithm 1" in the Gopal paper.

The reason that last bit doesn't actually matter is because the output
of the Barrett reduction is supposed to be fully reduced mod P(x), so
only 32 bits (the bits that represent the coefficients of x^0..x^31) are
supposed to be able to be nonzero. Therefore, in practice
implementations will just truncate the result to 32 bits. Incorrectly
using 0x1DB710640 rather than P(x) = 0x1DB710641 only causes the result
to be wrong by a factor of x^32, so it doesn't affect those 32 bits.
2021-03-18 11:21:51 +11:00
Nigel Tao 16b4b7554b Link to SIMD CRC-32 k6' magic constant blog post 2021-03-18 11:06:21 +11:00
Nigel Tao 79c7ba0ae9 Add script/print-crc32-x86-sse42-magic-numbers.go 2021-03-07 22:18:52 +11:00
Nigel Tao 5d4fbc3ac6 Implement std/crc32 ieee_hasher.up_x86_sse42
On a mid-range x86_64 laptop:

----

$ ls -lh linux-5.11.3.tar.gz | awk '{print $5 " " $9}'
178M linux-5.11.3.tar.gz

$ # Debian's /usr/bin/crc32
$ time /usr/bin/crc32 /dev/stdin < linux-5.11.3.tar.gz > /dev/null
real    0m0.411s

$ # Wuffs; before
$ time gen/bin/example-crc32     < linux-5.11.3.tar.gz > /dev/null
real    0m0.137s

$ # Wuffs; after
$ time gen/bin/example-crc32     < linux-5.11.3.tar.gz > /dev/null
real    0m0.060s

----

Wuffs benchmarks:

wuffs_crc32_ieee_10k/clang9                               1.71GB/s ± 0%   8.40GB/s ± 3%  +391.97%  (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/clang9                              1.73GB/s ± 1%  10.67GB/s ± 0%  +518.47%  (p=0.008 n=5+5)

wuffs_crc32_ieee_10k/gcc10                                2.08GB/s ± 0%   8.77GB/s ± 0%  +322.05%  (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/gcc10                               2.10GB/s ± 0%  11.23GB/s ± 0%  +435.56%  (p=0.008 n=5+5)

mimic_crc32_ieee_10k                                       774MB/s ± 0%    776MB/s ± 0%    +0.23%  (p=0.016 n=5+5)
mimic_crc32_ieee_100k                                      773MB/s ± 0%    775MB/s ± 0%    +0.24%  (p=0.032 n=5+5)

wuffs_gzip_decode_10k/clang9                               151MB/s ± 0%    166MB/s ± 0%    +9.81%  (p=0.008 n=5+5)
wuffs_gzip_decode_100k/clang9                              155MB/s ± 0%    178MB/s ± 0%   +14.53%  (p=0.008 n=5+5)

wuffs_gzip_decode_10k/gcc10                                150MB/s ± 0%    172MB/s ± 0%   +14.92%  (p=0.008 n=5+5)
wuffs_gzip_decode_100k/gcc10                               188MB/s ± 0%    198MB/s ± 0%    +5.37%  (p=0.008 n=5+5)

mimic_gzip_decode_10k                                      121MB/s ± 0%    121MB/s ± 0%      ~     (p=1.000 n=5+5)
mimic_gzip_decode_100k                                     143MB/s ± 0%    144MB/s ± 0%      ~     (p=0.056 n=5+5)

wuffs_png_decode_image_19k_8bpp/clang9                     111MB/s ± 0%    115MB/s ± 0%    +3.77%  (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/clang9                    147MB/s ± 0%    153MB/s ± 0%    +4.66%  (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/clang9                     404MB/s ± 0%    415MB/s ± 0%    +2.75%  (p=0.016 n=5+4)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9   344MB/s ± 0%    345MB/s ± 1%      ~     (p=0.151 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9   329MB/s ± 0%    335MB/s ± 0%    +1.78%  (p=0.016 n=5+4)
wuffs_png_decode_image_4002k_24bpp/clang9                  148MB/s ± 0%    153MB/s ± 0%    +3.38%  (p=0.008 n=5+5)

wuffs_png_decode_image_19k_8bpp/gcc10                      120MB/s ± 0%    123MB/s ± 0%    +2.62%  (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/gcc10                     153MB/s ± 0%    159MB/s ± 1%    +3.54%  (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/gcc10                      440MB/s ± 0%    452MB/s ± 0%    +2.82%  (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc10    357MB/s ± 0%    353MB/s ± 0%    -0.95%  (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc10    341MB/s ± 0%    341MB/s ± 0%      ~     (p=0.151 n=5+5)
wuffs_png_decode_image_4002k_24bpp/gcc10                   157MB/s ± 0%    161MB/s ± 0%    +2.85%  (p=0.016 n=5+4)

mimic_png_decode_image_19k_8bpp                           58.1MB/s ± 0%   58.1MB/s ± 0%      ~     (p=0.389 n=5+5)
mimic_png_decode_image_40k_24bpp                          73.2MB/s ± 0%   73.2MB/s ± 0%      ~     (p=0.548 n=5+5)
mimic_png_decode_image_77k_8bpp                            177MB/s ± 0%    177MB/s ± 0%    +0.14%  (p=0.008 n=5+5)
mimic_png_decode_image_552k_32bpp_ignore_checksum               skipped
mimic_png_decode_image_552k_32bpp_verify_checksum          146MB/s ± 0%    147MB/s ± 0%    +0.13%  (p=0.016 n=5+5)
mimic_png_decode_image_4002k_24bpp                         104MB/s ± 0%    104MB/s ± 0%      ~     (p=0.556 n=5+4)
2021-03-07 12:27:45 +11:00
Nigel Tao 44acd0e20a Add std/crc32 ieee_hasher.up_x86_sse42 placeholder 2021-03-06 11:20:50 +11:00