Nigel Tao
29c4122360
lang: have hasher_u32.update_u32 take a roslice
2023-04-10 21:16:53 +10:00
Nigel Tao
ea3fbc37f5
lang: change const array to const roarray
2023-04-10 16:06:10 +10:00
Nigel Tao
f5eb38d76e
wuffsfmt: indent with 4 spaces (not a tab)
2023-04-09 15:28:07 +10:00
Nigel Tao
9c03b8c71d
wuffsfmt: double-indents hanging lines
2023-04-09 15:22:25 +10:00
Nigel Tao
b24e046670
Replace SIMD CRC-32 k6' magic constant with Px'
...
This is just a change of name (and a change of comment), not a change of
value.
From my colleague, Eric Biggers:
The number is not k6' = (x^32 mod P(x) << 32)' << 1 = 0x1DB710640, but
rather P(x)' = 0x1DB710641. This value represents the generator
polynomial of CRC-32 as a 33-bit value, using the CRC-32 bit ordering
convention (lowest order bit = coefficient of highest order polynomial
term). The lowest order bit is the coefficient of x^32, which by
definition is 1 for CRC-32. P(x) is used as a multiplicand in Barrett
reduction, which is the step which reduces a 64-bit value to the final
32-bit remainder. See "Algorithm 1" in the Gopal paper.
The reason that last bit doesn't actually matter is because the output
of the Barrett reduction is supposed to be fully reduced mod P(x), so
only 32 bits (the bits that represent the coefficients of x^0..x^31) are
supposed to be able to be nonzero. Therefore, in practice
implementations will just truncate the result to 32 bits. Incorrectly
using 0x1DB710640 rather than P(x) = 0x1DB710641 only causes the result
to be wrong by a factor of x^32, so it doesn't affect those 32 bits.
2021-03-18 11:21:51 +11:00
Nigel Tao
16b4b7554b
Link to SIMD CRC-32 k6' magic constant blog post
2021-03-18 11:06:21 +11:00
Nigel Tao
79c7ba0ae9
Add script/print-crc32-x86-sse42-magic-numbers.go
2021-03-07 22:18:52 +11:00
Nigel Tao
5d4fbc3ac6
Implement std/crc32 ieee_hasher.up_x86_sse42
...
On a mid-range x86_64 laptop:
----
$ ls -lh linux-5.11.3.tar.gz | awk '{print $5 " " $9}'
178M linux-5.11.3.tar.gz
$ # Debian's /usr/bin/crc32
$ time /usr/bin/crc32 /dev/stdin < linux-5.11.3.tar.gz > /dev/null
real 0m0.411s
$ # Wuffs; before
$ time gen/bin/example-crc32 < linux-5.11.3.tar.gz > /dev/null
real 0m0.137s
$ # Wuffs; after
$ time gen/bin/example-crc32 < linux-5.11.3.tar.gz > /dev/null
real 0m0.060s
----
Wuffs benchmarks:
wuffs_crc32_ieee_10k/clang9 1.71GB/s ± 0% 8.40GB/s ± 3% +391.97% (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/clang9 1.73GB/s ± 1% 10.67GB/s ± 0% +518.47% (p=0.008 n=5+5)
wuffs_crc32_ieee_10k/gcc10 2.08GB/s ± 0% 8.77GB/s ± 0% +322.05% (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/gcc10 2.10GB/s ± 0% 11.23GB/s ± 0% +435.56% (p=0.008 n=5+5)
mimic_crc32_ieee_10k 774MB/s ± 0% 776MB/s ± 0% +0.23% (p=0.016 n=5+5)
mimic_crc32_ieee_100k 773MB/s ± 0% 775MB/s ± 0% +0.24% (p=0.032 n=5+5)
wuffs_gzip_decode_10k/clang9 151MB/s ± 0% 166MB/s ± 0% +9.81% (p=0.008 n=5+5)
wuffs_gzip_decode_100k/clang9 155MB/s ± 0% 178MB/s ± 0% +14.53% (p=0.008 n=5+5)
wuffs_gzip_decode_10k/gcc10 150MB/s ± 0% 172MB/s ± 0% +14.92% (p=0.008 n=5+5)
wuffs_gzip_decode_100k/gcc10 188MB/s ± 0% 198MB/s ± 0% +5.37% (p=0.008 n=5+5)
mimic_gzip_decode_10k 121MB/s ± 0% 121MB/s ± 0% ~ (p=1.000 n=5+5)
mimic_gzip_decode_100k 143MB/s ± 0% 144MB/s ± 0% ~ (p=0.056 n=5+5)
wuffs_png_decode_image_19k_8bpp/clang9 111MB/s ± 0% 115MB/s ± 0% +3.77% (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/clang9 147MB/s ± 0% 153MB/s ± 0% +4.66% (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/clang9 404MB/s ± 0% 415MB/s ± 0% +2.75% (p=0.016 n=5+4)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9 344MB/s ± 0% 345MB/s ± 1% ~ (p=0.151 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9 329MB/s ± 0% 335MB/s ± 0% +1.78% (p=0.016 n=5+4)
wuffs_png_decode_image_4002k_24bpp/clang9 148MB/s ± 0% 153MB/s ± 0% +3.38% (p=0.008 n=5+5)
wuffs_png_decode_image_19k_8bpp/gcc10 120MB/s ± 0% 123MB/s ± 0% +2.62% (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/gcc10 153MB/s ± 0% 159MB/s ± 1% +3.54% (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/gcc10 440MB/s ± 0% 452MB/s ± 0% +2.82% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc10 357MB/s ± 0% 353MB/s ± 0% -0.95% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc10 341MB/s ± 0% 341MB/s ± 0% ~ (p=0.151 n=5+5)
wuffs_png_decode_image_4002k_24bpp/gcc10 157MB/s ± 0% 161MB/s ± 0% +2.85% (p=0.016 n=5+4)
mimic_png_decode_image_19k_8bpp 58.1MB/s ± 0% 58.1MB/s ± 0% ~ (p=0.389 n=5+5)
mimic_png_decode_image_40k_24bpp 73.2MB/s ± 0% 73.2MB/s ± 0% ~ (p=0.548 n=5+5)
mimic_png_decode_image_77k_8bpp 177MB/s ± 0% 177MB/s ± 0% +0.14% (p=0.008 n=5+5)
mimic_png_decode_image_552k_32bpp_ignore_checksum skipped
mimic_png_decode_image_552k_32bpp_verify_checksum 146MB/s ± 0% 147MB/s ± 0% +0.13% (p=0.016 n=5+5)
mimic_png_decode_image_4002k_24bpp 104MB/s ± 0% 104MB/s ± 0% ~ (p=0.556 n=5+4)
2021-03-07 12:27:45 +11:00
Nigel Tao
44acd0e20a
Add std/crc32 ieee_hasher.up_x86_sse42 placeholder
2021-03-06 11:20:50 +11:00