Nigel Tao
4dfd70928b
std: add "get_quirk(key: u32) u64"
2023-07-16 12:04:09 +10:00
Nigel Tao
6e84987d33
std: change set_quirk_enabled to set_quirk
2023-04-14 21:42:15 +10:00
Nigel Tao
29c4122360
lang: have hasher_u32.update_u32 take a roslice
2023-04-10 21:16:53 +10:00
Nigel Tao
ea3fbc37f5
lang: change const array to const roarray
2023-04-10 16:06:10 +10:00
Nigel Tao
f5eb38d76e
wuffsfmt: indent with 4 spaces (not a tab)
2023-04-09 15:28:07 +10:00
Nigel Tao
9c03b8c71d
wuffsfmt: double-indents hanging lines
2023-04-09 15:22:25 +10:00
Nigel Tao
e9cb3c8b34
Use hd consistently, not xxd, in docs
2022-04-30 13:20:08 +10:00
Nigel Tao
2dddd388bc
Add crc32.ieee_hasher.up_x86_avx2
...
name old speed new speed delta
wuffs_crc32_ieee_10k/clang9 8.42GB/s ± 1% 8.12GB/s ± 1% -3.55% (p=0.000 n=9+8)
wuffs_crc32_ieee_100k/clang9 11.0GB/s ± 1% 10.9GB/s ± 1% ~ (p=0.931 n=9+9)
wuffs_crc32_ieee_10k/gcc10 9.20GB/s ± 0% 9.71GB/s ± 0% +5.54% (p=0.000 n=8+10)
wuffs_crc32_ieee_100k/gcc10 11.9GB/s ± 2% 13.1GB/s ± 1% +10.22% (p=0.000 n=10+8)
wuffs_gzip_decode_10k/clang9 219MB/s ± 1% 218MB/s ± 2% ~ (p=0.497 n=9+10)
wuffs_gzip_decode_100k/clang9 283MB/s ± 2% 281MB/s ± 1% ~ (p=0.063 n=10+10)
wuffs_gzip_decode_10k/gcc10 224MB/s ± 1% 223MB/s ± 1% ~ (p=0.489 n=9+9)
wuffs_gzip_decode_100k/gcc10 285MB/s ± 4% 290MB/s ± 0% +1.68% (p=0.004 n=10+8)
wuffs_png_decode_image_19k_8bpp/clang9 138MB/s ± 2% 138MB/s ± 1% ~ (p=0.156 n=10+9)
wuffs_png_decode_image_40k_24bpp/clang9 171MB/s ± 2% 172MB/s ± 1% ~ (p=0.971 n=10+10)
wuffs_png_decode_image_77k_8bpp/clang9 495MB/s ± 4% 504MB/s ± 1% +1.79% (p=0.000 n=9+9)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9 457MB/s ± 4% 463MB/s ± 0% +1.35% (p=0.002 n=9+9)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9 435MB/s ± 5% 445MB/s ± 1% +2.31% (p=0.000 n=10+9)
wuffs_png_decode_image_4002k_24bpp/clang9 172MB/s ± 2% 174MB/s ± 1% ~ (p=0.053 n=10+9)
wuffs_png_decode_image_19k_8bpp/gcc10 157MB/s ± 1% 157MB/s ± 1% ~ (p=0.549 n=9+10)
wuffs_png_decode_image_40k_24bpp/gcc10 187MB/s ± 1% 186MB/s ± 1% ~ (p=0.182 n=10+9)
wuffs_png_decode_image_77k_8bpp/gcc10 554MB/s ± 1% 559MB/s ± 0% +0.84% (p=0.000 n=9+9)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc10 502MB/s ± 1% 500MB/s ± 0% -0.38% (p=0.011 n=10+10)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc10 481MB/s ± 1% 478MB/s ± 0% -0.46% (p=0.002 n=8+9)
wuffs_png_decode_image_4002k_24bpp/gcc10 189MB/s ± 2% 190MB/s ± 0% +0.62% (p=0.008 n=10+9)
2021-04-09 23:12:32 +10:00
Nigel Tao
b24e046670
Replace SIMD CRC-32 k6' magic constant with Px'
...
This is just a change of name (and a change of comment), not a change of
value.
From my colleague, Eric Biggers:
The number is not k6' = (x^32 mod P(x) << 32)' << 1 = 0x1DB710640, but
rather P(x)' = 0x1DB710641. This value represents the generator
polynomial of CRC-32 as a 33-bit value, using the CRC-32 bit ordering
convention (lowest order bit = coefficient of highest order polynomial
term). The lowest order bit is the coefficient of x^32, which by
definition is 1 for CRC-32. P(x) is used as a multiplicand in Barrett
reduction, which is the step which reduces a 64-bit value to the final
32-bit remainder. See "Algorithm 1" in the Gopal paper.
The reason that last bit doesn't actually matter is because the output
of the Barrett reduction is supposed to be fully reduced mod P(x), so
only 32 bits (the bits that represent the coefficients of x^0..x^31) are
supposed to be able to be nonzero. Therefore, in practice
implementations will just truncate the result to 32 bits. Incorrectly
using 0x1DB710640 rather than P(x) = 0x1DB710641 only causes the result
to be wrong by a factor of x^32, so it doesn't affect those 32 bits.
2021-03-18 11:21:51 +11:00
Nigel Tao
16b4b7554b
Link to SIMD CRC-32 k6' magic constant blog post
2021-03-18 11:06:21 +11:00
Nigel Tao
79c7ba0ae9
Add script/print-crc32-x86-sse42-magic-numbers.go
2021-03-07 22:18:52 +11:00
Nigel Tao
5d4fbc3ac6
Implement std/crc32 ieee_hasher.up_x86_sse42
...
On a mid-range x86_64 laptop:
----
$ ls -lh linux-5.11.3.tar.gz | awk '{print $5 " " $9}'
178M linux-5.11.3.tar.gz
$ # Debian's /usr/bin/crc32
$ time /usr/bin/crc32 /dev/stdin < linux-5.11.3.tar.gz > /dev/null
real 0m0.411s
$ # Wuffs; before
$ time gen/bin/example-crc32 < linux-5.11.3.tar.gz > /dev/null
real 0m0.137s
$ # Wuffs; after
$ time gen/bin/example-crc32 < linux-5.11.3.tar.gz > /dev/null
real 0m0.060s
----
Wuffs benchmarks:
wuffs_crc32_ieee_10k/clang9 1.71GB/s ± 0% 8.40GB/s ± 3% +391.97% (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/clang9 1.73GB/s ± 1% 10.67GB/s ± 0% +518.47% (p=0.008 n=5+5)
wuffs_crc32_ieee_10k/gcc10 2.08GB/s ± 0% 8.77GB/s ± 0% +322.05% (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/gcc10 2.10GB/s ± 0% 11.23GB/s ± 0% +435.56% (p=0.008 n=5+5)
mimic_crc32_ieee_10k 774MB/s ± 0% 776MB/s ± 0% +0.23% (p=0.016 n=5+5)
mimic_crc32_ieee_100k 773MB/s ± 0% 775MB/s ± 0% +0.24% (p=0.032 n=5+5)
wuffs_gzip_decode_10k/clang9 151MB/s ± 0% 166MB/s ± 0% +9.81% (p=0.008 n=5+5)
wuffs_gzip_decode_100k/clang9 155MB/s ± 0% 178MB/s ± 0% +14.53% (p=0.008 n=5+5)
wuffs_gzip_decode_10k/gcc10 150MB/s ± 0% 172MB/s ± 0% +14.92% (p=0.008 n=5+5)
wuffs_gzip_decode_100k/gcc10 188MB/s ± 0% 198MB/s ± 0% +5.37% (p=0.008 n=5+5)
mimic_gzip_decode_10k 121MB/s ± 0% 121MB/s ± 0% ~ (p=1.000 n=5+5)
mimic_gzip_decode_100k 143MB/s ± 0% 144MB/s ± 0% ~ (p=0.056 n=5+5)
wuffs_png_decode_image_19k_8bpp/clang9 111MB/s ± 0% 115MB/s ± 0% +3.77% (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/clang9 147MB/s ± 0% 153MB/s ± 0% +4.66% (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/clang9 404MB/s ± 0% 415MB/s ± 0% +2.75% (p=0.016 n=5+4)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9 344MB/s ± 0% 345MB/s ± 1% ~ (p=0.151 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9 329MB/s ± 0% 335MB/s ± 0% +1.78% (p=0.016 n=5+4)
wuffs_png_decode_image_4002k_24bpp/clang9 148MB/s ± 0% 153MB/s ± 0% +3.38% (p=0.008 n=5+5)
wuffs_png_decode_image_19k_8bpp/gcc10 120MB/s ± 0% 123MB/s ± 0% +2.62% (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/gcc10 153MB/s ± 0% 159MB/s ± 1% +3.54% (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/gcc10 440MB/s ± 0% 452MB/s ± 0% +2.82% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc10 357MB/s ± 0% 353MB/s ± 0% -0.95% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc10 341MB/s ± 0% 341MB/s ± 0% ~ (p=0.151 n=5+5)
wuffs_png_decode_image_4002k_24bpp/gcc10 157MB/s ± 0% 161MB/s ± 0% +2.85% (p=0.016 n=5+4)
mimic_png_decode_image_19k_8bpp 58.1MB/s ± 0% 58.1MB/s ± 0% ~ (p=0.389 n=5+5)
mimic_png_decode_image_40k_24bpp 73.2MB/s ± 0% 73.2MB/s ± 0% ~ (p=0.548 n=5+5)
mimic_png_decode_image_77k_8bpp 177MB/s ± 0% 177MB/s ± 0% +0.14% (p=0.008 n=5+5)
mimic_png_decode_image_552k_32bpp_ignore_checksum skipped
mimic_png_decode_image_552k_32bpp_verify_checksum 146MB/s ± 0% 147MB/s ± 0% +0.13% (p=0.016 n=5+5)
mimic_png_decode_image_4002k_24bpp 104MB/s ± 0% 104MB/s ± 0% ~ (p=0.556 n=5+4)
2021-03-07 12:27:45 +11:00
Nigel Tao
44acd0e20a
Add std/crc32 ieee_hasher.up_x86_sse42 placeholder
2021-03-06 11:20:50 +11:00
Nigel Tao
21cc4fa8ba
Use more arm_neon_uAxB types and methods
2021-02-24 22:22:30 +11:00
Nigel Tao
9c1af68889
Factor out std/crc32/common_up_arm_crc32.wuffs
2021-02-16 22:27:33 +11:00
Nigel Tao
84f3592dbe
Add base.arm_crc32_u32 type
...
On a Raspberry Pi 4 (32-bit armv7l) with -march=native and -mfpu=neon
("native" means "armv8-a+crc+simd"):
wuffs_crc32_ieee_10k/clang9 825MB/s ± 0% 6802MB/s ± 0% +724.00% (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/clang9 837MB/s ± 0% 6199MB/s ± 0% +640.25% (p=0.008 n=5+5)
wuffs_crc32_ieee_10k/gcc8 960MB/s ± 0% 4495MB/s ± 0% +368.35% (p=0.016 n=4+5)
wuffs_crc32_ieee_100k/gcc8 961MB/s ± 0% 4447MB/s ± 0% +363.01% (p=0.016 n=5+4)
mimic_crc32_ieee_10k 638MB/s ± 0% 631MB/s ± 0% -1.17% (p=0.008 n=5+5)
mimic_crc32_ieee_100k 639MB/s ± 0% 639MB/s ± 0% +0.02% (p=0.008 n=5+5)
wuffs_gzip_decode_10k/clang9 83.0MB/s ± 0% 90.7MB/s ± 0% +9.26% (p=0.008 n=5+5)
wuffs_gzip_decode_100k/clang9 100MB/s ± 0% 112MB/s ± 0% +11.91% (p=0.008 n=5+5)
wuffs_gzip_decode_10k/gcc8 86.1MB/s ± 0% 91.9MB/s ± 0% +6.72% (p=0.008 n=5+5)
wuffs_gzip_decode_100k/gcc8 102MB/s ± 0% 112MB/s ± 0% +9.34% (p=0.008 n=5+5)
mimic_gzip_decode_10k 98.8MB/s ± 0% 98.2MB/s ± 0% -0.60% (p=0.008 n=5+5)
mimic_gzip_decode_100k 128MB/s ± 0% 128MB/s ± 0% -0.08% (p=0.008 n=5+5)
wuffs_png_decode_image_19k_8bpp/clang9 67.6MB/s ± 0% 70.8MB/s ± 0% +4.72% (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/clang9 92.8MB/s ± 0% 98.8MB/s ± 0% +6.49% (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/clang9 255MB/s ± 0% 269MB/s ± 0% +5.44% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9 123MB/s ± 0% 123MB/s ± 0% +0.13% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9 110MB/s ± 0% 112MB/s ± 0% +1.46% (p=0.008 n=5+5)
wuffs_png_decode_image_4002k_24bpp/clang9 88.4MB/s ± 0% 92.8MB/s ± 0% +4.90% (p=0.008 n=5+5)
wuffs_png_decode_image_19k_8bpp/gcc8 67.2MB/s ± 0% 70.8MB/s ± 0% +5.37% (p=0.008 n=5+5)
wuffs_png_decode_image_40k_24bpp/gcc8 94.0MB/s ± 0% 98.2MB/s ± 0% +4.45% (p=0.008 n=5+5)
wuffs_png_decode_image_77k_8bpp/gcc8 236MB/s ± 0% 248MB/s ± 0% +4.84% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc8 114MB/s ± 0% 115MB/s ± 0% +1.11% (p=0.008 n=5+5)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc8 102MB/s ± 0% 104MB/s ± 0% +2.17% (p=0.008 n=5+5)
wuffs_png_decode_image_4002k_24bpp/gcc8 88.8MB/s ± 0% 94.3MB/s ± 0% +6.21% (p=0.008 n=5+5)
mimic_png_decode_image_19k_8bpp 43.7MB/s ± 1% 43.4MB/s ± 1% ~ (p=0.095 n=5+5)
mimic_png_decode_image_40k_24bpp 54.6MB/s ± 0% 53.9MB/s ± 0% -1.25% (p=0.008 n=5+5)
mimic_png_decode_image_77k_8bpp 125MB/s ± 0% 124MB/s ± 0% -0.73% (p=0.008 n=5+5)
mimic_png_decode_image_552k_32bpp_ignore_checksum skipped
mimic_png_decode_image_552k_32bpp_verify_checksum 102MB/s ± 0% 101MB/s ± 0% -0.95% (p=0.008 n=5+5)
mimic_png_decode_image_4002k_24bpp 81.3MB/s ± 0% 81.0MB/s ± 0% -0.46% (p=0.016 n=4+5)
2021-02-13 22:54:47 +11:00
Nigel Tao
81ba41e251
Add slice uintptr_low_12_bits method
2021-02-13 17:19:35 +11:00
Nigel Tao
0834b40d22
Add std/crc32 ieee_hasher.up_arm_crc32
2021-02-13 17:02:02 +11:00
Nigel Tao
a1a70da539
Add iterate advance parameter
2021-01-11 00:03:44 +11:00
Nigel Tao
532e18c4a5
Add colons to const syntax
2020-04-14 14:57:30 +10:00
Nigel Tao
74871344f0
Capitalize Wuffs const names
2020-04-13 15:52:36 +10:00
Nigel Tao
4bc42db1d3
Add set_quirk_enabled interface methods
2020-04-12 20:53:54 +10:00
Nigel Tao
b40671ec84
Have wuffsfmt put underscores in long numbers
2020-02-08 18:21:29 +11:00
Nigel Tao
ea54c2ea11
Add syntax support for implementing interfaces
2020-01-09 15:45:29 +11:00
Nigel Tao
8ff9cdb9a6
Rename hasher update_u32 method
2019-11-20 21:07:22 +11:00
Nigel Tao
a21ad25bb8
Rename x T to x: T for vars
2019-10-12 21:55:13 +11:00
Nigel Tao
1520a1f345
Rename x T to x: T for fields and func args
2019-10-12 21:49:04 +11:00
Nigel Tao
5b4e3647c0
Reformat x:y as x: y
2019-10-12 21:21:39 +11:00
Nigel Tao
d552f88738
Make Iterate contain Assign's not Var's
2018-12-27 10:35:27 +11:00
Nigel Tao
25776a724e
Place more vars at the top of functions
2018-12-26 19:14:07 +11:00
Nigel Tao
03b046d774
Add script/print-crc32-example.go
2018-11-17 08:29:48 +11:00
Nigel Tao
34fe051cac
Add commentary about the CRC-32 implementation
2018-11-11 10:25:19 +11:00
Nigel Tao
99597e1ff1
Raise the N in crc32's slicing-by-N from 8 to 16
...
name old speed new speed delta
wuffs_crc32_ieee_10k/clang 2.35GB/s ± 0% 2.86GB/s ± 1% +21.45% (p=0.000 n=47+5)
wuffs_crc32_ieee_100k/clang 2.35GB/s ± 0% 2.86GB/s ± 2% +21.84% (p=0.000 n=48+5)
wuffs_crc32_ieee_10k/gcc 2.69GB/s ± 0% 3.36GB/s ± 1% +25.10% (p=0.000 n=47+5)
wuffs_crc32_ieee_100k/gcc 2.68GB/s ± 1% 3.39GB/s ± 0% +26.39% (p=0.000 n=44+5)
wuffs_gzip_decode_10k/clang 213MB/s ± 0% 215MB/s ± 0% +0.95% (p=0.000 n=46+5)
wuffs_gzip_decode_100k/clang 237MB/s ± 0% 240MB/s ± 0% +1.31% (p=0.000 n=48+5)
wuffs_gzip_decode_10k/gcc 241MB/s ± 1% 246MB/s ± 0% +2.00% (p=0.000 n=45+5)
wuffs_gzip_decode_100k/gcc 283MB/s ± 1% 290MB/s ± 0% +2.25% (p=0.000 n=48+5)
2018-10-07 21:01:18 +11:00
Nigel Tao
c1af1e2c82
Rename $(etc) to [etc]
2018-08-18 10:11:43 +10:00
Nigel Tao
6236e7205f
Remove packageid
2018-08-12 21:44:49 +10:00
Nigel Tao
ca06083f8a
Rename the implicit in variable to args
2018-07-29 10:26:22 +10:00
Nigel Tao
9919d78245
Change func out-type from struct to bare type
2018-07-29 10:02:06 +10:00
Nigel Tao
dfc8e14f4f
Fix some hasher methods' effects
2018-05-12 18:01:24 +10:00
Nigel Tao
b65fdee306
Make the else-length-1 iterate tail explicit
2018-05-11 10:22:24 +10:00
Nigel Tao
282b9c4b20
Place iterate params after vars, not before
2018-05-10 09:38:14 +10:00
Nigel Tao
79aded4797
Introduce a =: operator
2018-05-10 09:31:58 +10:00
Nigel Tao
1702c1e303
Rename iterate's step to length
2018-05-05 08:25:23 +10:00
Nigel Tao
58cd282138
Unroll the std/crc32 loop
...
name old speed new speed delta
wuffs_crc32_ieee_10k/clang 2.30GB/s ± 1% 2.37GB/s ± 0% +3.31% (p=0.016 n=5+4)
wuffs_crc32_ieee_100k/clang 2.30GB/s ± 0% 2.35GB/s ± 0% +2.39% (p=0.008 n=5+5)
wuffs_crc32_ieee_10k/gcc 2.49GB/s ± 1% 2.72GB/s ± 0% +9.19% (p=0.008 n=5+5)
wuffs_crc32_ieee_100k/gcc 2.50GB/s ± 0% 2.70GB/s ± 1% +8.18% (p=0.008 n=5+5)
wuffs_gzip_decode_10k/clang 206MB/s ± 2% 207MB/s ± 0% ~ (p=1.000 n=5+5)
wuffs_gzip_decode_100k/clang 231MB/s ± 2% 235MB/s ± 0% +1.57% (p=0.008 n=5+5)
wuffs_gzip_decode_10k/gcc 234MB/s ± 0% 235MB/s ± 1% ~ (p=0.095 n=5+5)
wuffs_gzip_decode_100k/gcc 267MB/s ± 0% 266MB/s ± 2% ~ (p=0.690 n=5+5)
2018-05-04 14:00:04 +10:00
Nigel Tao
81edda0413
Remove an obsolete TODO
2018-05-04 13:31:48 +10:00
Nigel Tao
96a549814d
Change iterate variable types from ptr to slice
2018-05-04 13:20:34 +10:00
Nigel Tao
6f6d4a2530
Use p[0] instead of deref p inside an iterate
2018-05-04 12:26:34 +10:00
Nigel Tao
c9ec81f0d6
Remove (ptr base.u8).read_u32le method
2018-05-04 12:11:42 +10:00
Nigel Tao
f64bf4af61
Allow p[i] for ptr-typed p; speed up std/crc32
...
name old speed new speed delta
wuffs_crc32_ieee_10k/clang 483MB/s ± 1% 2263MB/s ± 3% +368.94% (p=0.000 n=48+5)
wuffs_crc32_ieee_100k/clang 482MB/s ± 1% 2257MB/s ± 3% +367.96% (p=0.000 n=48+5)
wuffs_crc32_ieee_10k/gcc 419MB/s ± 1% 2473MB/s ± 1% +489.93% (p=0.000 n=48+5)
wuffs_crc32_ieee_100k/gcc 419MB/s ± 1% 2465MB/s ± 1% +488.02% (p=0.000 n=47+5)
wuffs_gzip_decode_10k/clang 153MB/s ± 1% 208MB/s ± 0% +35.30% (p=0.000 n=48+5)
wuffs_gzip_decode_100k/clang 168MB/s ± 1% 236MB/s ± 0% +40.67% (p=0.000 n=50+4)
wuffs_gzip_decode_10k/gcc 156MB/s ± 1% 231MB/s ± 3% +47.91% (p=0.000 n=50+5)
wuffs_gzip_decode_100k/gcc 172MB/s ± 1% 267MB/s ± 0% +55.64% (p=0.000 n=48+5)
2018-05-04 11:59:48 +10:00
Nigel Tao
e72f957eca
Add (ptr base.u8).read_u32le method
2018-05-04 10:42:05 +10:00
Nigel Tao
33746151aa
Generate placeholders for an iterate step:8 body
2018-05-04 09:18:44 +10:00