mirror of
https://github.com/ddidderr/wuffs.git
synced 2026-11-18 00:32:11 +01:00
std/softfloat: add bit-exact IEEE 754 binary64/binary32 arithmetic
Wuffs the Language has no floating point types, but some reference implementations (notably the brotli encoder) make decisions with double and float arithmetic and libm's log2. Porting them with byte-identical output needs bit-exact floating point results. This package implements f64 and f32 add, sub, mul, div, fma, comparisons, min/max, integer conversions and f64<->f32 conversions on bit patterns, with round-to-nearest-even, subnormals, infinities and NaNs, following the structure of Berkeley SoftFloat 3. It also ports glibc 2.39's x86-64 __log2_fma log2 implementation (the variant selected on CPUs with FMA and AVX2), following the shipped machine code's fused operations. Verification: test/c/std/softfloat.c checks every operation against the C compiler's native arithmetic on edge cases and random operands. The script/manual-test-softfloat-*.c programs run 10^8 random operations per operation group and compare log2_u64 against libm for every integer in [1, 2^32) plus 10^7 larger values; all with zero mismatches. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011NoosQNe7DKWnqDCBeqw1p
This commit is contained in:
9 files changed
+26111
No files matched your search
@@ -0,0 +1,189 @@
|
||||
# Soft Float
|
||||
|
||||
`std/softfloat` implements IEEE 754 binary64 (`f64`, C's `double`) and binary32
|
||||
(`f32`, C's `float`) arithmetic using only integer operations. Wuffs the
|
||||
Language has no floating point types, but some file formats' reference
|
||||
implementations make decisions (e.g. a compressor choosing between encodings)
|
||||
with floating point arithmetic. Porting them to Wuffs with byte-for-byte
|
||||
identical output needs bit-exact floating point results. The motivating user is
|
||||
a Wuffs port of the brotli encoder, whose cost models use `double` and `float`
|
||||
arithmetic and libm's `log2`.
|
||||
|
||||
Values are passed around as their bit patterns: an `f64` is a `base.u64` and an
|
||||
`f32` is a `base.u32`, the same bits that C's `memcpy` from a `double` or
|
||||
`float` would give. All of the functions are pure methods on the `arith`
|
||||
struct, which has no state (other than Wuffs' usual initialization check).
|
||||
|
||||
// C example. The bits of 0.1 and 0.2 are 0x3FB999999999999A and
|
||||
// 0x3FC999999999999A. Their sum is 0x3FD3333333333334 (0.30000000000000004).
|
||||
wuffs_softfloat__arith sf;
|
||||
wuffs_softfloat__arith__initialize(&sf, sizeof sf, WUFFS_VERSION, 0);
|
||||
uint64_t sum = wuffs_softfloat__arith__f64_add(
|
||||
&sf, 0x3FB999999999999A, 0x3FC999999999999A);
|
||||
|
||||
// Wuffs example, with a "sf : softfloat.arith" field in this struct.
|
||||
sum = this.sf.f64_add(a: 0x3FB9_9999_9999_999A, b: 0x3FC9_9999_9999_999A)
|
||||
|
||||
|
||||
## API
|
||||
|
||||
binary64:
|
||||
|
||||
- `f64_add(a, b)`, `f64_sub(a, b)`, `f64_mul(a, b)`, `f64_div(a, b)`: C's `a +
|
||||
b`, `a - b`, `a * b` and `a / b`.
|
||||
- `f64_fma(a, b, c)`: C's `fma(a, b, c)`, i.e. `(a * b) + c` with a single
|
||||
rounding.
|
||||
- `f64_neg(a)`, `f64_abs(a)`: flip or clear the sign bit (C's `-a` and
|
||||
`fabs(a)`).
|
||||
- `f64_is_nan(a)`: 1 if `a` is a NaN, else 0.
|
||||
- `f64_cmp(a, b)`: 0 if `a < b`, 1 if `a == b`, 2 if `a > b`, 3 if unordered
|
||||
(either is a NaN).
|
||||
- `f64_eq`, `f64_ne`, `f64_lt`, `f64_le`, `f64_gt`, `f64_ge`: C's `==`, `!=`,
|
||||
`<`, `<=`, `>` and `>=`, returning 1 (true) or 0 (false). Any comparison
|
||||
involving a NaN is false (except `!=`, which is true) and `-0 == +0`. These
|
||||
return `base.u32` instead of `base.bool` because the Wuffs-to-C code
|
||||
generator cannot (yet) emit public functions that return a `base.bool`.
|
||||
- `f64_min(a, b)`, `f64_max(a, b)`: exactly brotli's `BROTLI_MIN(double, a, b)`
|
||||
and `BROTLI_MAX(double, a, b)`, which are `(a < b) ? a : b` and `(a > b) ? a :
|
||||
b`. These are not IEEE 754 `minNum` / `maxNum`: given a NaN, or given `-0`
|
||||
and `+0`, they return the second argument.
|
||||
- `f64_from_u64(a)`, `f64_from_u32(a)`, `f64_from_i64(a)`: C's `(double)a`,
|
||||
rounding to nearest (exact for `u32`, and for 64-bit integers whose magnitude
|
||||
is at most `1 << 53`).
|
||||
- `f64_to_u64_trunc(a)`: C's `(uint64_t)a`, truncating towards zero. C only
|
||||
defines this for `a` in the range `(-1, 1 << 64)`. Out of that range, this
|
||||
function returns 0 for NaNs and negative values and `0xFFFF_FFFF_FFFF_FFFF`
|
||||
for large values, but callers should not rely on that.
|
||||
- `f64_to_f32(a)`: C's `(float)a`, rounding to nearest (ties to even),
|
||||
overflowing to infinity and underflowing to subnormals or zero.
|
||||
- `log2_u64(v)`: glibc's `log2((double)v)`, see below. `log2_u64(0)` is `-inf`.
|
||||
- `log2_f64(x)`: glibc's `log2(x)`, for any `x` (including subnormals, zeroes,
|
||||
negative numbers, infinities and NaNs).
|
||||
|
||||
binary32:
|
||||
|
||||
- `f32_add`, `f32_sub`, `f32_mul`, `f32_div`: C's `a + b`, `a - b`, `a * b` and
|
||||
`a / b` on `float`s (evaluated in `float` precision, i.e. C's
|
||||
`FLT_EVAL_METHOD == 0`, as on x86-64).
|
||||
- `f32_neg`, `f32_abs`, `f32_is_nan`, `f32_cmp`, `f32_eq`, `f32_ne`, `f32_lt`,
|
||||
`f32_le`, `f32_gt`, `f32_ge`, `f32_min`, `f32_max`: like their `f64`
|
||||
counterparts.
|
||||
- `f32_to_f64(a)`: C's `(double)a`, which is exact.
|
||||
- `f32_from_u32(a)`, `f32_from_i32(a)`, `f32_from_u64(a)`: C's `(float)a`,
|
||||
rounding to nearest.
|
||||
|
||||
All arithmetic rounds to nearest, ties to even (the default IEEE 754 rounding
|
||||
mode, and the only one implemented), and handles signed zeroes, subnormals
|
||||
(gradual underflow, no flush-to-zero), infinities and NaNs as IEEE 754
|
||||
requires. Exception flags (inexact, overflow, etc.) are not reported.
|
||||
|
||||
NaNs: whenever IEEE 754 (and x86 hardware) produces a NaN, so does this
|
||||
package, but the NaN's payload and sign bit are not guaranteed to match any
|
||||
particular hardware. Invalid operations like `0 / 0` produce `0xFFF8...` (x86's
|
||||
"default NaN") and NaN operands are propagated (quietened), preferring the
|
||||
first NaN operand.
|
||||
|
||||
|
||||
## Implementation
|
||||
|
||||
The `f64` operations follow the structure of John Hauser's Berkeley SoftFloat
|
||||
(release 3): unpack the sign, exponent and significand, handle special cases,
|
||||
compute an exact or sticky-bit-augmented integer significand (with 10 extra
|
||||
low bits for guard, round and sticky), then normalize and round in one place
|
||||
(`round_pack_f64`). Multiplication and fused multiply-add compute exact 128-bit
|
||||
products using 32-bit limbs (Wuffs has no 128-bit integers). Division is exact
|
||||
integer long division, 11 quotient bits per (64-bit integer) division step,
|
||||
with the final remainder as the sticky bit. Leading zero counts use a 256-entry
|
||||
table.
|
||||
|
||||
The `f32` add, sub, mul and div operations convert their operands exactly to
|
||||
`f64`, perform the `f64` operation and then round to `f32`. This double
|
||||
rounding is innocuous: binary64's 53 bits of precision are more than `(2 * 24)
|
||||
+ 2`, which suffices for `+`, `-`, `*` and `/` (S. A. Figueroa, "When is double
|
||||
rounding innocuous?", ACM SIGNUM Newsletter, 1995). The tests confirm this
|
||||
against hardware, including for subnormal `f32` results.
|
||||
|
||||
|
||||
## log2
|
||||
|
||||
The encoder being ported calls libm's `log2((double)v)` for integers `v`. glibc's
|
||||
`log2` is not correctly rounded (its documented maximum error is about 0.55
|
||||
ULP), so a correctly rounded `log2` would not match it for every input.
|
||||
Instead, `log2_f64` is a bit-exact port of the glibc function itself. (Note
|
||||
that brotli's `FastLog2(v)` only calls `log2` for `v >= 256`. For smaller `v`
|
||||
it reads `kBrotliLog2Table`, whose entries are `float` literals widened to
|
||||
`double`, which a port should copy verbatim rather than compute.)
|
||||
|
||||
Specifically, it ports glibc 2.39's `__log2_fma`, the variant of
|
||||
`sysdeps/ieee754/dbl-64/e_log2.c` (by Szabolcs Nagy, from ARM's
|
||||
optimized-routines) that glibc's x86-64 `log2` IFUNC resolves to on CPUs with
|
||||
FMA and AVX2. Compiling that C code with `-mfma` lets the compiler contract
|
||||
some (but not all) `a * b + c` expressions into fused multiply-adds, which
|
||||
changes the rounding, so the port follows the machine code of the shipped
|
||||
Ubuntu glibc 2.39 `libm.so.6`, not just the C source. The exact sequence of
|
||||
operations (9 `fma`s, 3 multiplications and 6 additions / subtractions on the
|
||||
main path; 13 `fma`s, 3 multiplications and 4 additions / subtractions on the
|
||||
separate near-1.0 path) was transcribed from its disassembly, and the
|
||||
constants (`InvLn2hi`, `InvLn2lo`, the two polynomials and the 64-entry
|
||||
`invc`/`logc` table) were read from its `__log2_data` and confirmed against
|
||||
glibc's `e_log2_data.c`. Since every softfloat operation is correctly rounded,
|
||||
the port then matches glibc by construction.
|
||||
|
||||
Other libm implementations (e.g. glibc's non-FMA variant on older x86-64 CPUs,
|
||||
musl, macOS or MSVC) can produce different `log2` results for some inputs. The
|
||||
port matches the configuration listed above, which is the reference
|
||||
configuration for the brotli port.
|
||||
|
||||
|
||||
## Verification
|
||||
|
||||
`test/c/std/softfloat.c` (run by `wuffs test std/softfloat`, under both gcc and
|
||||
clang) checks every operation against the C compiler's own `double` and `float`
|
||||
arithmetic on every pair of 34 (f64) or 23 (f32) edge-case values (±0,
|
||||
±min/max subnormal, ±min normal, ±1, ±max, ±inf, NaNs, f32 rounding
|
||||
boundaries, etc.), on those edge cases paired with pseudo-random values, and on
|
||||
250,000 pseudo-random (deterministic xorshift64*) operand pairs biased towards
|
||||
subnormals, huge values, nearby exponents (cancellation) and sparse or dense
|
||||
fractions (exact results and rounding ties). It also checks brotli-specific
|
||||
patterns such as `(size_t)(x + 0.5)`, `(float)FastLog2(n) + 2` and the
|
||||
`literal_costs` Kahan summation. `fma` and `log2` results are checked against
|
||||
golden checksums; with `wuffs test -mimic std/softfloat` the same checksums are
|
||||
also recomputed from libm's `fma` and `log2`.
|
||||
|
||||
`script/manual-test-softfloat-ops.c` runs much longer randomized comparisons
|
||||
(by default 10⁸ iterations per group of operations) against the hardware and
|
||||
libm. Result, on an x86-64 machine (gcc 13 `-O2`, no `-mfma`, no
|
||||
`-ffast-math`):
|
||||
|
||||
f64_add 100000000 iterations, 0 mismatches, 8.0 seconds
|
||||
f64_sub 100000000 iterations, 0 mismatches, 8.1 seconds
|
||||
f64_mul 100000000 iterations, 0 mismatches, 8.0 seconds
|
||||
f64_div 100000000 iterations, 0 mismatches, 11.7 seconds
|
||||
f64_fma 100000000 iterations, 0 mismatches, 13.0 seconds
|
||||
f64_cmp 100000000 iterations, 0 mismatches, 7.5 seconds
|
||||
f64_predicates 100000000 iterations, 0 mismatches, 7.9 seconds
|
||||
f64_min_max 100000000 iterations, 0 mismatches, 7.7 seconds
|
||||
f32_add 100000000 iterations, 0 mismatches, 10.3 seconds
|
||||
f32_sub 100000000 iterations, 0 mismatches, 10.3 seconds
|
||||
f32_mul 100000000 iterations, 0 mismatches, 10.0 seconds
|
||||
f32_div 100000000 iterations, 0 mismatches, 14.3 seconds
|
||||
f32_cmp 100000000 iterations, 0 mismatches, 8.3 seconds
|
||||
f32_predicates 100000000 iterations, 0 mismatches, 8.5 seconds
|
||||
f32_min_max 100000000 iterations, 0 mismatches, 8.3 seconds
|
||||
conversions 100000000 iterations, 0 mismatches, 23.3 seconds
|
||||
brotli_patterns 25000000 iterations, 0 mismatches, 14.4 seconds
|
||||
log2_f64 25000000 iterations, 0 mismatches, 8.0 seconds
|
||||
PASS: 0 mismatches in total
|
||||
(3 minutes 9 seconds in total, single-threaded)
|
||||
|
||||
`script/manual-test-softfloat-log2.c` compares `log2_u64(v)` to glibc's
|
||||
`log2((double)v)` for every `v` in `[1, 1 << 32)` and for 10⁷ pseudo-random `v`
|
||||
in `[1 << 32, 1 << 53)`. Result (glibc 2.39, x86-64 with FMA and AVX2, 4
|
||||
threads):
|
||||
|
||||
exhaustive: 4294967295 values checked, 0 mismatches, 376 seconds
|
||||
random: 10000000 values checked, 0 mismatches, 379 seconds in total
|
||||
PASS: 0 mismatches in total
|
||||
(6 minutes 20 seconds wall clock in total)
|
||||
|
||||
No exception table is needed.
|
||||
@@ -0,0 +1,950 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// Package softfloat implements IEEE 754 binary64 ("f64", C's double) and
|
||||
// binary32 ("f32", C's float) arithmetic using only integer operations.
|
||||
//
|
||||
// Values are passed around as their bit patterns: an f64 is a base.u64 and an
|
||||
// f32 is a base.u32. Every operation rounds to nearest, ties to even, and
|
||||
// handles signed zeroes, subnormals, infinities and NaNs. The results are
|
||||
// bit-for-bit identical to x86-64 SSE2 hardware (and any other conforming
|
||||
// IEEE 754 implementation), except that NaN payloads (and NaN signs) are not
|
||||
// guaranteed to match: callers should only rely on a result being "some NaN".
|
||||
//
|
||||
// See README.md for more details.
|
||||
|
||||
pub struct arith?(
|
||||
util : base.utility,
|
||||
)
|
||||
|
||||
pri const F64_SIGN : base.u64 = 0x8000_0000_0000_0000
|
||||
pri const F64_ABS : base.u64 = 0x7FFF_FFFF_FFFF_FFFF
|
||||
pri const F64_INF : base.u64 = 0x7FF0_0000_0000_0000
|
||||
pri const F64_FRAC : base.u64 = 0x000F_FFFF_FFFF_FFFF
|
||||
pri const F64_HIDDEN : base.u64 = 0x0010_0000_0000_0000
|
||||
pri const F64_QUIET : base.u64 = 0x0008_0000_0000_0000
|
||||
|
||||
// F64_DEFAULT_NAN is what x86 hardware produces for invalid operations such
|
||||
// as (0 / 0) or (inf - inf).
|
||||
pri const F64_DEFAULT_NAN : base.u64 = 0xFFF8_0000_0000_0000
|
||||
|
||||
// -------- Bit twiddling helpers.
|
||||
|
||||
// clz64 returns the number of leading zero bits of a, which must be non-zero.
|
||||
// If a is zero, it returns 63.
|
||||
pri func arith.clz64(a: base.u64) base.u32[..= 63] {
|
||||
var x : base.u64
|
||||
var n32 : base.u32[..= 32]
|
||||
var n16 : base.u32[..= 16]
|
||||
var n8 : base.u32[..= 8]
|
||||
|
||||
x = args.a
|
||||
if (x >> 32) == 0 {
|
||||
x = x ~mod<< 32
|
||||
n32 = 32
|
||||
}
|
||||
if (x >> 48) == 0 {
|
||||
x = x ~mod<< 16
|
||||
n16 = 16
|
||||
}
|
||||
if (x >> 56) == 0 {
|
||||
x = x ~mod<< 8
|
||||
n8 = 8
|
||||
}
|
||||
return (n32 + n16 + n8 + ((CLZ8[x >> 56] as base.u32) & 7)) & 63
|
||||
}
|
||||
|
||||
// shift_right_jam64 returns (a >> dist), except that if any non-zero bits
|
||||
// were shifted out then the least significant bit of the result is set. This
|
||||
// "sticky bit" is needed to round correctly.
|
||||
pri func arith.shift_right_jam64(a: base.u64, dist: base.u32) base.u64 {
|
||||
var d : base.u32
|
||||
|
||||
if args.dist == 0 {
|
||||
return args.a
|
||||
} else if args.dist < 64 {
|
||||
d = args.dist
|
||||
if (args.a ~mod<< (64 - d)) <> 0 {
|
||||
return (args.a >> d) | 1
|
||||
}
|
||||
return args.a >> d
|
||||
} else if args.a <> 0 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// jam128_hi and jam128_lo return the high and low 64-bit halves of the
|
||||
// 128-bit value ((hi << 64) | lo) shifted right by dist, with a sticky bit
|
||||
// (see shift_right_jam64).
|
||||
pri func arith.jam128_hi(hi: base.u64, lo: base.u64, dist: base.u32) base.u64 {
|
||||
if args.dist < 64 {
|
||||
return args.hi >> args.dist
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pri func arith.jam128_lo(hi: base.u64, lo: base.u64, dist: base.u32) base.u64 {
|
||||
var d : base.u32
|
||||
var sticky : base.u64
|
||||
|
||||
if args.dist == 0 {
|
||||
return args.lo
|
||||
} else if args.dist < 64 {
|
||||
d = args.dist
|
||||
if (args.lo ~mod<< (64 - d)) <> 0 {
|
||||
sticky = 1
|
||||
}
|
||||
return (args.hi ~mod<< (64 - d)) | (args.lo >> d) | sticky
|
||||
} else if args.dist < 128 {
|
||||
d = args.dist - 64
|
||||
if d > 0 {
|
||||
if ((args.hi ~mod<< (64 - d)) | args.lo) <> 0 {
|
||||
sticky = 1
|
||||
}
|
||||
} else if args.lo <> 0 {
|
||||
sticky = 1
|
||||
}
|
||||
return (args.hi >> d) | sticky
|
||||
} else if (args.hi | args.lo) <> 0 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// mul_hi64 returns the high 64 bits of the 128-bit product (a * b).
|
||||
pri func arith.mul_hi64(a: base.u64, b: base.u64) base.u64 {
|
||||
var a0 : base.u64[..= 0xFFFF_FFFF]
|
||||
var a1 : base.u64[..= 0xFFFF_FFFF]
|
||||
var b0 : base.u64[..= 0xFFFF_FFFF]
|
||||
var b1 : base.u64[..= 0xFFFF_FFFF]
|
||||
var p00 : base.u64
|
||||
var p01 : base.u64
|
||||
var p10 : base.u64
|
||||
var p11 : base.u64
|
||||
var mid : base.u64
|
||||
|
||||
a0 = args.a & 0xFFFF_FFFF
|
||||
a1 = args.a >> 32
|
||||
b0 = args.b & 0xFFFF_FFFF
|
||||
b1 = args.b >> 32
|
||||
p00 = a0 * b0
|
||||
p01 = a0 * b1
|
||||
p10 = a1 * b0
|
||||
p11 = a1 * b1
|
||||
mid = (p00 >> 32) + (p01 & 0xFFFF_FFFF) + (p10 & 0xFFFF_FFFF)
|
||||
return ((p11 ~mod+ (p01 >> 32)) ~mod+ (p10 >> 32)) ~mod+ (mid >> 32)
|
||||
}
|
||||
|
||||
// -------- f64 rounding and packing.
|
||||
//
|
||||
// Exponents are passed as base.u32 values holding two's complement signed
|
||||
// integers (so that 0xFFFF_FFFF means -1), manipulated with ~mod arithmetic.
|
||||
//
|
||||
// The round_pack_f64 sig argument holds a significand with its implicit
|
||||
// (hidden) bit at bit 62 and 10 extra bits of precision at the bottom (the
|
||||
// least significant of which is typically sticky). The value represented is
|
||||
// (sig * 2**(exp - 1084)), so exp is one less than the biased exponent.
|
||||
|
||||
pri func arith.round_pack_f64(sign: base.u64, exp: base.u32, sig: base.u64) base.u64 {
|
||||
var e : base.u32
|
||||
var s : base.u64
|
||||
var round_bits : base.u64
|
||||
|
||||
e = args.exp
|
||||
s = args.sig
|
||||
round_bits = s & 0x3FF
|
||||
if e >= 0x8000_0000 {
|
||||
// Negative exponent: the result is subnormal (or zero).
|
||||
s = this.shift_right_jam64(a: s, dist: 0 ~mod- e)
|
||||
e = 0
|
||||
round_bits = s & 0x3FF
|
||||
} else if e >= 0x7FD {
|
||||
if (e > 0x7FD) or (s >= 0x7FFF_FFFF_FFFF_FE00) {
|
||||
return args.sign | F64_INF
|
||||
}
|
||||
}
|
||||
s = (s ~mod+ 0x200) >> 10
|
||||
if round_bits == 0x200 {
|
||||
// Ties to even.
|
||||
s = s & 0xFFFF_FFFF_FFFF_FFFE
|
||||
}
|
||||
if s == 0 {
|
||||
e = 0
|
||||
}
|
||||
return args.sign | (((e as base.u64) ~mod<< 52) ~mod+ s)
|
||||
}
|
||||
|
||||
// norm_round_pack_f64 is like round_pack_f64 but sig does not need to be
|
||||
// normalized (its most significant set bit can be anywhere).
|
||||
pri func arith.norm_round_pack_f64(sign: base.u64, exp: base.u32, sig: base.u64) base.u64 {
|
||||
var c : base.u32[..= 63]
|
||||
var shift : base.u32[..= 62]
|
||||
var e : base.u32
|
||||
|
||||
if args.sig == 0 {
|
||||
return args.sign
|
||||
}
|
||||
c = this.clz64(a: args.sig)
|
||||
if c == 0 {
|
||||
return this.round_pack_f64(sign: args.sign,
|
||||
exp: args.exp ~mod+ 1,
|
||||
sig: (args.sig >> 1) | (args.sig & 1))
|
||||
}
|
||||
shift = c - 1
|
||||
e = args.exp ~mod- shift
|
||||
if (shift >= 10) and (e < 0x7FD) {
|
||||
return args.sign | (((e as base.u64) ~mod<< 52) ~mod+ (args.sig ~mod<< (shift - 10)))
|
||||
}
|
||||
return this.round_pack_f64(sign: args.sign, exp: e, sig: args.sig ~mod<< shift)
|
||||
}
|
||||
|
||||
// norm_sub_exp and norm_sub_sig normalize a subnormal's (non-zero) fraction
|
||||
// bits, returning the (two's complement) exponent and the significand such
|
||||
// that the significand's hidden bit is at bit 52.
|
||||
pri func arith.norm_sub_exp(frac: base.u64) base.u32 {
|
||||
return 12 ~mod- this.clz64(a: args.frac)
|
||||
}
|
||||
|
||||
pri func arith.norm_sub_sig(frac: base.u64) base.u64 {
|
||||
return args.frac ~mod<< ((this.clz64(a: args.frac) ~mod- 11) & 63)
|
||||
}
|
||||
|
||||
pri func arith.f64_propagate_nan(a: base.u64, b: base.u64) base.u64 {
|
||||
if (args.a & F64_ABS) > F64_INF {
|
||||
return args.a | F64_QUIET
|
||||
}
|
||||
return args.b | F64_QUIET
|
||||
}
|
||||
|
||||
// -------- f64 classification, sign and comparison.
|
||||
|
||||
// f64_is_nan returns 1 if a is a NaN (quiet or signaling) and 0 otherwise.
|
||||
pub func arith.f64_is_nan(a: base.u64) base.u32 {
|
||||
if (args.a & F64_ABS) > F64_INF {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// f64_neg returns -a (flipping the sign bit, even for NaNs and zeroes).
|
||||
pub func arith.f64_neg(a: base.u64) base.u64 {
|
||||
return args.a ^ F64_SIGN
|
||||
}
|
||||
|
||||
// f64_abs returns |a| (clearing the sign bit, even for NaNs).
|
||||
pub func arith.f64_abs(a: base.u64) base.u64 {
|
||||
return args.a & F64_ABS
|
||||
}
|
||||
|
||||
// f64_cmp compares a and b, returning 0 if (a < b), 1 if (a == b), 2 if (a
|
||||
// > b) and 3 if they are unordered (either is a NaN). -0 and +0 compare equal.
|
||||
pub func arith.f64_cmp(a: base.u64, b: base.u64) base.u32 {
|
||||
var ka : base.u64
|
||||
var kb : base.u64
|
||||
|
||||
if ((args.a & F64_ABS) > F64_INF) or ((args.b & F64_ABS) > F64_INF) {
|
||||
return 3
|
||||
} else if ((args.a | args.b) & F64_ABS) == 0 {
|
||||
return 1
|
||||
}
|
||||
// Map the bit patterns to unsigned integers that sort in the same order
|
||||
// as the f64 values they represent.
|
||||
ka = args.a | F64_SIGN
|
||||
if (args.a >> 63) <> 0 {
|
||||
ka = 0xFFFF_FFFF_FFFF_FFFF ^ args.a
|
||||
}
|
||||
kb = args.b | F64_SIGN
|
||||
if (args.b >> 63) <> 0 {
|
||||
kb = 0xFFFF_FFFF_FFFF_FFFF ^ args.b
|
||||
}
|
||||
if ka < kb {
|
||||
return 0
|
||||
} else if ka == kb {
|
||||
return 1
|
||||
}
|
||||
return 2
|
||||
}
|
||||
|
||||
// The f64_eq, f64_ne, f64_lt, f64_le, f64_gt and f64_ge functions implement
|
||||
// C's ==, !=, <, <=, > and >= operators. They return 1 for true and 0 for
|
||||
// false. Any comparison involving a NaN is false (except for !=, which is
|
||||
// true), and -0 equals +0.
|
||||
|
||||
pub func arith.f64_eq(a: base.u64, b: base.u64) base.u32 {
|
||||
if this.f64_cmp(a: args.a, b: args.b) == 1 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f64_ne(a: base.u64, b: base.u64) base.u32 {
|
||||
if this.f64_cmp(a: args.a, b: args.b) == 1 {
|
||||
return 0
|
||||
}
|
||||
return 1
|
||||
}
|
||||
|
||||
pub func arith.f64_lt(a: base.u64, b: base.u64) base.u32 {
|
||||
if this.f64_cmp(a: args.a, b: args.b) == 0 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f64_le(a: base.u64, b: base.u64) base.u32 {
|
||||
if this.f64_cmp(a: args.a, b: args.b) <= 1 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f64_gt(a: base.u64, b: base.u64) base.u32 {
|
||||
if this.f64_cmp(a: args.a, b: args.b) == 2 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f64_ge(a: base.u64, b: base.u64) base.u32 {
|
||||
var c : base.u32
|
||||
|
||||
c = this.f64_cmp(a: args.a, b: args.b)
|
||||
if (c == 1) or (c == 2) {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// f64_min returns ((a < b) ? a : b), like brotli's BROTLI_MIN(double, a, b).
|
||||
// Note that this is not IEEE 754 minNum: f64_min(NaN, x) and f64_min(x, NaN)
|
||||
// both return the second argument, and f64_min(-0, +0) returns +0.
|
||||
pub func arith.f64_min(a: base.u64, b: base.u64) base.u64 {
|
||||
if this.f64_cmp(a: args.a, b: args.b) == 0 {
|
||||
return args.a
|
||||
}
|
||||
return args.b
|
||||
}
|
||||
|
||||
// f64_max returns ((a > b) ? a : b), like brotli's BROTLI_MAX(double, a, b).
|
||||
pub func arith.f64_max(a: base.u64, b: base.u64) base.u64 {
|
||||
if this.f64_cmp(a: args.a, b: args.b) == 2 {
|
||||
return args.a
|
||||
}
|
||||
return args.b
|
||||
}
|
||||
|
||||
// -------- f64 arithmetic.
|
||||
|
||||
// f64_add returns (a + b).
|
||||
pub func arith.f64_add(a: base.u64, b: base.u64) base.u64 {
|
||||
if ((args.a ^ args.b) >> 63) == 0 {
|
||||
return this.f64_add_mags(a: args.a, b: args.b, sign: args.a & F64_SIGN)
|
||||
}
|
||||
return this.f64_sub_mags(a: args.a, b: args.b, sign: args.a & F64_SIGN)
|
||||
}
|
||||
|
||||
// f64_sub returns (a - b).
|
||||
pub func arith.f64_sub(a: base.u64, b: base.u64) base.u64 {
|
||||
if ((args.a ^ args.b) >> 63) == 0 {
|
||||
return this.f64_sub_mags(a: args.a, b: args.b, sign: args.a & F64_SIGN)
|
||||
}
|
||||
return this.f64_add_mags(a: args.a, b: args.b, sign: args.a & F64_SIGN)
|
||||
}
|
||||
|
||||
// f64_add_mags returns (sign * (|a| + |b|)).
|
||||
pri func arith.f64_add_mags(a: base.u64, b: base.u64, sign: base.u64) base.u64 {
|
||||
var exp_a : base.u32
|
||||
var exp_b : base.u32
|
||||
var exp_z : base.u32
|
||||
var sig_a : base.u64
|
||||
var sig_b : base.u64
|
||||
var sig_z : base.u64
|
||||
|
||||
exp_a = ((args.a >> 52) & 0x7FF) as base.u32
|
||||
exp_b = ((args.b >> 52) & 0x7FF) as base.u32
|
||||
sig_a = args.a & F64_FRAC
|
||||
sig_b = args.b & F64_FRAC
|
||||
|
||||
if exp_a == exp_b {
|
||||
if exp_a == 0 {
|
||||
// Both subnormal (or zero). The sum of the fractions might carry
|
||||
// into the exponent field, which is what we want.
|
||||
return args.sign | ((args.a & F64_ABS) ~mod+ sig_b)
|
||||
} else if exp_a == 0x7FF {
|
||||
if (sig_a | sig_b) <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return args.a
|
||||
}
|
||||
exp_z = exp_a
|
||||
sig_z = ((0x0020_0000_0000_0000 ~mod+ sig_a) ~mod+ sig_b) ~mod<< 9
|
||||
return this.round_pack_f64(sign: args.sign, exp: exp_z, sig: sig_z)
|
||||
}
|
||||
|
||||
sig_a = sig_a ~mod<< 9
|
||||
sig_b = sig_b ~mod<< 9
|
||||
if exp_a < exp_b {
|
||||
if exp_b == 0x7FF {
|
||||
if sig_b <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return args.sign | F64_INF
|
||||
}
|
||||
exp_z = exp_b
|
||||
if exp_a <> 0 {
|
||||
sig_a = sig_a ~mod+ 0x2000_0000_0000_0000
|
||||
} else {
|
||||
sig_a = sig_a ~mod<< 1
|
||||
}
|
||||
sig_a = this.shift_right_jam64(a: sig_a, dist: exp_b ~mod- exp_a)
|
||||
} else {
|
||||
if exp_a == 0x7FF {
|
||||
if sig_a <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return args.sign | F64_INF
|
||||
}
|
||||
exp_z = exp_a
|
||||
if exp_b <> 0 {
|
||||
sig_b = sig_b ~mod+ 0x2000_0000_0000_0000
|
||||
} else {
|
||||
sig_b = sig_b ~mod<< 1
|
||||
}
|
||||
sig_b = this.shift_right_jam64(a: sig_b, dist: exp_a ~mod- exp_b)
|
||||
}
|
||||
sig_z = (0x2000_0000_0000_0000 ~mod+ sig_a) ~mod+ sig_b
|
||||
if sig_z < 0x4000_0000_0000_0000 {
|
||||
exp_z = exp_z ~mod- 1
|
||||
sig_z = sig_z ~mod<< 1
|
||||
}
|
||||
return this.round_pack_f64(sign: args.sign, exp: exp_z, sig: sig_z)
|
||||
}
|
||||
|
||||
// f64_sub_mags returns (sign * (|a| - |b|)).
|
||||
pri func arith.f64_sub_mags(a: base.u64, b: base.u64, sign: base.u64) base.u64 {
|
||||
var sign_z : base.u64
|
||||
var exp_a : base.u32
|
||||
var exp_b : base.u32
|
||||
var exp_z : base.u32
|
||||
var sig_a : base.u64
|
||||
var sig_b : base.u64
|
||||
var sig_z : base.u64
|
||||
var shift : base.u32
|
||||
|
||||
sign_z = args.sign
|
||||
exp_a = ((args.a >> 52) & 0x7FF) as base.u32
|
||||
exp_b = ((args.b >> 52) & 0x7FF) as base.u32
|
||||
sig_a = args.a & F64_FRAC
|
||||
sig_b = args.b & F64_FRAC
|
||||
|
||||
if exp_a == exp_b {
|
||||
if exp_a == 0x7FF {
|
||||
if (sig_a | sig_b) <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return F64_DEFAULT_NAN
|
||||
} else if sig_a == sig_b {
|
||||
// Exact cancellation gives +0 when rounding to nearest.
|
||||
return 0
|
||||
}
|
||||
if exp_a <> 0 {
|
||||
exp_a = exp_a ~mod- 1
|
||||
}
|
||||
if sig_a < sig_b {
|
||||
sign_z ^= F64_SIGN
|
||||
sig_z = sig_b ~mod- sig_a
|
||||
} else {
|
||||
sig_z = sig_a ~mod- sig_b
|
||||
}
|
||||
// sig_z is non-zero and less than (1 << 52), so shift is at least 1.
|
||||
shift = (this.clz64(a: sig_z) ~mod- 11) & 63
|
||||
if shift > exp_a {
|
||||
shift = exp_a & 63
|
||||
exp_z = 0
|
||||
} else {
|
||||
exp_z = exp_a ~mod- shift
|
||||
}
|
||||
return sign_z | (((exp_z as base.u64) ~mod<< 52) ~mod+ (sig_z ~mod<< shift))
|
||||
}
|
||||
|
||||
sig_a = sig_a ~mod<< 10
|
||||
sig_b = sig_b ~mod<< 10
|
||||
if exp_a < exp_b {
|
||||
sign_z ^= F64_SIGN
|
||||
if exp_b == 0x7FF {
|
||||
if sig_b <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return sign_z | F64_INF
|
||||
}
|
||||
if exp_a <> 0 {
|
||||
sig_a = sig_a ~mod+ 0x4000_0000_0000_0000
|
||||
} else {
|
||||
sig_a = sig_a ~mod+ sig_a
|
||||
}
|
||||
sig_a = this.shift_right_jam64(a: sig_a, dist: exp_b ~mod- exp_a)
|
||||
sig_b = sig_b | 0x4000_0000_0000_0000
|
||||
exp_z = exp_b
|
||||
sig_z = sig_b ~mod- sig_a
|
||||
} else {
|
||||
if exp_a == 0x7FF {
|
||||
if sig_a <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return sign_z | F64_INF
|
||||
}
|
||||
if exp_b <> 0 {
|
||||
sig_b = sig_b ~mod+ 0x4000_0000_0000_0000
|
||||
} else {
|
||||
sig_b = sig_b ~mod+ sig_b
|
||||
}
|
||||
sig_b = this.shift_right_jam64(a: sig_b, dist: exp_a ~mod- exp_b)
|
||||
sig_a = sig_a | 0x4000_0000_0000_0000
|
||||
exp_z = exp_a
|
||||
sig_z = sig_a ~mod- sig_b
|
||||
}
|
||||
return this.norm_round_pack_f64(sign: sign_z, exp: exp_z ~mod- 1, sig: sig_z)
|
||||
}
|
||||
|
||||
// f64_mul returns (a * b).
|
||||
pub func arith.f64_mul(a: base.u64, b: base.u64) base.u64 {
|
||||
var sign_z : base.u64
|
||||
var exp_a : base.u32
|
||||
var exp_b : base.u32
|
||||
var exp_z : base.u32
|
||||
var sig_a : base.u64
|
||||
var sig_b : base.u64
|
||||
var sig_z : base.u64
|
||||
|
||||
sign_z = (args.a ^ args.b) & F64_SIGN
|
||||
exp_a = ((args.a >> 52) & 0x7FF) as base.u32
|
||||
exp_b = ((args.b >> 52) & 0x7FF) as base.u32
|
||||
sig_a = args.a & F64_FRAC
|
||||
sig_b = args.b & F64_FRAC
|
||||
|
||||
if exp_a == 0x7FF {
|
||||
if (sig_a <> 0) or ((exp_b == 0x7FF) and (sig_b <> 0)) {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
} else if (args.b & F64_ABS) == 0 {
|
||||
return F64_DEFAULT_NAN
|
||||
}
|
||||
return sign_z | F64_INF
|
||||
} else if exp_b == 0x7FF {
|
||||
if sig_b <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
} else if (args.a & F64_ABS) == 0 {
|
||||
return F64_DEFAULT_NAN
|
||||
}
|
||||
return sign_z | F64_INF
|
||||
}
|
||||
|
||||
if exp_a == 0 {
|
||||
if sig_a == 0 {
|
||||
return sign_z
|
||||
}
|
||||
exp_a = this.norm_sub_exp(frac: sig_a)
|
||||
sig_a = this.norm_sub_sig(frac: sig_a)
|
||||
}
|
||||
if exp_b == 0 {
|
||||
if sig_b == 0 {
|
||||
return sign_z
|
||||
}
|
||||
exp_b = this.norm_sub_exp(frac: sig_b)
|
||||
sig_b = this.norm_sub_sig(frac: sig_b)
|
||||
}
|
||||
|
||||
exp_z = (exp_a ~mod+ exp_b) ~mod- 0x3FF
|
||||
sig_a = (sig_a | F64_HIDDEN) ~mod<< 10
|
||||
sig_b = (sig_b | F64_HIDDEN) ~mod<< 11
|
||||
sig_z = this.mul_hi64(a: sig_a, b: sig_b)
|
||||
if (sig_a ~mod* sig_b) <> 0 {
|
||||
sig_z |= 1
|
||||
}
|
||||
if sig_z < 0x4000_0000_0000_0000 {
|
||||
exp_z = exp_z ~mod- 1
|
||||
sig_z = sig_z ~mod<< 1
|
||||
}
|
||||
return this.round_pack_f64(sign: sign_z, exp: exp_z, sig: sig_z)
|
||||
}
|
||||
|
||||
// f64_div returns (a / b).
|
||||
pub func arith.f64_div(a: base.u64, b: base.u64) base.u64 {
|
||||
var sign_z : base.u64
|
||||
var exp_a : base.u32
|
||||
var exp_b : base.u32
|
||||
var exp_z : base.u32
|
||||
var sig_a : base.u64
|
||||
var sig_b : base.u64
|
||||
var q : base.u64
|
||||
var r : base.u64
|
||||
var i : base.u32
|
||||
|
||||
sign_z = (args.a ^ args.b) & F64_SIGN
|
||||
exp_a = ((args.a >> 52) & 0x7FF) as base.u32
|
||||
exp_b = ((args.b >> 52) & 0x7FF) as base.u32
|
||||
sig_a = args.a & F64_FRAC
|
||||
sig_b = args.b & F64_FRAC
|
||||
|
||||
if exp_a == 0x7FF {
|
||||
if sig_a <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
} else if exp_b == 0x7FF {
|
||||
if sig_b <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return F64_DEFAULT_NAN
|
||||
}
|
||||
return sign_z | F64_INF
|
||||
} else if exp_b == 0x7FF {
|
||||
if sig_b <> 0 {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
}
|
||||
return sign_z
|
||||
}
|
||||
|
||||
if exp_b == 0 {
|
||||
if sig_b == 0 {
|
||||
if (args.a & F64_ABS) == 0 {
|
||||
return F64_DEFAULT_NAN
|
||||
}
|
||||
return sign_z | F64_INF
|
||||
}
|
||||
exp_b = this.norm_sub_exp(frac: sig_b)
|
||||
sig_b = this.norm_sub_sig(frac: sig_b)
|
||||
}
|
||||
if exp_a == 0 {
|
||||
if sig_a == 0 {
|
||||
return sign_z
|
||||
}
|
||||
exp_a = this.norm_sub_exp(frac: sig_a)
|
||||
sig_a = this.norm_sub_sig(frac: sig_a)
|
||||
}
|
||||
|
||||
exp_z = (exp_a ~mod- exp_b) ~mod+ 0x3FE
|
||||
sig_a = (sig_a & F64_FRAC) | F64_HIDDEN
|
||||
sig_b = (sig_b & F64_FRAC) | F64_HIDDEN
|
||||
if sig_a < sig_b {
|
||||
exp_z = exp_z ~mod- 1
|
||||
sig_a = sig_a ~mod<< 1
|
||||
}
|
||||
|
||||
// Long division, 11 quotient bits at a time. The remainder r is always
|
||||
// less than sig_b (and so less than (1 << 53)), so (r << 11) never
|
||||
// overflows. The final quotient q has 56 bits: 1 integer bit and 55
|
||||
// fraction bits.
|
||||
q = 1
|
||||
r = sig_a ~mod- sig_b
|
||||
i = 0
|
||||
while i < 5 {
|
||||
r = r ~mod<< 11
|
||||
if sig_b > 0 {
|
||||
q = (q ~mod<< 11) | (r / sig_b)
|
||||
r = r % sig_b
|
||||
}
|
||||
i += 1
|
||||
}
|
||||
q = q ~mod<< 7
|
||||
if r <> 0 {
|
||||
q |= 1
|
||||
}
|
||||
return this.round_pack_f64(sign: sign_z, exp: exp_z, sig: q)
|
||||
}
|
||||
|
||||
// f64_fma returns ((a * b) + c), computed with a single rounding (a fused
|
||||
// multiply-add), like C's fma function.
|
||||
pub func arith.f64_fma(a: base.u64, b: base.u64, c: base.u64) base.u64 {
|
||||
var sign_p : base.u64
|
||||
var sign_c : base.u64
|
||||
var sign_z : base.u64
|
||||
var exp_a : base.u32
|
||||
var exp_b : base.u32
|
||||
var exp_c : base.u32
|
||||
var exp_z : base.u32
|
||||
var sig_a : base.u64
|
||||
var sig_b : base.u64
|
||||
var sig_c : base.u64
|
||||
var p_hi : base.u64
|
||||
var p_lo : base.u64
|
||||
var c_hi : base.u64
|
||||
var c_lo : base.u64
|
||||
var z_hi : base.u64
|
||||
var z_lo : base.u64
|
||||
var tmp : base.u64
|
||||
var d : base.u32
|
||||
var top : base.u32
|
||||
|
||||
if ((args.a & F64_ABS) > F64_INF) or ((args.b & F64_ABS) > F64_INF) {
|
||||
return this.f64_propagate_nan(a: args.a, b: args.b)
|
||||
} else if (args.c & F64_ABS) > F64_INF {
|
||||
return args.c | F64_QUIET
|
||||
}
|
||||
|
||||
sign_p = (args.a ^ args.b) & F64_SIGN
|
||||
sign_c = args.c & F64_SIGN
|
||||
exp_a = ((args.a >> 52) & 0x7FF) as base.u32
|
||||
exp_b = ((args.b >> 52) & 0x7FF) as base.u32
|
||||
exp_c = ((args.c >> 52) & 0x7FF) as base.u32
|
||||
sig_a = args.a & F64_FRAC
|
||||
sig_b = args.b & F64_FRAC
|
||||
sig_c = args.c & F64_FRAC
|
||||
|
||||
if (exp_a == 0x7FF) or (exp_b == 0x7FF) {
|
||||
if ((args.a & F64_ABS) == 0) or ((args.b & F64_ABS) == 0) {
|
||||
// (inf * 0) is invalid.
|
||||
return F64_DEFAULT_NAN
|
||||
} else if (exp_c == 0x7FF) and (sign_c <> sign_p) {
|
||||
// (inf - inf) is invalid.
|
||||
return F64_DEFAULT_NAN
|
||||
}
|
||||
return sign_p | F64_INF
|
||||
} else if exp_c == 0x7FF {
|
||||
return args.c
|
||||
}
|
||||
|
||||
if ((args.a & F64_ABS) == 0) or ((args.b & F64_ABS) == 0) {
|
||||
// The product is an exact (signed) zero.
|
||||
if (args.c & F64_ABS) == 0 {
|
||||
if sign_p == sign_c {
|
||||
return args.c
|
||||
}
|
||||
return 0
|
||||
}
|
||||
return args.c
|
||||
}
|
||||
|
||||
// Normalize a and b and compute their exact product, as a 128-bit
|
||||
// integer (p_hi, p_lo), whose value (in 106 bits) is shifted left by 20
|
||||
// so that it is in [1 << 124, 1 << 126).
|
||||
if exp_a == 0 {
|
||||
exp_a = this.norm_sub_exp(frac: sig_a)
|
||||
sig_a = this.norm_sub_sig(frac: sig_a)
|
||||
}
|
||||
if exp_b == 0 {
|
||||
exp_b = this.norm_sub_exp(frac: sig_b)
|
||||
sig_b = this.norm_sub_sig(frac: sig_b)
|
||||
}
|
||||
sig_a = (sig_a & F64_FRAC) | F64_HIDDEN
|
||||
sig_b = (sig_b & F64_FRAC) | F64_HIDDEN
|
||||
p_hi = this.mul_hi64(a: sig_a, b: sig_b)
|
||||
p_lo = sig_a ~mod* sig_b
|
||||
p_hi = (p_hi ~mod<< 20) | (p_lo >> 44)
|
||||
p_lo = p_lo ~mod<< 20
|
||||
// The product's value is ((p_hi, p_lo) * 2**(exp_a + exp_b - 2170)).
|
||||
|
||||
if (args.c & F64_ABS) == 0 {
|
||||
// Find the top bit (in 0 ..= 127) and the round_pack_f64 exponent.
|
||||
top = 127 - this.clz64(a: p_hi)
|
||||
exp_z = ((exp_a ~mod+ exp_b) ~mod+ top) ~mod- 1148
|
||||
return this.round_pack_f64(sign: sign_p, exp: exp_z,
|
||||
sig: this.jam128_lo(hi: p_hi, lo: p_lo, dist: top ~mod- 62))
|
||||
}
|
||||
|
||||
// Normalize c, as a 128-bit integer (c_hi, c_lo) in [1 << 125, 1 << 126)
|
||||
// whose value is ((c_hi, c_lo) * 2**(exp_c - 1148)).
|
||||
if exp_c == 0 {
|
||||
exp_c = this.norm_sub_exp(frac: sig_c)
|
||||
sig_c = this.norm_sub_sig(frac: sig_c)
|
||||
}
|
||||
sig_c = (sig_c & F64_FRAC) | F64_HIDDEN
|
||||
c_hi = sig_c ~mod<< 9
|
||||
c_lo = 0
|
||||
|
||||
// Align the two operands to the larger exponent. d is the (two's
|
||||
// complement) difference in exponents.
|
||||
d = ((exp_a ~mod+ exp_b) ~mod- exp_c) ~mod- 1022
|
||||
if d < 0x8000_0000 {
|
||||
tmp = this.jam128_hi(hi: c_hi, lo: c_lo, dist: d)
|
||||
c_lo = this.jam128_lo(hi: c_hi, lo: c_lo, dist: d)
|
||||
c_hi = tmp
|
||||
// The common exponent is (exp_a + exp_b - 2170). We will add (top +
|
||||
// 1022) to it, below. Set exp_z to their difference.
|
||||
exp_z = (exp_a ~mod+ exp_b) ~mod- 1148
|
||||
} else {
|
||||
d = 0 ~mod- d
|
||||
tmp = this.jam128_hi(hi: p_hi, lo: p_lo, dist: d)
|
||||
p_lo = this.jam128_lo(hi: p_hi, lo: p_lo, dist: d)
|
||||
p_hi = tmp
|
||||
// The common exponent is (exp_c - 1148).
|
||||
exp_z = exp_c ~mod- 126
|
||||
}
|
||||
|
||||
// Add or subtract the 128-bit values.
|
||||
if sign_p == sign_c {
|
||||
sign_z = sign_p
|
||||
z_lo = p_lo ~mod+ c_lo
|
||||
z_hi = p_hi ~mod+ c_hi
|
||||
if z_lo < p_lo {
|
||||
z_hi = z_hi ~mod+ 1
|
||||
}
|
||||
} else {
|
||||
if (p_hi > c_hi) or ((p_hi == c_hi) and (p_lo >= c_lo)) {
|
||||
sign_z = sign_p
|
||||
z_lo = p_lo ~mod- c_lo
|
||||
z_hi = p_hi ~mod- c_hi
|
||||
if p_lo < c_lo {
|
||||
z_hi = z_hi ~mod- 1
|
||||
}
|
||||
} else {
|
||||
sign_z = sign_c
|
||||
z_lo = c_lo ~mod- p_lo
|
||||
z_hi = c_hi ~mod- p_hi
|
||||
if c_lo < p_lo {
|
||||
z_hi = z_hi ~mod- 1
|
||||
}
|
||||
}
|
||||
if (z_hi | z_lo) == 0 {
|
||||
// Exact cancellation gives +0 when rounding to nearest.
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
// Normalize and round.
|
||||
if z_hi <> 0 {
|
||||
top = 127 - this.clz64(a: z_hi)
|
||||
} else {
|
||||
top = 63 - this.clz64(a: z_lo)
|
||||
}
|
||||
exp_z = exp_z ~mod+ top
|
||||
if top >= 62 {
|
||||
return this.round_pack_f64(sign: sign_z, exp: exp_z,
|
||||
sig: this.jam128_lo(hi: z_hi, lo: z_lo, dist: top - 62))
|
||||
}
|
||||
return this.round_pack_f64(sign: sign_z, exp: exp_z,
|
||||
sig: z_lo ~mod<< (62 - top))
|
||||
}
|
||||
|
||||
// -------- f64 conversions.
|
||||
|
||||
// f64_from_u64 returns the f64 nearest to the integer a (C's (double)a).
|
||||
pub func arith.f64_from_u64(a: base.u64) base.u64 {
|
||||
return this.norm_round_pack_f64(sign: 0, exp: 0x43C, sig: args.a)
|
||||
}
|
||||
|
||||
// f64_from_u32 returns the f64 equal to the integer a (C's (double)a).
|
||||
pub func arith.f64_from_u32(a: base.u32) base.u64 {
|
||||
return this.norm_round_pack_f64(sign: 0, exp: 0x43C, sig: args.a as base.u64)
|
||||
}
|
||||
|
||||
// f64_from_i64 returns the f64 nearest to the integer a (C's (double)a).
|
||||
pub func arith.f64_from_i64(a: base.i64) base.u64 {
|
||||
var m : base.u64
|
||||
|
||||
if args.a >= 0 {
|
||||
return this.norm_round_pack_f64(sign: 0, exp: 0x43C, sig: args.a as base.u64)
|
||||
}
|
||||
m = ((0 - (args.a + 1)) as base.u64) + 1
|
||||
return this.norm_round_pack_f64(sign: F64_SIGN, exp: 0x43C, sig: m)
|
||||
}
|
||||
|
||||
// f64_to_u64_trunc returns the integer part of a, truncating towards zero,
|
||||
// like C's (uint64_t)a. That C conversion is undefined behavior unless a is
|
||||
// in the range (-1, 1 << 64): this function returns 0 for NaNs and for
|
||||
// negative values and returns 0xFFFF_FFFF_FFFF_FFFF for values at or above
|
||||
// (1 << 64), but callers should not rely on that.
|
||||
pub func arith.f64_to_u64_trunc(a: base.u64) base.u64 {
|
||||
var exp : base.u32
|
||||
var sig : base.u64
|
||||
|
||||
if args.a >= F64_SIGN {
|
||||
return 0
|
||||
} else if args.a > F64_INF {
|
||||
return 0
|
||||
}
|
||||
exp = ((args.a >> 52) & 0x7FF) as base.u32
|
||||
sig = (args.a & F64_FRAC) | F64_HIDDEN
|
||||
if exp < 0x3FF {
|
||||
return 0
|
||||
} else if exp >= (0x3FF + 64) {
|
||||
return 0xFFFF_FFFF_FFFF_FFFF
|
||||
} else if exp >= 0x433 {
|
||||
return sig ~mod<< ((exp - 0x433) & 63)
|
||||
}
|
||||
return sig >> ((0x433 - exp) & 63)
|
||||
}
|
||||
|
||||
// f64_to_f32 returns the f32 nearest to a (C's (float)a), rounding to
|
||||
// nearest, ties to even, overflowing to infinity and underflowing to
|
||||
// subnormals or zero.
|
||||
pub func arith.f64_to_f32(a: base.u64) base.u32 {
|
||||
var sign : base.u32
|
||||
var exp : base.u32
|
||||
var frac : base.u64
|
||||
var sig : base.u32
|
||||
|
||||
sign = ((args.a >> 32) as base.u32) & 0x8000_0000
|
||||
exp = ((args.a >> 52) & 0x7FF) as base.u32
|
||||
frac = args.a & F64_FRAC
|
||||
|
||||
if exp == 0x7FF {
|
||||
if frac <> 0 {
|
||||
return sign | 0x7FC0_0000 | ((frac >> 29) as base.u32)
|
||||
}
|
||||
return sign | 0x7F80_0000
|
||||
}
|
||||
// Keep the top 30 bits of the 52 fraction bits, plus a sticky bit.
|
||||
sig = (this.shift_right_jam64(a: frac, dist: 22) & 0x3FFF_FFFF) as base.u32
|
||||
if (exp == 0) and (sig == 0) {
|
||||
return sign
|
||||
}
|
||||
return this.round_pack_f32(sign: sign, exp: exp ~mod- 0x381, sig: sig | 0x4000_0000)
|
||||
}
|
||||
|
||||
// round_pack_f32 is the f32 equivalent of round_pack_f64. The sig argument
|
||||
// has its implicit bit at bit 30 and 7 extra bits of precision at the
|
||||
// bottom. The value represented is (sig * 2**(exp - 156)).
|
||||
pri func arith.round_pack_f32(sign: base.u32, exp: base.u32, sig: base.u32) base.u32 {
|
||||
var e : base.u32
|
||||
var s : base.u32
|
||||
var round_bits : base.u32
|
||||
|
||||
e = args.exp
|
||||
s = args.sig
|
||||
round_bits = s & 0x7F
|
||||
if e >= 0x8000_0000 {
|
||||
s = (this.shift_right_jam64(a: s as base.u64, dist: 0 ~mod- e) & 0xFFFF_FFFF) as base.u32
|
||||
e = 0
|
||||
round_bits = s & 0x7F
|
||||
} else if e >= 0xFD {
|
||||
if (e > 0xFD) or (s >= 0x7FFF_FFC0) {
|
||||
return args.sign | 0x7F80_0000
|
||||
}
|
||||
}
|
||||
s = (s ~mod+ 0x40) >> 7
|
||||
if round_bits == 0x40 {
|
||||
s = s & 0xFFFF_FFFE
|
||||
}
|
||||
if s == 0 {
|
||||
e = 0
|
||||
}
|
||||
return args.sign | ((e ~mod<< 23) ~mod+ s)
|
||||
}
|
||||
|
||||
// -------- Tables.
|
||||
|
||||
// CLZ8[i] is the number of leading zero bits in the 8-bit value i.
|
||||
pri const CLZ8 : roarray[256] base.u8 = [
|
||||
0x08, 0x07, 0x06, 0x06, 0x05, 0x05, 0x05, 0x05, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04,
|
||||
0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03,
|
||||
0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02,
|
||||
0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02,
|
||||
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
|
||||
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
|
||||
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
|
||||
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
]
|
||||
@@ -0,0 +1,225 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// -------- f32 (binary32) operations.
|
||||
//
|
||||
// The f32 add, sub, mul and div functions convert their (exactly
|
||||
// representable) operands to f64, perform the f64 operation and round the
|
||||
// result to f32. Double rounding is innocuous here: binary64 has more than
|
||||
// (2 * 24) + 2 bits of precision, which suffices for +, -, * and / (Figueroa,
|
||||
// "When is double rounding innocuous?", 1995). This is also exhaustively
|
||||
// cross-checked against hardware by the tests.
|
||||
|
||||
pri const F32_SIGN : base.u32 = 0x8000_0000
|
||||
pri const F32_ABS : base.u32 = 0x7FFF_FFFF
|
||||
pri const F32_INF : base.u32 = 0x7F80_0000
|
||||
|
||||
// f32_to_f64 returns the f64 equal to a (C's (double)a). It is exact.
|
||||
pub func arith.f32_to_f64(a: base.u32) base.u64 {
|
||||
var sign : base.u64
|
||||
var exp : base.u32
|
||||
var frac : base.u64
|
||||
var top : base.u32[..= 63]
|
||||
|
||||
sign = ((args.a as base.u64) >> 31) << 63
|
||||
exp = (args.a >> 23) & 0xFF
|
||||
frac = (args.a & 0x7F_FFFF) as base.u64
|
||||
|
||||
if exp == 0xFF {
|
||||
if frac <> 0 {
|
||||
return sign | 0x7FF8_0000_0000_0000 | (frac << 29)
|
||||
}
|
||||
return sign | 0x7FF0_0000_0000_0000
|
||||
} else if exp == 0 {
|
||||
if frac == 0 {
|
||||
return sign
|
||||
}
|
||||
// Subnormal: the value is (frac * 2**-149) and frac's most
|
||||
// significant bit is at bit position top (in 0 ..= 22).
|
||||
top = 63 - this.clz64(a: frac)
|
||||
if top > 22 {
|
||||
return 0
|
||||
}
|
||||
return sign |
|
||||
(((top + 874) as base.u64) << 52) |
|
||||
((frac ~mod<< (52 - top)) & 0x000F_FFFF_FFFF_FFFF)
|
||||
}
|
||||
return sign | (((exp + 896) as base.u64) << 52) | (frac << 29)
|
||||
}
|
||||
|
||||
// f32_is_nan returns 1 if a is a NaN (quiet or signaling) and 0 otherwise.
|
||||
pub func arith.f32_is_nan(a: base.u32) base.u32 {
|
||||
if (args.a & F32_ABS) > F32_INF {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// f32_neg returns -a (flipping the sign bit, even for NaNs and zeroes).
|
||||
pub func arith.f32_neg(a: base.u32) base.u32 {
|
||||
return args.a ^ F32_SIGN
|
||||
}
|
||||
|
||||
// f32_abs returns |a| (clearing the sign bit, even for NaNs).
|
||||
pub func arith.f32_abs(a: base.u32) base.u32 {
|
||||
return args.a & F32_ABS
|
||||
}
|
||||
|
||||
// f32_add returns (a + b).
|
||||
pub func arith.f32_add(a: base.u32, b: base.u32) base.u32 {
|
||||
return this.f64_to_f32(a: this.f64_add(
|
||||
a: this.f32_to_f64(a: args.a),
|
||||
b: this.f32_to_f64(a: args.b)))
|
||||
}
|
||||
|
||||
// f32_sub returns (a - b).
|
||||
pub func arith.f32_sub(a: base.u32, b: base.u32) base.u32 {
|
||||
return this.f64_to_f32(a: this.f64_sub(
|
||||
a: this.f32_to_f64(a: args.a),
|
||||
b: this.f32_to_f64(a: args.b)))
|
||||
}
|
||||
|
||||
// f32_mul returns (a * b).
|
||||
pub func arith.f32_mul(a: base.u32, b: base.u32) base.u32 {
|
||||
return this.f64_to_f32(a: this.f64_mul(
|
||||
a: this.f32_to_f64(a: args.a),
|
||||
b: this.f32_to_f64(a: args.b)))
|
||||
}
|
||||
|
||||
// f32_div returns (a / b).
|
||||
pub func arith.f32_div(a: base.u32, b: base.u32) base.u32 {
|
||||
return this.f64_to_f32(a: this.f64_div(
|
||||
a: this.f32_to_f64(a: args.a),
|
||||
b: this.f32_to_f64(a: args.b)))
|
||||
}
|
||||
|
||||
// f32_cmp compares a and b, returning 0 if (a < b), 1 if (a == b), 2 if (a
|
||||
// > b) and 3 if they are unordered (either is a NaN). -0 and +0 compare equal.
|
||||
pub func arith.f32_cmp(a: base.u32, b: base.u32) base.u32 {
|
||||
var ka : base.u32
|
||||
var kb : base.u32
|
||||
|
||||
if ((args.a & F32_ABS) > F32_INF) or ((args.b & F32_ABS) > F32_INF) {
|
||||
return 3
|
||||
} else if ((args.a | args.b) & F32_ABS) == 0 {
|
||||
return 1
|
||||
}
|
||||
ka = args.a | F32_SIGN
|
||||
if (args.a >> 31) <> 0 {
|
||||
ka = 0xFFFF_FFFF ^ args.a
|
||||
}
|
||||
kb = args.b | F32_SIGN
|
||||
if (args.b >> 31) <> 0 {
|
||||
kb = 0xFFFF_FFFF ^ args.b
|
||||
}
|
||||
if ka < kb {
|
||||
return 0
|
||||
} else if ka == kb {
|
||||
return 1
|
||||
}
|
||||
return 2
|
||||
}
|
||||
|
||||
// The f32_eq, f32_ne, f32_lt, f32_le, f32_gt and f32_ge functions implement
|
||||
// C's ==, !=, <, <=, > and >= operators. They return 1 for true and 0 for
|
||||
// false. Any comparison involving a NaN is false (except for !=, which is
|
||||
// true), and -0 equals +0.
|
||||
|
||||
pub func arith.f32_eq(a: base.u32, b: base.u32) base.u32 {
|
||||
if this.f32_cmp(a: args.a, b: args.b) == 1 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f32_ne(a: base.u32, b: base.u32) base.u32 {
|
||||
if this.f32_cmp(a: args.a, b: args.b) == 1 {
|
||||
return 0
|
||||
}
|
||||
return 1
|
||||
}
|
||||
|
||||
pub func arith.f32_lt(a: base.u32, b: base.u32) base.u32 {
|
||||
if this.f32_cmp(a: args.a, b: args.b) == 0 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f32_le(a: base.u32, b: base.u32) base.u32 {
|
||||
if this.f32_cmp(a: args.a, b: args.b) <= 1 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f32_gt(a: base.u32, b: base.u32) base.u32 {
|
||||
if this.f32_cmp(a: args.a, b: args.b) == 2 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
pub func arith.f32_ge(a: base.u32, b: base.u32) base.u32 {
|
||||
var c : base.u32
|
||||
|
||||
c = this.f32_cmp(a: args.a, b: args.b)
|
||||
if (c == 1) or (c == 2) {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// f32_min returns ((a < b) ? a : b), like brotli's BROTLI_MIN(float, a, b).
|
||||
// Note that this is not IEEE 754 minNum: f32_min(NaN, x) and f32_min(x, NaN)
|
||||
// both return the second argument, and f32_min(-0, +0) returns +0.
|
||||
pub func arith.f32_min(a: base.u32, b: base.u32) base.u32 {
|
||||
if this.f32_cmp(a: args.a, b: args.b) == 0 {
|
||||
return args.a
|
||||
}
|
||||
return args.b
|
||||
}
|
||||
|
||||
// f32_max returns ((a > b) ? a : b), like brotli's BROTLI_MAX(float, a, b).
|
||||
pub func arith.f32_max(a: base.u32, b: base.u32) base.u32 {
|
||||
if this.f32_cmp(a: args.a, b: args.b) == 2 {
|
||||
return args.a
|
||||
}
|
||||
return args.b
|
||||
}
|
||||
|
||||
// f32_from_u32 returns the f32 nearest to the integer a (C's (float)a).
|
||||
pub func arith.f32_from_u32(a: base.u32) base.u32 {
|
||||
// The u32 to f64 conversion is exact, so there is only one rounding.
|
||||
return this.f64_to_f32(a: this.f64_from_u32(a: args.a))
|
||||
}
|
||||
|
||||
// f32_from_i32 returns the f32 nearest to the integer a (C's (float)a).
|
||||
pub func arith.f32_from_i32(a: base.i32) base.u32 {
|
||||
return this.f64_to_f32(a: this.f64_from_i64(a: args.a as base.i64))
|
||||
}
|
||||
|
||||
// f32_from_u64 returns the f32 nearest to the integer a (C's (float)a).
|
||||
pub func arith.f32_from_u64(a: base.u64) base.u32 {
|
||||
var top : base.u32[..= 63]
|
||||
var x : base.u64
|
||||
|
||||
// Converting via f64 could double-round, so first reduce a to at most
|
||||
// 53 significant bits, the last of which is a sticky bit. The f64
|
||||
// conversion of that is exact.
|
||||
x = args.a
|
||||
if x >= 0x0020_0000_0000_0000 {
|
||||
top = 63 - this.clz64(a: x)
|
||||
if top >= 53 {
|
||||
x = this.shift_right_jam64(a: x, dist: top - 52) ~mod<< (top - 52)
|
||||
}
|
||||
}
|
||||
return this.f64_to_f32(a: this.f64_from_u64(a: x))
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
// Copyright 2026 The Wuffs Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
|
||||
// https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
|
||||
// <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
|
||||
// option. This file may not be copied, modified, or distributed
|
||||
// except according to those terms.
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
|
||||
// -------- log2.
|
||||
//
|
||||
// This is a bit-exact port of glibc 2.39's x86-64 "__log2_fma" function
|
||||
// (sysdeps/ieee754/dbl-64/e_log2.c compiled with -mfma -mavx2), which is what
|
||||
// glibc's log2 resolves to at run time on x86-64 CPUs with FMA and AVX2. The
|
||||
// sequence of f64 operations (including where the compiler contracted a
|
||||
// multiply and add into a single fused multiply-add) was transcribed from the
|
||||
// machine code of the shipped libm.so.6, and the constants were copied from
|
||||
// its __log2_data table. Since every softfloat operation is correctly
|
||||
// rounded, the result matches glibc bit-for-bit for every input.
|
||||
//
|
||||
// The algorithm is by Szabolcs Nagy (ARM's optimized-routines, MIT licensed).
|
||||
// Its maximum error is about 0.55 ULP: it is not always correctly rounded,
|
||||
// which is why this port reproduces glibc's exact operation sequence instead
|
||||
// of computing a correctly rounded logarithm.
|
||||
|
||||
// LOG2_INVLN2_HI and LOG2_INVLN2_LO sum to approximately (1 / ln(2)).
|
||||
pri const LOG2_INVLN2_HI : base.u64 = 0x3FF7_1547_6520_0000 // 0x1.7154765200000p+0
|
||||
pri const LOG2_INVLN2_LO : base.u64 = 0x3DE7_05FC_2EEF_A200 // 0x1.705fc2eefa200p-33
|
||||
|
||||
// LOG2_A0 ..= LOG2_A5 are the "poly" coefficients (used when x is not near 1).
|
||||
pri const LOG2_A0 : base.u64 = 0xBFE7_1547_652B_8339 // -0x1.71547652b8339p-1
|
||||
pri const LOG2_A1 : base.u64 = 0x3FDE_C709_DC3A_04BE // 0x1.ec709dc3a04bep-2
|
||||
pri const LOG2_A2 : base.u64 = 0xBFD7_1547_6470_2FFB // -0x1.7154764702ffbp-2
|
||||
pri const LOG2_A3 : base.u64 = 0x3FD2_776C_5003_4C48 // 0x1.2776c50034c48p-2
|
||||
pri const LOG2_A4 : base.u64 = 0xBFCE_C7B3_28EA_92BC // -0x1.ec7b328ea92bcp-3
|
||||
pri const LOG2_A5 : base.u64 = 0x3FCA_6225_E117_F92E // 0x1.a6225e117f92ep-3
|
||||
|
||||
// LOG2_B0 ..= LOG2_B9 are the "poly1" coefficients (used when x is near 1).
|
||||
pri const LOG2_B0 : base.u64 = 0xBFE7_1547_652B_82FE // -0x1.71547652b82fep-1
|
||||
pri const LOG2_B1 : base.u64 = 0x3FDE_C709_DC3A_03F7 // 0x1.ec709dc3a03f7p-2
|
||||
pri const LOG2_B2 : base.u64 = 0xBFD7_1547_652B_7C3F // -0x1.71547652b7c3fp-2
|
||||
pri const LOG2_B3 : base.u64 = 0x3FD2_776C_50F0_5BE4 // 0x1.2776c50f05be4p-2
|
||||
pri const LOG2_B4 : base.u64 = 0xBFCE_C709_DD76_8FE5 // -0x1.ec709dd768fe5p-3
|
||||
pri const LOG2_B5 : base.u64 = 0x3FCA_6176_1EC4_E736 // 0x1.a61761ec4e736p-3
|
||||
pri const LOG2_B6 : base.u64 = 0xBFC7_153F_BC64_A79B // -0x1.7153fbc64a79bp-3
|
||||
pri const LOG2_B7 : base.u64 = 0x3FC4_84D1_54F0_1B4A // 0x1.484d154f01b4ap-3
|
||||
pri const LOG2_B8 : base.u64 = 0xBFC2_89E4_A72C_383C // -0x1.289e4a72c383cp-3
|
||||
pri const LOG2_B9 : base.u64 = 0x3FC0_B32F_285A_EE66 // 0x1.0b32f285aee66p-3
|
||||
|
||||
pri const F64_ONE : base.u64 = 0x3FF0_0000_0000_0000
|
||||
pri const F64_NEG_ONE : base.u64 = 0xBFF0_0000_0000_0000
|
||||
|
||||
// log2_u64 returns the f64 bits of glibc's log2((double)v), where (double)v
|
||||
// is C's (rounding) conversion. In particular, it returns -inf for v == 0.
|
||||
pub func arith.log2_u64(v: base.u64) base.u64 {
|
||||
return this.log2_f64(x: this.f64_from_u64(a: args.v))
|
||||
}
|
||||
|
||||
// log2_f64 returns the f64 bits of glibc's log2(x), for any f64 x, including
|
||||
// special cases: log2(±0) is -inf, log2(+inf) is +inf, log2(1) is +0 and
|
||||
// log2 of a NaN or of a negative number is a NaN.
|
||||
pub func arith.log2_f64(x: base.u64) base.u64 {
|
||||
var ix : base.u64
|
||||
var top : base.u64
|
||||
var tmp : base.u64
|
||||
var i : base.u64
|
||||
var k : base.u64
|
||||
var kd : base.u64
|
||||
var invc : base.u64
|
||||
var logc : base.u64
|
||||
var z : base.u64
|
||||
var r : base.u64
|
||||
var r2 : base.u64
|
||||
var r4 : base.u64
|
||||
var t1 : base.u64
|
||||
var t2 : base.u64
|
||||
var t3 : base.u64
|
||||
var hi : base.u64
|
||||
var lo : base.u64
|
||||
var p : base.u64
|
||||
var y : base.u64
|
||||
|
||||
ix = args.x
|
||||
|
||||
// (ix - LO) < (HI - LO), where LO and HI are the bits of
|
||||
// (1.0 - 0x1.5b51p-5) and (1.0 + 0x1.6ab2p-5).
|
||||
if (ix ~mod- 0x3FEE_A4AF_0000_0000) < (0x3FF0_B559_0000_0000 - 0x3FEE_A4AF_0000_0000) {
|
||||
if ix == F64_ONE {
|
||||
return 0
|
||||
}
|
||||
r = this.f64_sub(a: ix, b: F64_ONE)
|
||||
hi = this.f64_mul(a: LOG2_INVLN2_HI, b: r)
|
||||
r2 = this.f64_mul(a: r, b: r)
|
||||
lo = this.f64_fma(a: LOG2_INVLN2_HI, b: r, c: this.f64_neg(a: hi))
|
||||
r4 = this.f64_mul(a: r2, b: r2)
|
||||
p = this.f64_fma(a: r, b: LOG2_B1, c: LOG2_B0)
|
||||
lo = this.f64_fma(a: r, b: LOG2_INVLN2_LO, c: lo)
|
||||
y = this.f64_fma(a: p, b: r2, c: hi)
|
||||
lo = this.f64_add(a: lo, b: this.f64_fma(a: p, b: r2, c: this.f64_sub(a: hi, b: y)))
|
||||
t1 = this.f64_fma(
|
||||
a: this.f64_fma(a: r, b: LOG2_B5, c: LOG2_B4),
|
||||
b: r2,
|
||||
c: this.f64_fma(a: r, b: LOG2_B3, c: LOG2_B2))
|
||||
t2 = this.f64_fma(
|
||||
a: this.f64_fma(a: r, b: LOG2_B9, c: LOG2_B8),
|
||||
b: r2,
|
||||
c: this.f64_fma(a: r, b: LOG2_B7, c: LOG2_B6))
|
||||
t3 = this.f64_fma(a: this.f64_fma(a: t2, b: r4, c: t1), b: r4, c: lo)
|
||||
return this.f64_add(a: y, b: t3)
|
||||
}
|
||||
|
||||
top = ix >> 48
|
||||
if (top ~mod- 0x0010) >= (0x7FF0 - 0x0010) {
|
||||
// x < 0x1p-1022 or inf or nan.
|
||||
if (ix & F64_ABS) == 0 {
|
||||
// log2(±0) is -inf (and raises divide-by-zero).
|
||||
return 0xFFF0_0000_0000_0000
|
||||
} else if ix == F64_INF {
|
||||
return ix
|
||||
} else if ((top & 0x8000) <> 0) or ((top & 0x7FF0) == 0x7FF0) {
|
||||
// log2 of a negative number or NaN is invalid.
|
||||
if (ix & F64_ABS) > F64_INF {
|
||||
return ix | F64_QUIET
|
||||
}
|
||||
return F64_DEFAULT_NAN
|
||||
}
|
||||
// x is subnormal. Normalize it.
|
||||
ix = this.f64_mul(a: ix, b: 0x4330_0000_0000_0000) ~mod- (52 << 52)
|
||||
}
|
||||
|
||||
// x = 2**k * z, where z is in the range [OFF, 2 * OFF) and exact. The
|
||||
// range is split into 64 subintervals. The i'th subinterval contains z
|
||||
// and c is near its center.
|
||||
tmp = ix ~mod- 0x3FE6_0000_0000_0000
|
||||
i = (tmp >> 46) & 63
|
||||
k = tmp >> 52
|
||||
if k >= 0x800 {
|
||||
// k is negative (as a 12-bit two's complement integer).
|
||||
kd = this.f64_neg(a: this.f64_from_u64(a: 0x1000 - k))
|
||||
} else {
|
||||
kd = this.f64_from_u64(a: k)
|
||||
}
|
||||
z = ix ~mod- (tmp & 0xFFF0_0000_0000_0000)
|
||||
invc = LOG2_TAB_INVC[i]
|
||||
logc = LOG2_TAB_LOGC[i]
|
||||
|
||||
// log2(x) = log2(z/c) + log2(c) + k.
|
||||
// r ~= z/c - 1, |r| < 1/(2*N).
|
||||
t3 = this.f64_add(a: kd, b: logc)
|
||||
r = this.f64_fma(a: z, b: invc, c: F64_NEG_ONE)
|
||||
t1 = this.f64_mul(a: LOG2_INVLN2_HI, b: r)
|
||||
t2 = this.f64_fma(a: r, b: LOG2_INVLN2_LO,
|
||||
c: this.f64_fma(a: LOG2_INVLN2_HI, b: r, c: this.f64_neg(a: t1)))
|
||||
|
||||
// hi + lo = r/ln2 + log2(c) + k.
|
||||
hi = this.f64_add(a: t1, b: t3)
|
||||
lo = this.f64_add(a: this.f64_add(a: this.f64_sub(a: t3, b: hi), b: t1), b: t2)
|
||||
|
||||
// log2(r+1) = r/ln2 + r^2*poly(r).
|
||||
r2 = this.f64_mul(a: r, b: r)
|
||||
r4 = this.f64_mul(a: r2, b: r2)
|
||||
p = this.f64_fma(
|
||||
a: this.f64_fma(a: r, b: LOG2_A3, c: LOG2_A2),
|
||||
b: r2,
|
||||
c: this.f64_fma(a: r, b: LOG2_A1, c: LOG2_A0))
|
||||
p = this.f64_fma(a: this.f64_fma(a: r, b: LOG2_A5, c: LOG2_A4), b: r4, c: p)
|
||||
y = this.f64_fma(a: r2, b: p, c: lo)
|
||||
return this.f64_add(a: y, b: hi)
|
||||
}
|
||||
|
||||
// LOG2_TAB_INVC[i] and LOG2_TAB_LOGC[i] are glibc's __log2_data.tab[i].invc
|
||||
// and .logc: c is near the center of the i'th subinterval, invc is (1 / c)
|
||||
// and logc is log2(c).
|
||||
|
||||
pri const LOG2_TAB_INVC : roarray[64] base.u64 = [
|
||||
0x3FF7_2428_6BB1_ACF8, 0x3FF6_E1F7_66D2_CCA1, 0x3FF6_A13D_0E30_D48A, 0x3FF6_61EC_32D0_6C85,
|
||||
0x3FF6_23FA_9511_98F8, 0x3FF5_E75B_A4CF_026C, 0x3FF5_AC05_5A21_4FB8, 0x3FF5_71ED_0F16_6E1E,
|
||||
0x3FF5_3909_590B_F835, 0x3FF5_014F_ED61_ADDD, 0x3FF4_CAB8_8E48_7BD0, 0x3FF4_9539_B433_4FEE,
|
||||
0x3FF4_60CB_DFAF_D569, 0x3FF4_2D66_4EE4_B953, 0x3FF3_FB01_111D_D8A6, 0x3FF3_C995_B70C_5836,
|
||||
0x3FF3_991C_4AB6_FD4A, 0x3FF3_698E_0CE0_99B5, 0x3FF3_3AE4_8213_E7B2, 0x3FF3_0D19_1985_BDB1,
|
||||
0x3FF2_E025_CAB2_71D7, 0x3FF2_B404_CF13_CD82, 0x3FF2_88B0_2C7C_CB50, 0x3FF2_5E22_6394_4DE5,
|
||||
0x3FF2_3456_3D86_15B1, 0x3FF2_0B46_E33E_AF38, 0x3FF1_E2EE_FDCD_A3DD, 0x3FF1_BB4A_580B_3930,
|
||||
0x3FF1_9453_847F_2200, 0x3FF1_6E06_C0D5_D73C, 0x3FF1_485F_47B7_E4C2, 0x3FF1_2358_AD00_85D1,
|
||||
0x3FF0_FEF0_0F53_2227, 0x3FF0_DB20_77D0_3A8F, 0x3FF0_B7E6_D659_80D9, 0x3FF0_953E_FE7B_408D,
|
||||
0x3FF0_7325_CAC5_3B83, 0x3FF0_5197_E40D_1B5C, 0x3FF0_3091_C120_8EA2, 0x3FF0_1010_25B3_7E21,
|
||||
0x3FEF_C07E_F9CA_A76B, 0x3FEF_4465_D3F6_F184, 0x3FEE_CC07_9F84_107F, 0x3FEE_573A_9997_5AE8,
|
||||
0x3FED_E5D6_F0BD_3DE6, 0x3FED_77B6_81FF_38B3, 0x3FED_0CB5_724D_E943, 0x3FEC_A4B2_DC0E_7563,
|
||||
0x3FEC_3F8E_E8D6_CB51, 0x3FEB_DD2B_4F02_0C4C, 0x3FEB_7D6C_0060_15CA, 0x3FEB_2036_6E2E_338F,
|
||||
0x3FEA_C570_2629_5039, 0x3FEA_6D01_BC27_31DD, 0x3FEA_16D3_BC3F_F18B, 0x3FE9_C2D1_4967_FEAD,
|
||||
0x3FE9_70E4_F47C_9902, 0x3FE9_20FB_3982_BCF2, 0x3FE8_D301_87F7_59F1, 0x3FE8_86E5_EBB9_F66D,
|
||||
0x3FE8_3C97_B658_B994, 0x3FE7_F405_FFC6_1022, 0x3FE7_AD22_1814_15CA, 0x3FE7_67DC_F99E_FF8C,
|
||||
]
|
||||
|
||||
pri const LOG2_TAB_LOGC : roarray[64] base.u64 = [
|
||||
0xBFE1_095F_EECD_B000, 0xBFE0_8494_BD76_D000, 0xBFE0_0143_AEE8_F800, 0xBFDE_FEC5_360B_4000,
|
||||
0xBFDD_FDD9_1AB7_E000, 0xBFDC_FFAE_0CC7_9000, 0xBFDC_0438_11FD_A000, 0xBFDB_0B67_323A_E000,
|
||||
0xBFDA_152F_5A2D_B000, 0xBFD9_217F_5AF8_6000, 0xBFD8_304D_B071_9000, 0xBFD7_4189_F9A9_E000,
|
||||
0xBFD6_552B_B519_9000, 0xBFD5_6B23_A29B_1000, 0xBFD4_8365_0F5F_A000, 0xBFD3_9DE9_37F6_A000,
|
||||
0xBFD2_BAA1_538D_6000, 0xBFD1_D983_40CA_4000, 0xBFD0_FA85_3A40_E000, 0xBFD0_1D9C_32E7_3000,
|
||||
0xBFCE_857D_A2FA_6000, 0xBFCC_D3C8_633D_8000, 0xBFCB_2603_4C14_A000, 0xBFC9_7C1C_2F4F_E000,
|
||||
0xBFC7_D602_3F80_0000, 0xBFC6_33A7_1A05_E000, 0xBFC4_94F5_E957_0000, 0xBFC2_F9E4_24E0_A000,
|
||||
0xBFC1_6259_5AFD_C000, 0xBFBF_9C9A_75BD_8000, 0xBFBC_7B57_5BF9_C000, 0xBFB9_60C6_0FF4_8000,
|
||||
0xBFB6_4CE2_47B6_0000, 0xBFB3_3F78_B201_4000, 0xBFB0_387D_1A42_C000, 0xBFAA_6F92_08B5_0000,
|
||||
0xBFA4_7A95_4F77_0000, 0xBF9D_23A8_C50C_0000, 0xBF91_6A26_2978_0000, 0xBF77_20F8_D8E8_0000,
|
||||
0x3F86_FE53_B150_0000, 0x3FA1_1CCC_E10F_8000, 0x3FAC_4DFC_8C8B_8000, 0x3FB3_AA32_1E57_4000,
|
||||
0x3FB9_18A0_D08B_8000, 0x3FBE_72E9_DA04_4000, 0x3FC1_DCD2_507F_6000, 0x3FC4_76AB_03DE_A000,
|
||||
0x3FC7_0743_77E2_2000, 0x3FC9_8EDE_8BA9_4000, 0x3FCC_0DB8_6AD2_E000, 0x3FCE_840A_AFCE_E000,
|
||||
0x3FD0_790A_B467_8000, 0x3FD1_AC05_6801_C000, 0x3FD2_DB11_D4FE_E000, 0x3FD4_0646_4EC5_8000,
|
||||
0x3FD5_2DBE_093A_F000, 0x3FD6_5190_2050_D000, 0x3FD7_71D2_CDEA_F000, 0x3FD8_8E9C_857D_9000,
|
||||
0x3FD9_A801_55E1_6000, 0x3FDA_BE18_6ED3_D000, 0x3FDB_D0F2_AEA0_E000, 0x3FDC_E0A4_3DBF_4000,
|
||||
]
|
||||
Reference in new issue
Block a user