IEEE754r floating point conformance tests.
A series of floating point conformance tests for Haskell. Currently implemented are: QTrial, the floating point benchmark described in / IEEE Standard 754 for Binary Floating-Point Arithmetic by Prof. W. Kahan <http:www.eecs.berkeley.edu~wkahanieee754statusIEEE754.PDF> This can be run with a simple cabal run qtrial. FPTest runs a sequence of floating point test vectors generated by IBM's FPGen Floating-Point Test Generator. This is done either by interpreting the tests or by translating the tests into a Haskell HUnit test suite script that can just be executed using runhaskell. The IBM test vectors are available from here. You will, most likely, want the binary tests unless you have a decimal machine. Only 32-bit float tests are provided (not 64-bit double ones) although FPTest supports Doubles. The syntax of the test vectors can be found here or in Chapter 4 <https://www.research.ibm.com/haifa/projects/verification/fpgen/papers/ieee-test-suite-v2.pdfs Floating-Point Test-Suite for IEEE here>
IEEE754 conformance tests for Haskell
This project contains IEEE754R floating point conformance tests in Haskell.
There are two sets of tests so far:
QTrial
QTrial is the floating point benchmark described in IEEE Standard 754 for Binary Floating-Point Arithmetic by Prof. W. Kahan Download link
You should just be able to cabal run qtrial from the main directory or runhaskell QTrial.hs from src.
An extract from the result is:
Principal Tests:
Results for Float:
r = 4098.0 produces 12.0 and 12.0 sig. bits
r = 4098.25 fails: root 0.99989897 isn't at least 1
r = 4097.004 produces 12.0 and 11.999298 sig. bits
r =1.6777218e7 is too big for qtrail
r =1.6777218e7 is too big for qtrail
r =1.677722e7 is too big for qtrail
r =9.4906264e7 is too big for qtrail
r =9.4906264e7 is too big for qtrail
r =2.6843546e8 is too big for qtrail
r =2.6843546e8 is too big for qtrail
r =2.6843546e8 is too big for qtrail
r =2.6843546e8 is too big for qtrail
r =2.6843546e8 is too big for qtrail
r =4.2949673e9 is too big for qtrail
r =4.2949673e9 is too big for qtrail
Worst accuracy is 11.999298 sig. bits
Results for Double:
r = 4098.0 produces Infinity and Infinity sig. bits
r = 4098.25 produces Infinity and 53.0 sig. bits
r = 4097.00390625 produces Infinity and 53.451178091541244 sig. bits
r = 1.6777218e7 produces Infinity and Infinity sig. bits
r = 1.677721825e7 produces Infinity and 75.0 sig. bits
r = 1.6777219e7 produces Infinity and 71.0 sig. bits
r = 9.4906267e7 produces 26.499999994288153 and 26.499999986733027 sig. bits
r = 9.490626725e7 fails: root 0.999999995635551 isn't at least 1
r = 2.684354505e8 produces 28.0 and 27.999999919383132 sig. bits
r = 2.684354515e8 produces 28.0 and 27.99999993013205 sig. bits
r = 2.68435458e8 produces 28.0 and 28.0 sig. bits
r = 2.6843545825e8 produces 28.0 and 28.00000000268723 sig. bits
r = 2.6843545700000006e8 produces 28.0 and 27.999999989251084 sig. bits
r = 4.294967298e9 produces 32.0 and 32.0 sig. bits
r = 4.29496729825e9 produces 32.0 and 32.00000000016795 sig. bits
Worst accuracy is 26.499999986733027 sig. bits
This is fine for the Float cases, but the test illustrates an issue with the Double case that is the subject of this ticket.
FPTest
FPTest runs a sequence of floating point test vectors generated by IBM's FPGen Floating-Point Test Generator, which is a commercial product.
This is done in two ways:
- By interpreting the tests
- By translating the tests into a Haskell HUnit test suite script that can just be run using
runhaskell
The IBM test vectors are contained in the test_suite directory, but you will want to get the latest version from the link above.
You will, most likely, want the binary tests unless you have a decimal machine. Only 32-bit float test vectors are provided (not 64-bit double ones).
The syntax of the test vectors can be found here or in Chapter 4 of Floating-Point Test-Suite for IEEE
NB: The test vectors do not use the normal syntax for hexadecimal floating point literals. The literals represent normal numbers as '1.fPsigned exponent' and subnormal numbers by '0.fPsigned exponent'. The significand 'f' represents the 23-bit significand for Floats and the 52-bit significand for Doubles. Since a hexadecimal digit represents 4 bits, the question arises of where the spare bit for Floats should sit. In the standard representation (ie, set out in IEEE754 and implemented in the C 'dtoa' and 'strtod' libraries, the spare 0 sits to the right (so the 23-bit significand 'f' is left justified in the 24-bit hexadecimal representation). The IBM representation right justifies the 23 bits of significand in the 24-bit hexadecimal representation; the first hexadecimal digit of the significand represents 3 bits. The IBM representation also has a fixed length format, including trailing zeros.
Some test vectors also uses lower case versions of certain specification elements.
Results
The tests correctly identify some issues with GHC 7.8.3 that are being fixed or looked at:
Caveats
A fair proportion of the test vector types is not implemented, because there is no corresponding Haskell operation. For example, Haskell has no direct support for
- signalling NaNs
- the reporting of underflow or other exceptions
- rounding mode specification
- non-standard operations such as fused multiply add or operations that are applicable only to decimal floating point
There is not a great deal of point in testing some of these facilities from Haskell; where they are accessible it would just be testing an underlying C library, which it might be better to to directly.