randomascii.wordpress.com/2014/10/09/intel-underestimates-error-bounds-by-1-3...

Intel’s manuals for their x86/x64 processor clearly state that the fsin instruction (calculating the trigonometric sin) has a maximum error, in round-to-nearest mode, of one unit in the last place....


Comments (0)

Sign in to post comments.