Number Representations & States

"how numbers are stored in computers""how numbers are stored in computers"

If is computed using the formula

then the relative error is at most when . Provided subtraction is performed with a guard digit, , is computed to within 0.5 ulp.

This formula will work for any value of , but is only interesting for , which is where catastrophic cancellation occurs in the formula . The reason this formula works is because can be rewritten as

The left hand factor can be computed exactly, but the right hand factor will suffer a large rounding error when adding to . However, is almost constant, since . Therefore, changing slightly will not introduce much error.

In other words, if , computing will be a good approximation to . Is there a value for for which and can be computed accurately? There is; namely , because then is exactly equal to .