Number Representations & States

"how numbers are stored in computers""how numbers are stored in computers"

When subtracting two floating-point numbers x - y, we usually lose some precision, especially if x and y are very close. However, under certain conditions where x and y are close but not too close, a guard digit allows us to perform an exact subtraction when .

Detecting precision loss

We can create a rudimentary mechanism for detecting precision loss during subtraction by scaling up the values before we subtract them, then scaling down the difference by the same factor.

code.ts
1function checkSubtraction(x: number, y: number, factor = 1000) {
2    const diff = x - y
3    const scaledDiff = (x * factor - y * factor) / factor
4
5    console.log(`normal: ${ x } - ${ y } = ${ diff }`)
6    console.log(`scaled: ${ x } - ${ y } = ${ scaledDiff }`)
7}
8
9// Try values satisfying y/2 <= x <= 2y
10checkSubtraction(0.3, 0.15)
11checkSubtraction(0.29, 0.15)
12checkSubtraction(0.15, 0.08)
13checkSubtraction(0.15, 0.075)

Intuition suggests that this is true - now let's examine how we would construct a rigorous proof of this.

Proof

If and have the same exponent, then certainly is exact. Otherwise, from the condition of the theorem, the exponents can differ by at most . Scale and interchange and if necessary so that , and is represented as and as . Then the algorithm for computing will compute exactly and round to a floating-point number. If the difference is of the form , the difference will already be digits long, and no rounding is necessary. Since , , and since is of the form , so is .

Notes

When , the hypothesis of Theorem 11 cannot be replaced by ; the stronger condition is still necessary. The analysis of the error in , immediately following the proof of Theorem 10, used the fact that the relative error in the basic operations of addition and subtraction is small, which is the most common kind of error analysis.