TAOCP 4.2.2 Accuracy of Floating Point Arithmetic: reading guide

Error analysis for floating point: which algebraic laws survive (commutativity does), which break (associativity, distributivity), and how to reason about the error of a sequence of operations.

Time About 8 hours this week

Why it matters

This is the section that teaches you why summing an array in a different order changes the answer, and how to write numeric code that does not drift.

Before you start

4.2.1 Single-Precision Calculations

How to approach it

Write programs that demonstrate non-associativity and catastrophic cancellation. Implement Kahan compensated summation (widely documented) and compare to naive summation.

Exercises

Do every exercise rated 00 to 20 in your head or on paper; they are quick and they check you actually absorbed the definitions. Then pick two or three rated 20 to 25 (M or not) and grind them out fully. Attempt one M25 to M30 and give yourself a hard 90-minute cap before reading the answer. Always read the answer to every exercise you attempted, even the ones you got, because Knuth's answers often teach a second technique.

Traps

Comparing floats with == in tests.