FP16: How a Number Becomes 16 Bits
Convert 0.421875 to binary by hand — sign, exponent, mantissa. Why half precision halves your VRAM, what it costs you, and why every GGUF file starts from an FP16 checkpoint.
What you'll learn
- Convert decimal weights to FP16 by hand
- Understand sign / exponent / mantissa trade-offs
- Know when FP16 is safe and when it overflows
Sixteen bits, three jobs
A model weight like 0.421875 is stored in half precision as 1 sign bit + 5 exponent bits + 10 mantissa bits. The exponent places the number on a scale of powers of two; the mantissa fills in the detail. Half the bytes of FP32 — which is why an 8B-parameter model needs ~16GB in FP32 but ~8GB in FP16 — at the price of a little precision and a much smaller maximum (65504) before overflow.
The converter
The whiteboard example is 0.421875 — try it, then try 0.1 (never exact in binary) and 70000 (overflow). Every step below is the real IEEE 754 conversion, computed live.
Step 1 · Sign.Positive → sign bit =
0Step 2 · Normalize.Find e with 2ᵉ ≤ |x| < 2ᵉ⁺¹: here e =
-2, so |x| =1.68750× 2-2Step 3 · Exponent bits.Add the bias 15: -2 + 15 =
13→01101(5 bits)Step 4 · Mantissa.Drop the implicit leading 1, keep the fraction: round((frac − 1) × 1024) =
704→1011000000(10 bits)
sign · 1 bit
+
exponent · 5 bits
13 − 15 = -2
mantissa · 10 bits
704 / 1024
0.421875 — exact, zero rounding error.The worked example, by hand
0.421875 = 0.011011 in binary = 1.1011 x 2^-2 # normalize: shift until one 1 leads sign = 0 # positive exponent = -2 + 15 = 13 # add the bias -> 01101 mantissa = 1011000000 # the ".1011", padded to 10 bits FP16 = 0 01101 1011000000 # exact! no rounding error for this one
Why care? Because every GGUF quantization you'll meet in the next lesson starts from an FP16 checkpoint. When you know what those 16 bits mean, "Q4 quantization" stops being magic and becomes arithmetic you can check yourself.