PreviousContentsNext

E Floating Point Arithmetic

Real number representation in fixed point (unsigned) is as follows:

\[ x : (a_i \dots a_0 . b_1 \dots b_j) = \sum _{k=0}^{i} a_k 2^k + \sum _{k=1}^{j} b_k 2^{-k} \]

e.g.

\[ (00000.001)_2 = 0.125 \]

In this case the smallest number (non zero) is \( 2^{-j} \) if \(a_i = 0, \forall i \) and \(b_1 = \dots = b_{j-1} = 0\), \(b_j = 1\).

The largest number is

\[ (2^i - 1) + 2^{-1} + \dots + 2^{-j}, \]

i.e. \(a_k = 1\) for all \(k\), and \(b_k = 1\) for all \(k\).

If we increase \(i\) we increase the range; if we increase \(j\) we increase the precision!! There is a tradeoff!!

The solution is to let the number float!!

We can represent

\[ x = \pm q \, 2^m \]

(the exponent)

\(q\) is the significand \(\in [1,2)\) (a fractional value)

\[ x = \pm q \cdot 2^m = \pm b_0 . b_1 \dots b_n \, 2^m \]

(fractional point)

\[ b_i \in \{0,1\} \]
\[ m \in [L, U] \]

Precision:

\[ p = n + 1 \]

We can convert a binary number, e.g.

\[ (100111.1011)_2 \]

into a floating point representation:

\[ (100111.1011)_2 = (1.001111011)_2 \cdot 2^5 \]

A normalized floating point number can be formally expressed as

\[ x = \pm 1.b_1 \ldots b_n \times 2^m = \pm 1.f \times 2^m \]

\(f\) is the fractional part, \(m\) is the exponent, \(b_i \in \{0,1\}\).

If \(m \in [L,U]\), \(p = n+1\),

then the smallest positive floating point number is

\[ 2^L \]

Largest positive normalized floating point number is

\[ (1 - 2^{-(n+1)}) \, 2^{U+1} \]

Example:

\[ x = \pm 1.b_1 b_2 \, 2^m, \qquad m \in [-4,4], \qquad b_i \in \{0,1\} \]
\[ (1.01)_2 \cdot 2^0 = 1.25 \]

Smallest is

\[ (1.00)_2 \cdot 2^{-4} = 0.0625 \]

Largest is

\[ (1.11)_2 \cdot 2^4 = 28.0 \]

If \(|x|\) is closer to zero than \(0.0625\), then it underflows to \(0\).

If \(x \notin [-28, 28]\), then \(x\) overflows to infinity.

In hardware floats are stored as follows:

\[ x = \pm 1.f \, 2^m \]

First attempt:

\[ x = \boxed {\;\pm \;|\; m \;|\; f\;} \]

but

\[ x = \boxed {\;m \;|\; f\;} \;\pm \]

is also valid!

We need a standard!!! IEEE

\[ x = \pm 1.f \, 2^m \]

In memory this becomes

\[ x = \boxed {\;s \;|\; c \;|\; f\;} \]

where \(s\) is the sign bit, c is the bias exponent, and \(f\) the significant.

\[ x = (-1)^s \, 1.f \, 2^{\,c - \text {shift}} \]
\[ m = c - \text {shift}, \qquad c = m + \text {shift} \]

We store a biased exponent in order to represent negative numbers, i.e.

\[ [0,255] \;\longrightarrow _{shift: -127}\; [-127,128] \]

bias \(= 127\)

e.g. IEEE–754 single precision (32 bits)

\[ \boxed {\, s \;|\; c = m + 127 \;|\; f \,} \]
\[ \text {1 bit} \qquad \text {8 bit exponent} \qquad \text {23 bit significand} \]

IEEE–754 double precision (64 bits)

\[ \boxed {\, s \;|\; c = m + 1023 \;|\; f \,} \]
\[ \text {1 bit} \qquad \text {11 bit exponent} \qquad \text {52 bit significand} \]

Zero is represented as

\[ \boxed {\, s \;|\; 0 \;|\; 0 \,} \qquad \text {i.e. } c = f = 0 \]

\(\pm \infty \)

\[ c = 11\ldots 11, \qquad f = 00\ldots 00 \]
\[ \boxed {\, s \;|\; 11\ldots 11 \;|\; 0\ldots 0 \,} \]
\[ +\infty \text { if } s = 0, \qquad -\infty \text { if } s = 1 \]

NaN: undefined, e.g. \(\dfrac {0}{0}\)

\[ x = \boxed {\, s \;|\; 11\ldots 11 \;|\; f \neq 0 \,} \]

So \(c = (00\ldots 00)\) and \(c = (11\ldots 11)\) are reserved and thus limit the exponent range!!

e.g. for 32-bit IEEE–754:

\[ \boxed {\, s \;|\; c \;|\; f \,} \]
\[ \text {1 bit} \qquad \text {8 bit} \qquad \text {23 bit} \]
\[ s = 0 \Rightarrow \text {positive}, \qquad s = 1 \Rightarrow \text {negative} \]
\[ c = (11111111)_2 = 255, \qquad c = (00000000)_2 = 0 \text { are reserved} \]
\[ c \in [1,254] \text { and not} [0,255] \]

Bias \(= 127\)

\[ U = 254 - 127 = 127 \]
\[ L = 1 - 127 = -126 \]

e.g. \(-67.125\)

tt in binary:

\[ 67.125 = (1000011.001)_2 = (1.000011001)_2 \cdot 2^6 \]
\[ c = 6 + 127 \quad (\text {shift}) = 133 = (10000101)_2 \]

IEEE–754 single precision representation:

\[ \boxed { 1 \;|\; 10000101 \;|\; 00001100100000000000000 } \]
\[ \text {1 bit} \qquad \text {8 bit} \qquad \text {23 bit} \]

PIC

Smallest positive normalized number (UFL):

\[ 2^L = 2^{-126} \approx 1.2 \times 10^{-38} \]

Largest positive normalized number (OFL) IEEE 32-bit number:

\[ 2^U (2 - 2^{-23}) = 2^{127} (1 - 2^{-24}) \approx 3.4 \times 10^{38} \]

For 64-bit (double precision):

\[ c \in [1,2046] \]
\[ U = 2046 - 1023 = 1023 \]
\[ L = 1 - 1023 = -1022 \]
\[ c = 0 \quad \text {and} \quad c = (11111111111)_2 = 2047 \]

are reserved.

Smallest positive normalized FP number:

\[ \mathrm {UFL} = 2^{L} = 2^{-1022} \approx 2.2 \times 10^{-308} \]

Largest positive normalized FP number:

\[ \mathrm {OFL} = 2^{U+1}(1 - 2^{-p}) = 2^{1024}(1 - 2^{-53}) \approx 1.8 \times 10^{308} \]

PIC

Note again that overflow and underflow is different in floating pointer land as explained earlier:

PIC

PreviousContentsNext