Question

I was reading about floating-point representation and underflow/overflow and I ecnountered something interesting – gradual underflow. As I understand gradual underflow means that the result of, for example substraction x-y is so small that it could be flushed to 0 but floating-point system produces number that is smaller then UFL. Everywhere I read that it is made by losing some precission, it means that some bits of mantissa goes to exponent so we can have smaller exponent?

Answer 1

Effectively the answer is yes — the bits of the mantissa go to the exponent. These are called subnormal (AKA denormal) numbers. For example, in IEEE double-precision, the smallest power of two exponent for a normal number — a number with a full 53 bits of precision — is 2^-1022. But powers of two up to 2^-1074 can effectively be represented, as dictated by the location of the first 1 bit in the unnormalized significand. So exponent 2^-1023 has 52 bits of precision, 2^-1024 has 51 bits of precision, … , 2^-1074 has 1 bit of precision.

(See my article What Powers of Two Look Like Inside a Computer to visualize this better.)