Floating-Point Numbers

Nowadays floats have roughly the same performance as integers: addition, subtraction, or multiplication all take about a nanosecond.  That is, floats are now cheap, and you can consider using floats for all sorts of stuff--even when you don't care about fractions!  The advantages of using floats are:

• Floats can store fractional numbers.
  
• Floats never overflow; they hit "infinity" instead.
  
• "double" has more bits than "int" (but less than "long").

Floats: Normalized Numbers

• You only need one bit to represent the sign--plus or minus.
• The exponent's just an integer, so you can store it as an integer (in binary).
• The 3.785746 part, called the "mantissa" or just "fraction" part, can be stored as the integer 3785746 (at least as long as you can figure out where the decimal point goes!).  Of course, we store the fraction part as a binary number.
  

1. There's exactly one digit to the left of the decimal point.
2. And that digit is not zero.   In binary, a "normalized" number *always* has a 1 at the left of the decimal point (if it ain't zero, it's gotta be one).  So most machines don't even store the 1, it's implicit--you just know it's there!

Bits in a Floating-Point Number

void print_bits(float f) {
	int i=*reinterpret_cast<int *>(&f); /* read bits with "pointer shuffle" */
	std::cout<<" float "<<std::setw(10)<<f<<" = ";
	for (int bit=31;bit>=0;bit--) {
		if (i&(1<<bit)) std::cout<<"1"; else std::cout<<"0";
		if (bit==31) std::cout<<" ";
		if (bit==23) std::cout<<" (implicit 1).";
	}
	std::cout<<std::endl;
}

int foo(void) {
	print_bits(0.0);
	print_bits(-1.0);
	print_bits(1.0);
	print_bits(2.0);
	print_bits(4.0);
	print_bits(8.0);
	print_bits(1.125);
	print_bits(1.25);
	print_bits(1.5);
	print_bits(1+1.0/10);
	return sizeof(float);
}

(Try this in NetRun now!)

/* IEEE floating-point number's bits:  sign  exponent   mantissa */
struct float_bits {
	unsigned int fraction:23; /**< Value is binary 1.fraction ("mantissa") */
	unsigned int exp:8; /**< Value is 2^(exp-127) */
	unsigned int sign:1; /**< 0 for positive, 1 for negative */
};

/* A union is a struct where all the fields *overlap* each other */
union float_dissector {
	float f;
	float_bits b;
};

float_dissector s;
s.f=8.0;
std::cout<<s.f<<"= sign "<<s.b.sign<<" exp "<<s.b.exp<<"  fract "<<s.b.fraction<<"\n";
return 0;

Many of the various new 16 or 8 bit float types are just truncated versions of larger floats, so the only circuit change is to throw away some fraction bits.

(Image adapted from the NVIDIA fp8 primer.)

They're pure research for now, but in the 2020 AI era there's some interest in formats that provide more precision with fewer bits. A "posit" adds a "regime" field to the usual float exponent field, which can then be shorter, leaving more precision for reasonable-sized numbers. This gives a numer format that smoothly scales down to even 8 bit floats (posit8_t) while maintaining a consistent format.

Roundoff in Arithmetic

float f=0.73;
while (1) {
	volatile float g=f+1;
	if (g==f) {
		std::cout<<"f+1 == f  at f="<< f <<", or 2^"<< log(f)/log(2.0) <<std::endl;
		return 0;
	}
	else f=g;
}

(Try this in NetRun now!)

double lil=1.0;
double big=pow(2.0,53); //<- carefully chosen for IEEE 64-bit float (52 bits of fraction + implicit 1)
std::cout<<" big+(lil+lil) -big = "<<  big+(lil+lil) -big <<std::endl;
std::cout<<"(big+lil)+lil  -big = "<< (big+lil)+lil  -big <<std::endl;

(Try this in NetRun now!)

float gnats=1.0;
volatile float windshield=1<<24;
float orig=windshield;
for (int i=0;i<1000;i++)
	windshield += gnats;

if (windshield==orig) std::cout<<"You puny bugs can't harm me!\n";
else std::cout<<"Gnats added "<<windshield-orig<<" to the windshield\n";

float gnats=1.0;
volatile float windshield=1<<24;
float orig=windshield;
volatile float gnatcup=0.0;
for (int i=0;i<1000;i++)
	gnatcup += gnats;
windshield+=gnatcup; /* add all gnats to the windshield at once */

if (windshield==orig) std::cout<<"You puny bugs can't harm me!\n";
else std::cout<<"Gnats added "<<windshield-orig<<" to the windshield\n";

Roundoff in Representation

for (int i=1;i<1000000000;i*=10) {
	double mul01=i*0.1;
	double div10=i/10.0;
	double diff=mul01-div10;
	std::cout<<"i="<<i<<"  diff="<<diff<<"\n";
}

i=1  diff=5.54976e-18
i=10  diff=5.55112e-17
i=100  diff=5.55112e-16
i=1000  diff=5.55112e-15
i=10000  diff=5.55112e-14
i=100000  diff=5.55112e-13
i=1000000  diff=5.54934e-12
i=10000000  diff=5.5536e-11
i=100000000  diff=5.54792e-10
Program complete.  Return 0 (0x0)