Why Does a Float Variable Stop Incrementing at 16777216 in C#

Why does a float variable stop incrementing at 16777216 in C#?

Short roundup of IEEE-754 floating point numbers (32-bit) off the top of my head:

• 1 bit sign (0 means positive number, 1 means negative number)
• 8 bit exponent (with -127 bias, not important here)
• 23 bits "mantissa"
• With exceptions for the exponent values 0 and 255, you can calculate the value as: (sign ? -1 : +1) * 2^exponent * (1.0 + mantissa)
  
  • The mantissa bits represent binary digits after the decimal separator, e.g. 1001 0000 0000 0000 0000 000 = 2^-1 + 2^-4 = .5 + .0625 = .5625 and the value in front of the decimal separator is not stored but implicitly assumed as 1 (if exponent is 255, 0 is assumed but that's not important here), so for an exponent of 30, for instance, this mantissa example represents the value 1.5625

Now to your example:

16777216 is exactly 2²⁴, and would be represented as 32-bit float like so:

• sign = 0 (positive number)
• exponent = 24 (stored as 24+127=151=10010111)
• mantissa = .0
• As 32 bits floating-point representation: 0 10010111 00000000000000000000000
• Therefore: Value = (+1) * 2^24 * (1.0 + .0) = 2^24 = 16777216

Now let's look at the number 16777217, or exactly 2²⁴+1:

• sign and exponent are the same
• mantissa would have to be exactly 2^-24 so that (+1) * 2^24 * (1.0 + 2^-24) = 2^24 + 1 = 16777217
• And here's the problem. The mantissa cannot have the value 2^-24 because it only has 23 bits, so the number 16777217 just cannot be represented with the accuracy of 32-bit floating points numbers!

Can't increment value of a floating-point variable

You have mantissa overflow and as the result a presision loss. float (or Single) type has 24 bits mantissa (up to 16777216):

https://en.wikipedia.org/wiki/Single-precision_floating-point_format

Let's see what's going on:

  private static String MakeReport(float value) {
    return String.Join(" ", BitConverter
     .GetBytes(value)
     .Select(b => Convert.ToString(b, 2).PadLeft(8, '0')));
  }

...

  float f = 16777215;

  // Mantissa (first 3 bytes) is full of 1's except the last one bit
  // 11111111 11111111 01111111 01001011
  Console.Write(MakeReport(f)); 

  // Overflow! Presision loss
  // 00000000 00000000 10000000 01001011  
  Console.Write(MakeReport(f + 1)); 

  // Overflow! Presision loss
  // 00000000 00000000 10000000 01001011  
  Console.Write(MakeReport(f + 2)); 

  // Overflow! Presision loss
  // 00000100 00000000 10000000 01001011  
  Console.Write(MakeReport(f + 10)); 

Remedy: do not use floating points as counter but integer:

 int counter = 0;

To avoid integer division cast value into double

  float x = (float) ((((double)counter * blockSizeBytes) / fileSize) * fullPerc);

Understanding floating point representation errors; what's wrong with my thinking?

If your exponent is decimal (i.e. it represents 10^X), you can precisely represent 0.1 -- however, most floating point formats use binary exponents (i.e. they represent 2^X). Since there are no integers X and Y such that Y * (2 ^ X) = 0.1, you cannot precisely represent 0.1 in most floating point formats.

Some languages have types with both exponents. In C#, for example, there is a data type aptly named decimal which is a floating point format with a decimal exponent so it will support storing a number like 0.1, although it has other uncommon properties: The decimal type can distinguish between 0.1 and 0.10, and it is always true that x + 1 != x for all values of x.

For most common purposes, though, C# also has the float and double floating point types that cannot precisely store 0.1 because they use a binary exponent (as defined in IEEE-754). The binary floating point types use less storage, are faster because they are easier to implement, and have more operations defined on them. In general decimal is only used for financial values where the exact representation of all decimal values is important and the storage, speed, and range of operations are not.

How To Represent 0.1 In Floating Point Arithmetic And Decimal

I've always pointed people towards Harald Schmidt's online converter, along with the Wikipedia IEEE754-1985 article with its nice pictures.

For those two specific values, you get (for 0.1):

s Why does a float variable stop incrementing at 16777216 in C#? Can't increment value of a floating-point variable Understanding floating point representation errors; wh mmmmmmmmmmmmmmmmmmmmmmm    1/n
0 01111011 10011001100110011001101
           |  ||  ||  ||  ||  || +- 8388608
           |  ||  ||  ||  ||  |+--- 2097152
           |  ||  ||  ||  ||  +---- 1048576
           |  ||  ||  ||  |+-------  131072
           |  ||  ||  ||  +--------   65536
           |  ||  ||  |+-----------    8192
           |  ||  ||  +------------    4096
           |  ||  |+---------------     512
           |  ||  +----------------     256
           |  |+-------------------      32
           |  +--------------------      16
           +-----------------------       2

The sign is positive, that's pretty easy.

The exponent is 64+32+16+8+2+1 = 123 - 127 bias = -4, so the multiplier is 2-4 or 1/16.

The mantissa is chunky. It consists of 1 (the implicit base) plus (for all those bits with each being worth 1/(2n) as n starts at 1 and increases to the right), {1/2, 1/16, 1/32, 1/256, 1/512, 1/4096, 1/8192, 1/65536, 1/131072, 1/1048576, 1/2097152, 1/8388608}.

When you add all these up, you get 1.60000002384185791015625.

When you multiply that by the multiplier, you get 0.100000001490116119384765625, which is why they say you cannot represent 0.1 exactly as an IEEE754 float, and provides so much opportunity on SO for people answering "why doesn't 0.1 + 0.1 + 0.1 == 0.3?"-type questions :-)

---

The 0.5 example is substantially easier. It's represented as:

s Why does a float variable stop incrementing at 16777216 in C#? Can't increment value of a floating-point variable Understanding floating point representation errors; wh mmmmmmmmmmmmmmmmmmmmmmm
0 01111110 00000000000000000000000

which means it's the implicit base, 1, plus no other additives (all the mantissa bits are zero).

The sign is again positive. The exponent is 64+32+16+8+4+2 = 126 - 127 bias = -1. Hence the multiplier is 2-1 which is 1/2 or 0.5.

So the final value is 1 multiplied by 0.5, or 0.5. Voila!

---

I've sometimes found it easier to think of it in terms of decimal.

The number 1.345 is equivalent to

1 + 3/10   + 4/100 + 5/1000

or:

        -1       -2      -3
1 + 3*10   + 4*10  + 5*10

---

Similarly, the IEEE754 representation for decimal 0.8125 is:

s Why does a float variable stop incrementing at 16777216 in C#? Can't increment value of a floating-point variable Understanding floating point representation errors; wh mmmmmmmmmmmmmmmmmmmmmmm
0 01111110 10100000000000000000000

With the implicit base of 1, that's equivalent to the binary:

         01111110-01111111
1.101 * 2

or:

                     -1
(1   + 1/2 + 1/8) * 2     (no 1/4 since that bit is 0)

which becomes:

(8/8 + 4/8 + 1/8) * 1/2

and then becomes:

13/8 * 1/2 = 0.8125

Avoid floating point precision issues

An interesting way would be to implement this into the movement function:

float result = //multiply force by time
float modulus = result%0.01f;
result -= modulus; //you will do this in any case
if(modulus>=0.005f) {/*round up. if you want it to only round down, remove
  the next 2 lines, if you want it to only round up, remove
  the conditional statement*/
  result+=0.01f; }

I can't think about how to optimize it further, I removed the else statement and have it take away the modulus without condition as it will be done anyway.

Int vs Float: Counter

The float data type is only precise to 6 significant figures and is inappropriate for counter and total. Any floating point type would be inappropriate in any case. Ther are a number of issues with this, not least that ++ for example is an integer operator, the implicit conversion from float to int, increment, then back to float will fail for integer values with greater than 6 digits.

I assume you chose such a type because it has greater range that unsigned int perhaps? I suggest that you use unsigned long long for these variables.

unsigned long long counter = 0;
unsigned long long total = 0;

...

float percent = (float)counter / (float)total * 100.0f ;

Understanding casts from integer to float

In the IEEE-754 basic 32-bit binary floating-point format, all integers from −16,777,216 to +16,777,216 are representable. From 16,777,216 to 33,554,432, only even integers are representable. Then, from 33,554,432 to 67,108,864, only multiples of four are representable. (Since the question does not necessitate discussion of which numbers are representable, I will omit explanation and just take this for granted.)

The most common default rounding mode is to round the exact mathematical result to the nearest representable value and, in case of a tie, to round to the representable value which has zero in the low bit of its significand.

16,777,217 is equidistant between the two representable values 16,777,216 and 16,777,218. These values are represented as 100000000000000000000000₂•2¹ and 100000000000000000000001₂•2¹. The former has 0 in the low bit of its significand, so it is chosen as the result.

16,777,219 is equidistant between the two representable values 16,777,218 and 16,777,220. These values are represented as 100000000000000000000001₂•2¹ and 100000000000000000000010₂•2¹. The latter has 0 in the low bit of its significand, so it is chosen as the result.

What's the rationale for null terminated strings?

From the horse's mouth

None of BCPL, B, or C supports
character data strongly in the
language; each treats strings much
like vectors of integers and
supplements general rules by a few
conventions. In both BCPL and B a
string literal denotes the address of
a static area initialized with the
characters of the string, packed into
cells. In BCPL, the first packed byte
contains the number of characters in
the string; in B, there is no count
and strings are terminated by a
special character, which B spelled
*e. This change was made partially
to avoid the limitation on the length
of a string caused by holding the
count in an 8- or 9-bit slot, and
partly because maintaining the count
seemed, in our experience, less
convenient than using a terminator.

_{Dennis M Ritchie, Development of the C Language}

Efficient implementation of natural logarithm (ln) and exponentiation

The Taylor series for e^x converges extremely quickly, and you can tune your implementation to the precision that you need. (http://en.wikipedia.org/wiki/Taylor_series)

The Taylor series for log is not as nice...

---

Related Topics