Dealing with Float Precision in JavaScript

Dealing with float precision in Javascript

From this post: How to deal with floating point number precision in JavaScript?

You have a few options:

• Use a special datatype for decimals, like decimal.js
• Format your result to some fixed number of significant digits, like this:
  (Math.floor(y/x) * x).toFixed(2)
• Convert all your numbers to integers

How to deal with floating point number precision in JavaScript?

From the Floating-Point Guide:

What can I do to avoid this problem?

That depends on what kind of
calculations you’re doing.

• If you really need your results to add up exactly, especially when you
  work with money: use a special decimal
  datatype.
• If you just don’t want to see all those extra decimal places: simply
  format your result rounded to a fixed
  number of decimal places when
  displaying it.
• If you have no decimal datatype available, an alternative is to work
  with integers, e.g. do money
  calculations entirely in cents. But
  this is more work and has some
  drawbacks.

Note that the first point only applies if you really need specific precise decimal behaviour. Most people don't need that, they're just irritated that their programs don't work correctly with numbers like 1/10 without realizing that they wouldn't even blink at the same error if it occurred with 1/3.

If the first point really applies to you, use BigDecimal for JavaScript, which is not elegant at all, but actually solves the problem rather than providing an imperfect workaround.

How to deal with floating point number precision in JavaScript?

From the Floating-Point Guide:

What can I do to avoid this problem?

That depends on what kind of
calculations you’re doing.

• If you really need your results to add up exactly, especially when you
  work with money: use a special decimal
  datatype.
• If you just don’t want to see all those extra decimal places: simply
  format your result rounded to a fixed
  number of decimal places when
  displaying it.
• If you have no decimal datatype available, an alternative is to work
  with integers, e.g. do money
  calculations entirely in cents. But
  this is more work and has some
  drawbacks.

Note that the first point only applies if you really need specific precise decimal behaviour. Most people don't need that, they're just irritated that their programs don't work correctly with numbers like 1/10 without realizing that they wouldn't even blink at the same error if it occurred with 1/3.

If the first point really applies to you, use BigDecimal for JavaScript, which is not elegant at all, but actually solves the problem rather than providing an imperfect workaround.

Algorithm to correct the floating point precision error in JavaScript

When a decimal numeral is rounded to binary floating-point, there is no way to know, just from the result, what the original number was or how many significant digits it had. Infinitely many decimal numerals will round to the same result.

However, the rounding error is bounded. If it is known that the original number had at most a certain number of digits, then only decimal numerals with that number of digits are candidates. If only one of those candidates differs from the binary value by less than the maximum rounding error, then that one must be the original number.

If I recall correctly (I do not use JavaScript regularly), JavaScript uses IEEE-754 64-bit binary. For this format, it is known that any 15-digit decimal numeral may be converted to this binary floating-point format and back without error. Thus, if the original input was a decimal numeral with at most 15 significant digits, and it was converted to 64-bit binary floating-point (and no other operations were performed on it that could have introduced additional error), and you format the binary floating-point value as a 15-digit decimal numeral, you will have the original number.

The resulting decimal numeral may have trailing zeroes. It is not possible to know (from the binary floating-point value alone) whether those were in the original numeral.

How to format a float in javascript?

var result = Math.round(original*100)/100;

The specifics, in case the code isn't self-explanatory.

edit: ...or just use toFixed, as proposed by Tim Büthe. Forgot that one, thanks (and an upvote) for reminder :)

What's the maximum precision (after the decimal point) of a float in Javascript

Short answer: you can probably squeeze out 15 "safe" digits, and it doesn't matter where you place your decimal point.

It's anyone's guess how the JavaScript standard is going to evolve and use other number representations.

Notice how the MDN doc says "about 17 decimals"? Right, it's because sometimes you can represent that many digits, and sometimes less. It's because the floating point representation doesn't map 1-to-1 to our decimal system.

Even numbers with seemingly less information will give rounding errors.

For example
0.1 + 0.2 => 0.30000000000000004

console.log(0.1 + 0.2);

Restrict Float Precision in JavaScript

Try this it is rounding to 3 numbers after coma:

(x/y).toFixed(3);

Now your result will be a string. If you need it to be float just do:

parseFloat((x/y).toFixed(3));

---

Related Topics