Double.Epsilon for Equality, Greater Than, Less Than, Less Than or Equal To, Greater Than or Equal To

Double.Epsilon for equality, greater than, less than, less than or equal to, greater than or equal to

I don't know what they were smoking when they wrote that. Double.Epsilon is the smallest representable non-denormal floating point value that isn't 0. All you know is that, if there's a truncation error, it will always be larger than this value. Much larger.

The System.Double type can represent values accurate to up to 15 digits. So a simple first order estimate if a double value x is equal to some constant is to use an epsilon of constant * 1E-15

public static bool AboutEqual(double x, double y) {
    double epsilon = Math.Max(Math.Abs(x), Math.Abs(y)) * 1E-15;
    return Math.Abs(x - y) <= epsilon;
}

You have to watch out though, truncation errors can accumulate. If both x and y are computed values then you have to increase the epsilon.

Rule of thumb to test the equality of two doubles in C#?

Using double.Epsilon does NOT necessarily work. double.Epsilon gives the smallest representable value that is greater than zero. However, because of the way that floating point numbers are implemented, they have less precision the further away from zero they are, so checking for a difference of double.Epsilon could fail for two large numbers that are very close to each other.

Details: A base-2 floating point number is represented as a significand - a number between 1 and 2 - multiplied by two raised to some exponent. A double has 52 bits for the fractional portion of the significand plus 11 bits of precision for the exponent. If the exponent is a very large negative value and the significand is 0, then you get values close to double.Epsilon, but if your exponent is big enough, then even a very small difference in two significands' values will result in a value much larger than double.Epsilon.

For a full discussion on how to test two floating point numbers for equality, see "Comparing Floating Point Numbers, 2012 Edition", by Bruce Dawson. To summarize, there are three main methods of comparison:

Use an absolute difference

As in Joel Coehoorn's example, but be very careful to select a value that's of an appropriate magnitude, unlike Joel's example.

Use a relative difference

Something like the following:

if (Math.Abs(a - b) / b <= maxRelativeError)
{
    return true;
}

However, there are complications; you should divide by the larger of the two values, and this function performs poorly for values close to zero unless you also add a check for a maximum absolute difference. See the paper for details.

Using units of last place

Comparison using units of last place (ULPs) means checking the last portion of the significand. (The paper refers to this as "Comparing using integers.") This is a more complicated approach but is very robust. The paper provides source code in C; for C#, you could probably use BitConverter.DoubleToInt64Bits.

In response to your edit

"How many times greater?" This is really a question of your application domain, which is probably why the .NET Framework doesn't provide a default method, but I've had good luck using the ULPs comparison with a max ULPs difference of 4.

Modern practice to compare double/float for equality in modern C++

Is this code with modern C++11/14/17/21 is still the way we should compare float and doubles, or now it's ok just to write if (double1 == double2) And compiler will handle the epsilon issue for us?

Both approaches function the same in modern C++ as they did in early C++.

Both approaches are also flawed.

• Using == assumes that your code has accounted for any floating point rounding errors, and it's very rare/difficult for code to do that.

• Comparing against epsilon assumes that a reasonable amount of rounding error will be less than the constant epsilon, and that is very likely a wrong assumption!

  • If your numbers have magnitude greater than 2.0, your epsilon trick will be no different from direct comparison, and have the same flaws. Regardless of whether you use < or <=.
  • If your numbers have the same sign and a magnitude smaller than epsilon, your epsilon trick will say they are always equal, even if one is hundreds of times larger than the other. They would both be equal to zero, too.

A wise approach may be to avoid writing code that depends on whether floating point numbers are equal. Instead test if they are relatively close, by some factor.

The code below will test whether two numbers are within about 0.01% of each other. Regardless of their scale.

const auto relative_difference_factor = 0.0001.    // 0.01%
const auto greater_magnitude = std::max(std::abs(double1),std::abs(double2));

if ( std::abs(double1-double2) < relative_difference_factor * greater_magnitude )
  std::cout<<"Relatively close";
else
  std::cout<<"Not relatively close";

Is the use of machine epsilon appropriate for floating-point equality tests?

How to choose a value for epsilon?

Short Answer: You take a small value which fits your applications needs.

Long Answer: Nobody can know which calculations your application does and how accurate you expect your results to be. Since rounding errors sum up machine epsilon will be almost all times far too big so you have to chose your own value. Depending on your needs, 0.01 be be sufficient, or maybe 0.00000000000001 or less will.

The question is, do you really want/need to do equality tests on floating point values? Maybe you should redesign your algorithms.

What is correct way to compare 2 doubles values?

So the question will be is there a generic solution/workaround for this

There will not be a universal solution for finite precision floating point that would apply to all use cases. There cannot be, because the correct threshold is specific to each calculation, and cannot generally be known automatically.

You have to know what you are comparing and what you are expecting from the comparison. Full explanation won't fit in this answer, but you can find most information from this blog: https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ (not mine).

---

There is however a generic solution/workaround that side-steps the issue: Use infinite precision arithmetic. The C++ standard library does not provide an implementation of infinite precision arithmetic.

Why does adding double.epsilon to a value result in the same value, perfectly equal?

Per the documentation of Double.Epsilon:

The value of the Epsilon property reflects the smallest positive
Double value that is significant in numeric operations or comparisons
when the value of the Double instance is zero.

(Emphasis mine.)

Adding it to 90.0 does not produce "the next smallest value after 90.0", this just yields 90.0 again.

Choosing an Epsilon Value for Floating Point Comparisons

Your reasoning seems sound, but as you have already discovered it is a complicated issue. You might want to read What Every Computer Scientist Should Know About Floating-Point Arithmetic. You do have a minimum of 15 digits of precision using 64 bit doubles. However, you will also want to validate your inputs as floats can contain Nan, +/- Infinity, negative zero and a considerably larger "range" than 15 decimal digits. If someone hands your library a value like 1.2E102, should you process it or consider it out of range? Ditto with very small values. Garbage In, Garbage out, but it might be nice if you code detected the "smell" of garbage and at very least logged it.

You might also want to consider providing a property for setting precision as well as different forms of rounding. That depends largely on the specifications you are working with. You might also want to determine if these values can represent currencies other than dollars (1 dollar is currently >112 yen).

Long and the short of it choosing your epsilon a digit below your needs (so four digits to the right of the decimal) is sound and gives you a digit to use for consistent rounding. Otherwise $10.0129 and $10.0121 would be equal but their sum would be $20.025 rather than $20.024 ... accountants like things that "foot".

---

Related Topics