How to Correctly and Standardly Compare Floats

How to correctly and standardly compare floats?

Thanks for your answers, they helped me a lot. I've read these materials:first and second

The answer is to use my own function for relative comparison:

bool areEqualRel(float a, float b, float epsilon) {
    return (fabs(a - b) <= epsilon * std::max(fabs(a), fabs(b)));
}

This is the most suitable solution for my needs. However I've wrote some tests and other comparison methods. I hope this will be useful for somebody. areEqualRel passes these tests, others don't.

#include <iostream>
#include <limits>
#include <algorithm>

using std::cout;
using std::max;

bool areEqualAbs(float a, float b, float epsilon) {
    return (fabs(a - b) <= epsilon);
}

bool areEqual(float a, float b, float epsilon) {
    return (fabs(a - b) <= epsilon * std::max(1.0f, std::max(a, b)));
}

bool areEqualRel(float a, float b, float epsilon) {
    return (fabs(a - b) <= epsilon * std::max(fabs(a), fabs(b)));
}

int main(int argc, char *argv[])
{
    cout << "minimum: " << FLT_MIN      << "\n";
    cout << "maximum: " << FLT_MAX      << "\n";
    cout << "epsilon: " << FLT_EPSILON  << "\n";

    float a = 0.0000001f;
    float b = 0.0000002f;
    if (areEqualRel(a, b, FLT_EPSILON)) {
        cout << "are equal a: " << a << " b: " << b << "\n";
    }
    a = 1000001.f;
    b = 1000002.f;
    if (areEqualRel(a, b, FLT_EPSILON)) {
        cout << "are equal a: " << a << " b: " << b << "\n";
    }
}

How should I do floating point comparison?

Comparing for greater/smaller is not really a problem unless you're working right at the edge of the float/double precision limit.

For a "fuzzy equals" comparison, this (Java code, should be easy to adapt) is what I came up with for The Floating-Point Guide after a lot of work and taking into account lots of criticism:

public static boolean nearlyEqual(float a, float b, float epsilon) {
    final float absA = Math.abs(a);
    final float absB = Math.abs(b);
    final float diff = Math.abs(a - b);

    if (a == b) { // shortcut, handles infinities
        return true;
    } else if (a == 0 || b == 0 || diff < Float.MIN_NORMAL) {
        // a or b is zero or both are extremely close to it
        // relative error is less meaningful here
        return diff < (epsilon * Float.MIN_NORMAL);
    } else { // use relative error
        return diff / (absA + absB) < epsilon;
    }
}

It comes with a test suite. You should immediately dismiss any solution that doesn't, because it is virtually guaranteed to fail in some edge cases like having one value 0, two very small values opposite of zero, or infinities.

An alternative (see link above for more details) is to convert the floats' bit patterns to integer and accept everything within a fixed integer distance.

In any case, there probably isn't any solution that is perfect for all applications. Ideally, you'd develop/adapt your own with a test suite covering your actual use cases.

What is the best way to compare floats for almost-equality in Python?

Python 3.5 adds the math.isclose and cmath.isclose functions as described in PEP 485.

If you're using an earlier version of Python, the equivalent function is given in the documentation.

def isclose(a, b, rel_tol=1e-09, abs_tol=0.0):
    return abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)

rel_tol is a relative tolerance, it is multiplied by the greater of the magnitudes of the two arguments; as the values get larger, so does the allowed difference between them while still considering them equal.

abs_tol is an absolute tolerance that is applied as-is in all cases. If the difference is less than either of those tolerances, the values are considered equal.

What is a standard way to compare float with zero?

To compare a floating-point value with 0, just compare it:

if (f == 0)
    // whatever

There is nothing wrong with this comparison. If it doesn't do what you expect it's because the value of f is not what you thought it was. Its essentially the same problem as this:

int i = 1/3;
i *= 3;
if (i == 1)
    // whatever

There's nothing wrong with that comparison, but the value of i is not 1. Almost all programmers understand the loss of precision with integer values; many don't understand it with floating-point values.

Using "nearly equal" instead of == is an advanced technique; it often leads to unexpected problems. For example, it is not transitive; that is, a nearly equals b and b nearly equals c does not mean that a nearly equals c.

Is it safe to compare floats strictly, given we do no operations on them?

A mass of "should"s follows.

I don't think there's anything that says an assignment from float to float (current_width = new_width) couldn't alter the value, but I would be surprised if such a thing existed. There should be no reason to make assignment between variables of the same type anything else than a direct copy.

If the incoming new_width and new_height keep their values until they change, then this comparison should not have any issues. But if they are calculated before every call they might change their value, depending on how the calculation is done. So it's not only this function that needs to be checked.

The C 2011 standard says that calculations may use bigger precision than the format you assign to, but nothing specific about assigning a variable to another. So the only imprecision should be in the calculation stage. The "simple assignment" part (6.5.16.1) says:

In simple assignment (=), the value of the right operand is converted to the type of the assignment expression and replaces the value stored in the object designated by the left operand.

So if the types already match, there should be no need for conversion.

So, simply put: if you don't recalculate the incoming value on every call the comparison for equality should hold true. But is there really a case where your framebuffers are sized as floats and not integers?

comparing float/double values using == operator

IBM has a recommendation for comparing two floats, using division rather than subtraction - this makes it easier to select an epsilon that works for all ranges of input.

if (abs(a/b - 1) < epsilon)

As for the value of epsilon, I would use 5.96e-08 as given in this Wikipedia table, or perhaps 2x that value.

Compare floats in R

.Machine$double.eps is the difference between 1 and the smallest representable value greater than 1. The difference between 0.1 and the smallest representable value greater than 0.1 is smaller than .Machine$double.eps and the difference between 100 and the smallest representable value greater than 100 is larger than .Machine$double.eps. Have a look at: What is the correct/standard way to check if difference is smaller than machine precision?.

.Machine$double.eps is

.Machine$double.eps
[1] 2.220446e-16

When you make the calculations the intern sotred values will be approximately:

print(1.0 + .Machine$double.eps, 20)
#[1] 1.000000000000000222
print(1.0 - .Machine$double.eps, 20)
#[1] 0.99999999999999977796
print(0.9 + .Machine$double.eps, 20)
#[1] 0.90000000000000024425
print(2.0 + .Machine$double.eps, 20)
#[1] 2

Using tolerance = .Machine$double.eps all.equal returns TRUE or FALSE depending if the difference of the intern stored values is lager or not than tolerance.

To compare 2 numbers in R if the are intern stored equal use ==.

How dangerous is it to compare floating point values?

First of all, floating point values are not "random" in their behavior. Exact comparison can and does make sense in plenty of real-world usages. But if you're going to use floating point you need to be aware of how it works. Erring on the side of assuming floating point works like real numbers will get you code that quickly breaks. Erring on the side of assuming floating point results have large random fuzz associated with them (like most of the answers here suggest) will get you code that appears to work at first but ends up having large-magnitude errors and broken corner cases.

First of all, if you want to program with floating point, you should read this:

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Yes, read all of it. If that's too much of a burden, you should use integers/fixed point for your calculations until you have time to read it. :-)

Now, with that said, the biggest issues with exact floating point comparisons come down to:

1. The fact that lots of values you may write in the source, or read in with scanf or strtod, do not exist as floating point values and get silently converted to the nearest approximation. This is what demon9733's answer was talking about.

2. The fact that many results get rounded due to not having enough precision to represent the actual result. An easy example where you can see this is adding x = 0x1fffffe and y = 1 as floats. Here, x has 24 bits of precision in the mantissa (ok) and y has just 1 bit, but when you add them, their bits are not in overlapping places, and the result would need 25 bits of precision. Instead, it gets rounded (to 0x2000000 in the default rounding mode).

3. The fact that many results get rounded due to needing infinitely many places for the correct value. This includes both rational results like 1/3 (which you're familiar with from decimal where it takes infinitely many places) but also 1/10 (which also takes infinitely many places in binary, since 5 is not a power of 2), as well as irrational results like the square root of anything that's not a perfect square.

4. Double rounding. On some systems (particularly x86), floating point expressions are evaluated in higher precision than their nominal types. This means that when one of the above types of rounding happens, you'll get two rounding steps, first a rounding of the result to the higher-precision type, then a rounding to the final type. As an example, consider what happens in decimal if you round 1.49 to an integer (1), versus what happens if you first round it to one decimal place (1.5) then round that result to an integer (2). This is actually one of the nastiest areas to deal with in floating point, since the behaviour of the compiler (especially for buggy, non-conforming compilers like GCC) is unpredictable.

5. Transcendental functions (trig, exp, log, etc.) are not specified to have correctly rounded results; the result is just specified to be correct within one unit in the last place of precision (usually referred to as 1ulp).

When you're writing floating point code, you need to keep in mind what you're doing with the numbers that could cause the results to be inexact, and make comparisons accordingly. Often times it will make sense to compare with an "epsilon", but that epsilon should be based on the magnitude of the numbers you are comparing, not an absolute constant. (In cases where an absolute constant epsilon would work, that's strongly indicative that fixed point, not floating point, is the right tool for the job!)

Edit: In particular, a magnitude-relative epsilon check should look something like:

if (fabs(x-y) < K * FLT_EPSILON * fabs(x+y))

Where FLT_EPSILON is the constant from float.h (replace it with DBL_EPSILON fordoubles or LDBL_EPSILON for long doubles) and K is a constant you choose such that the accumulated error of your computations is definitely bounded by K units in the last place (and if you're not sure you got the error bound calculation right, make K a few times bigger than what your calculations say it should be).

Finally, note that if you use this, some special care may be needed near zero, since FLT_EPSILON does not make sense for denormals. A quick fix would be to make it:

if (fabs(x-y) < K * FLT_EPSILON * fabs(x+y) || fabs(x-y) < FLT_MIN)

and likewise substitute DBL_MIN if using doubles.

---

Related Topics