Increment a Python Floating Point Value by the Smallest Possible Amount

Increment a Python floating point value by the smallest possible amount

Python 3.9 and above

Starting with Python 3.9, released 2020-10-05, you can use the math.nextafter function:

math.nextafter(x, y)

Return the next floating-point value after x towards y.

If x is equal to y, return y.

Examples:

• math.nextafter(x, math.inf) goes up: towards positive infinity.

• math.nextafter(x, -math.inf) goes down: towards minus infinity.

• math.nextafter(x, 0.0) goes towards zero.

• math.nextafter(x, math.copysign(math.inf, x)) goes away from zero.

See also math.ulp().

increment float32 by smallest possible amount (using numpy currently)

Seems to work fine:

>>> x = np.float32(1.)
>>> y = np.nextafter(x, np.float32(2.))
>>> y
1.0000001
>>> type(y)
numpy.float32

Python increment float by smallest step possible predetermined by its number of decimals

As the other commenters have noted: You should not operate with floats because a given number 0.1234 is converted into an internal representation and you cannot further process it the way you want. This is deliberately vaguely formulated. Floating points is a subject for itself. This article explains the topic very well and is a good primer on the topic.

That said, what you could do instead is to have the input as strings (e.g. do not convert it to float when reading from input). Then you could do this:

from decimal import Decimal

def add_one(v):
    after_comma = Decimal(v).as_tuple()[-1]*-1
    add = Decimal(1) / Decimal(10**after_comma)
    return Decimal(v) + add

if __name__ == '__main__':
    print(add_one("0.00531"))
    print(add_one("0.051959"))
    print(add_one("0.0067123"))
    print(add_one("1"))

This prints

0.00532
0.051960
0.0067124
2

Update:

If you need to operate on floats, you could try to use a fuzzy logic to come to a close presentation. decimal offers a normalize function which lets you downgrade the precision of the decimal representation so that it matches the original number:

from decimal import Decimal, Context

def add_one_float(v):
    v_normalized = Decimal(v).normalize(Context(prec=16))
    after_comma = v_normalized.as_tuple()[-1]*-1
    add = Decimal(1) / Decimal(10**after_comma)
    return Decimal(v_normalized) + add

But please note that the precision of 16 is purely experimental, you need to play with it to see if it yields the desired results. If you need correct results, you cannot take this path.

How to alter a float by its smallest increment (or close to it)?

Check your math.h file. If you're lucky you have the nextafter and nextafterf functions defined. They do exactly what you want in a portable and platform independent way and are part of the C99 standard.

Another way to do it (could be a fallback solution) is to decompose your float into the mantissa and exponent part. Incrementing is easy: Just add one to the mantissa. If you get an overflow you have to handle this by incrementing your exponent. Decrementing works the same way.

EDIT: As pointed out in the comments it is sufficient to just increment the float in it's binary representation. The mantissa-overflow will increment the exponent, and that's exactly what we want.

That's in a nutshell the same thing that nextafter does.

This won't be completely portable though. You would have to deal with endianess and the fact that not all machines do have IEEE floats (ok - the last reason is more academic).

Also handling NAN's and infinites can be a bit tricky. You cannot simply increment them as they are by definition not numbers.

Smallest positive float64 number

Use np.nextafter.

>>> import numpy as np
>>> np.nextafter(0, 1)
4.9406564584124654e-324
>>> np.nextafter(np.float32(0), np.float32(1))
1.4012985e-45

In Python, why does the lowest (smallest) floating point value have different significant-digits than the largest?

All floating point numbers (double is a double-precision float) are written as a product of two values, the mantissa and the exponent.

The way the floating point number exponent is stored uses 8 bits (for floats) or 11 bits (for doubles), meaning you get exponent values of -127 to +128 (float) or -1023 to +1024 (double).

And 2^1024 gives us a value of 1.797693134862315907729305190789 * 10^308, which is the largest exponent of a double precision float.

Have a demo in IEEE-754 Floating Point Converter

For Python, a detail explanation can be found in the page "Floating Point Arithmetic: Issues and Limitations" in the Python tutorial.

How do you get the next value in the floating-point sequence?

Here are five (really four-and-a-half) possible solutions.

Solution 1: use Python 3.9 or later

Python 3.9, released in October 2020, includes a new standard library function math.nextafter which provides this functionality directly: use math.nextafter(x, math.inf) to get the next floating-point number towards positive infinity. For example:

>>> from math import nextafter, inf
>>> nextafter(100.0, inf)
100.00000000000001

It's a bit easier to verify that this function really is producing the next float up if you look at the hexadecimal representation, provided by the float.hex method:

>>> 100.0.hex()
'0x1.9000000000000p+6'
>>> nextafter(100.0, inf).hex()
'0x1.9000000000001p+6'

Python 3.9 also introduces a closely related and frequently useful companion function math.ulp which gives the difference between a value and the next value away from zero:

>>> from math import ulp
>>> nextafter(100.0, inf) - 100.0
1.4210854715202004e-14
>>> ulp(100.0)
1.4210854715202004e-14

Solution 2: use NumPy

If you don't have Python 3.9 or later, but you do have access to NumPy, then you can use numpy.nextafter. For regular Python floats, the semantics match those of math.nextafter (though it would be fairer to say that Python's semantics match NumPy's, since NumPy had this functionality available long before Python did).

>>> from numpy import nextafter, inf
>>> nextafter(100.0, inf)
100.00000000000001

Solution 3: wrap C's nextafter yourself

C specifies a nextafter function in math.h (see for example section 7.12.11.3 of C99); this is exactly the function that Python >= 3.9 wraps and exposes in its math module. If you don't have Python 3.9 or later, you can either use ctypes or cffi to dynamically call C's nextafter, or alternatively write a simple Cython wrapper or Python C extension that exposes C's nextafter. The details of how to do this are already well-explained elsewhere: in @Endophage's answer to this question, and in this answer to a similar StackOverflow question (the one that this question is closed as a duplicate of).

Solution 4: bit manipulation via the struct module

If you're willing to make the (almost always safe in practice) assumption that Python is using IEEE 754 floating-point, it's quite easy to write a
Python function to provide nextafter. A little bit of care is needed to get all the corner cases right.

The IEEE 754 binary floating-point formats are cleverly designed so that moving from one floating-point number to the 'next' one is as simple as incrementing the bit representation. This works for any number in the range [0, infinity), right across exponent boundaries and subnormals. To produce a version of nextUp that covers the complete floating-point range, you also need to deal with negative numbers, infinities, nans, and one special case involving negative zero. Below is a standards compliant version of IEEE 754's nextUp function in Python. It covers all the corner cases.

import math
import struct

def nextup(x):
    # NaNs and positive infinity map to themselves.
    if math.isnan(x) or (math.isinf(x) and x > 0):
        return x

    # 0.0 and -0.0 both map to the smallest +ve float.
    if x == 0.0:
        x = 0.0

    n = struct.unpack('<q', struct.pack('<d', x))[0]
    if n >= 0:
        n += 1
    else:
        n -= 1
    return struct.unpack('<d', struct.pack('<q', n))[0]

The implementations of nextDown and nextAfter then look like this. (Note that nextAfter is not a function specified by IEEE 754, so there's a little bit of guesswork as to what should happen with IEEE special values. Here I'm following the IBM Decimal Arithmetic standard that Python's decimal.Decimal class is based on.)

def nextdown(x):
    return -nextup(-x)

def nextafter(x, y):
    # If either argument is a NaN, return that argument.
    # This matches the implementation in decimal.Decimal
    if math.isnan(x):
        return x
    if math.isnan(y):
        return y

    if y == x:
        return y
    elif y > x:
        return nextup(x)
    else:
        return nextdown(x)

(Partial) solution 5: floating-point operations

If x is a positive not-too-tiny float and you're willing to assume IEEE 754 binary64 format and semantics, there's a surprisingly simple solution: the next float up from x is x / (1 - 2**-53), and the next float down from x is x * (1 - 2**-53).

In more detail, suppose that all of the following are true:

• You don't care about IEEE 754 corner cases (zeros, infinities, subnormals, nans)
• You can assume not only IEEE 754 binary64 floating-point format, but also IEEE 754 binary64 semantics: namely that all basic arithmetic operations are correctly rounded according to the current rounding mode
• You can further assume that the current rounding mode is the IEEE 754 default round-ties-to-even mode.

Then the quantity 1 - 2**-53 is exactly representable as a float, and given a positive non-subnormal Python float x, x / (1 - 2**-53) will match nextafter(x, inf). Similarly, x * (1 - 2**-53) will match nextafter(x, -inf), except in the corner case where x is the smallest positive normal value, 2**-1022.

There's one thing to be careful of when using this: the expression 2**-53 will invoke your pow from your system's math library, and it's generally not safe to expect pow to be correctly rounded. There are many safer ways to compute this constant, one of which is to use float.fromhex. Here's an example:

>>> d = float.fromhex('0x1.fffffffffffffp-1')  # 1 - 2**-53, safely
>>> d
0.9999999999999999
>>> x = 100.0
>>> x / d  # nextup(x), or nextafter(x, inf)
100.00000000000001
>>> x * d  # nextdown(x), or nextafter(x, -inf)
99.99999999999999

These tricks work right across the normal range of floats, including for awkward cases like exact powers of two.

For a sketch of a proof: to show that x / d matches nextafter(x, inf) for positive normal x, we can scale by a power of two without affecting correctness, so in the proof we can assume without loss of generality that 0.5 <= x < 1.0. If we write z for the exact mathematical value of x / d (thought of as a real number, not a floating-point number), then z - x is equal to x * 2**-53 / (1 - 2**-53). Combining with the inequality 0.5 <= x <= 1 - 2**-53, we can conclude that 2**-54 < z - x <= 2**-53, which since floats are spaced exactly 2**-53 apart in the interval [0.5, 1.0], is enough to guaranteed that the closest float to z is nextafter(x, inf). The proof for x * d is similar.

What is the range of values a float can have in Python?

See this post.

Relevant parts of the post:

In [2]: import kinds 
In [3]: kinds.default_float_kind.M 
kinds.default_float_kind.MAX         kinds.default_float_kind.MIN 
kinds.default_float_kind.MAX_10_EXP  kinds.default_float_kind.MIN_10_EXP 
kinds.default_float_kind.MAX_EXP     kinds.default_float_kind.MIN_EXP 
In [3]: kinds.default_float_kind.MIN 
Out[3]: 2.2250738585072014e-308 

---

Related Topics