Deal With Overflow in Exp Using Numpy

Overflow Error in Python's numpy.exp function

As fuglede says, the issue here is that np.float64 can't handle a number as large as exp(1234.1). Try using np.float128 instead:

>>> cc = np.array([[0.120,0.34,-1234.1]], dtype=np.float128)
>>> cc
array([[ 0.12,  0.34, -1234.1]], dtype=float128)
>>> 1 / (1 + np.exp(-cc))
array([[ 0.52996405,  0.58419052,  1.0893812e-536]], dtype=float128)

Note however, that there are certain quirks with using extended precision. It may not work on Windows; you don't actually get the full 128 bits of precision; and you might lose the precision whenever the number passes through pure python. You can read more about the details here.

For most practical purposes, you can probably approximate 1 / (1 + <a large number>) to zero. That is to say, just ignore the warning and move on. Numpy takes care of the approximation for you (when using np.float64):

>>> 1 / (1 + np.exp(-cc))
/usr/local/bin/ipython3:1: RuntimeWarning: overflow encountered in exp
  #!/usr/local/bin/python3.4
array([[ 0.52996405,  0.58419052,  0.        ]])

If you want to suppress the warning, you could use scipy.special.expit, as suggested by WarrenWeckesser in a comment to the question:

>>> from scipy.special import expit
>>> expit(cc)
array([[ 0.52996405,  0.58419052,  0.        ]])

Deal with overflow in exp using numpy

You can use the bigfloat package. It supports arbitrary precision floating point operations.

http://packages.python.org/bigfloat/

import bigfloat
bigfloat.exp(5000,bigfloat.precision(100))
# -> BigFloat.exact('2.9676283840236670689662968052896e+2171', precision=100)

Are you using a function optimization framework? They usually implement value boundaries (using penalty terms). Try that. Are the relevant values really that extreme? In optimization it's not uncommon to minimize log(f). (approximate log likelihood etc etc). Are you sure you want to optimize on that exp value and not log(exp(f)) == f. ?

Have a look at my answer to this question: logit and inverse logit functions for extreme values

Btw, if all you do is minimize powellBadlyScaled(x,y) then the minimum is at x -> + inf and y -> + inf, so no need for numerics.

Overflow in exp in scipy/numpy in Python?

In your case, it means that b is very small somewhere in your array, and you're getting a number (a/b or exp(log(a) - log(b))) that is too large for whatever dtype (float32, float64, etc) the array you're using to store the output is.

Numpy can be configured to

1. Ignore these sorts of errors,
2. Print the error, but not raise a warning to stop the execution (the default)
3. Log the error,
4. Raise a warning
5. Raise an error
6. Call a user-defined function

See numpy.seterr to control how it handles having under/overflows, etc in floating point arrays.

How to avoid an overflow in numpy.exp()

SciPy comes with a function to do that, which won't give you that warning:

scipy.special.expit(x)

numpy exp overflow with negative input in numpy where

np.exp(1493) is e^1493, right? We can rebase that to be equal to 10^648.

When we ask numpy what the maximum value of a float64 is, it's 2e308, which is wildly less than 1e648.

>>> np.finfo(float)
finfo(resolution=1e-15, min=-1.7976931348623157e+308, max=1.7976931348623157e+308, dtype=float64)

The where clause doesn't evaluate till after you calculate the np.exp(Z) - 1 array.

You could fix this with np.exp(Z, where=Z<0) - 1.

In response to your comment:

>>> import numpy as np
>>> Z = np.array([[1e11,-1e11]])
>>> np.where(Z>=0, Z, 1*(np.exp(Z)-1) )
__main__:1: RuntimeWarning: overflow encountered in exp
array([[ 1.e+11, -1.e+00]])
>>> 

Clearly it does raise the same warning.

Numpy: How do I get inside an "overflow encountered in exp"?

I'm guessing the problem is that you cannot figure out when warning happens. To do that use try/except but in slightly modified way:

import warnings
warnings.filterwarnings("error")
...
try:
    1/(np.exp((j-w)/x)-1)
except RuntimeWarning:
    print(j)
    print(w)
    print(x)
    print((j-w)/x)

This should give a hint, there the problem lies

How to deal with exponent overflow of 64float precision in python?

You can use the function np.logaddexp() to do such operations. It computes logaddexp(x1, x2) == log(exp(x1) + exp(x2)) without explicitly computing the intermediate exp() values. This avoids the overflow. Since exp(0.0) == 1, you would compute np.logaddexp(0.0, 1000.0) and get the result of 1000.0, as expected.

How to fix "overflow encounter in exp" when curve fitting data in Scipy?

The problem is the small value for a. The minimization process tries to compensate via b resulting in an overflow. I get good results with starting values p0=( 3.2e6, -4000 ) Alternatively, you can define the function to be exp( a - b / t ) which the coverges well with p0=( 15, -4000 ) or even without providing a p0

---

Related Topics