How to Check For Nan Values

How can I check for NaN values?

Use math.isnan:

>>> import math
>>> x = float('nan')
>>> math.isnan(x)
True

How to check for float('nan') in Python?

It's very pedestrian, and a bit ugly, but why not just do the following?

import math
import numpy as np

if math.isnan(x) or x in ['nan', np.nan, ... ]:
    # Do something
    pass

I want to recommend a Yoda expression but haven't quite worked it out yet.

If you want to sweep everything under the carpet put it in a lambda or function.

Following on from https://stackoverflow.com/a/64090763/1021819, you can try to get the iterator to evaluate any in a lazy fashion. The problem then is that if none of the first conditions evaluates to True then the math.isnan() call is executed and can still throw the TypeError. If you evaluate lazily you can guard the math.isnan() call with a type check against str:

fn_list_to_check=[
    lambda x: x in ['nan', np.nan, ... ],
    lambda x: not isinstance(x, str),
    lambda x: math.isnan(x)
    ]

if any(f(x) for f in fn_list_to_check):
    # Do something
    pass

Note the absence of square list brackets in the any i.e. any() not any([]) (who knew?).

I think it's quite brilliant but equally as ugly - choose your poison.

For the second part of the question (why float('nan') != float('nan')), see

What is the rationale for all comparisons returning false for IEEE754 NaN values?

How to check if any value is NaN in a Pandas DataFrame

jwilner's response is spot on. I was exploring to see if there's a faster option, since in my experience, summing flat arrays is (strangely) faster than counting. This code seems faster:

df.isnull().values.any()

import numpy as np
import pandas as pd
import perfplot


def setup(n):
    df = pd.DataFrame(np.random.randn(n))
    df[df > 0.9] = np.nan
    return df


def isnull_any(df):
    return df.isnull().any()


def isnull_values_sum(df):
    return df.isnull().values.sum() > 0


def isnull_sum(df):
    return df.isnull().sum() > 0


def isnull_values_any(df):
    return df.isnull().values.any()


perfplot.save(
    "out.png",
    setup=setup,
    kernels=[isnull_any, isnull_values_sum, isnull_sum, isnull_values_any],
    n_range=[2 ** k for k in range(25)],
)

df.isnull().sum().sum() is a bit slower, but of course, has additional information -- the number of NaNs.

How do you check that a number is NaN in JavaScript?

Try this code:

isNaN(parseFloat("geoff"))

For checking whether any value is NaN, instead of just numbers, see here: How do you test for NaN in Javascript?

How can I check if a value is nan or a list?

Use or:

for i, row in df.iterrows():
    value = row["Name"]
    if pd.isnull(value) or (type(value) == list):
        dosomething()

Another way for check is isinstance:

for i, row in df.iterrows():
    value = row["Name"]
    if pd.isnull(value) or isinstance(value, list):
        dosomething()

Checking if particular value (in cell) is NaN in pandas DataFrame not working using ix or iloc

Try this:

In [107]: pd.isnull(df.iloc[1,0])
Out[107]: True

---

UPDATE: in a newer Pandas versions use pd.isna():

In [7]: pd.isna(df.iloc[1,0])
Out[7]: True

---

Related Topics