How to Check List Containing Nan

How to Check list containing NaN

I think this makes sense because of your pulling numpy into scope indirectly via the star import.

>>> import numpy as np
>>> [0.0,0.0]/0
Traceback (most recent call last):
  File "<ipython-input-3-aae9e30b3430>", line 1, in <module>
    [0.0,0.0]/0
TypeError: unsupported operand type(s) for /: 'list' and 'int'

>>> [0.0,0.0]/np.float64(0)
array([ nan,  nan])

When you did

from matplotlib.pylab import *

it pulled in numpy.sum:

>>> from matplotlib.pylab import *
>>> sum is np.sum
True
>>> [0.0,0.0]/sum([0.0, 0.0])
array([ nan,  nan])

You can test that this nan object (nan isn't unique in general) is in a list via identity, but if you try it in an array it seems to test via equality, and nan != nan:

>>> nan == nan
False
>>> nan == nan, nan is nan
(False, True)
>>> nan in [nan]
True
>>> nan in np.array([nan])
False

You could use np.isnan:

>>> np.isnan([nan, nan])
array([ True,  True], dtype=bool)
>>> np.isnan([nan, nan]).any()
True

Python check if all elements in a list are Nan

You need all:

np.isnan(op_list).all()
# True

For a solution using lists you can do:

all(i != i for i in op_list)
# True

how to check for np.nan in list comprehension?

Why are you comparing with is? Different values may not necessarily be the same instance. This is true for ints as well as floats. (CPython's quirk for small integers is the only exception that I know of, and is strictly an implementation detail.)

>>> import math
>>> float('nan') is math.nan
False

You can use np.isnan to check for nan, or the built-in math.isnan, or check that the value is equal to itself.

>>> np.isnan([12.6,np.nan,0.5,4.6])
array([False,  True, False, False], dtype=bool)
>>> [math.isnan(x) for x in [12.6,np.nan,0.5,4.6]]
[False, True, False, False]
>>> [x for x in [12.6,np.nan,0.5,4.6] if x == x]
[12.6, 0.5, 4.6]

How to get list of strings from list-like string that includes nan?

What about:

eval(z,{'nan':'nan'}) # if you can tolerate then: 
[i for i in eval(z,{'nan':'nan'}) if i != 'nan']

It may have security considerations.

How can I get all the index of all the NaNs of a list?

If you're using NumPy, you should really start using arrays, and get out of the habit of manually looping at Python level. Manual loops are typically about 100 times slower than letting NumPy handle things, and lists of floats take about 4 times the memory of an array.

In this case, NumPy can give you an array of NaN indices quite simply:

ind = numpy.where(numpy.isnan(a))[0]

numpy.isnan gives an array of booleans telling which elements of a are NaN. numpy.where gives an array of indices of True elements, but wrapped in a 1-element tuple for consistency with the behavior on multidimensional arrays, so [0] extracts the array from the tuple.

This works when a is a list, but you should really use arrays.

---

Your attempt fails because NaN values aren't equal to each other or themselves:

>>> numpy.nan == numpy.nan
False
>>> numpy.nan == float('nan')
False

NaN was designed this way for algorithmic convenience, to make x != x a simple check for NaN values in environments where producing a NaN to compare against is awkward, and because NaNs have a rarely-used payload component that may be different between different NaNs.

---

The other answer recommends an is numpy.nan test, but this is buggy and unreliable. It only works if your NaNs happen to be the specific object numpy.nan, which is rarely the case:

>>> float('nan') is numpy.nan
False
>>> numpy.float64(0)/0 is numpy.nan
__main__:1: RuntimeWarning: invalid value encountered in double_scalars
False
>>> numpy.array([numpy.nan])[0] is numpy.nan
False

Rely on is numpy.nan checks, and they will bite you.

flatten nested list in pandas containing nan

This should work for any nested lists

from collections.abc import Iterable
def flatten(l):
    for el in l:
        if isinstance(el, Iterable) and not isinstance(el, (str, bytes)):
            yield from flatten(el)
        else:
            yield el

So recreating your df

import pandas as pd
df = pd.DataFrame([[[[float('nan')],[float('nan'), 'DE']]],
                   [[[float('nan'), ['IT', 'DE']]]],
                   [[[['FR']]]],
                   [[[['AE'], float('nan'), ['AE',  'MT'], ['MX']]]]],columns=['country'])

df['country'] = df['country'].apply(lambda x:list(set(flatten(x)))).apply(lambda x: [i for i in x if str(i) != 'nan'])

gives the following output

    country
0   [DE]
1   [IT, DE]
2   [FR]
3   [AE, MT, MX]

Retrieving the index of NaN in a list

Take advantage of the fact that NaN != NaN, mathematically, so you can pass a generator to next to get the first index of NaN, or -1 if it doesn't exist.

nan_idx = next((i for i, v in enumerate(li) if v != v), -1)

print(nan_idx) 
0

How can I check for NaN values?

Use math.isnan:

>>> import math
>>> x = float('nan')
>>> math.isnan(x)
True

---

Related Topics