﻿ How to Check List Containing Nan - ITCodar

# How to Check List Containing Nan

## How to Check list containing NaN

I think this makes sense because of your pulling `numpy` into scope indirectly via the star import.

``>>> import numpy as np>>> [0.0,0.0]/0Traceback (most recent call last):  File "<ipython-input-3-aae9e30b3430>", line 1, in <module>    [0.0,0.0]/0TypeError: unsupported operand type(s) for /: 'list' and 'int'>>> [0.0,0.0]/np.float64(0)array([ nan,  nan])``

When you did

``from matplotlib.pylab import *``

it pulled in `numpy.sum`:

``>>> from matplotlib.pylab import *>>> sum is np.sumTrue>>> [0.0,0.0]/sum([0.0, 0.0])array([ nan,  nan])``

You can test that this `nan` object (`nan` isn't unique in general) is in a list via identity, but if you try it in an `array` it seems to test via equality, and `nan != nan`:

``>>> nan == nanFalse>>> nan == nan, nan is nan(False, True)>>> nan in [nan]True>>> nan in np.array([nan])False``

You could use `np.isnan`:

``>>> np.isnan([nan, nan])array([ True,  True], dtype=bool)>>> np.isnan([nan, nan]).any()True``

## Python check if all elements in a list are Nan

You need `all`:

``np.isnan(op_list).all()# True``

For a solution using lists you can do:

``all(i != i for i in op_list)# True``

## how to check for np.nan in list comprehension?

Why are you comparing with `is`? Different values may not necessarily be the same instance. This is true for ints as well as floats. (CPython's quirk for small integers is the only exception that I know of, and is strictly an implementation detail.)

``>>> import math>>> float('nan') is math.nanFalse``

You can use `np.isnan` to check for nan, or the built-in `math.isnan`, or check that the value is equal to itself.

``>>> np.isnan([12.6,np.nan,0.5,4.6])array([False,  True, False, False], dtype=bool)>>> [math.isnan(x) for x in [12.6,np.nan,0.5,4.6]][False, True, False, False]>>> [x for x in [12.6,np.nan,0.5,4.6] if x == x][12.6, 0.5, 4.6]``

## How to get list of strings from list-like string that includes nan?

``eval(z,{'nan':'nan'}) # if you can tolerate then: [i for i in eval(z,{'nan':'nan'}) if i != 'nan']``

It may have security considerations.

## How can I get all the index of all the NaNs of a list?

If you're using NumPy, you should really start using arrays, and get out of the habit of manually looping at Python level. Manual loops are typically about 100 times slower than letting NumPy handle things, and lists of floats take about 4 times the memory of an array.

In this case, NumPy can give you an array of NaN indices quite simply:

``ind = numpy.where(numpy.isnan(a))[0]``

`numpy.isnan` gives an array of booleans telling which elements of `a` are NaN. `numpy.where` gives an array of indices of `True` elements, but wrapped in a 1-element tuple for consistency with the behavior on multidimensional arrays, so `[0]` extracts the array from the tuple.

This works when `a` is a list, but you should really use arrays.

Your attempt fails because NaN values aren't equal to each other or themselves:

``>>> numpy.nan == numpy.nanFalse>>> numpy.nan == float('nan')False``

NaN was designed this way for algorithmic convenience, to make `x != x` a simple check for NaN values in environments where producing a NaN to compare against is awkward, and because NaNs have a rarely-used payload component that may be different between different NaNs.

The other answer recommends an `is numpy.nan` test, but this is buggy and unreliable. It only works if your NaNs happen to be the specific object `numpy.nan`, which is rarely the case:

``>>> float('nan') is numpy.nanFalse>>> numpy.float64(0)/0 is numpy.nan__main__:1: RuntimeWarning: invalid value encountered in double_scalarsFalse>>> numpy.array([numpy.nan])[0] is numpy.nanFalse``

Rely on `is numpy.nan` checks, and they will bite you.

## flatten nested list in pandas containing nan

This should work for any nested lists

``from collections.abc import Iterabledef flatten(l):    for el in l:        if isinstance(el, Iterable) and not isinstance(el, (str, bytes)):            yield from flatten(el)        else:            yield el``

``import pandas as pddf = pd.DataFrame([[[[float('nan')],[float('nan'), 'DE']]],                   [[[float('nan'), ['IT', 'DE']]]],                   [[[['FR']]]],                   [[[['AE'], float('nan'), ['AE',  'MT'], ['MX']]]]],columns=['country'])df['country'] = df['country'].apply(lambda x:list(set(flatten(x)))).apply(lambda x: [i for i in x if str(i) != 'nan'])``

gives the following output

``    country0   [DE]1   [IT, DE]2   [FR]3   [AE, MT, MX]``

## Retrieving the index of NaN in a list

Take advantage of the fact that NaN != NaN, mathematically, so you can pass a generator to `next` to get the first index of NaN, or -1 if it doesn't exist.

``nan_idx = next((i for i, v in enumerate(li) if v != v), -1)``
``print(nan_idx) 0``

## How can I check for NaN values?

Use `math.isnan`:

``>>> import math>>> x = float('nan')>>> math.isnan(x)True``

Submit