How to Remove Nan from List Python/Numpy

How to delete numpy nan from a list of strings in Python?

If you have a numpy array you can simply check the item is not the string nan, but if you have a list you can check the identity with is and np.nan since it's a singleton object.

In [25]: x = np.array(['A', 'B', np.nan, 'D'])

In [26]: x
Out[26]:
array(['A', 'B', 'nan', 'D'],
dtype='<U3')

In [27]: x[x != 'nan']
Out[27]:
array(['A', 'B', 'D'],
dtype='<U3')


In [28]: x = ['A', 'B', np.nan, 'D']

In [30]: [i for i in x if i is not np.nan]
Out[30]: ['A', 'B', 'D']

Or as a functional approach in case you have a python list:

In [34]: from operator import is_not

In [35]: from functools import partial

In [37]: f = partial(is_not, np.nan)

In [38]: x = ['A', 'B', np.nan, 'D']

In [39]: list(filter(f, x))
Out[39]: ['A', 'B', 'D']

Remove NaN from lists in python

It is not working because you are trying to compare it to string 'nan'.

If excel cell is empty it is returned as NaN value in pandas.

You can use numpy library, to compare it with NaN:

import numpy as np

for l in df_list:
newlist = [x for x in l if x != np.nan]
print(newlist)

EDIT:

If you want to get all values from the dataframe, which are not NaN, you can just do:

df.stack().tolist()

If you want to print values with the loop (as in your example), you can do:

for l in df.columns:
print(list(df[l][df[l].notna()]))

To create nested list with a loop:

main = []

for l in df.T.columns:
new_list = list(df.T[l][df.T[l].notna()])
main.append(new_list)

print(main)

How to remove nan's from list of lists?

Try iterating over the list and building a new list using numpy's isnan method.

import numpy as np
nan = np.nan
l = [[1,2,3,nan,nan,nan],[4,5,nan],[9,8,7,6,5,4,3, nan, nan, nan, nan, nan]]
[[x for x in y if not np.isnan(x)] for y in l]

Removing a nan from a list

What you can do is simply get a cleaned list where you don't put the values that, once converted to strings, are 'nan'.

The code would be :

incoms = [incom for incom in incoms if str(incom) != 'nan']

Removing nan from list - Python

ztt = []
for z_i in z:
row = []
for z_ij in z_i:
if math.isnan(z_ij):
row.append(z_ij)
# If you want to replace with, for example, 0:
# else:
# row.append(0)
ztt.append(row)

Alternatively, with nested list comprehensions:

ztt = [[z_ij for z_ij in zi if math.isnan(z_ij)] for z_i in z]

Btw, if you are using NumPy, you probably can just do:

import numpy as np

ztt =[value[~np.isnan(value)] for value in z]

Removing nan values from a Python List

The control flow in your function makes no sense - you set a variable x to be nan, and then check if it is indeed nan in your loop and set it to 0. You never touch nor check any of the elements of the array.

To properly convert your nan values to 0, you could simply use numpy.nan_to_num as it appears you're working with NumPy arrays.

Demo

In[37]: arr
Out[37]:
array([ nan, 19523.32112031, 19738.42763774, 19654.84783027,
119.63673757, 19712.43294378, nan, 20052.36456133,
19846.4815936 , 20041.86766194, 19921.81269442, nan,
20030.50736357])

In[38]: np.nan_to_num(arr)
Out[38]:
array([ 0. , 19523.32112031, 19738.42763774, 19654.84783027,
119.63673757, 19712.43294378, 0. , 20052.36456133,
19846.4815936 , 20041.86766194, 19921.81269442, 0. ,
20030.50736357])

If you're more interested in having a functioning version of an approach for a regular Python list, you might try something like this, or a list comprehension as fafl has provided.

In[39]: list(map(lambda x: 0.0 if math.isnan(x) else x, oldlist))
Out[39]:
[0.0,
19523.3211203121,
19738.4276377355,
19654.8478302742,
119.63673757136,
19712.432943781,
0.0,
20052.3645613346,
19846.4815936009,
20041.8676619438,
19921.8126944154,
0.0,
20030.5073635719]

how to remove NaN from numpy subarray

First for some reason, the provided array is an array of strings. So before proceeding further we need to convert it to an array of floats:

# assuming your original array is arr
new_arr = arr.astype(float)

Then, we can filter the list elements, in a way to only keep the subarrays which second element is not NaN

filtered_list = np.array(list(filter(lambda x: not np.isnan(x[1]), new_arr)))

Remove all nan from nested list

nan is not equal to itself (this goes for float('nan') as well as np.nan). So, we can use filter(), removing elements which are not equal to itself.

l1 = [['a', 'b', 'c', 'd', 'e', 'f'], 
[1.0, 2.0, 3.0, 4.0, 5.0, nan],
['red', 'orange', 'blue', nan, nan, nan]]

result = [list(filter(lambda x: x == x, inner_list)) for inner_list in l1]

print(result)

Removing nan values from an array

If you're using numpy for your arrays, you can also use

x = x[numpy.logical_not(numpy.isnan(x))]

Equivalently

x = x[~numpy.isnan(x)]

[Thanks to chbrown for the added shorthand]

Explanation

The inner function, numpy.isnan returns a boolean/logical array which has the value True everywhere that x is not-a-number. As we want the opposite, we use the logical-not operator, ~ to get an array with Trues everywhere that x is a valid number.

Lastly we use this logical array to index into the original array x, to retrieve just the non-NaN values.



Related Topics



Leave a reply



Submit