How to delete numpy nan from a list of strings in Python?
If you have a numpy array you can simply check the item is not the string nan
, but if you have a list you can check the identity with is
and np.nan
since it's a singleton object.
In [25]: x = np.array(['A', 'B', np.nan, 'D'])
In [26]: x
Out[26]:
array(['A', 'B', 'nan', 'D'],
dtype='<U3')
In [27]: x[x != 'nan']
Out[27]:
array(['A', 'B', 'D'],
dtype='<U3')
In [28]: x = ['A', 'B', np.nan, 'D']
In [30]: [i for i in x if i is not np.nan]
Out[30]: ['A', 'B', 'D']
Or as a functional approach in case you have a python list:
In [34]: from operator import is_not
In [35]: from functools import partial
In [37]: f = partial(is_not, np.nan)
In [38]: x = ['A', 'B', np.nan, 'D']
In [39]: list(filter(f, x))
Out[39]: ['A', 'B', 'D']
Remove NaN from lists in python
It is not working because you are trying to compare it to string 'nan'.
If excel cell is empty it is returned as NaN
value in pandas.
You can use numpy library, to compare it with NaN
:
import numpy as np
for l in df_list:
newlist = [x for x in l if x != np.nan]
print(newlist)
EDIT:
If you want to get all values from the dataframe, which are not NaN
, you can just do:
df.stack().tolist()
If you want to print values with the loop (as in your example), you can do:
for l in df.columns:
print(list(df[l][df[l].notna()]))
To create nested list with a loop:
main = []
for l in df.T.columns:
new_list = list(df.T[l][df.T[l].notna()])
main.append(new_list)
print(main)
How to remove nan's from list of lists?
Try iterating over the list and building a new list using numpy's isnan method.
import numpy as np
nan = np.nan
l = [[1,2,3,nan,nan,nan],[4,5,nan],[9,8,7,6,5,4,3, nan, nan, nan, nan, nan]]
[[x for x in y if not np.isnan(x)] for y in l]
Removing a nan from a list
What you can do is simply get a cleaned list where you don't put the values that, once converted to strings, are 'nan'.
The code would be :
incoms = [incom for incom in incoms if str(incom) != 'nan']
Removing nan from list - Python
ztt = []
for z_i in z:
row = []
for z_ij in z_i:
if math.isnan(z_ij):
row.append(z_ij)
# If you want to replace with, for example, 0:
# else:
# row.append(0)
ztt.append(row)
Alternatively, with nested list comprehensions:
ztt = [[z_ij for z_ij in zi if math.isnan(z_ij)] for z_i in z]
Btw, if you are using NumPy, you probably can just do:
import numpy as np
ztt =[value[~np.isnan(value)] for value in z]
Removing nan values from a Python List
The control flow in your function makes no sense - you set a variable x
to be nan
, and then check if it is indeed nan
in your loop and set it to 0. You never touch nor check any of the elements of the array.
To properly convert your nan
values to 0, you could simply use numpy.nan_to_num
as it appears you're working with NumPy arrays.
Demo
In[37]: arr
Out[37]:
array([ nan, 19523.32112031, 19738.42763774, 19654.84783027,
119.63673757, 19712.43294378, nan, 20052.36456133,
19846.4815936 , 20041.86766194, 19921.81269442, nan,
20030.50736357])
In[38]: np.nan_to_num(arr)
Out[38]:
array([ 0. , 19523.32112031, 19738.42763774, 19654.84783027,
119.63673757, 19712.43294378, 0. , 20052.36456133,
19846.4815936 , 20041.86766194, 19921.81269442, 0. ,
20030.50736357])
If you're more interested in having a functioning version of an approach for a regular Python list, you might try something like this, or a list comprehension as fafl has provided.
In[39]: list(map(lambda x: 0.0 if math.isnan(x) else x, oldlist))
Out[39]:
[0.0,
19523.3211203121,
19738.4276377355,
19654.8478302742,
119.63673757136,
19712.432943781,
0.0,
20052.3645613346,
19846.4815936009,
20041.8676619438,
19921.8126944154,
0.0,
20030.5073635719]
how to remove NaN from numpy subarray
First for some reason, the provided array is an array of strings. So before proceeding further we need to convert it to an array of floats:
# assuming your original array is arr
new_arr = arr.astype(float)
Then, we can filter the list elements, in a way to only keep the subarrays which second element is not NaN
filtered_list = np.array(list(filter(lambda x: not np.isnan(x[1]), new_arr)))
Remove all nan from nested list
nan
is not equal to itself (this goes for float('nan')
as well as np.nan
). So, we can use filter()
, removing elements which are not equal to itself.
l1 = [['a', 'b', 'c', 'd', 'e', 'f'],
[1.0, 2.0, 3.0, 4.0, 5.0, nan],
['red', 'orange', 'blue', nan, nan, nan]]
result = [list(filter(lambda x: x == x, inner_list)) for inner_list in l1]
print(result)
Removing nan values from an array
If you're using numpy for your arrays, you can also use
x = x[numpy.logical_not(numpy.isnan(x))]
Equivalently
x = x[~numpy.isnan(x)]
[Thanks to chbrown for the added shorthand]
Explanation
The inner function, numpy.isnan
returns a boolean/logical array which has the value True
everywhere that x
is not-a-number. As we want the opposite, we use the logical-not operator, ~
to get an array with True
s everywhere that x
is a valid number.
Lastly we use this logical array to index into the original array x
, to retrieve just the non-NaN values.
Related Topics
How to Divide Each Column of Pandas Dataframe by a Series
Saving the Output of a Python Program
Django: Check Whether an Object Already Exists Before Adding
How to Transfer Data from One Worksheet into Another Using Python in the Same Workbook
How to Check If a String Column in Pyspark Dataframe Is All Numeric
How to Get the Response Json Data from Network Call in Xhr Using Python Selenium Web Driver Chorme
Removing Rows and Columns in Python CSV Module
Add Numpy Array as Column to Pandas Data Frame
How to Execute Two Commands in Terminal Using Python'S Subprocess Module
Key Error When Selecting Columns in Pandas Dataframe After Read_Csv
Python: [Errno 10054] an Existing Connection Was Forcibly Closed by the Remote Host
How to Concatenate Multiple Column Values into a Single Column in Pandas Dataframe
Using SQL Server Stored Procedures from Python (Pyodbc)
Vscode Import Error for Python Module
How to Close a Tkinter Window by Pressing a Button
Save Variables in Every Iteration of for Loop and Load Them Later