Checking If Particular Value (In Cell) Is Nan in Pandas Dataframe Not Working Using Ix or Iloc

Checking if particular value (in cell) is NaN in pandas DataFrame not working using ix or iloc

Try this:

In [107]: pd.isnull(df.iloc[1,0])
Out[107]: True

---

UPDATE: in a newer Pandas versions use pd.isna():

In [7]: pd.isna(df.iloc[1,0])
Out[7]: True

Check if single cell value is NaN in Pandas

Try this:

import pandas as pd
import numpy as np
from pandas import *

>>> L = [4, nan ,6]
>>> df = Series(L)

>>> df
0     4
1   NaN
2     6

>>> if(pd.isnull(df[1])):
        print "Found"

Found

>>> if(np.isnan(df[1])):
        print "Found"

Found

Using isnull() in a pandas data frame to check a particular value is null or not

Using pandas, you should avoid loop. Use mask filtering and slicing to fill your flag column. In order to detect null values, use .isnull() directly on pandas dataframe or series (when you select a column), not on a value as you did. Then use .fillna() if you want to replace null values with something else.

Based on your code (but not sure it will works, it could be helpfull you share some input data and expected output), the solution may look as follow.

First create empty column as you did:

data['Flags'] = None

Then fill this columns based on condition on "Temperature phase" column (using fillna(0) to replace all null values by 0 allow you to only test if values are <= 0, this replacement is not applied on the final dataframe):

data.loc[data['Temperature phase'].fillna(0) <= 0, "Flags"] = 1
data.loc[data['Temperature phase'] > 200, "Flags"] = 2

And now replace Temperature phase values.

For the values equal to 0 or null, you seems to have choosen to replace them with the previous value in dataframe. You maybe could achieve this part using this.

data.loc[data['Temperature phase'].isnull(), 'Temperature phase'] = data['Temperature phase'].shift().loc[data.loc[data['Temperature phase'].isnull()].index]

First, this command use .shift() to shift all values in column Temperature phase by one, then filtering rows where Temperature phase is null and replace values by corresponding index in shifted Temperature phase values.

Finaly, replace other Temperature phase values:

data.loc[data['Temperature phase'] < 0, "Temperature phase"] = 0
data.loc[data['Temperature phase'] > 200, "Temperature phase"] = 130

You don't need flag index so on as the Flag is directly fill in the final dataframe.

How to check if a particular cell in pandas DataFrame isnull?

Use pd.isnull, for select use loc or iloc:

print (df)
   0  A   B  C
0  1  2 NaN  8

print (df.loc[0, 'B'])
nan

a = pd.isnull(df.loc[0, 'B'])
print (a)
True

print (df['B'].iloc[0])
nan

a = pd.isnull(df['B'].iloc[0])
print (a)
True

Find empty or NaN entry in Pandas Dataframe

np.where(pd.isnull(df)) returns the row and column indices where the value is NaN:

In [152]: import numpy as np
In [153]: import pandas as pd
In [154]: np.where(pd.isnull(df))
Out[154]: (array([2, 5, 6, 6, 7, 7]), array([7, 7, 6, 7, 6, 7]))

In [155]: df.iloc[2,7]
Out[155]: nan

In [160]: [df.iloc[i,j] for i,j in zip(*np.where(pd.isnull(df)))]
Out[160]: [nan, nan, nan, nan, nan, nan]

Finding values which are empty strings could be done with applymap:

In [182]: np.where(df.applymap(lambda x: x == ''))
Out[182]: (array([5]), array([7]))

Note that using applymap requires calling a Python function once for each cell of the DataFrame. That could be slow for a large DataFrame, so it would be better if you could arrange for all the blank cells to contain NaN instead so you could use pd.isnull.

Cannot calculate mean in datasetHow to iterate a pandas column and update contents

You can't check item == np.NaN, you have to use pd.isna(item):

for item in ht_df["sex"]:
    if pd.isna(item):
        print(f"{item} is NaN\n")
    print(f"{item} {type(item)}")

Output:

...
M <class 'str'>
F <class 'str'>
nan is NaN

nan <class 'float'>
F <class 'str'>
...

Find integer index of rows with NaN in pandas dataframe

For DataFrame df:

import numpy as np
index = df['b'].index[df['b'].apply(np.isnan)]

will give you back the MultiIndex that you can use to index back into df, e.g.:

df['a'].ix[index[0]]
>>> 1.452354

For the integer index:

df_index = df.index.values.tolist()
[df_index.index(i) for i in index]
>>> [3, 6]

Pandas - Edit a single dataframe value from a returned subset based on condition WITHOUT a for loop

You could get the minimal index where the conditions are met and use it to set the new values:

ix = df[(df.iloc[:,3] == condition_a)&(df.iloc[:,4]==condition_b)].index.min()
df.loc[ix, df.columns[[0, 1]]] = [value_1, value_2]

---

Related Topics