Extract Values from Column of Dictionaries Using Pandas

Extract dictionary value from column in data frame

You can use a list comprehension to extract feature 3 from each row in your dataframe, returning a list.

feature3 = [d.get('Feature3') for d in df.dic]

If 'Feature3' is not in dic, it returns None by default.

You don't even need pandas, as you can again use a list comprehension to extract the feature from your original dictionary a.

feature3 = [d.get('Feature3') for d in a]

How to extract values from column of dictionaries in pandas?

You can use applymap:

import pandas as pd
df = pd.DataFrame([
    {'node': 'A', 'read': {'value': 2343}, 'write': {'value': 23}},
    {'node': 'B', 'read': {'value': 334}, 'write': {'value': 233444}},
])

cols = ['read', 'write']
df[cols] = df[cols].applymap(lambda x: x['value'])

print(df)

[Out]:

  node  read   write
0    A  2343      23
1    B   334  233444

Extract value from list of dictionaries in dataframe using pandas

df['ID'] = df['Resources'].apply(lambda x: ','.join([i['resourceName'] for i in eval(x)]))

         Date  ...                                   ID
0  28-02-2022  ...                            i-05fbb7a
1  28-02-2022  ...  i-08bd2475,i-0fd69dc1,i-0174dd38aea

Pandas - Extract value from dictionary in a df from a column containing list of dictionaries

You can explode and apply pd.Series constructor:

df['values'] = df['values'].explode().apply(pd.Series)

or apply a function that gets the value of value key in each cell in values column:

df["values"] = df['values'].apply(lambda x: x[0]["value"])

Output:

           name  values                                     id
0   impressions    7352   179286/insights/impressions/lifetime
1         reach    7352         179286/insights/reach/lifetime
2  taps_forward    6280  179286/insights/taps_forward/lifetime
3     taps_back     134     179286/insights/taps_back/lifetime
4         exits     610         179286/insights/exits/lifetime
5       replies       0       179286/insights/replies/lifetime

Extract values from dictionaries in dataframe columns

IIUC, you can use an applymap and extract the value associated with the value key from every dictionary.

import operator

df = pd.DataFrame(res['results']['bindings'], columns=res['head']['vars']) 
df = df.applymap(operator.itemgetter('value'))

This operates under the assumption the each cell value is a dictionary.

---

It could be possible some of your dictionaries do not contain value as a key. In that case, a slight modification is required, using dict.get:

df = df.applymap(lambda x: x.get('value', np.nan) \
                        if isinstance(x, dict) else np.nan)

This will also handle the potential problems that arise when x is not a dict.

Extract values from column of dictionaries using pandas

If there are no NaNs, use json_normalize.

pd.io.json.json_normalize(df.name.tolist())['Name']

0    Kevin
1    Scott
Name: Name, dtype: object

---

If there are NaNs, you will need to drop them first. However, it is easy to retain the indices.

df

   emp_id                                               name
0   101.0  {'Name': 'Kevin', 'attributes': {'type': 'Cont...
1   102.0                                                NaN
2   103.0  {'Name': 'Scott', 'attributes': {'type': 'Cont...

idx = df.index[df.name.notna()]
names = pd.io.json.json_normalize(df.name.dropna().tolist())['Name']  
names.index = idx

names

0    Kevin
2    Scott
Name: Name, dtype: object

How extract values of dictionary column in pandas dataframe

Use .get for get value from dict with default value None if non match, last remove Nones by Series.dropna:

s = df['info_dict'].apply(lambda x: x.get('IDS')).dropna()
print (s)
chr1  2337194    1026660,1026661
      2338126      652130,652129
Name: col, dtype: object

How to extract list of dictionaries from Pandas column

You can use pandas. First cast your data to pd.DataFrame, then use apply(pd.Series) to expand lists inside 'values' column to separate columns and set_axis method to change column names:

import pandas as pd
data = {'rows': [{'values': ['Tesla Inc (TSLA)', '$1056.78', '$1199.78', '13.53%'], 'children': []}, {'values': ['Taiwan Semiconductor Manufacturing Company Limited (TSM)', '$120.31', '$128.80', '7.06%'], 'children': []}]}
out = pd.DataFrame(data['rows'])['values'].apply(pd.Series).set_axis(['name','price','price_n','pct'], axis=1)

Output:

                                                    name     price   price_n     pct
    0                                   Tesla Inc (TSLA)  $1056.78  $1199.78   13.53%  
    1  Taiwan Semiconductor Manufacturing Company Lim...   $120.31   $128.80    7.06%  
      

---

Related Topics