Elegant Way to Check If a Nested Key Exists in a Dict

Elegant way to check if a nested key exists in a dict?

To be brief, with Python you must trust it is easier to ask for forgiveness than permission

try:
    x = s['mainsnak']['datavalue']['value']['numeric-id']
except KeyError:
    pass

---

The answer

Here is how I deal with nested dict keys:

def keys_exists(element, *keys):
    '''
    Check if *keys (nested) exists in `element` (dict).
    '''
    if not isinstance(element, dict):
        raise AttributeError('keys_exists() expects dict as first argument.')
    if len(keys) == 0:
        raise AttributeError('keys_exists() expects at least two arguments, one given.')

    _element = element
    for key in keys:
        try:
            _element = _element[key]
        except KeyError:
            return False
    return True

Example:

data = {
    "spam": {
        "egg": {
            "bacon": "Well..",
            "sausages": "Spam egg sausages and spam",
            "spam": "does not have much spam in it"
        }
    }
}

print 'spam (exists): {}'.format(keys_exists(data, "spam"))
print 'spam > bacon (do not exists): {}'.format(keys_exists(data, "spam", "bacon"))
print 'spam > egg (exists): {}'.format(keys_exists(data, "spam", "egg"))
print 'spam > egg > bacon (exists): {}'.format(keys_exists(data, "spam", "egg", "bacon"))

Output:

spam (exists): True
spam > bacon (do not exists): False
spam > egg (exists): True
spam > egg > bacon (exists): True

It loop in given element testing each key in given order.

I prefere this to all variable.get('key', {}) methods I found because it follows EAFP.

Function except to be called like: keys_exists(dict_element_to_test, 'key_level_0', 'key_level_1', 'key_level_n', ..). At least two arguments are required, the element and one key, but you can add how many keys you want.

If you need to use kind of map, you can do something like:

expected_keys = ['spam', 'egg', 'bacon']
keys_exists(data, *expected_keys)

Check if a key exists in a nested dictionary python based on a list

You can use functools.reduce() for this, you just need to anticipate the KeyError. For example:

from functools import reduce

d = {"key1" : {"key2": {"key3": 4 } } }

def findindict(d, l):
    try:
        return reduce(lambda current_dict, key: current_dict[key], l, d)
    except (KeyError, TypeError):
        return None

findindict(d, ["key1","key2", "key3"])
# 4
findindict(d, ["key1","key2", "abc"])
# None
findindict(d, ["key1","key2", "key3", "key6"])
#None

Python : check if the nested dictionary exist

Here's the explicit short code with try/except:

try:
    dict1['layer1']['layer2'][0]['layer3']
except KeyError:
    present = False
else:
    present = True

if present: 
    ...

To get the element:

try:
    obj = dict1['layer1']['layer2'][0]['layer3']
except KeyError:
    obj = None  # or whatever

How to check if a key exists in an inner dictionary inside a dictionary in python?

IF you can't use an exception for some reason (eg. lambda func, list comprehension, generator expression etc)

value = a.get(b, {}).get(c, {}).get("z", None)

But normally you should prefer to use the exception handler

Check key existence in nested dictionaries

If you are working with JSON, you can write a simple class to use with the dict as it is imported.

Given the following bit of JSON:

>>> js='{"a": {"b": {"c": {"d": "e"}}}}'

It would usually be decoded into a Python dict if it is comprised of object pairs:

>>> import json
>>> json.loads(js)
{u'a': {u'b': {u'c': {u'd': u'e'}}}}

As a normal Python dict, it is subject to KeyError with missing keys. You can use the __missing__ hook to override KeyErrors and achieve your original structure:

class Mdict(dict):
    def __missing__(self, key):
        return False

Now test that:

>>> md=Mdict({'a':Mdict({'b':Mdict({'c':Mdict({'d':'e'})})})})
>>> if md['a']['b']['d']:
...    print md['a']['b']['d']
... elif md['a']['b']['c']:
...    print 'elif', md['a']['b']['c']  
... 
elif {'d': 'e'}

Each level of the dict needs to be an Mdict vs a normal Python dict. If you are working with JSON, however, this is really easy to achieve. Just apply object_pairs_hook as you decode the JSON:

>>> js
'{"a": {"b": {"c": {"d": "e"}}}}'
>>> md=json.loads(js, object_pairs_hook=Mdict)

And that applies the class Mdict instead the default Python dict as the JSON is decoded.

>>> md
{u'a': {u'b': {u'c': {u'd': u'e'}}}}
>>> md['a']
{u'b': {u'c': {u'd': u'e'}}}
>>> md['a']['c']
False

The rest of the example here remains the same.

How to know if a value exists in a list within a nested dictionary

You can use any with a list comprehension to check if 'Anna' exists as either a key in the dictionary or appears as an element in a subdictionary:

data = {"Pam": {"Job":"Pilot", 
          "YOB": "1986", 
          "Children": ["Greg", "Anna","Sima"]}, 
    "Joe": {"Job":"Engineer", 
          "YOB": "1987"}}
if any(a == 'Anna' or 'Anna' in b.get('children', []) for a, b in data.items()):
   pass

Find all occurrences of a key in nested dictionaries and lists

I found this Q/A very interesting, since it provides several different solutions for the same problem. I took all these functions and tested them with a complex dictionary object. I had to take two functions out of the test, because they had to many fail results and they did not support returning lists or dicts as values, which i find essential, since a function should be prepared for almost any data to come.

So i pumped the other functions in 100.000 iterations through the timeit module and output came to following result:

0.11 usec/pass on gen_dict_extract(k,o)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
6.03 usec/pass on find_all_items(k,o)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
0.15 usec/pass on findkeys(k,o)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1.79 usec/pass on get_recursively(k,o)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
0.14 usec/pass on find(k,o)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
0.36 usec/pass on dict_extract(k,o)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All functions had the same needle to search for ('logging') and the same dictionary object, which is constructed like this:

o = { 'temparature': '50', 
      'logging': {
        'handlers': {
          'console': {
            'formatter': 'simple', 
            'class': 'logging.StreamHandler', 
            'stream': 'ext://sys.stdout', 
            'level': 'DEBUG'
          }
        },
        'loggers': {
          'simpleExample': {
            'handlers': ['console'], 
            'propagate': 'no', 
            'level': 'INFO'
          },
         'root': {
           'handlers': ['console'], 
           'level': 'DEBUG'
         }
       }, 
       'version': '1', 
       'formatters': {
         'simple': {
           'datefmt': "'%Y-%m-%d %H:%M:%S'", 
           'format': '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
         }
       }
     }, 
     'treatment': {'second': 5, 'last': 4, 'first': 4},   
     'treatment_plan': [[4, 5, 4], [4, 5, 4], [5, 5, 5]]
}

All functions delivered the same result, but the time differences are dramatic! The function gen_dict_extract(k,o) is my function adapted from the functions here, actually it is pretty much like the find function from Alfe, with the main difference, that i am checking if the given object has iteritems function, in case strings are passed during recursion:

# python 2
def gen_dict_extract(key, var):
    if hasattr(var,'iteritems'): # hasattr(var,'items') for python 3
        for k, v in var.iteritems(): # var.items() for python 3
            if k == key:
                yield v
            if isinstance(v, dict):
                for result in gen_dict_extract(key, v):
                    yield result
            elif isinstance(v, list):
                for d in v:
                    for result in gen_dict_extract(key, d):
                        yield result

So this variant is the fastest and safest of the functions here. And find_all_items is incredibly slow and far off the second slowest get_recursivley while the rest, except dict_extract, is close to each other. The functions fun and keyHole only work if you are looking for strings.

Interesting learning aspect here :)

Safe method to get value of nested dictionary

You could use get twice:

example_dict.get('key1', {}).get('key2')

This will return None if either key1 or key2 does not exist.

Note that this could still raise an AttributeError if example_dict['key1'] exists but is not a dict (or a dict-like object with a get method). The try..except code you posted would raise a TypeError instead if example_dict['key1'] is unsubscriptable.

Another difference is that the try...except short-circuits immediately after the first missing key. The chain of get calls does not.

---

If you wish to preserve the syntax, example_dict['key1']['key2'] but do not want it to ever raise KeyErrors, then you could use the Hasher recipe:

class Hasher(dict):
    # https://stackoverflow.com/a/3405143/190597
    def __missing__(self, key):
        value = self[key] = type(self)()
        return value

example_dict = Hasher()
print(example_dict['key1'])
# {}
print(example_dict['key1']['key2'])
# {}
print(type(example_dict['key1']['key2']))
# <class '__main__.Hasher'>

Note that this returns an empty Hasher when a key is missing.

Since Hasher is a subclass of dict you can use a Hasher in much the same way you could use a dict. All the same methods and syntax is available, Hashers just treat missing keys differently.

You can convert a regular dict into a Hasher like this:

hasher = Hasher(example_dict)

and convert a Hasher to a regular dict just as easily:

regular_dict = dict(hasher)

---

Another alternative is to hide the ugliness in a helper function:

def safeget(dct, *keys):
    for key in keys:
        try:
            dct = dct[key]
        except KeyError:
            return None
    return dct

So the rest of your code can stay relatively readable:

safeget(example_dict, 'key1', 'key2')

Check or set entries in nested dict from list of keys of variable length

my_dict = {}
path = "a/e/f/g"
wd = my_dict  # Set working dictionary to my_dict
path_split = path.split('/')
for char in path_split[:-1]:  # Loop through all but the final char in the path
    if char not in wd or not isinstance(wd[char], dict):
        wd[char] = {}
    wd = wd[char]  # Set new working dictionary
if path_split[-1] not in wd:  # Check if final char is in the wd, if not set it to 1
    wd[path_split[-1]] = 1

You need to check that a key exists in the working dictionary and that the corresponding value is a dictionary, then set the working dictionary after each check.

---

Related Topics