Is There a Recursive Version of the Dict.Get() Built-In

Is there a recursive version of the dict.get() built-in?

A very common pattern to do this is to use an empty dict as your default:

d.get('foo', {}).get('bar')

If you have more than a couple of keys, you could use reduce (note that in Python 3 reduce must be imported: from functools import reduce) to apply the operation multiple times

reduce(lambda c, k: c.get(k, {}), ['foo', 'bar'], d)

Of course, you should consider wrapping this into a function (or a method):

def recursive_get(d, *keys):
    return reduce(lambda c, k: c.get(k, {}), keys, d)

recursive n-th child dict.get() - efficiency?

One suggestion I have is to give split() a second argument. You can do something simpler like:

parent, rest = string.split(".", 1)

Other than that, I see no immediate issues with the code.

You can also do this without recursion:

def recursive_dict_get(item, string, default=False):
    for s in string.split('.'):
        if (isinstance(item, dict) and s in item):
            item = item[s]
        else:
            return default
    return item

Recursive DotDict

I don't see where you are copying the values in the constructor. Here DotDict is always empty because of that. When I added the key assignment, it worked:

class DotDict(dict):
    """
    a dictionary that supports dot notation 
    as well as dictionary access notation 
    usage: d = DotDict() or d = DotDict({'val1':'first'})
    set attributes: d.val2 = 'second' or d['val2'] = 'second'
    get attributes: d.val2 or d['val2']
    """
    __getattr__ = dict.__getitem__
    __setattr__ = dict.__setitem__
    __delattr__ = dict.__delitem__

    def __init__(self, dct):
        for key, value in dct.items():
            if hasattr(value, 'keys'):
                value = DotDict(value)
            self[key] = value

dct = {'scalar_value':1, 'nested_dict':{'value':2, 'nested_nested': {'x': 21}}}
dct = DotDict(dct)

print dct.nested_dict.nested_nested.x

It looks a bit dangerous and error prone, not to mention source of countless surprises to other developers, but seems to be working.

Recursive function to return a dictionary in Python

Your code is almost correct. It has to be adjusted a little, though.
More specifically,

1. element is a file or directory name (not path). If it is a subdirectory or file in a subdirectory the value of if os.path.isfile(element) and elif os.path.isdir(element) will be always False. Hence, replace them with if os.path.isfile(os.path.join(root, element)) and elif os.path.isdir(os.path.join(root, element)) respectively.

2. Similarly, with open(element) should be replaced by with open(os.path.join(root,element)).

3. When reading the file's first line, you have to store the path and that line in a dictionary.

4. That dictionary has to be updated when calling the recursive function in elif os.path.isdir(element).

See below for the complete snippet:

import os

def search(root):

    my_dict = {}   # this is the final dictionary to be populated

    for element in os.listdir(root):
        
        if os.path.isfile(os.path.join(root, element)):
            try: 
                with open(os.path.join(root, element)) as file:
                    my_dict[os.path.join(root, element)] = file.readline() # populate the dictionary
            except UnicodeDecodeError: 
                # This exception handling has been put here to ignore decode errors (some files cannot be read)
                pass

        elif os.path.isdir(os.path.join(root, element)):
            my_dict.update(search(os.path.join(root,element)))  # update the current dictionary with the one resulting from the recursive call

    return my_dict

print(search('.'))

It prints a dictionary like below:

{
 "path/file.csv": "name,surname,grade",
 "path/to/file1.txt": "this is the first line of file 1",
 "path/to/file2.py": "import os"
}

For the sake of readability, os.path.join(root, element) can be stored in a variable, then:

import os

def search(root):

    my_dict = {}   # this is the final dictionary to be populated

    for element in os.listdir(root):
        path = os.path.join(root, element)

        if os.path.isfile(path):
            with open(path) as file:
                my_dict[path] = file.readline()

        elif os.path.isdir(path):
            my_dict.update(search(path))

    return my_dict

print(search('.'))

recursive access to dictionary and modification

You can use the reduce() function to traverse a series of nested dictionaries:

def get_nested(d, path):
    return reduce(dict.__getitem__, path, d)

Demo:

>>> def get_nested(d, path):
...     return reduce(dict.__getitem__, path, d)
... 
>>> my_dict = {'key1': {'key2': {'foo': 'bar', 'key3': {'key4': {'key5': 'blah'}}}}}
>>> get_nested(my_dict, ('key1', 'key2', 'key3', 'key4', 'key5'))
'blah'

This version throws an exception when a key doesn't exist:

>>> get_nested(my_dict, ('key1', 'nonesuch'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in get_nested
KeyError: 'nonesuch'

but you could replace dict.__getitem__ with lambda d, k: d.setdefault(k, {}) to have it create empty dictionaries instead:

def get_nested_default(d, path):
    return reduce(lambda d, k: d.setdefault(k, {}), path, d)

Demo:

>>> def get_nested_default(d, path):
...     return reduce(lambda d, k: d.setdefault(k, {}), path, d)
... 
>>> get_nested_default(my_dict, ('key1', 'nonesuch'))
{}
>>> my_dict
{'key1': {'key2': {'key3': {'key4': {'key5': 'blah'}}, 'foo': 'bar'}, 'nonesuch': {}}}

To set a value at a given path, traverse over all keys but the last one, then use the final key in a regular dictionary assignment:

def set_nested(d, path, value):
    get_nested_default(d, path[:-1])[path[-1]] = value

This uses the get_nested_default() function to add empty dictionaries as needed:

>>> def set_nested(d, path, value):
...     get_nested_default(d, path[:-1])[path[-1]] = value
... 
>>> my_dict = {'key1': {'key2': {'foo': 'bar'}}}
>>> set_nested(my_dict, ('key1', 'key2', 'key3', 'key4', 'key5'), 'blah')
>>> my_dict
{'key1': {'key2': {'key3': {'key4': {'key5': 'blah'}}, 'foo': 'bar'}}}

Python Recursive function to fetch data from dict

The problem is in these lines -

for key in items:
    items += get_data(key)

Here you are modifying items as you're iterating over it. So in the last iteration, your items ends up getting with the same key multiple times; you can add a logging statement to see which key is being used to call get_data.

You want to obtain all the new items separately, and then update items after the iteration is done -

new_items = []
for key in items:
    new_items += get_data(key)
items += new_items

Accessing value inside nested dictionaries

You can use the get() on each dict. Make sure that you have added the None check for each access.

How to return nested dictionary from function

When you give ['c']['d'], you slice the list ['c'] using the letter d, which isin't possible. So what you can do is, correct the slicing:

foo('c')['d']

Or you could alter your function to slice it:

def foo(*args):
    d = {'a': 1, 'b': 2, 'c': {'d': 3, 'e': 4}, }
    d_old = dict(d) # if case you have to store the dict for other operations in the fucntion
    for i in args:
        d = d[i]
    return d
>>> foo('c','d')
3

In Python, why isn't there a built in function to find a key in a nested dictionary?

Suppose, as several commenters pointed out, the key pointing to your "value of interest" appears multiple times in the nested hierarchy:

sample_dict = {'key1': {'key1': 'value_of_interest'}}
val = sample_dict.recursive_get('key1', None)

What is the value of val? The inner dict? 'value_of_interest'? What should it be? Which answer violates the principle of least astonishment?

What about a dict like this?

sample_dict = {
        'key1': [
                {'key1': 'value_of_interest'},
                {'key2': 'less_interesting_value'},
                ],
        }

Should the recursion check at every level for a possible list or array? It's not uncommon to see dicts like this, especially when formed from JSON.

The point is there's a lot of different dicts out there, which can be nested in a lot of different ways. Python is pretty well-equipped to handle them all fairly elegantly. No definition of recursive_get that I can think of, though, is.

Finding a key recursively in a dictionary

when you recurse, you need to return the result of _finditem

def _finditem(obj, key):
    if key in obj: return obj[key]
    for k, v in obj.items():
        if isinstance(v,dict):
            return _finditem(v, key)  #added return statement

To fix the actual algorithm, you need to realize that _finditem returns None if it didn't find anything, so you need to check that explicitly to prevent an early return:

def _finditem(obj, key):
    if key in obj: return obj[key]
    for k, v in obj.items():
        if isinstance(v,dict):
            item = _finditem(v, key)
            if item is not None:
                return item

Of course, that will fail if you have None values in any of your dictionaries. In that case, you could set up a sentinel object() for this function and return that in the case that you don't find anything -- Then you can check against the sentinel to know if you found something or not.

---

Related Topics