How do you create nested dict in Python?
A nested dict is a dictionary within a dictionary. A very simple thing.
>>> d = {}
>>> d['dict1'] = {}
>>> d['dict1']['innerkey'] = 'value'
>>> d['dict1']['innerkey2'] = 'value2'
>>> d
{'dict1': {'innerkey': 'value', 'innerkey2': 'value2'}}
You can also use a defaultdict
from the collections
package to facilitate creating nested dictionaries.
>>> import collections
>>> d = collections.defaultdict(dict)
>>> d['dict1']['innerkey'] = 'value'
>>> d # currently a defaultdict type
defaultdict(<type 'dict'>, {'dict1': {'innerkey': 'value'}})
>>> dict(d) # but is exactly like a normal dictionary.
{'dict1': {'innerkey': 'value'}}
You can populate that however you want.
I would recommend in your code something like the following:
d = {} # can use defaultdict(dict) instead
for row in file_map:
# derive row key from something
# when using defaultdict, we can skip the next step creating a dictionary on row_key
d[row_key] = {}
for idx, col in enumerate(row):
d[row_key][idx] = col
According to your comment:
may be above code is confusing the question. My problem in nutshell: I
have 2 files a.csv b.csv, a.csv has 4 columns i j k l, b.csv also has
these columns. i is kind of key columns for these csvs'. j k l column
is empty in a.csv but populated in b.csv. I want to map values of j k
l columns using 'i` as key column from b.csv to a.csv file
My suggestion would be something like this (without using defaultdict):
a_file = "path/to/a.csv"
b_file = "path/to/b.csv"
# read from file a.csv
with open(a_file) as f:
# skip headers
f.next()
# get first colum as keys
keys = (line.split(',')[0] for line in f)
# create empty dictionary:
d = {}
# read from file b.csv
with open(b_file) as f:
# gather headers except first key header
headers = f.next().split(',')[1:]
# iterate lines
for line in f:
# gather the colums
cols = line.strip().split(',')
# check to make sure this key should be mapped.
if cols[0] not in keys:
continue
# add key to dict
d[cols[0]] = dict(
# inner keys are the header names, values are columns
(headers[idx], v) for idx, v in enumerate(cols[1:]))
Please note though, that for parsing csv files there is a csv module.
What is the best way to implement nested dictionaries?
What is the best way to implement nested dictionaries in Python?
This is a bad idea, don't do it. Instead, use a regular dictionary and use dict.setdefault
where apropos, so when keys are missing under normal usage you get the expected KeyError
. If you insist on getting this behavior, here's how to shoot yourself in the foot:
Implement __missing__
on a dict
subclass to set and return a new instance.
This approach has been available (and documented) since Python 2.5, and (particularly valuable to me) it pretty prints just like a normal dict, instead of the ugly printing of an autovivified defaultdict:
class Vividict(dict):
def __missing__(self, key):
value = self[key] = type(self)() # retain local pointer to value
return value # faster to return than dict lookup
(Note self[key]
is on the left-hand side of assignment, so there's no recursion here.)
and say you have some data:
data = {('new jersey', 'mercer county', 'plumbers'): 3,
('new jersey', 'mercer county', 'programmers'): 81,
('new jersey', 'middlesex county', 'programmers'): 81,
('new jersey', 'middlesex county', 'salesmen'): 62,
('new york', 'queens county', 'plumbers'): 9,
('new york', 'queens county', 'salesmen'): 36}
Here's our usage code:
vividict = Vividict()
for (state, county, occupation), number in data.items():
vividict[state][county][occupation] = number
And now:
>>> import pprint
>>> pprint.pprint(vividict, width=40)
{'new jersey': {'mercer county': {'plumbers': 3,
'programmers': 81},
'middlesex county': {'programmers': 81,
'salesmen': 62}},
'new york': {'queens county': {'plumbers': 9,
'salesmen': 36}}}
Criticism
A criticism of this type of container is that if the user misspells a key, our code could fail silently:
>>> vividict['new york']['queens counyt']
{}
And additionally now we'd have a misspelled county in our data:
>>> pprint.pprint(vividict, width=40)
{'new jersey': {'mercer county': {'plumbers': 3,
'programmers': 81},
'middlesex county': {'programmers': 81,
'salesmen': 62}},
'new york': {'queens county': {'plumbers': 9,
'salesmen': 36},
'queens counyt': {}}}
Explanation:
We're just providing another nested instance of our class Vividict
whenever a key is accessed but missing. (Returning the value assignment is useful because it avoids us additionally calling the getter on the dict, and unfortunately, we can't return it as it is being set.)
Note, these are the same semantics as the most upvoted answer but in half the lines of code - nosklo's implementation:
class AutoVivification(dict):
"""Implementation of perl's autovivification feature."""
def __getitem__(self, item):
try:
return dict.__getitem__(self, item)
except KeyError:
value = self[item] = type(self)()
return value
Demonstration of Usage
Below is just an example of how this dict could be easily used to create a nested dict structure on the fly. This can quickly create a hierarchical tree structure as deeply as you might want to go.
import pprint
class Vividict(dict):
def __missing__(self, key):
value = self[key] = type(self)()
return value
d = Vividict()
d['foo']['bar']
d['foo']['baz']
d['fizz']['buzz']
d['primary']['secondary']['tertiary']['quaternary']
pprint.pprint(d)
Which outputs:
{'fizz': {'buzz': {}},
'foo': {'bar': {}, 'baz': {}},
'primary': {'secondary': {'tertiary': {'quaternary': {}}}}}
And as the last line shows, it pretty prints beautifully and in order for manual inspection. But if you want to visually inspect your data, implementing __missing__
to set a new instance of its class to the key and return it is a far better solution.
Other alternatives, for contrast:
dict.setdefault
Although the asker thinks this isn't clean, I find it preferable to the Vividict
myself.
d = {} # or dict()
for (state, county, occupation), number in data.items():
d.setdefault(state, {}).setdefault(county, {})[occupation] = number
and now:
>>> pprint.pprint(d, width=40)
{'new jersey': {'mercer county': {'plumbers': 3,
'programmers': 81},
'middlesex county': {'programmers': 81,
'salesmen': 62}},
'new york': {'queens county': {'plumbers': 9,
'salesmen': 36}}}
A misspelling would fail noisily, and not clutter our data with bad information:
>>> d['new york']['queens counyt']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'queens counyt'
Additionally, I think setdefault works great when used in loops and you don't know what you're going to get for keys, but repetitive usage becomes quite burdensome, and I don't think anyone would want to keep up the following:
d = dict()
d.setdefault('foo', {}).setdefault('bar', {})
d.setdefault('foo', {}).setdefault('baz', {})
d.setdefault('fizz', {}).setdefault('buzz', {})
d.setdefault('primary', {}).setdefault('secondary', {}).setdefault('tertiary', {}).setdefault('quaternary', {})
Another criticism is that setdefault requires a new instance whether it is used or not. However, Python (or at least CPython) is rather smart about handling unused and unreferenced new instances, for example, it reuses the location in memory:
>>> id({}), id({}), id({})
(523575344, 523575344, 523575344)
An auto-vivified defaultdict
This is a neat looking implementation, and usage in a script that you're not inspecting the data on would be as useful as implementing __missing__
:
from collections import defaultdict
def vivdict():
return defaultdict(vivdict)
But if you need to inspect your data, the results of an auto-vivified defaultdict populated with data in the same way looks like this:
>>> d = vivdict(); d['foo']['bar']; d['foo']['baz']; d['fizz']['buzz']; d['primary']['secondary']['tertiary']['quaternary']; import pprint;
>>> pprint.pprint(d)
defaultdict(<function vivdict at 0x17B01870>, {'foo': defaultdict(<function vivdict
at 0x17B01870>, {'baz': defaultdict(<function vivdict at 0x17B01870>, {}), 'bar':
defaultdict(<function vivdict at 0x17B01870>, {})}), 'primary': defaultdict(<function
vivdict at 0x17B01870>, {'secondary': defaultdict(<function vivdict at 0x17B01870>,
{'tertiary': defaultdict(<function vivdict at 0x17B01870>, {'quaternary': defaultdict(
<function vivdict at 0x17B01870>, {})})})}), 'fizz': defaultdict(<function vivdict at
0x17B01870>, {'buzz': defaultdict(<function vivdict at 0x17B01870>, {})})})
This output is quite inelegant, and the results are quite unreadable. The solution typically given is to recursively convert back to a dict for manual inspection. This non-trivial solution is left as an exercise for the reader.
Performance
Finally, let's look at performance. I'm subtracting the costs of instantiation.
>>> import timeit
>>> min(timeit.repeat(lambda: {}.setdefault('foo', {}))) - min(timeit.repeat(lambda: {}))
0.13612580299377441
>>> min(timeit.repeat(lambda: vivdict()['foo'])) - min(timeit.repeat(lambda: vivdict()))
0.2936999797821045
>>> min(timeit.repeat(lambda: Vividict()['foo'])) - min(timeit.repeat(lambda: Vividict()))
0.5354437828063965
>>> min(timeit.repeat(lambda: AutoVivification()['foo'])) - min(timeit.repeat(lambda: AutoVivification()))
2.138362169265747
Based on performance, dict.setdefault
works the best. I'd highly recommend it for production code, in cases where you care about execution speed.
If you need this for interactive use (in an IPython notebook, perhaps) then performance doesn't really matter - in which case, I'd go with Vividict for readability of the output. Compared to the AutoVivification object (which uses __getitem__
instead of __missing__
, which was made for this purpose) it is far superior.
Conclusion
Implementing __missing__
on a subclassed dict
to set and return a new instance is slightly more difficult than alternatives but has the benefits of
- easy instantiation
- easy data population
- easy data viewing
and because it is less complicated and more performant than modifying __getitem__
, it should be preferred to that method.
Nevertheless, it has drawbacks:
- Bad lookups will fail silently.
- The bad lookup will remain in the dictionary.
Thus I personally prefer setdefault
to the other solutions, and have in every situation where I have needed this sort of behavior.
How to create nested dictionaries PYTHON
Use a list comprehension. Iterate over the zipped lists and create the nested dictionaries:
a = ['A', 'B', 'C', 'D']
b = [1, 2, 3, 4]
c = ['N1', 'N2', 'N3', 'N4']
>>> [{k1: {k2: v}} for k1, v, k2 in zip(a, b, c)]
[{'A': {'N1': 1}}, {'B': {'N2': 2}}, {'C': {'N3': 3}}, {'D': {'N4': 4}}]
Create a nested Dictionary with two dictionaries
You can use defaultdict
here. multi
need not be sorted.
from collections import defaultdict
out=defaultdict(dict)
for v,k,vs in multi:
out[k]={**out[k],**{v:vs[0]}}
Output
defaultdict(dict,
{77766: {14: 2, 15: 1},
88866: {70: 1, 71: 2, 72: 5, 73: 4, 74: 3},
99966: {79: 1, 80: 2}})
EDIT:
Sorting the inner dictionaries.
out={k:dict(sorted(v.items(),key=lambda x:x[1])) for k,v in out.items()}
Output:
{77766: {15: 1, 14: 2},
88866: {70: 1, 71: 2, 74: 3, 73: 4, 72: 5},
99966: {79: 1, 80: 2}}
How to create a nested dictionary in python with 6 lists
Maybe something like:
out = {k: {k1: v1, k2: v2} for k, k1, v1, k2, v2 in zip(a, b, c, d, e)}
If we want to use excessive number of zip
s, we could also do:
out = {k: dict(v) for k,v in zip(a, zip(zip(b, c), zip(d, e)))}
Output:
{'A':{1 :9, 0:11} , 'B':{2:8, 3:13}, 'C':{3:7, 5:13} , 'D':{4:6, 7:15}}
How to combine two nested dictionaries with same master keys
Yes:
dict3 = {k: {**v, **dict2[k]} for k, v in dict1.items()}
Firstly, use .items()
to iterate over both keys and values at the same time.
Then, for each key k
you want the value to be a new dict that is created by dumping — or destructuring — both v
and dict2[k]
in it.
UPDATE for Python >= 3.9:
Thanks @mwo for mentioning the pipe |
operand:
dict3 = {k: v | dict2[k] for k, v in dict1.items()}
How to create dynamic nested dictionaries in Python
from functools import reduce
list_name=["node1","node2","node3","nodeN"]
print(reduce(lambda k, v: {v: k}, reversed(list_name), True))
Output:
{'node1': {'node2': {'node3': {'nodeN': True}}}}
Related Topics
How to Pass a Variable Between Flask Pages
How to Execute a String Containing Python Code in Python
Converting Unix Timestamp String to Readable Date
String Comparison in Python: Is Vs. ==
Setting the Correct Encoding When Piping Stdout in Python
How to Properly Ignore Exceptions
How to Get the Last Element of a List
Scatter Plot With Different Text At Each Data Point
Updating Gui Elements in Multithreaded Pyqt
How to Print a Number Using Commas as Thousands Separators
Return, Return None, and No Return At All