Python: Load Variables in a Dict into Namespace

Python: load variables in a dict into namespace

Consider the Bunch alternative:

class Bunch(object):
  def __init__(self, adict):
    self.__dict__.update(adict)

so if you have a dictionary d and want to access (read) its values with the syntax x.foo instead of the clumsier d['foo'], just do

x = Bunch(d)

this works both inside and outside functions -- and it's enormously cleaner and safer than injecting d into globals()! Remember the last line from the Zen of Python...:

>>> import this
The Zen of Python, by Tim Peters
   ...
Namespaces are one honking great idea -- let's do more of those!

Easily dumping variables from/to namespaces/dictionaries in Python

The solution below provides syntax very close to your requirement, the only difference is that you have to pass to the function environment where the variables are defined explicitly:

x = 10
y = 20

class dump_into_namespace:
    def __init__(self, env, *vars):
        self.vars = dict([(x, env[x]) for v in vars for x in env if v is env[x]])
    def __getattr__(self, name): return self.vars[name]

o = dump_into_namespace(locals(), x, y)
print o.x, o.y

You can then 'dump' back the variables to your locals (say, in a different function):

>>> locals().update(o.vars)
>>> x
10

EDIT:

Thanks to the suggestion of eyquem this can be even shorter. The idea is to put variables into self.__dict__ of the 'dump' object (note: syntax of update changes here):

class dump_into_namespace:
    def __init__(self, env, *vs):
        vars(self).update(dict([(x, env[x]) for v in vs for x in env if v is env[x]]))

def f():
    x = 10
    y = 20
    return dump_into_namespace(locals(), x, y)

o = f() 
print o.x, o.y 
globals().update(vars(o))
print x

Convert dictionary entries into variables

This was what I was looking for:

>>> d = {'a':1, 'b':2}
>>> for key,val in d.items():
        exec(key + '=val')

Argparse Dictionary to Namespace

Just use the ** unpacking:

In [57]: adict={'foo':1, 'bar':'astring', 'baz':[1,2,3]}
In [59]: argparse.Namespace(**adict)
Out[59]: Namespace(bar='astring', baz=[1, 2, 3], foo=1)
In [60]: args = argparse.Namespace(**adict)
In [61]: args
Out[61]: Namespace(bar='astring', baz=[1, 2, 3], foo=1)
In [62]: args.bar
Out[62]: 'astring'

Its docs:

In [63]: argparse.Namespace?
Init signature: argparse.Namespace(**kwargs)
Docstring:     
Simple object for storing attributes.

Implements equality by attribute names and values, and provides a simple string representation.

It's a simple object subclass that assigns its **kwargs to its attributes. And provides a basic display method.

vars is the standard Python method of reversing that:

In [65]: vars(args)
Out[65]: {'foo': 1, 'bar': 'astring', 'baz': [1, 2, 3]}

Internally, parser.parse_args creates an argparse.Namespace() and assigns values with setattr(args, dest, value).

Python: include entries of a dictionary in the local namespace of a function

It is impossible in the general case, because the dict can have keys which are not valid variable names, e.g. Python keywords or non-strings.

If you desperately need this, and you can guarantee that the dict keys are all valid variable names, and you don't need it to work on Python 3, it's possible to loop over the dict items and use an exec statement. There are some weird scoping consequences of this approach, it's very ugly, and should be strongly discouraged.

An acceptable alternative is to create a dummy object and set them as attributes, for example:

>>> data = dict(a=1,b=2,c=3,d=4)
>>> from types import SimpleNamespace
>>> v = SimpleNamespace(**data)
>>> v.a
1
>>> v.d
4

But it's only really like a syntactic sugar for dict access. Read zen of Python #19. Whichever way you look at it, you will need namespacing!

Another idea: create a callable class instead of a function (by inheriting collections.Callable), and unpack the dict into attributes on the class. Then at least your variables will be namespaced by the class.

Divide a dictionary into variables

Problem is that dicts are unordered, so you can't use simple unpacking of d.values(). You could of course first sort the dict by key, then unpack the values:

# Note: in python 3, items() functions as iteritems() did
#       in older versions of Python; use it instead
ds = sorted(d.iteritems())
name0, name1, name2..., namen = [v[1] for v in ds]

You could also, at least within an object, do something like:

for k, v in dict.iteritems():
    setattr(self, k, v)

Additionally, as I mentioned in the comment above, if you can get all your logic that needs your unpacked dictionary as variables in to a function, you could do:

def func(**kwargs):
    # Do stuff with labeled args

func(**d)

How to convert a nested python dictionary into a simple namespace?

2022 answer: now there is a tiny, relatively fast library I have published, called dotwiz, which alternatively can be used to provide easy dot access for a python dict object.

It should, coincidentally, be a little faster than the other options -- I've added a quick and dirty benchmark code I put together using the timeit module below, timing against both a attrdict and SimpleNamespace approach -- the latter of which actually performs pretty solid in times.

Note that I had to modify the parse function slightly, so that it handles nested dicts within a list object, for example.

from timeit import timeit
from types import SimpleNamespace

from attrdict import AttrDict
from dotwiz import DotWiz

example_input = {'key0a': "test", 'key0b': {'key1a': [{'key2a': 'end', 'key2b': "test"}], 'key1b': "test"},
                 "something": "else"}

def parse(d):
    x = SimpleNamespace()
    _ = [setattr(x, k,
                 parse(v) if isinstance(v, dict)
                 else [parse(e) for e in v] if isinstance(v, list)
                 else v) for k, v in d.items()]
    return x

print('-- Create')
print('attrdict:         ', round(timeit('AttrDict(example_input)', globals=globals()), 2))
print('dotwiz:           ', round(timeit('DotWiz(example_input)', globals=globals()), 2))
print('SimpleNamespace:  ', round(timeit('parse(example_input)', globals=globals()), 2))
print()

dw = DotWiz(example_input)
ns = parse(example_input)
ad = AttrDict(example_input)

print('-- Get')
print('attrdict:         ', round(timeit('ad.key0b.key1a[0].key2a', globals=globals()), 2))
print('dotwiz:           ', round(timeit('dw.key0b.key1a[0].key2a', globals=globals()), 2))
print('SimpleNamespace:  ', round(timeit('ns.key0b.key1a[0].key2a', globals=globals()), 2))
print()

print(ad)
print(dw)
print(ns)

assert ad.key0b.key1a[0].key2a \
       == dw.key0b.key1a[0].key2a \
       == ns.key0b.key1a[0].key2a \
       == 'end'

Here are the results, on my M1 Mac Pro laptop:

attrdict:          0.69
dotwiz:            1.3
SimpleNamespace:   1.38

-- Get
attrdict:          6.06
dotwiz:            0.06
SimpleNamespace:   0.06

---

The dotwiz library can be installed with pip:

$ pip install dotwiz

Injecting variables into an import namespace

I came up with a solution based on this answer and the importlib docs. Basically, I have access to the module object before it is loaded by using the correct sequence of calls to importlib:

from importlib.util import spec_from_file_location, module_from_spec
from os.path import splitext, basename

def loadConfig(fileName):
    test = 'This is a test'
    name = splitext(basename(fileName))[0]
    spec = spec_from_file_location(name, fileName)
    config = module_from_spec(spec)
    config.test = test
    spec.loader.exec_module(config)
    return config

testmod = loadConfig('./testmod.py')

This is a bit better than modifying builtins, which may have unintended consequences in other parts of the program, and may also restrict the names I can pass in to the module.

I decided to put all the configuration items into a single field accessible at load time, which I named config. This allows me to do the following in testmod:

if 'test' in config:
    x = config['test']

The loader now looks like this:

from importlib.util import spec_from_file_location, module_from_spec
from os.path import splitext, basename

def loadConfig(fileName, **kwargs):
    name = splitext(basename(fileName))[0]
    spec = spec_from_file_location(name, fileName)
    config = module_from_spec(spec)
    config.config = kwargs
    spec.loader.exec_module(config)
    return config

testmod = loadConfig('./testmod.py', test='This is a test')

After finding myself using this a bunch of times, I finally ended up adding this functionality to the utility library I maintain, haggis. haggis.load.load_module loads a text file as a module with injection, while haggis.load.module_as_dict does a more advanced version of the same that loads it as a potentially nested configuration file into a dict.

Python variables as keys to dict

for i in ('apple', 'banana', 'carrot'):
    fruitdict[i] = locals()[i]

---

Related Topics