Storing Python Dictionaries

Elegant way to store dictionary permanently with Python?

Why not dump it to a JSON file, and then load it from there where you need it?

import json

with open('my_dict.json', 'w') as f:
    json.dump(my_dict, f)

# elsewhere...

with open('my_dict.json') as f:
    my_dict = json.load(f)

Loading from JSON is fairly efficient.

Another option would be to use pickle, but unlike JSON, the files it generates aren't human-readable so you lose out on the visual verification you liked from your old method.

Storing Python dictionaries

Pickle save:

try:
    import cPickle as pickle
except ImportError:  # Python 3.x
    import pickle

with open('data.p', 'wb') as fp:
    pickle.dump(data, fp, protocol=pickle.HIGHEST_PROTOCOL)

See the pickle module documentation for additional information regarding the protocol argument.

Pickle load:

with open('data.p', 'rb') as fp:
    data = pickle.load(fp)

---

JSON save:

import json

with open('data.json', 'w') as fp:
    json.dump(data, fp)

Supply extra arguments, like sort_keys or indent, to get a pretty result. The argument sort_keys will sort the keys alphabetically and indent will indent your data structure with indent=N spaces.

json.dump(data, fp, sort_keys=True, indent=4)

JSON load:

with open('data.json', 'r') as fp:
    data = json.load(fp)

How to store a dictionary as a separate file, and read the file in a python script to use the variables

You can replace this line:
gene_data = open("gene_data1.txt", "r")

with this:

import ast

with open('dict.txt') as f:
    gene_data = f.read()
gene_data = ast.literal_eval(gene_data)

but make sure the text file just contains the dictionary, not the assignment of the dictionary:

{ 'ham_pb_length':2973, 'ham_pb_bitscore':5664,'cg2225_ph_length':3303, 'cg2225_ph_bitscore':6435,'lrp1_pf_length':14259, 'lrp1_pf_bitscore':28010,}

As pointed out by others, allowing your script to execute any command in a file can be dangerous. With this method, at least it won't execute anything in the external file, if the contents don't evaluate nicely the script will just throw an error.

Storing code in a list or dictionary?

Yes, there is a way but by using functions:

def up():
    y=y+1

def down():
    y=y-1

def left():
    x=x-1

def right():
    x=x+1

d = {'up': up, 'down': down, 'left': left, 'right': right};

then call it like d['up']().

To avoid using global variables, you may use OOP approach:

class player:
    def __init__(self, x=0, y=0):
       self.x = x
       self.y = y

    def up(self):
        self.y += 1

    def down(self):
        self.y -= 1

    def left(self):
        self.x -= 1

    def right(self):
        self.x += 1

    def move(self, direction):
        # call the appropriate method that corresponds to given direction
        getattr(self, direction)()

Example use:

p = player(0,0)
p.move('up') # and so on..

Store a dictionary in a file for later retrieval

A few people have recommended shelve - I haven't used it, and I'm not knocking it. I have used pickle/cPickle and I'll offer the following approach:

How to use Pickle/cPickle (the abridged version)...

There are many reasons why you would use Pickle (or its noticable faster variant, cPickle). Put tersely Pickle is a way to store objects outside of your process.

Pickle not only gives you the options to store objects outside your python process, but also does so in a serialized fashion. Meaning, First In, First Out behavior (FIFO).

import pickle

## I am making up a dictionary here to show you how this works...
## Because I want to store this outside of this single run, it could be that this
## dictionary is dynamic and user based - so persistance beyond this run has
## meaning for me.  
myMadeUpDictionary = {"one": "banana", "two": "banana", "three": "banana", "four": "no-more"}

with open("mySavedDict.txt", "wb") as myFile:
    pickle.dump(myMadeUpDictionary, myFile)

So what just happened?

• Step1: imported a module named 'pickle'
• Step2: created my dictionary object
• Step3: used a context manager to handle the opening/closing of a new file...
• Step4: dump() the contents of the dictionary (which is referenced as 'pickling' the object) and then write it to a file (mySavedDict.txt).

If you then go into the file that was just created (located now on your filesystem), you can see the contents. It's messy - ugly - and not very insightlful.

nammer@crunchyQA:~/workspace/SandBox/POSTS/Pickle & cPickle$ cat mySavedDict.txt 
(dp0
S'four'
p1
S'no-more'
p2
sS'three'
p3
S'banana'
p4
sS'two'
p5
g4
sS'one'
p6
g4
s.

So what's next?

To bring that BACK into our program we simply do the following:

import pickle

with open("mySavedDict.txt", "rb") as myFile:
    myNewPulledInDictionary = pickle.load(myFile)

print myNewPulledInDictionary

Which provides the following return:

{'four': 'no-more', 'one': 'banana', 'three': 'banana', 'two': 'banana'}

cPickle vs Pickle

You won't see many people use pickle these days - I can't think off the top of my head why you would want to use the first implementation of pickle, especially when there is cPickle which does the same thing (more or less) but a lot faster!

So you can be lazy and do:

import cPickle as pickle

Which is great if you have something already built that uses pickle... but I argue that this is a bad recommendation and I fully expect to get scolded for even recommending that! (you should really look at your old implementation that used the original pickle and see if you need to change anything to follow cPickle patterns; if you have legacy code or production code you are working with, this saves you time refactoring (finding/replacing all instances of pickle with cPickle).

Otherwise, just:

import cPickle

and everywhere you see a reference to the pickle library, just replace accordingly. They have the same load() and dump() method.

Warning Warning I don't want to write this post any longer than it is, but I seem to have this painful memory of not making a distinction between load() and loads(), and dump() and dumps(). Damn... that was stupid of me! The short answer is that load()/dump() does it to a file-like object, wheres loads()/dumps() will perform similar behavior but to a string-like object (read more about it in the API, here).

Again, I haven't used shelve, but if it works for you (or others) - then yay!

RESPONSE TO YOUR EDIT

You need to remove the dict_items_read = dict_items_open.read() from your context-manager at the end. The file is already open and read in. You don't read it in like you would a text file to pull out strings... it's storing pickled python objects. It's not meant for eyes! It's meant for load().

Your code modified... works just fine for me (copy/paste and run the code below and see if it works). Notice near the bottom I've removed your read() of the file object.

import cPickle as pickle

BDICT = {}

## Automatically generated START
name = "BOB"
name_title = name.title()
count = 5
BDICT[name_title] = count

name = "TOM"
name_title = name.title()
count = 5
BDICT[name_title] = count

name = "TIMMY JOE"
name_title = name.title()
count = 5
BDICT[name_title] = count
## Automatically generated END

if BDICT:
    with open('DICT_ITEMS.txt', 'wb') as dict_items_save:
        pickle.dump(BDICT, dict_items_save)

BDICT = {} ## Wiping the dictionary

## Usually in a loop
firstrunDICT = True

if firstrunDICT:
    with open('DICT_ITEMS.txt', 'rb') as dict_items_open:
        BDICT = pickle.load(dict_items_open)
        firstrunDICT = False
        print BDICT

---

Related Topics