How to make a class JSON serializable
Do you have an idea about the expected output? For example, will this do?
>>> f = FileItem("/foo/bar")
>>> magic(f)
'{"fname": "/foo/bar"}'
In that case you can merely call json.dumps(f.__dict__)
.
If you want more customized output then you will have to subclass JSONEncoder
and implement your own custom serialization.
For a trivial example, see below.
>>> from json import JSONEncoder
>>> class MyEncoder(JSONEncoder):
def default(self, o):
return o.__dict__
>>> MyEncoder().encode(f)
'{"fname": "/foo/bar"}'
Then you pass this class into the json.dumps()
method as cls
kwarg:
json.dumps(cls=MyEncoder)
If you also want to decode then you'll have to supply a custom object_hook
to the JSONDecoder
class. For example:
>>> def from_json(json_object):
if 'fname' in json_object:
return FileItem(json_object['fname'])
>>> f = JSONDecoder(object_hook = from_json).decode('{"fname": "/foo/bar"}')
>>> f
<__main__.FileItem object at 0x9337fac>
>>>
Serializing class instance to JSON
The basic problem is that the JSON encoder json.dumps()
only knows how to serialize a limited set of object types by default, all built-in types. List here: https://docs.python.org/3.3/library/json.html#encoders-and-decoders
One good solution would be to make your class inherit from JSONEncoder
and then implement the JSONEncoder.default()
function, and make that function emit the correct JSON for your class.
A simple solution would be to call json.dumps()
on the .__dict__
member of that instance. That is a standard Python dict
and if your class is simple it will be JSON serializable.
class Foo(object):
def __init__(self):
self.x = 1
self.y = 2
foo = Foo()
s = json.dumps(foo) # raises TypeError with "is not JSON serializable"
s = json.dumps(foo.__dict__) # s set to: {"x":1, "y":2}
The above approach is discussed in this blog posting:
Serializing arbitrary Python objects to JSON using _dict_
And, of course, Python offers a built-in function that accesses .__dict__
for you, called vars()
.
So the above example can also be done as:
s = json.dumps(vars(foo)) # s set to: {"x":1, "y":2}
Make a Custom Class JSON serializable
Subclass json.JSONEncoder, and then construct a suitable dictionary or array.
See "Extending JSONEncoder" behind this link
Like this:
>>> class A: pass
...
>>> a = A()
>>> a.foo = "bar"
>>> import json
>>>
>>> class MyEncoder(json.JSONEncoder):
... def default(self, obj):
... if isinstance(obj, A):
... return { "foo" : obj.foo }
... return json.JSONEncoder.default(self, obj)
...
>>> json.dumps(a, cls=MyEncoder)
'{"foo": "bar"}'
json serialization of a list of objects of a custom class
I've solved this by adding an encode method to the class:
def encode(self):
return self.__dict__
and adding some arguments to json.dumps:
jsontracks = json.dumps(tracklist, default=lambda o: o.encode(), indent=4)
This will "crawl" down your class tree (if you have any child classes) and encode every object as a json list/object automatically. This should work with just about any class and is fast to type. You may also want to control which class parameters get encoded with something like:
def encode(self):
return {'name': self.name,
'code': self.code,
'amount': self.amount,
'minimum': self.minimum,
'maximum': self.maximum}
or a little bit faster to edit (if you're lazy like me):
def encode(self):
encoded_items = ['name', 'code', 'batch_size', 'cost',
'unit', 'ingredients', 'nutrients']
return {k: v for k, v in self.__dict__.items() if k in encoded_items}
full code:
import json
class Song:
def __init__(self, sname, sartist, coverart, albname, albartist, spotid):
self.sname = sname
self.sartist = sartist
self.coverart = coverart
self.albname = albname
self.albartist = albartist
self.spotid = spotid
def encode(self):
return self.__dict__
tracklist = [
Song('Imagine', 'John Lennon', None, None, None, None),
Song('Hey Jude', 'The Beatles', None, None, None, None),
Song('(I Can\'t Get No) Satisfaction', 'The Rolling Stones', None, None, None, None),
]
jsontracks = json.dumps(tracklist, default=lambda o: o.encode(), indent=4)
print(jsontracks)
output:
[
{
"sname": "Imagine",
"sartist": "John Lennon",
"coverart": null,
"albname": null,
"albartist": null,
"spotid": null
},
{
"sname": "Hey Jude",
"sartist": "The Beatles",
"coverart": null,
"albname": null,
"albartist": null,
"spotid": null
},
{
"sname": "(I Can't Get No) Satisfaction",
"sartist": "The Rolling Stones",
"coverart": null,
"albname": null,
"albartist": null,
"spotid": null
}
]
Python: converting class objects to JSON - object is not JSON serializable
You wish the encoder to support both Decimal
and dataclass
es. You can do it like so:
import dataclasses, json
class JSONEncoder(json.JSONEncoder):
def default(self, o):
if dataclasses.is_dataclass(o):
return dataclasses.asdict(o)
if isinstance(obj, Decimal):
return str(obj)
return super().default(o)
json.dumps(foo, cls=JSONEncoder)
How to make a class JSON serializable
Do you have an idea about the expected output? For example, will this do?
>>> f = FileItem("/foo/bar")
>>> magic(f)
'{"fname": "/foo/bar"}'
In that case you can merely call json.dumps(f.__dict__)
.
If you want more customized output then you will have to subclass JSONEncoder
and implement your own custom serialization.
For a trivial example, see below.
>>> from json import JSONEncoder
>>> class MyEncoder(JSONEncoder):
def default(self, o):
return o.__dict__
>>> MyEncoder().encode(f)
'{"fname": "/foo/bar"}'
Then you pass this class into the json.dumps()
method as cls
kwarg:
json.dumps(cls=MyEncoder)
If you also want to decode then you'll have to supply a custom object_hook
to the JSONDecoder
class. For example:
>>> def from_json(json_object):
if 'fname' in json_object:
return FileItem(json_object['fname'])
>>> f = JSONDecoder(object_hook = from_json).decode('{"fname": "/foo/bar"}')
>>> f
<__main__.FileItem object at 0x9337fac>
>>>
How to make a class with an inner class JSON serializable
While myClass.__dict__
does not work on inner classes, you can define your own method to convert your class into a dictionary, as long as you know what fields are an Object.
The follow example...
class Thing():
def __init__(self, name):
self.name = name
self.children = [self.Thing2(self)]
def asDict(self):
dict = self.__dict__
dict["children"] = [child.__dict__ for child in dict["children"]]
return dict
class Thing2():
def __init__(self, parent):
self.name = parent.name + "'s child"
myThing = Thing("Clay")
print(myThing.__dict__)
print(myThing.asDict())
yields the result...
{'name': 'Clay', 'children': [<__main__.Thing.Thing2 object at 0x00000257C4358B00>]}
{'name': 'Clay', 'children': [{'name': "Clay's child"}]}
which can be converted into JSON using json.dumps()
.
If you do not know what fields in your class are JSON serializable and which are inner classes, you could iterate over the values in your class's dictionary, check if they are JSON serializable, and convert them to dictionaries (value.__dict__
) as necessary.
Making object JSON serializable with regular encoder
As I said in a comment to your question, after looking at the json
module's source code, it does not appear to lend itself to doing what you want. However the goal could be achieved by what is known as monkey-patching
(see question What is a monkey patch?).
This could be done in your package's __init__.py
initialization script and would affect all subsequent json
module serialization since modules are generally only loaded once and the result is cached in sys.modules
.
The patch changes the default json encoder's default
method—the default default()
.
Here's an example implemented as a standalone module for simplicity's sake:
Module: make_json_serializable.py
""" Module that monkey-patches json module when it's imported so
JSONEncoder.default() automatically checks for a special "to_json()"
method and uses it to encode the object if found.
"""
from json import JSONEncoder
def _default(self, obj):
return getattr(obj.__class__, "to_json", _default.default)(obj)
_default.default = JSONEncoder.default # Save unmodified default.
JSONEncoder.default = _default # Replace it.
Using it is trivial since the patch is applied by simply importing the module.
Sample client script:
import json
import make_json_serializable # apply monkey-patch
class Foo(object):
def __init__(self, name):
self.name = name
def to_json(self): # New special method.
""" Convert to JSON format string representation. """
return '{"name": "%s"}' % self.name
foo = Foo('sazpaz')
print(json.dumps(foo)) # -> "{\"name\": \"sazpaz\"}"
To retain the object type information, the special method can also include it in the string returned:
return ('{"type": "%s", "name": "%s"}' %
(self.__class__.__name__, self.name))
Which produces the following JSON that now includes the class name:
"{\"type\": \"Foo\", \"name\": \"sazpaz\"}"
Magick Lies Here
Even better than having the replacement default()
look for a specially named method, would be for it to be able to serialize most Python objects automatically, including user-defined class instances, without needing to add a special method. After researching a number of alternatives, the following — based on an answer by @Raymond Hettinger to another question — which uses the pickle
module, seemed closest to that ideal to me:
Module: make_json_serializable2.py
""" Module that imports the json module and monkey-patches it so
JSONEncoder.default() automatically pickles any Python objects
encountered that aren't standard JSON data types.
"""
from json import JSONEncoder
import pickle
def _default(self, obj):
return {'_python_object': pickle.dumps(obj)}
JSONEncoder.default = _default # Replace with the above.
Of course everything can't be pickled—extension types for example. However there are ways defined to handle them via the pickle protocol by writing special methods—similar to what you suggested and I described earlier—but doing that would likely be necessary for a far fewer number of cases.
Deserializing
Regardless, using the pickle protocol also means it would be fairly easy to reconstruct the original Python object by providing a custom object_hook
function argument on any json.loads()
calls that used any '_python_object'
key in the dictionary passed in, whenever it has one. Something like:
def as_python_object(dct):
try:
return pickle.loads(str(dct['_python_object']))
except KeyError:
return dct
pyobj = json.loads(json_str, object_hook=as_python_object)
If this has to be done in many places, it might be worthwhile to define a wrapper function that automatically supplied the extra keyword argument:
json_pkloads = functools.partial(json.loads, object_hook=as_python_object)
pyobj = json_pkloads(json_str)
Naturally, this could be monkey-patched it into the json
module as well, making the function the default object_hook
(instead of None
).
I got the idea for using pickle
from an answer by Raymond Hettinger to another JSON serialization question, whom I consider exceptionally credible as well as an official source (as in Python core developer).
Portability to Python 3
The code above does not work as shown in Python 3 because json.dumps()
returns a bytes
object which the JSONEncoder
can't handle. However the approach is still valid. A simple way to workaround the issue is to latin1
"decode" the value returned from pickle.dumps()
and then "encode" it from latin1
before passing it on to pickle.loads()
in the as_python_object()
function. This works because arbitrary binary strings are valid latin1
which can always be decoded to Unicode and then encoded back to the original string again (as pointed out in this answer by Sven Marnach).
(Although the following works fine in Python 2, the latin1
decoding and encoding it does is superfluous.)
from decimal import Decimal
class PythonObjectEncoder(json.JSONEncoder):
def default(self, obj):
return {'_python_object': pickle.dumps(obj).decode('latin1')}
def as_python_object(dct):
try:
return pickle.loads(dct['_python_object'].encode('latin1'))
except KeyError:
return dct
class Foo(object): # Some user-defined class.
def __init__(self, name):
self.name = name
def __eq__(self, other):
if type(other) is type(self): # Instances of same class?
return self.name == other.name
return NotImplemented
__hash__ = None
data = [1,2,3, set(['knights', 'who', 'say', 'ni']), {'key':'value'},
Foo('Bar'), Decimal('3.141592653589793238462643383279502884197169')]
j = json.dumps(data, cls=PythonObjectEncoder, indent=4)
data2 = json.loads(j, object_hook=as_python_object)
assert data == data2 # both should be same
How to JSON serialize a python built-in class (eg. int)?
There is no way to serialize a JSON or Python type
as JSON value. As described in RFC7159 Section 3 the only available values are:
false / null / true / object / array / number / string
However you could serialize a Python type
as a JSON string. For instance, a Python int
would become JSON string value "int"
.
Since Python keyword int
is object of type type
. You can use __name__
to get its string name. For instance: print(int.__name__)
.
To automatically encode it, I let you check this answer which use a custom JSONEncoder
.
JSON serialization of dictionary with complex objects
Maybe this can be a starting spot for you. The serializer grabs the __dict__
attribute from the object and makes a new dict-of-dicts, then writes it to JSON. The deserializer creates a dummy object, then updates the __dict__
on the way in.
import json
class PlayerElo:
"""
A class to represent a player in the Elo Rating System
"""
def __init__(self, name: str, id: str, rating):
self.id = id
self.name = name
self.eloratings = {0: 1500}
self.elomatches = {0: 0}
self.initialrating = rating
playersElo={} # dictionary of {<int> : <PlayerElo>}
playersElo[1] = PlayerElo('Joe','123',999)
playersElo[2] = PlayerElo('Bill','456',1999)
def serialize(ratings):
newdict = {i:j.__dict__ for i,j in ratings.items()}
json.dump( newdict, open('x.json','w') )
def deserialize():
o = json.load(open('x.json'))
pe = {}
for k,v in o.items():
obj = PlayerElo('0','0',0)
obj.__dict__.update( v )
pe[int(k)] = obj
return pe
print(playersElo)
serialize( playersElo )
pe = deserialize( )
print(pe)
Related Topics
Modifying List While Iterating
How to Find All Occurrences of a Substring
Link to Flask Static Files With Url_For
Why Does Python Use 'Else' After For and While Loops
Why Does Concatenation of Dataframes Get Exponentially Slower
Find the First Instance of a Nonzero Number in a List in Python
How to Watch a File For Changes
How to Split a Dataframe String Column into Two Columns
How Are Iloc and Loc Different
Lazy Method For Reading Big File in Python
How to Randomly Select an Item from a List
How to Use Multiprocessing Pool.Map With Multiple Arguments
How Do "And" and "Or" Act With Non-Boolean Values
Fatal Error: Python.H: No Such File or Directory
Passing HTML to Template Using Flask/Jinja2
What Does _All_ Mean in Python
How to Control the Source Ip Address of a Zeromq Packet on a Machine With Multiple Ips