How to JSON serialize sets?
JSON notation has only a handful of native datatypes (objects, arrays, strings, numbers, booleans, and null), so anything serialized in JSON needs to be expressed as one of these types.
As shown in the json module docs, this conversion can be done automatically by a JSONEncoder and JSONDecoder, but then you would be giving up some other structure you might need (if you convert sets to a list, then you lose the ability to recover regular lists; if you convert sets to a dictionary using dict.fromkeys(s)
then you lose the ability to recover dictionaries).
A more sophisticated solution is to build-out a custom type that can coexist with other native JSON types. This lets you store nested structures that include lists, sets, dicts, decimals, datetime objects, etc.:
from json import dumps, loads, JSONEncoder, JSONDecoder
import pickle
class PythonObjectEncoder(JSONEncoder):
def default(self, obj):
try:
return {'_python_object': pickle.dumps(obj).decode('latin-1')}
except pickle.PickleError:
return super().default(obj)
def as_python_object(dct):
if '_python_object' in dct:
return pickle.loads(dct['_python_object'].encode('latin-1'))
return dct
Here is a sample session showing that it can handle lists, dicts, and sets:
>>> data = [1,2,3, set(['knights', 'who', 'say', 'ni']), {'key':'value'}, Decimal('3.14')]
>>> j = dumps(data, cls=PythonObjectEncoder)
>>> loads(j, object_hook=as_python_object)
[1, 2, 3, set(['knights', 'say', 'who', 'ni']), {'key': 'value'}, Decimal('3.14')]
Alternatively, it may be useful to use a more general purpose serialization technique such as YAML, Twisted Jelly, or Python's pickle module. These each support a much greater range of datatypes.
set object is not JSON serializable
Turn sets into lists before serializing, or use a custom default
handler to do so:
def set_default(obj):
if isinstance(obj, set):
return list(obj)
raise TypeError
result = json.dumps(yourdata, default=set_default)
How to make a class JSON serializable
Do you have an idea about the expected output? For example, will this do?
>>> f = FileItem("/foo/bar")
>>> magic(f)
'{"fname": "/foo/bar"}'
In that case you can merely call json.dumps(f.__dict__)
.
If you want more customized output then you will have to subclass JSONEncoder
and implement your own custom serialization.
For a trivial example, see below.
>>> from json import JSONEncoder
>>> class MyEncoder(JSONEncoder):
def default(self, o):
return o.__dict__
>>> MyEncoder().encode(f)
'{"fname": "/foo/bar"}'
Then you pass this class into the json.dumps()
method as cls
kwarg:
json.dumps(cls=MyEncoder)
If you also want to decode then you'll have to supply a custom object_hook
to the JSONDecoder
class. For example:
>>> def from_json(json_object):
if 'fname' in json_object:
return FileItem(json_object['fname'])
>>> f = JSONDecoder(object_hook = from_json).decode('{"fname": "/foo/bar"}')
>>> f
<__main__.FileItem object at 0x9337fac>
>>>
How to serialize SqlAlchemy result to JSON?
A flat implementation
You could use something like this:
from sqlalchemy.ext.declarative import DeclarativeMeta
class AlchemyEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj.__class__, DeclarativeMeta):
# an SQLAlchemy class
fields = {}
for field in [x for x in dir(obj) if not x.startswith('_') and x != 'metadata']:
data = obj.__getattribute__(field)
try:
json.dumps(data) # this will fail on non-encodable values, like other classes
fields[field] = data
except TypeError:
fields[field] = None
# a json-encodable dict
return fields
return json.JSONEncoder.default(self, obj)
and then convert to JSON using:
c = YourAlchemyClass()
print json.dumps(c, cls=AlchemyEncoder)
It will ignore fields that are not encodable (set them to 'None').
It doesn't auto-expand relations (since this could lead to self-references, and loop forever).
A recursive, non-circular implementation
If, however, you'd rather loop forever, you could use:
from sqlalchemy.ext.declarative import DeclarativeMeta
def new_alchemy_encoder():
_visited_objs = []
class AlchemyEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj.__class__, DeclarativeMeta):
# don't re-visit self
if obj in _visited_objs:
return None
_visited_objs.append(obj)
# an SQLAlchemy class
fields = {}
for field in [x for x in dir(obj) if not x.startswith('_') and x != 'metadata']:
fields[field] = obj.__getattribute__(field)
# a json-encodable dict
return fields
return json.JSONEncoder.default(self, obj)
return AlchemyEncoder
And then encode objects using:
print json.dumps(e, cls=new_alchemy_encoder(), check_circular=False)
This would encode all children, and all their children, and all their children... Potentially encode your entire database, basically. When it reaches something its encoded before, it will encode it as 'None'.
A recursive, possibly-circular, selective implementation
Another alternative, probably better, is to be able to specify the fields you want to expand:
def new_alchemy_encoder(revisit_self = False, fields_to_expand = []):
_visited_objs = []
class AlchemyEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj.__class__, DeclarativeMeta):
# don't re-visit self
if revisit_self:
if obj in _visited_objs:
return None
_visited_objs.append(obj)
# go through each field in this SQLalchemy class
fields = {}
for field in [x for x in dir(obj) if not x.startswith('_') and x != 'metadata']:
val = obj.__getattribute__(field)
# is this field another SQLalchemy object, or a list of SQLalchemy objects?
if isinstance(val.__class__, DeclarativeMeta) or (isinstance(val, list) and len(val) > 0 and isinstance(val[0].__class__, DeclarativeMeta)):
# unless we're expanding this field, stop here
if field not in fields_to_expand:
# not expanding this field: set it to None and continue
fields[field] = None
continue
fields[field] = val
# a json-encodable dict
return fields
return json.JSONEncoder.default(self, obj)
return AlchemyEncoder
You can now call it with:
print json.dumps(e, cls=new_alchemy_encoder(False, ['parents']), check_circular=False)
To only expand SQLAlchemy fields called 'parents', for example.
How to JSONL serialize sets in YAML style?
The "YAML printed in one line" format similar to JSONL is called "inline syntax" and is achievable with default_flow_style=True
, however, this is very badly documented. The result is not JSON, but still standard compliant and does not require custom encoder/decoder.
See the example:
from yaml import dump, load
data = [1, 2, 3, {'say', 'knights', 'ni', 'who'}, {'key': 'value'}, 3.14]
data_dumped = dump(data, default_flow_style=True)
print(data_dumped)
[1, 2, 3, !!set {knights: null, ni: null, say: null, who: null}, {"key": "value"}, 3.14]
data_loaded = load(data_dumped)
assert data == data_loaded
How to JSON serialize sets?
JSON notation has only a handful of native datatypes (objects, arrays, strings, numbers, booleans, and null), so anything serialized in JSON needs to be expressed as one of these types.
As shown in the json module docs, this conversion can be done automatically by a JSONEncoder and JSONDecoder, but then you would be giving up some other structure you might need (if you convert sets to a list, then you lose the ability to recover regular lists; if you convert sets to a dictionary using dict.fromkeys(s)
then you lose the ability to recover dictionaries).
A more sophisticated solution is to build-out a custom type that can coexist with other native JSON types. This lets you store nested structures that include lists, sets, dicts, decimals, datetime objects, etc.:
from json import dumps, loads, JSONEncoder, JSONDecoder
import pickle
class PythonObjectEncoder(JSONEncoder):
def default(self, obj):
try:
return {'_python_object': pickle.dumps(obj).decode('latin-1')}
except pickle.PickleError:
return super().default(obj)
def as_python_object(dct):
if '_python_object' in dct:
return pickle.loads(dct['_python_object'].encode('latin-1'))
return dct
Here is a sample session showing that it can handle lists, dicts, and sets:
>>> data = [1,2,3, set(['knights', 'who', 'say', 'ni']), {'key':'value'}, Decimal('3.14')]
>>> j = dumps(data, cls=PythonObjectEncoder)
>>> loads(j, object_hook=as_python_object)
[1, 2, 3, set(['knights', 'say', 'who', 'ni']), {'key': 'value'}, Decimal('3.14')]
Alternatively, it may be useful to use a more general purpose serialization technique such as YAML, Twisted Jelly, or Python's pickle module. These each support a much greater range of datatypes.
solidity TypeError: Object of type set is not JSON serializable
Instead of this:
{"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}
you should use this:
["abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"]
Sets in Python aren't JSON serializable.
Python json.dumps TypeError: Object of type 'set' is not JSON serializable , when trying convert from variable , working when hardcoded
Your problem is that your code doesn't do what you expect it to do.
the curly brackets in Python are used for both dict
and set
objects.
if you are using them like that:
{'message': 'Hello World'}
an object of type dict
is created.
However, if you use them like that:
{'Hello World'}
an object of type set
is created.
Your pro
variable, is a single str
variable, therefore, doing the following:
pro_ = {"'message': 'no verstion found'"}
would create a set
with one value only - your string.
If you would like to serialize that set into the json, you can do it with list(_pro)
but that wouldn't really make any sense, as it will also contain your quotation marks.
Serializing class instance to JSON
The basic problem is that the JSON encoder json.dumps()
only knows how to serialize a limited set of object types by default, all built-in types. List here: https://docs.python.org/3.3/library/json.html#encoders-and-decoders
One good solution would be to make your class inherit from JSONEncoder
and then implement the JSONEncoder.default()
function, and make that function emit the correct JSON for your class.
A simple solution would be to call json.dumps()
on the .__dict__
member of that instance. That is a standard Python dict
and if your class is simple it will be JSON serializable.
class Foo(object):
def __init__(self):
self.x = 1
self.y = 2
foo = Foo()
s = json.dumps(foo) # raises TypeError with "is not JSON serializable"
s = json.dumps(foo.__dict__) # s set to: {"x":1, "y":2}
The above approach is discussed in this blog posting:
Serializing arbitrary Python objects to JSON using _dict_
And, of course, Python offers a built-in function that accesses .__dict__
for you, called vars()
.
So the above example can also be done as:
s = json.dumps(vars(foo)) # s set to: {"x":1, "y":2}
Related Topics
In Python, How to Read the Exif Data for an Image
What Does a Python Process Return Code -9 Mean
Loading .Rdata Files into Python
Importing Variables from Another File
Matplotlib Make Tick Labels Font Size Smaller
Merge Lists That Share Common Elements
Python List Subtraction Operation
Tkinter: "Python May Not Be Configured for Tk"
Find Column Whose Name Contains a Specific String
How to Replace Text in a String Column of a Pandas Dataframe
Popen.Communicate() Throws Oserror: "[Errno 10] No Child Processes"
Translating Function for Finding All Partitions of a Set from Python to Ruby
Regular Expression Matching a Multiline Block of Text
Get List from Pandas Dataframe Column or Row
How to Include Third Party Python Libraries in Google App Engine
Executing Multi-Line Statements in the One-Line Command-Line