How to Overcome "Datetime.Datetime Not Json Serializable"

How to overcome datetime.datetime not JSON serializable?

Updated for 2018

The original answer accommodated the way MongoDB "date" fields were represented as:

{"$date": 1506816000000}

If you want a generic Python solution for serializing datetime to json, check out @jjmontes' answer for a quick solution which requires no dependencies.

---

As you are using mongoengine (per comments) and pymongo is a dependency, pymongo has built-in utilities to help with json serialization:

http://api.mongodb.org/python/1.10.1/api/bson/json_util.html

Example usage (serialization):

from bson import json_util
import json

json.dumps(anObject, default=json_util.default)

Example usage (deserialization):

json.loads(aJsonString, object_hook=json_util.object_hook)

---

Django

Django provides a native DjangoJSONEncoder serializer that deals with this kind of properly.

See https://docs.djangoproject.com/en/dev/topics/serialization/#djangojsonencoder

from django.core.serializers.json import DjangoJSONEncoder

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  cls=DjangoJSONEncoder
)

One difference I've noticed between DjangoJSONEncoder and using a custom default like this:

import datetime
import json

def default(o):
    if isinstance(o, (datetime.date, datetime.datetime)):
        return o.isoformat()

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  default=default
)

Is that Django strips a bit of the data:

 "last_login": "2018-08-03T10:51:42.990", # DjangoJSONEncoder 
 "last_login": "2018-08-03T10:51:42.990239", # default

So, you may need to be careful about that in some cases.

How to fix Python at Object of type datetime is not JSON serializable error

Answer:

In order to fix the Object of type datetime is not JSON serializable error, you need to convert all instances of datetime objects in your object to string.

There are other errors in your code, however, meaning this alone will not make your program run.

Converting the datetime objects to string objects:

In python, you can natively convert JSON data to string using json.dumps() with a default conversion of string.

You can do this by adding this line before the service.spreadsheets().values().append() call:

//rngData at this point has already been assigned
rngData = json.dumps(rngData, indent = 4, sort_keys = True, default = str)

NB: This on its own will not fix your code!

Other Matters:

When making calls to the Google Sheets API, it is very important that you make the requests in the way that the servers are expecting to receive those requests. That is to say, it is important to follow the documentation for making requests.

I am on a linux machine, and so I can not test the output format of win32.Dispatch().Workbooks.Open().Worksheets().Range().CurrentRegion(), but if the Microsoft Documentation on the Worksheet.Range property of Excel is anything to go by, I can safely assume that it's output isn't in the format required by the spreadsheets.values.append method:

array (ListValue format):

The data that was read or to be written. This is an array of arrays, the outer array representing all the data and each inner array representing a major dimension. Each item in the inner array corresponds with one cell.

For output, empty trailing rows and columns will not be included.

For input, supported value types are: bool, string, and double. Null values will be skipped. To set a cell to an empty value, set the string value to an empty string.

I'm not 100% sure if the output is the same, but to try and emulate what you're trying I used the python package xlrd to get the values from the Excel file you provided like so:

workbook = xlrd.open_workbook("twitter.xlsx")
sheet = workbook.sheet_by_index(0)
data = [sheet.row_values(rowx) for rowx in range(sheet.nrows)]

And, as in the screenshot you provided in a comment (seen below):

I had the same response. Scrolling up, the error was due to a bad request:

googleapiclient.errors.HttpError: <HttpError 400 when requesting https://sheets.googleapis.com/v4/spreadsheets/XXXXX/values/Sheet1%21A1:append?alt=json&valueInputOption=RAW returned "Invalid value at 'data.values' (type.googleapis.com/google.protobuf.ListValue)..."

specifically, Invalid value at 'data.values'. You will need to adhere to the Google Sheets API request specification for this method.

References:

• Method: spreadsheets.values.append | Sheets API - Response body
• Worksheet.Range property (Excel) | Microsoft Docs
• REST Resource: spreadsheets.values | Sheets API - Resource: ValueRange
• xlrd documentation - xlrd 1.1.0 documentation

TypeError: Object of type 'datetime' is not JSON serializable (with serialize function)

See below (using custom JSON encoder)

See http://flask.pocoo.org/snippets/119/ as well

import datetime
import json


class DateTimeEncoder(json.JSONEncoder):
    def default(self, z):
        if isinstance(z, datetime.datetime):
            return (str(z))
        else:
            return super().default(z)


my_dict = {'date': datetime.datetime.now()}

print(json.dumps(my_dict,cls=DateTimeEncoder))

output

{"date": "2019-06-12 15:44:14.978766"}

datetime.datetime is not JSON serializable

JSON doesn't have a default datetime type, so this is why Python can't handle it automatically. So you need to make the datetime into a string one way or another. I think the best way is to write a custom handler to help the json module.

import datetime
import json

def datetime_handler(x):
    if isinstance(x, datetime.datetime):
        return x.isoformat()
    raise TypeError("Unknown type")

json.dumps(data, default=datetime_handler)

How to overcome TypeError: Object of type datetime is not JSON serializable

For this line:

PrefFile["Preferences"]["Quotas"]["Reset time"] = data_option

Change to this or similar:

PrefFile["Preferences"]["Quotas"]["Reset time"] = data_option.isoformat()

https://docs.python.org/library/datetime.html#datetime.datetime.isoformat

How to overcome datetime.datetime not JSON serializable?

Updated for 2018

The original answer accommodated the way MongoDB "date" fields were represented as:

{"$date": 1506816000000}

If you want a generic Python solution for serializing datetime to json, check out @jjmontes' answer for a quick solution which requires no dependencies.

---

As you are using mongoengine (per comments) and pymongo is a dependency, pymongo has built-in utilities to help with json serialization:

http://api.mongodb.org/python/1.10.1/api/bson/json_util.html

Example usage (serialization):

from bson import json_util
import json

json.dumps(anObject, default=json_util.default)

Example usage (deserialization):

json.loads(aJsonString, object_hook=json_util.object_hook)

---

Django

Django provides a native DjangoJSONEncoder serializer that deals with this kind of properly.

See https://docs.djangoproject.com/en/dev/topics/serialization/#djangojsonencoder

from django.core.serializers.json import DjangoJSONEncoder

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  cls=DjangoJSONEncoder
)

One difference I've noticed between DjangoJSONEncoder and using a custom default like this:

import datetime
import json

def default(o):
    if isinstance(o, (datetime.date, datetime.datetime)):
        return o.isoformat()

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  default=default
)

Is that Django strips a bit of the data:

 "last_login": "2018-08-03T10:51:42.990", # DjangoJSONEncoder 
 "last_login": "2018-08-03T10:51:42.990239", # default

So, you may need to be careful about that in some cases.

Object of type datetime is not JSON serializable error

As the error message says, datetime objects cannot be converted by json automatically to a string or a dictionary. It seems your view function is converting everything to json before sending it to the front end, or to rendering.

All you need to do is to explicitly convert the DateTime object to a string for your code to work:

for obj in objects_with_category_id_2:
        data =  {'label': str(obj.session_start),
                 'value': obj.input_input_value}
        dataSourceBar['data'].append(data) 

Or use the built in functions from datetime to format it. For ISO Format use .isoformat():

for obj in objects_with_category_id_2:
        data =  {'label': obj.session_start.isoformat(),
                 'value': obj.input_input_value}
        dataSourceBar['data'].append(data) 

If you want the date to have a different format, you can use the datetime.strftime(format) function, that takes a string containing the format of the resulting date string. Check the datetime package documentation: https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior

maybe something like this:

for obj in objects_with_category_id_2:
        data =  {'label': obj.session_start.strftime("%d.%m.%Y"),
                 'value': obj.input_input_value}
        dataSourceBar['data'].append(data) 

Good luck!

---

Related Topics