get UTC timestamp in python with datetime
Naïve datetime
versus aware datetime
Default datetime
objects are said to be "naïve": they keep time information without the time zone information. Think about naïve datetime
as a relative number (ie: +4
) without a clear origin (in fact your origin will be common throughout your system boundary).
In contrast, think about aware datetime
as absolute numbers (ie: 8
) with a common origin for the whole world.
Without timezone information you cannot convert the "naive" datetime towards any non-naive time representation (where does +4
targets if we don't know from where to start ?). This is why you can't have a datetime.datetime.toutctimestamp()
method. (cf: http://bugs.python.org/issue1457227)
To check if your datetime
dt
is naïve, check dt.tzinfo
, if None
, then it's naïve:
datetime.now() ## DANGER: returns naïve datetime pointing on local time
datetime(1970, 1, 1) ## returns naïve datetime pointing on user given time
I have naïve datetimes, what can I do ?
You must make an assumption depending on your particular context:
The question you must ask yourself is: was your datetime
on UTC ? or was it local time ?
If you were using UTC (you are out of trouble):
import calendar
def dt2ts(dt):
"""Converts a datetime object to UTC timestamp
naive datetime will be considered UTC.
"""
return calendar.timegm(dt.utctimetuple())If you were NOT using UTC, welcome to hell.
You have to make your
datetime
non-naïve prior to using the former
function, by giving them back their intended timezone.You'll need the name of the timezone and the information about
if DST was in effect when producing the target naïve datetime (the
last info about DST is required for cornercases):import pytz ## pip install pytz
mytz = pytz.timezone('Europe/Amsterdam') ## Set your timezone
dt = mytz.normalize(mytz.localize(dt, is_dst=True)) ## Set is_dst accordinglyConsequences of not providing
is_dst
:Not using
is_dst
will generate incorrect time (and UTC timestamp)
if target datetime was produced while a backward DST was put in place
(for instance changing DST time by removing one hour).Providing incorrect
is_dst
will of course generate incorrect
time (and UTC timestamp) only on DST overlap or holes. And, when
providing
also incorrect time, occuring in "holes" (time that never existed due
to forward shifting DST),is_dst
will give an interpretation of
how to consider this bogus time, and this is the only case where.normalize(..)
will actually do something here, as it'll then
translate it as an actual valid time (changing the datetime AND the
DST object if required). Note that.normalize()
is not required
for having a correct UTC timestamp at the end, but is probably
recommended if you dislike the idea of having bogus times in your
variables, especially if you re-use this variable elsewhere.and AVOID USING THE FOLLOWING: (cf: Datetime Timezone conversion using pytz)
dt = dt.replace(tzinfo=timezone('Europe/Amsterdam')) ## BAD !!
Why? because
.replace()
replaces blindly thetzinfo
without
taking into account the target time and will choose a bad DST object.
Whereas.localize()
uses the target time and youris_dst
hint
to select the right DST object.
OLD incorrect answer (thanks @J.F.Sebastien for bringing this up):
Hopefully, it is quite easy to guess the timezone (your local origin) when you create your naive datetime
object as it is related to the system configuration that you would hopefully NOT change between the naive datetime object creation and the moment when you want to get the UTC timestamp. This trick can be used to give an imperfect question.
By using time.mktime
we can create an utc_mktime
:
def utc_mktime(utc_tuple):
"""Returns number of seconds elapsed since epoch
Note that no timezone are taken into consideration.
utc tuple must be: (year, month, day, hour, minute, second)
"""
if len(utc_tuple) == 6:
utc_tuple += (0, 0, 0)
return time.mktime(utc_tuple) - time.mktime((1970, 1, 1, 0, 0, 0, 0, 0, 0))
def datetime_to_timestamp(dt):
"""Converts a datetime object to UTC timestamp"""
return int(utc_mktime(dt.timetuple()))
You must make sure that your datetime
object is created on the same timezone than the one that has created your datetime
.
This last solution is incorrect because it makes the assumption that the UTC offset from now is the same than the UTC offset from EPOCH. Which is not the case for a lot of timezones (in specific moment of the year for the Daylight Saving Time (DST) offsets).
Python date object Convert to UTC timestamp at midnight
assuming your date object d
is in UTC, you could simply add midnight time (00:00:00) and timezone info UTC and then apply timestamp()
method to get the desired timestamp. Example:
import datetime as dt
d = dt.date(2020, 3, 29)
ts_utc = dt.datetime.combine(d, dt.time(0,0,0), tzinfo=dt.timezone.utc).timestamp()
print(ts_utc)
# 1585440000.0
check = dt.datetime.utcfromtimestamp(ts_utc)
print(check)
# 2020-03-29 00:00:00
Note: it is important to set the timezone, otherwise, the timestamp()
method assumes that the input is in the timezone that the machine you run this code on is set to!
How to convert local time string to UTC?
Thanks @rofly, the full conversion from string to string is as follows:
time.strftime("%Y-%m-%d %H:%M:%S",
time.gmtime(time.mktime(time.strptime("2008-09-17 14:04:00",
"%Y-%m-%d %H:%M:%S"))))
My summary of the time
/calendar
functions:
time.strptime
string --> tuple (no timezone applied, so matches string)
time.mktime
local time tuple --> seconds since epoch (always local time)
time.gmtime
seconds since epoch --> tuple in UTC
and
calendar.timegm
tuple in UTC --> seconds since epoch
time.localtime
seconds since epoch --> tuple in local timezone
Converting python datetime to timestamp and back in UTC still uses local timezone
Hmm I found the answer here: How to specify time zone (UTC) when converting to Unix time? (Python)
In [101]: ts = calendar.timegm(datetime(2010, 7, 1, tzinfo=pytz.utc).timetuple())
In [102]: datetime.fromtimestamp(ts, tz=pytz.utc)
Out[102]: datetime.datetime(2010, 7, 1, 0, 0, tzinfo=<UTC>)
Convert date to timestamp in Python
import time
timestamp = time.mktime(time.strptime('2015-10-20 22:24:46', '%Y-%m-%d %H:%M:%S'))
For more on the format string with all the % symbols, see python's time library.
Convert datetime to Unix timestamp and convert it back in python
What you missed here is timezones.
Presumably you've five hours off UTC, so 2013-09-01T11:00:00 local and 2013-09-01T06:00:00Z are the same time.
You need to read the top of the datetime
docs, which explain about timezones and "naive" and "aware" objects.
If your original naive datetime was UTC, the way to recover it is to use utcfromtimestamp
instead of fromtimestamp
.
On the other hand, if your original naive datetime was local, you shouldn't have subtracted a UTC timestamp from it in the first place; use datetime.fromtimestamp(0)
instead.
Or, if you had an aware datetime object, you need to either use a local (aware) epoch on both sides, or explicitly convert to and from UTC.
If you have, or can upgrade to, Python 3.3 or later, you can avoid all of these problems by just using the timestamp
method instead of trying to figure out how to do it yourself. And even if you don't, you may want to consider borrowing its source code.
(And if you can wait for Python 3.4, it looks like PEP 341 is likely to make it into the final release, which means all of the stuff J.F. Sebastian and I were talking about in the comments should be doable with just the stdlib, and working the same way on both Unix and Windows.)
Python convert timestamps with specific timezone to datetime in UTC
I am not sure what timestamp_received
is, but I think what you want is utcfromtimestamp()
import pytz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_naive_utc = datetime.utcfromtimestamp(timestamp_received)
return dt_naive_utc.replace(tzinfo=pytz.utc)
For completeness, here is another way to accomplish the same thing by referencing python-dateutil
's tzlocal
time zone:
from dateutil import tz
from datetime import datetime
def convert_timestamp_in_datetime_utc(timestamp_received):
dt_local = datetime.fromtimestamp(timestamp_received, tz.tzlocal())
if tz.datetime_ambiguous(dt_local):
raise AmbiguousTimeError
if tz.datetime_imaginary(dt_local):
raise ImaginaryTimeError
return dt_local.astimezone(tz.tzutc())
class AmbiguousTimeError(ValueError):
pass
class ImaginaryTimeError(ValueError):
pass
(I added in the AmbiguousTimeError
and ImaginaryTimeError
conditions to mimic the pytz
interface.) Note that I'm including this just in case you have a similar problem that needs to make reference to the local time zone for some reason - if you have something that will give you the right answer in UTC, it's best to use that and then use astimezone
to get it into whatever local zone you want it in.
How it works
Since you expressed that you were still a bit confused about how this works in the comments, I thought I would clarify why this works. There are two functions that convert timestamps to datetime.datetime
objects, datetime.datetime.fromtimestamp(timestamp, tz=None)
and datetime.datetime.utcfromtimestamp(timestamp)
:
utcfromtimestamp(timestamp)
will give you a naivedatetime
that represents the time in UTC. You can then dodt.replace(tzinfo=pytz.utc)
(or any otherutc
implementation -datetime.timezone.utc
,dateutil.tz.tzutc()
, etc) to get an aware datetime and convert it to whatever time zone you want.fromtimestamp(timestamp, tz=None)
, whentz
is notNone
, will give you an awaredatetime
equivalent toutcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(tz)
. Iftz
isNone
, instead of converting too the specified time zone, it converts to your local time (equivalent todateutil.tz.tzlocal()
), and then returns a naivedatetime
.
Starting in Python 3.6, you can use datetime.datetime.astimezone(tz=None)
on naive datetimes, and the time zone will be assumed to be system local time. So if you're developing a Python >= 3.6 application or library, you can use datetime.fromtimestamp(timestamp).astimezone(whatever_timezone)
or datetime.utcfromtimestamp(timestamp).replace(tzinfo=timezone.utc).astimezone(whatever_timezone)
as equivalents.
Convert UTC datetime string to local datetime
If you don't want to provide your own tzinfo
objects, check out the python-dateutil library. It provides tzinfo
implementations on top of a zoneinfo (Olson) database such that you can refer to time zone rules by a somewhat canonical name.
from datetime import datetime
from dateutil import tz
# METHOD 1: Hardcode zones:
from_zone = tz.gettz('UTC')
to_zone = tz.gettz('America/New_York')
# METHOD 2: Auto-detect zones:
from_zone = tz.tzutc()
to_zone = tz.tzlocal()
# utc = datetime.utcnow()
utc = datetime.strptime('2011-01-21 02:37:21', '%Y-%m-%d %H:%M:%S')
# Tell the datetime object that it's in UTC time zone since
# datetime objects are 'naive' by default
utc = utc.replace(tzinfo=from_zone)
# Convert time zone
central = utc.astimezone(to_zone)
Edit Expanded example to show strptime
usage
Edit 2 Fixed API usage to show better entry point method
Edit 3 Included auto-detect methods for timezones (Yarin)
Related Topics
Does Python Have a Ternary Conditional Operator
Unboundlocalerror on Local Variable When Reassigned After First Use
Converting String "Jun 1 2005 1:33Pm" into Datetime
Difference Between _Str_ and _Repr_
How to Unnest (Explode) a Column in a Pandas Dataframe, into Multiple Rows
Get the Data Received in a Flask Request
How to Pandas Group-By to Get Sum
What Is the Meaning of Single and Double Underscore Before an Object Name
What Does "List Comprehension" and Similar Mean? How Does It Work and How to Use It
Importing Files from Different Folder
Lazy Method For Reading Big File in Python
Does Pandas Iterrows Have Performance Issues
What's With the Integer Cache Maintained by the Interpreter
How to Randomly Select an Item from a List