Calculate Time Difference Between Two Pandas Columns in Hours and Minutes
Pandas timestamp differences returns a datetime.timedelta object. This can easily be converted into hours by using the *as_type* method, like so
import pandas
df = pandas.DataFrame(columns=['to','fr','ans'])
df.to = [pandas.Timestamp('2014-01-24 13:03:12.050000'), pandas.Timestamp('2014-01-27 11:57:18.240000'), pandas.Timestamp('2014-01-23 10:07:47.660000')]
df.fr = [pandas.Timestamp('2014-01-26 23:41:21.870000'), pandas.Timestamp('2014-01-27 15:38:22.540000'), pandas.Timestamp('2014-01-23 18:50:41.420000')]
(df.fr-df.to).astype('timedelta64[h]')
to yield,
0 58
1 3
2 8
dtype: float64
How to calculate the time difference in a data frame in python?
Assuming this is your dataset
data = {'date': ['2020/06/24', '2020/06/25', '2020/06/27', '2020/06/30'],
'time': ['23:00:28', '09:10:55', '03:42:58','16:45:51']}
df = pd.DataFrame(data)
print(df)
date time
0 2020/06/24 23:00:28
1 2020/06/25 09:10:55
2 2020/06/27 03:42:58
3 2020/06/30 16:45:51
You can use pandas .diff
after converting your data to proper datetime format using pd.to_datetime
df['date_time'] = pd.to_datetime(df['date'] + ' ' + df['time'])
df['time_diff'] = df['date_time'].diff()
print(df)
date time date_time time_diff
0 2020/06/24 23:00:28 2020-06-24 23:00:28 NaT
1 2020/06/25 09:10:55 2020-06-25 09:10:55 0 days 10:10:27
2 2020/06/27 03:42:58 2020-06-27 03:42:58 1 days 18:32:03
3 2020/06/30 16:45:51 2020-06-30 16:45:51 3 days 13:02:53
Calculate the time difference between two hh:mm columns in a pandas dataframe
You can use the technique (a+x)%x
with a timedelta
of 24h (or 1d, same)
- the
+ timedelta(hours=24)
makes all values becomes positive - the
% timedelta(hours=24)
makes the ones above24h
back of24h
df['duration'] = (pd.to_datetime(df['End']) - pd.to_datetime(df['Start']) + timedelta(hours=24)) \
% timedelta(hours=24)
Gives
Start End duration
0 11:15 15:00 0 days 03:45:00
1 22:30 2:00 0 days 03:30:00
How do I find the time difference between two datetime objects in python?
>>> import datetime
>>> first_time = datetime.datetime.now()
>>> later_time = datetime.datetime.now()
>>> difference = later_time - first_time
datetime.timedelta(0, 8, 562000)
>>> seconds_in_day = 24 * 60 * 60
>>> divmod(difference.days * seconds_in_day + difference.seconds, 60)
(0, 8) # 0 minutes, 8 seconds
Subtracting the later time from the first time difference = later_time - first_time
creates a datetime object that only holds the difference.
In the example above it is 0 minutes, 8 seconds and 562000 microseconds.
How to calculate time difference between two pandas column
You need omit pd.Timedelta
, because difference of times return timedeltas:
df_time['td'] = df_time['stop']-df_time['start']
print (df_time)
start stop td
0 2015-11-04 10:12:00 2015-11-06 06:38:00 1 days 20:26:00
1 2015-11-04 10:23:00 2015-11-05 08:30:00 0 days 22:07:00
2 2015-11-04 14:01:00 2015-11-17 10:34:00 12 days 20:33:00
EDIT: Another solution is subtract numpy arrays:
df_time['td'] = df_time['stop'].values - df_time['start'].values
print (df_time)
start stop td
0 2015-11-04 10:12:00 2015-11-06 06:38:00 1 days 20:26:00
1 2015-11-04 10:23:00 2015-11-05 08:30:00 0 days 22:07:00
2 2015-11-04 14:01:00 2015-11-17 10:34:00 12 days 20:33:00
Calculate time difference between two dates in the same column in Pandas
Depending on how you'd care to store the differences, either
df = pd.DataFrame(data=['01-01-2006 00:53:00',
'01-01-2006 01:53:00',
'01-01-2006 02:53:00',
'01-01-2006 03:53:00',
'01-01-2006 04:53:00'],
columns=['DATE'])
df['DATE'] = pd.to_datetime(df['DATE'])
df['time_interval'] = df['DATE'].diff().fillna(timedelta(0)).apply(lambda x: x.total_seconds() / 60)
to get
DATE time_interval
0 2006-01-01 00:53:00 0.0
1 2006-01-01 01:53:00 60.0
2 2006-01-01 02:53:00 60.0
3 2006-01-01 03:53:00 60.0
4 2006-01-01 04:53:00 60.0
or alternatively
df['time_interval'] = df['DATE'].diff().shift(-1).fillna(timedelta(0)).apply(lambda x: x.total_seconds() / 60)
to get
DATE time_interval
0 2006-01-01 00:53:00 60.0
1 2006-01-01 01:53:00 60.0
2 2006-01-01 02:53:00 60.0
3 2006-01-01 03:53:00 60.0
4 2006-01-01 04:53:00 0.0
Related Topics
How to Find and Replace a Part of a Value in Json File
Splitting a Phone Number into a List of Digits: Python
Find the Index of the First Digit in a String
Pandas Convert from Datetime to Integer Timestamp
How to Get One Key and Value from a Json in Python
How to Find a Minimum Value in a 2D Array Without Using Numpy or Flattened in Python
How to Convert Data from Txt Files to Excel Files Using Python
Reading a CSV File into Pandas Dataframe With Quotation in Some Entries
Easiest Way to Convert Two Columns to Python Dictionary
How to Name a File by a Variable Name in Python
Can Anyone Explain Me What This Python 3 Command Do
Importing Local Module (Python Script) in Airflow Dag
Python - Regex Match Multiple Patterns in Multiple Lines
Python: How to Turn CSV Data in to Array
How to Change Python Version in Anaconda Spyder
Save Variables in Every Iteration of for Loop and Load Them Later