Convert Dataframe Column Type from String to Datetime

Convert DataFrame column type from string to datetime

The easiest way is to use to_datetime:

df['col'] = pd.to_datetime(df['col'])

It also offers a dayfirst argument for European times (but beware this isn't strict).

Here it is in action:

In [11]: pd.to_datetime(pd.Series(['05/23/2005']))
Out[11]:
0   2005-05-23 00:00:00
dtype: datetime64[ns]

You can pass a specific format:

In [12]: pd.to_datetime(pd.Series(['05/23/2005']), format="%m/%d/%Y")
Out[12]:
0   2005-05-23
dtype: datetime64[ns]

Convert the column type from string to datetime format in Pandas dataframe

The above error was resolved with the below code, as there was some mismatch in the format of the date.

DOB_Permits["job_start_date"] = pd.to_datetime(DOB_Permits["job_start_date"], errors='coerce', format="%y%m%d")

Converting a datetime column to a string column

If you're using version 0.17.0 or higher then you can call this using .dt.strftime which is vectorised:

all_data['Order Day new'] = all_data['Order Day new'].dt.strftime('%Y-%m-%d')

** If your pandas version is older than 0.17.0 then you have to call apply and pass the data to strftime:

In [111]:

all_data = pd.DataFrame({'Order Day new':[dt.datetime(2014,5,9), dt.datetime(2012,6,19)]})
print(all_data)
all_data.info()
  Order Day new
0    2014-05-09
1    2012-06-19
<class 'pandas.core.frame.DataFrame'>
Int64Index: 2 entries, 0 to 1
Data columns (total 1 columns):
Order Day new    2 non-null datetime64[ns]
dtypes: datetime64[ns](1)
memory usage: 32.0 bytes

In [108]:

all_data['Order Day new'] = all_data['Order Day new'].apply(lambda x: dt.datetime.strftime(x, '%Y-%m-%d'))
all_data
Out[108]:
  Order Day new
0    2014-05-09
1    2012-06-19
In [109]:

all_data.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 2 entries, 0 to 1
Data columns (total 1 columns):
Order Day new    2 non-null object
dtypes: object(1)
memory usage: 32.0+ bytes

You can't call strftime on the column as it doesn't understand Series as a param hence the error

Convert Pandas Column to DateTime

Use the to_datetime function, specifying a format to match your data.

raw_data['Mycol'] =  pd.to_datetime(raw_data['Mycol'], format='%d%b%Y:%H:%M:%S.%f')

Convert String Column directly to Date format (not Datetime) in Pandas DataFrame

pandas.DataFrame.apply is essentially a native python for loop.

pandas.to_datetime is a vectorized function, meaning it's meant to operate on sequences/lists/arrays/series by doing the inner loop in C

If we start with a larger dataframe:

import pandas
df = pandas.DataFrame({'a': ['2020-01-02', '2020-01-02'] * 5000})

And then do (in a jupyter notebook)

%%timeit
df['a'].apply(pandas.to_datetime).dt.date

We get a pretty slow result:

1.03 s ± 48.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

But if we rearrange just slightly to pass the entire column:

%%timeit
pandas.to_datetime(df['a']).dt.date

We get a much faster result:

6.07 ms ± 232 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Convert Pandas column to datetime for a specific datetime format

You can use pd.to_datetime for this.

import pandas as pd
df = pd.DataFrame({"date": ["2011-06-12T01:17:56"]})

Conversion using map method:

df["date"].map(pd.to_datetime)

or

Conversion using apply method:

df["date"].apply(pd.to_datetime)

or

Conversion using function on column series:

df["date"] = pd.to_datetime(df["date"])

Python/Pandas convert string to time only

These two lines:

dfc['Time_of_Sail'] = pd.to_datetime(dfc['Time_of_Sail'])
dfc['Time_of_Sail'] = [time.time() for time in dfc['Time_of_Sail']]

Can be written as:

dfc['Time_of_Sail'] = pd.to_datetime(dfc['Time_of_Sail'],format= '%H:%M:%S' ).dt.time

Changing a column type from a Pandas dataframe to datetime changes the values on the other columns

Have you tried adding the infer_date_format option to pandas.to_datetime?

Something like this:

dique_aforador['Fecha y Hora'] = pd.to_datetime(dique_aforador['Fecha y Hora'], infer_datetime_format=True)

Pandas can take the first value in your date field and infer the remaining from it

Convert a series of type string to type Date (not Datetime)

Try pandas.to_datetime() and extract the date:

import pandas as pd
df = pd.DataFrame({'Dates':['2020-01-22', '2021-05-06', '2022-01-01']})
df.Dates.apply(lambda x: pd.to_datetime(x).date())

outputs:

0    2020-01-22
1    2021-05-06
2    2022-01-01
Name: Dates, dtype: object

How to convert string to datetime format in pandas python?

Use to_datetime, there is no need for a format string the parser is man/woman enough to handle it:

In [51]:
pd.to_datetime(df['I_DATE'])

Out[51]:
0   2012-03-28 14:15:00
1   2012-03-28 14:17:28
2   2012-03-28 14:50:50
Name: I_DATE, dtype: datetime64[ns]

To access the date/day/time component use the dt accessor:

In [54]:
df['I_DATE'].dt.date

Out[54]:
0    2012-03-28
1    2012-03-28
2    2012-03-28
dtype: object

In [56]:    
df['I_DATE'].dt.time

Out[56]:
0    14:15:00
1    14:17:28
2    14:50:50
dtype: object

You can use strings to filter as an example:

In [59]:
df = pd.DataFrame({'date':pd.date_range(start = dt.datetime(2015,1,1), end = dt.datetime.now())})
df[(df['date'] > '2015-02-04') & (df['date'] < '2015-02-10')]

Out[59]:
         date
35 2015-02-05
36 2015-02-06
37 2015-02-07
38 2015-02-08
39 2015-02-09

---

Related Topics