How to Shift a Column in Pandas Dataframe

How to shift a column in Pandas DataFrame

In [18]: a
Out[18]: 
   x1  x2
0   0   5
1   1   6
2   2   7
3   3   8
4   4   9

In [19]: a['x2'] = a.x2.shift(1)

In [20]: a
Out[20]: 
   x1  x2
0   0 NaN
1   1   5
2   2   6
3   3   7
4   4   8

move column in pandas dataframe

You can rearrange columns directly by specifying their order:

df = df[['a', 'y', 'b', 'x']]

In the case of larger dataframes where the column titles are dynamic, you can use a list comprehension to select every column not in your target set and then append the target set to the end.

>>> df[[c for c in df if c not in ['b', 'x']] 
       + ['b', 'x']]
   a  y  b   x
0  1 -1  2   3
1  2 -2  4   6
2  3 -3  6   9
3  4 -4  8  12

To make it more bullet proof, you can ensure that your target columns are indeed in the dataframe:

cols_at_end = ['b', 'x']
df = df[[c for c in df if c not in cols_at_end] 
        + [c for c in cols_at_end if c in df]]

Shift column in pandas dataframe up by one?

In [44]: df['gdp'] = df['gdp'].shift(-1)

In [45]: df
Out[45]: 
   y  gdp  cap
0  1    3    5
1  2    7    9
2  8    4    2
3  3    7    7
4  6  NaN    7

In [46]: df[:-1]                                                                                                                                                                                                                                                                                                               
Out[46]: 
   y  gdp  cap
0  1    3    5
1  2    7    9
2  8    4    2
3  3    7    7

How to shift all columns through a loop in Pandas?

You can access the ordered sequence of column names using df.columns. From there it is quick to iterate over them and shift the column with the help of enumerate. Since your first column is your timestamp, set the enumerate start value to -1 to align the column shift.

for i, col in enumerate(df.columns, -1):
    if i > 0:
        df[col] = df[col].shift(i)

Shift only one column in DataFrame

Do,

df['col2'] = df['col2'].shift(5)

You will get the following output

   col1  col2
0     A   NaN
1     B   NaN
2     C   NaN
3     D   NaN
4     E   NaN
5     F   5.0
6     G   3.0
7     H   8.0
8     I   2.0
9     J   9.0
10    K   9.0
11    L   4.0
12    M   9.0

But from the updated Q, I can see that the col1 should be extended as well. So do this instead,

df['col2'].index = df['col2'].index + 5
pd.concat([df['col1'], df['col2']], axis=1)

   col1  col2
0     A   NaN
1     B   NaN
2     C   NaN
3     D   NaN
4     E   NaN
5     F   5.0
6     G   3.0
7     H   8.0
8     I   2.0
9     J   9.0
10    K   9.0
11    L   4.0
12    M   9.0
13  NaN   3.0
14  NaN   5.0
15  NaN   7.0
16  NaN   3.0
17  NaN   7.0

Shift multiple daily value columns forward by one year in Pandas

Code below seems works:

df[['B', 'C']] = df[['B', 'C']].shift(freq=pd.DateOffset(years=1))
print(df)

Out:

             A     B     C
2013-01-01  85   NaN   NaN
2013-01-02  94   NaN   NaN
2013-01-03  62   NaN   NaN
2013-01-04  21   NaN   NaN
2013-01-05  12   NaN   NaN
         ..   ...   ...
2014-12-17  38  33.0  79.0
2014-12-18  67  24.0  53.0
2014-12-19  27  54.0  39.0
2014-12-20  18  68.0  80.0
2014-12-21  90  65.0  65.0

---

Related Topics