Question

Suppose I have a Pandas dataframe df has columns a,b,c,d…z . And I want to: df.groupby('a').apply(my_func()) for columns d-z, while leave column 'b' & 'c' unchanged . How to do that ?

I notice Pandas can apply different function to different column by passing a dict . But I have a long column list and just want parameters to set or tip to simply tell Pandas to bypass some columns and apply my_func() to rest of columns ? (Otherwise I have to build a long dict)

Answer 1

One simple (and general) approach is to create a view of the dataframe with the subset you are interested in (or, stated for your case, a view with all columns except the ones you want to ignore), and then use APPLY for that view.

In [116]: df
Out[116]: 
     a  b         c  d        f
0  one  3  0.493808  a      bob
1  two  8  0.150585  b    alice
2  one  6  0.641816  c  michael
3  two  5  0.935653  d      joe
4  one  1  0.521159  e     kate

Use your favorite methods to create the view you need. You could select a range of columns like so df_view = df.ix[:,'b':'d'], but the following might be more useful for your scenario:

#I want all columns except two 
cols = df.columns.tolist()    
mycols = [x for x in cols if not x in ['a','f']]
df_view = df[mycols]

Apply your function to that view. (Note this doesn’t yet change anything in df.)

In [158]: df_view.apply(lambda x: x /2)
Out[158]: 
   b         c   d
0  1  0.246904  20
1  4  0.075293  25
2  3  0.320908  28
3  2  0.467827  28
4  0  0.260579  24

Update the df using update()

In [156]: df.update(df_view.apply(lambda x: x/2))

In [157]: df
Out[157]: 
     a  b         c   d        f
0  one  1  0.246904  20      bob
1  two  4  0.075293  25    alice
2  one  3  0.320908  28  michael
3  two  2  0.467827  28      joe
4  one  0  0.260579  24     kate