Question

I have several data frames that contain all of the same column names. I want to append them into a master data frame. I also want to create a column that denotes the original field and then flood it with the original data frames name. I have some code that works.

df_combine = df_breakfast.copy()
df_combine['X_ORIG_DF'] = 'Breakfast'
df_combine = df_combine.append(df_lunch, ignore_index=True)
df_combine['X_ORIG_DF'] = df_combine['X_ORIG_DF'].fillna('Lunch')
# Rinse and repeat

However, it seems inelegant. I was hoping someone could point me to a more elegant solution. Thank you in advance for your time!

Note: Edited to reflect comment!

Answer 1

I would definitely consider restructuring you data in a way the names can be accessed neatly rather than as variable names (if they must be separate to begin with).
For example a dictionary:

d = {'breakfast': df_breakfast, 'lunch': df_lunch}

Create a function to give each DataFrame a new column:

def add_col(df, col_name, col_entry):
    df = df.copy() # so as not to change df_lunch etc.
    df[col_name] = col_entry
    return df

and combine the list of DataFrame each with the appended column ('X_ORIG_DF'):

In [3]: df_combine = pd.DataFrame().append(list(add_col(v, 'X_ORIG_DF', k)
                                           for k, v in d.items()))
Out[3]: 
   0  1  X_ORIG_DF
0  1  2      lunch
1  3  4      lunch
0  1  2  breakfast
1  3  4  breakfast

In this example: df_lunch = df_breakfast = pd.DataFrame([[1, 2], [3, 4]]).