Question

I have two CSV files with different number of columns and rows. The first CSV file has M columns and N rows, the second has H columns and G rows. Some of the columns have the same name.

I’d like to combine the two into data frame with following properties:

• N+G rows
• Union of (M, H) columns
• if column A is element of first CSV file but not of second, the data frame should contain the same values in first N entries of A as in first CSV, and for the rest (since there is no A data in second CSV) should be NA.

Here is an example:

CSV1
City, Population,
Zagreb, 700000,
Rijeka, 142000

CSV2
City, Area,
Split, 200.00
Osijek, 171.00
Dubrovnik, 143.35

I’d like build a data frame that looks like this:

City      Population   Area
Zagreb    700000       NA
Rijeka    142000       NA
Split     NA           200.00
Osijek    NA           171.00
Dubrovnik NA           143.35

Also what if instead two CSV files I had two data frames and wanted to do the same, for example if I loaded first csv to df1 and second one in df2 and then wanted to make a merge to df3 that would look like example above.

Answer 1

Why not try the concat function:

In [25]: df1
Out[25]: 
     City  Population
0  Zagreb      700000
1  Rijeka      142000

In [26]: df2
Out[26]: 
        City    Area
0      Split  200.00
1     Osijek  171.00
2  Dubrovnik  143.35

In [27]: pd.concat([df1,df2])
Out[27]: 
     Area       City  Population
0     NaN     Zagreb      700000
1     NaN     Rijeka      142000
0  200.00      Split         NaN
1  171.00     Osijek         NaN
2  143.35  Dubrovnik         NaN

In [28]: pd.concat([df1,df2], ignore_index=True)
Out[28]: 
     Area       City  Population
0     NaN     Zagreb      700000
1     NaN     Rijeka      142000
2  200.00      Split         NaN
3  171.00     Osijek         NaN
4  143.35  Dubrovnik         NaN

Note: the concat does have some additional options if you have slightly different requirements.