Question

How do you read a csv file into a two dimensional array in BASH? The script needs to be dynamic enough where it can take csv files with variable number of rows and columns.

For example, if I have a csv file that looks like

AVERAGE     STDEV     MAX
17          18        19

or

AVERAGE     STDEV     MAX     MIN
17          18        19      1

Answer 1

One way to simulate a two-dimensional array is to keep the rows as strings in a one-dimensional array and unpack them at each iteration. You will have to choose a suitable delimiter that doesn’t appear in the data. Since you mention CSV, I’ll use a comma, but this won’t be smart enough to handle data like this with embedded commas:

name, start date, visits, games, balance
"Williamson, Dennis", "January 11, 2007", 12, 42, 17000

Here’s a simple example of iterating over the values in a simulated two-dimensional array:

# avg, stddev, max, min
data_array=(
            "17,18,19,1"
            "12,14,16,2"
            "6,8,10,3"
            )

saveIFS=$IFS

for row in ${data_array[@]}
do
    IFS=","
    cols=($row)
    IFS=$saveIFS
    for col in ${cols[@]}
    do
        newval=$(do_something $col)
    done
done

Making changes to the contents of the array is possible:

rowidx=2
colidx=2
IFS=","
cols=(${data_array[rowidx]})
cols[colidx]=$some_value
data_array[rowidx]="${cols[*]}"
IFS=$saveIFS

As you can see, it gets complicated fast and there are lots of gotchas which I haven’t mentioned. Use Python.