Question

let’s say I have a folder with a thousand files named File1.csv, File2.csv, …, File1000.csv and every one contains some lines of semicolon-separated (;) data values.

I need a Perl script to “merge” all the csv files in that folder into one by appending each file one after another and also adding another data column at the end of each line with name of the file being currently processed (without the ending, e.g. “;File2”).

Steve

Answer 1

Text::CSV can be used to parse CSV. The following script is to be run from within the directory containing the CSV files. It is not recursive (a glob has been used). If you require it to recursively find files, you can use the File::Find Perl module.

#!/usr/bin/env perl

use strict;
use warnings;

use Text::CSV;

my $csv = Text::CSV->new( { 'sep_char' => ';' } );

open my $fho, '>', 'combined.csv' or die "Error opening file: $!";

while ( my $file = <*.csv> ) {
    open my $fhi, '<', $file or die "Error opening file: $!";
    ( my $last_field = $file ) =~ s/\.[^\.]+$//;  # Strip the file extension off

    while ( my $row = $csv->getline($fhi) ) {
        $csv->combine( @$row, $last_field );  # Construct new row by appending the file name without the extension
        print $fho $csv->string, "\n";        # Write the combined string to combined.csv
    }
}