Question

I would like to filter out particular columns with a regex and discard others. For example, if I had the following column names:

date
mem_total
cpu.usagemhz.average_0
cpu.usagemhz.average_1
cpu.usagemhz.average_2

I would like to capture only columns that begin with “cpu.usage.mhz.average”

Is their a particular function of text::csv that will help me do a quick check of the column names?

Thanks!
JD

* Update **

I tried jimtut answer and it is extremely close to what I am looking for. Thanks Again Everyone!

Here is the code from jimtut with one small edit on the print statement at the bottom. I added the print $colCount just to see what was going on with the data;

use Text::CSV;

my $file = "foo.csv";
my $pattern = ".*In";
open(F, $file) or warn "Warning! Unable to open $file\n";

my $lineCount = 0;
my %desiredColumns;
while(<F>) {
  $lineCount++;
  my $csv = Text::CSV->new();
  my $status = $csv->parse($_); # should really check this!
  my @fields = $csv->fields();
  my $colCount = 0;

  if ($lineCount == 1) {
    # Let's look at the column headings.
    foreach my $field (@fields) {
      $colCount++;
      if ($field =~ m/$pattern/) {
        # This heading matches, save the column #.
        $desiredColumns{$colCount} = 1;
      }
    }
  }
  else {
    # Not the header row.  Parse the body of the file.
    foreach my $field (@fields) {
      $colCount++;
      if (exists $desiredColumns{$colCount}) {
        # This is one of the desired columns.
        # Do whatever you want to do with this column!
        print "$colCount\t$field\n";
      }
    }
  }
}
close(F);

Here is the results

colCount |  $field

12      565
13      73
14      36
15      32
16      127
17      40
18      32
19      42
20      171
12      464
13      62
14      32
15      24
16      109
17      21
18      19
19      39
20      150
12      515
13      76
14      28
15      30
16      119
17      15
18      25
19      46
20      169
12      500
13      71
14      30
15      28
16      111
17      20
18      18
19      40
20      167

I would like to add this data to individual arrays or hashes. what do you think? something like…

foreach column {
check to see if a hash already exists with that column number. If not then create hash.
}

Then go through each field and add the field data to the appropriate hash.

Do you think this is the right way to go about solving this?

Answer 1

No, not a specific function in Text::CSV. I would do something like this:

use Text::CSV;

my $file = "foo.csv";
my $pattern = "cpu.usage.mhz.average.*";
open(F, $file) or die "Unable to open $file: $!\n";

my $lineCount = 0;
my %desiredColumns;
my %columnContents;

while(<F>) {
  $lineCount++;
  my $csv = Text::CSV->new();
  my $status = $csv->parse($_); # should really check this!
  my @fields = $csv->fields();
  my $colCount = 0;

  if ($lineCount == 1) {
    # Let's look at the column headings.
    foreach my $field (@fields) {
      $colCount++;
      if ($field =~ m/$pattern/) {
        # This heading matches, save the column #.
        $desiredColumns{$colCount} = 1;
      }
    }
  }
  else {
    # Not the header row.  Parse the body of the file.
    foreach my $field (@fields) {
      $colCount++;
      if (exists $desiredColumns{$colCount}) {
        # This is one of the desired columns.
        # Do whatever you want to do with this column!
        push(@{$columnContents{$colCount}}, $field);
      }
    }
  }
}
close(F);

foreach my $key (sort keys %columnContents) {
  print "Column $key: " . join(",", @{$columnContents{$key}}) . "\n\n";
}

Hope that helps! I’m sure someone can write that in a Perl one-liner, but that’s easier (for me) to read…