Question

Let’s say I have a tab-delimited text file that contains data arranged in columns (with headers).

It is possible that different columns may be ‘stacked’ into a ‘worksheet’-like arrangement, i.e. there is some divider (that may or may not be known ahead of time) that allows different columns to be arranged vertically.

Is there a Perl module that facilitates parsing of columnar data in this text file into a data structure (e.g., a hash table with the key being the column header, and the value being an array of column data scalars)?

EDIT By ‘stacked’, I mean that a column of text may include multiple, individual ‘vectors’ of data, each with different headers and different lengths. Admittedly, this complicates parsing.

EDIT I’m honestly not sure where the confusion is. Nonetheless, here’s an example:

header_one\theader_three data_1\tdata_7 data_2\tdata_8 data_3\tdata_9 \tdata_10 header_two\tdata_11 data_4\theader_four data_5\tdata_12 data_6\tdata_13 \tdata_14 

The script would turn this into a hash table with four keys: header_one, header_two, header_three, and header_four, each key referencing an array reference pointing to the data_n elements underneath the header.

Answer 1

I think this is close to what you are talking about. If the number of columns changes then the input is treated as if it were a different table. This code could be easily modified to recognize some other marker (such as a line of equal signs) instead of using the column counts.

#!/usr/bin/perl  use strict; use warnings;  use Text::CSV_XS;  #setup the parser, here we want tab separated and we allow #loose quoting, so qq/foo\t'bar\tbaz'\tquux/ is  #('foo', 'bar\tbaz', 'quux') my $p = Text::CSV_XS->new(     {         sep_char           => '\t',         allow_loose_quotes => 1,     } );  my @stacked; my $cur = 0; while (<>) {     $p->parse($_) or die $p->error_input;     my @rec = $p->fields;     #normal case, just add the record to the last     #section in @stacked     if (@rec == $cur) {         push @{$stacked[-1]}, \@rec;         next;     }     #if the number of columns don't match then     #we have a new section     push @stacked, [\@rec];     $cur = @rec; #set the new number of columns }  for my $table (@stacked) {     print 'header: ', join('::', @{$table->[0]}), '\n';     for my $i (1 .. $#$table) {         print 'data: ', join('::', @{$table->[$i]}), '\n';     }     print '\n'; }