Question

Requirements

• I have a very large CSV file to read. (about 3 GB)
• I won’t need all records, I mean, there are some conditionals that we can use, for example, if the 3rd CSV column content has ‘XXXX’ and 4th column has ‘999’.

Question:
Can I use these conditionals to improve the read process? If so, how can I do that using Perl?

I need an example (Perl Script) in your answer.

Answer 1

Here’s a solution:

#!/usr/bin/env perl
use warnings;
use strict;
use Text::CSV_XS;
use autodie;
my $csv = Text::CSV_XS->new();
open my $FH, "<", "file.txt";
while (<$FH>) {
    $csv->parse($_);
    my @fields = $csv->fields;
    next unless $fields[1] =~ /something I want/;
    # do the stuff to the fields you want here
}