Question

Is there any way I could optimize the following script to run faster?

foreach my $arg (@data){ #  
   @score=();
   `program $arg $arg1 > $result`; #!!! $arg1 is a very large file with lots of data!!!
   open(FH,$result);
   while(<FH>){
      chomp;
      if($_ =~ /\d+.+\s+(\d+\.\d+|\d+\.|\.\d+).+/){ #here i'm looking for any number such as: 21.343 or 12 or 0.22 or -3.0
         push(@score, $1);
      }
   }
   close FH;
   @sorted = sort{$a <=> $b} @score; #a sorted score is what i actually want
}

Answer 1

There are a few things I can see (for instance not loading your result into the file immediately), but I suspect the main performance benefit you will get will probably be from using a different regex. To that end, do you have a better idea what the data output format from your program is?

Here’s some sample perl that may run a little bit quicker:

use strict;
foreach my $arg (@data){
  my @score=();
  open(my $fh, "program $arg $arg1 |");
  while (<$fh>) {
    chomp;
    if (/\d+.+\s+((\d+)?\.?\d+)/o) {
      push(@score, $1);
    }
  }
  close($fh);
  my @sorted = sort { $a <=> $b } @score;
}

Notice a few things here:

1. I’m using a program file handler so that I’m not using a temporary file, thus skipping a whole pass of data.
2. I changed the regex to use nested groups rather than multiple options.
3. I use strict and keep package names (for the love of God use strict in your perl).

The other people have said to use threads. You DO NOT need to do this, as running the process as I have done with the trailing pipe (|) in the open function causes perl to fork a process for you. Then you use standard unix pipes to read from the program asynchronously.