Question

In the below perl script, I check my folder name (which is in the date format like 11-08-31) with the current date. If it matches, I process the folder. It also checks the previous day folder if there is no folder in today’s date. I already asked this type of question here but I need to make some changes here and add new features as well:

• The script checks for the previous date if todays not find. But I need to check if the previous date has already been processed or not so that I donot process it again. So, Do I need to create a list for it?

• This script checks only for the one previous date. What if I have to check for the 2 previous days? Thanks for your help. hope you understand my doubts.

Updated: This perl script run automatically when It checks the curent date with the folder name. The folder is a tar folder which is loaded from other server.

So, basically I need to run the script if it matched with the folder name and current date.

Problem: Sometimes, I used to get the folder next day and my perl script checks only for the current date. The folder i get has the name which is previous date (not the current date).So, I need to do processing of the folder manually. I need to automate it in my perl script

---

#!/usr/bin/perl
use strict;
use warnings;
use Cwd;
use DateTime;
use File::Copy;

# set to your desired time zone
my $today = DateTime->now( time_zone => "America/New_York" );
my $td = $today->strftime("%y-%m-%d");

# strongly recommended to do date math in the 'floating'/UTC zone
my $yesterday = $today->set_time_zone('floating')->subtract( days => 1);
my $yd = $yesterday->set_time_zone('America/New_York')->strftime("%y-%m-%d");

my $dir = shift or die "Provide path on command line. $!";

if ($dir eq '.') {
    $dir = cwd;
}
elsif ($dir !~ /^\//) {
    $dir = cwd() . "/$dir"; 
}

opendir my $dh, $dir or die $!;
my @dir = sort grep {-d and /$td/ || /$yd/} readdir $dh;
closedir $dh or die $!;
@dir or die "Found no date directories. $!";

my $dday = "$dir/$dir[-1]"; # is today unless today not found, then yesterday
my $fdir = '/some/example/path/';    
my @gzfiles = glob("$dday/*tar.gz");

foreach my $zf (@gzfiles) {  
    next if (($zf =~ /BMP/) || ($zf =~ /LG/) || ($zf =~ /MAP/) || ($zf =~ /STR/)); 
    print "$zf\n";
    copy($zf, $fdir) or die "Unable to copy. $!";
}

Answer 1

Well, another way to do it, as suggested by mugen kenichi, is to use Storable. This way stores a hash with all processed directories in it. Then when you run your program, it can check the hash to see if they have been processed.

You would need a one-time script to set up the hash of processed directories.

#!/usr/bin/perl
use strict;
use warnings;
use Storable;

# This script to be run 1 time only. Sets up 'processed' directories hash.
# After this script is run, ready to run the daily script.

my $dir = '.'; # or what ever directory the date-directories are stored in

opendir my $dh, $dir or die "Opening failed for directory $dir $!";
my @dir = grep {-d && /^\d\d-\d\d-\d\d$/ && $_ le '11-04-21'} readdir $dh;
closedir $dh or die "Unable to close $dir $!";

my %processed = map {$_ => 1} @dir;

store \%processed, 'processed_dirs.dat';

Then, a script to be run periodically to find and process your date directories.

#!/usr/bin/perl
use strict;
use warnings;
use File::Copy;
use Storable;

my $dir = shift or die "Provide path on command line. $!";

my $processed = retrieve('processed_dirs.dat'); # $processed is a hashref

opendir my $dh, $dir or die "Opening failed for directory $dir $!";
my @dir = grep {-d && /^\d\d-\d\d-\d\d$/ && !$processed->{$_} } readdir $dh;
closedir $dh or die "Unable to close $dir $!";
@dir or die "Found no unprocessed date directories";

my $fdir = '/some/example/path';

for my $date (@dir) {
    my $dday = "$dir/$date";
    my @gzfiles = glob("$dday/*tar.gz");

    foreach my $zf (@gzfiles) {  
        next if $zf =~ /BMP/ || $zf =~ /LG/ || $zf =~ /MAP/ || $zf =~ /STR/; 
        print "$zf\n";
        copy($zf, $fdir) or die "Unable to copy $zf to $fdir. $!";
    }
    $processed->{ $date } = 1;
}

store $processed, 'processed_dirs.dat';