Question

I’m looking for good timer implementation in perl. The situation I met is like: I need to keep track of I/O activities of many files and for thoes files keep untouched for enough time a remove action will be taken upon them, so an efficient timer implementation is really vital for
the app I’m involved right now. To avoid recreate the wheel, ask you guys for help first.

Answer 1

Time::HiRes comes with perl.

Furthermore, your application sounds like it could benefit from Linux::Inotify (note the Linux:: in front). When setting the timer for a file that you want to remove after a certain time of inactivity, remember the last access. In an inotify event hook, update this time to the current time. Then, you can periodically check whether the file’s lifetime expired without doing a stat on all of the files you track. On expiration, you could add a final check just to make sure nothing went wrong, of course.

If you have huge numbers of files in flight, you may want to keep the list of files sorted by expiration time. That makes the periodic check for expiration trivial.

Update: I just did a little experimentation with Linux::Inotify. Things aren’t as easy with that approach as I thought. First, here’s the partially working code that I didn’t have time to finish.

#!/usr/bin/env perl
use strict;
use warnings;
use List::Util qw/min max/;
use Time::HiRes qw/time sleep/;
use Data::Dumper;
use Linux::Inotify;

# [s], but handles subsecond granularity, too
use constant CLEANUP_INTERVAL    => 1.;
use constant FILE_ACCESS_TIMEOUT => 5.;

# for fast and readable struct access
use constant FILENAME   => 0;
use constant ACCESSTIME => 1;
use constant WATCHER    => 2;

my $notifier = Linux::Inotify->new;
my @tracked_files = populate_tracked_files(\@ARGV, $notifier);
warn Dumper \@tracked_files;

while (1) {
  # update the tracked files according to inotify events
  my @events = $notifier->read;

  my %files_seen_this_round;
  foreach my $event (@events) {
    $event->print();
    my $ev_filename = $event->{name}; # part of the API, apparently

    # we mave have multiple events per file.
    next if $files_seen_this_round{$ev_filename}++;

    # find and update the right tracked file
    # TODO: this could be optimized to O(1) with a hash at
    #       the cost of more bookkeeping
    foreach my $tfile (@tracked_files) {
      if ($tfile->[FILENAME] eq $ev_filename) {
        my $atime = $^T + 60*60*24 * -A $ev_filename; # update access time
        $tfile->[ACCESSTIME] = $atime;
        # a partial bubble sort would be hugely more efficient here!
        # => O(n) from O(n*log(n))
        @tracked_files = sort {$a->[ACCESSTIME] <=> $b->[ACCESSTIME]} 
                         @tracked_files;
        last;
      }
    } # end foreach tracked file

  } # end foreach event

  cleanup_files(\@tracked_files);

  sleep(CLEANUP_INTERVAL);

  last if not @tracked_files;
} # end while(1)


$notifier->close;

sub cleanup_files {
  my $files = shift;

  my $now = time();
  for (my $fileno = 0; $fileno < $#{$files}; ++$fileno) {
    my $file = $files->[$fileno];
    if ($now - $file->[ACCESSTIME] > FILE_ACCESS_TIMEOUT) {
      warn "File '" . $file->[FILENAME] . "' timed out";
      # remove this file from the watch list
      # (and delete in your scenario)
      $file->[WATCHER]->remove;
      splice @$files, $fileno, 1;
      $fileno--;
    }
  }
}

sub populate_tracked_files {
  my $files = shift;
  my $notifier = shift;

  my @tracked_files;
  foreach my $file (@$files) {
    die "Not a file: '$file'" if not -f $file;

    my $watch = $notifier->add_watch($file, Linux::Inotify::ALL_EVENTS);
    push @tracked_files, [$file, $^T + 60*60*24*-A $file, $watch];
  }
  @tracked_files = sort {$a->[ACCESSTIME] <=> $b->[ACCESSTIME]} 
                   @tracked_files;

  return @tracked_files;
}

There’s still some bug in the time-checking logic. But the main problem is that $notifier->read() will block until a new event. Whereas we really just want to see whether there’s a new event and then proceed to cleanup. This would have to be added to Linux::Inotify as a non-blocking read of the file descriptor. Anybody can take over maintenance of the module since the author is no longer interested.