Question

I have a daemon process written in Perl that uses Inotify2 to watch directories for incoming files. On arrival of each file, the daemon will fork a child process. Now, it seems too many files are arriving at the same time (hence too many forks), because I got this error in my log file:

Cannot allocate memory at notifyd.pl line ...

which is the result of fork().

Basically I have the following code:

    my $inotify = new Linux::Inotify2() or die($!);

    foreach my $k (@PATHS) {
        $inotify->watch($k,
IN_MOVE_SELF|IN_DELETE_SELF|IN_CLOSE_WRITE, \&watcher) or die($!);
    }

    $inotify->blocking(1) or die($!);

    for(;;) {
        $inotify->poll() or die($!);
    }

with the watcher function doing fork and then execv:

sub watcher {
        my $e = shift;
        my $pid = fork();
        if(!defined $pid) {
            print "[ERROR]", $!;
        }
        elsif($pid == 0) {
            my @args = ($e->fullname, $e->mask);
            exec($childprocess, @args) or die($!);
        }
}

I cannot afford to miss events by not forking a process.

Does anyone have suggestions how I can improve this and make sure fork will not fail?

---

Edit: it seems like the child processes were becoming zombies once they exited since the daemon would not respond to SIGCHLD. So a lot of zombie child processes could have been the reason why fork() failed. The daemon now does $SIG{CHLD} = 'IGNORE'; before forking.

Answer 1

Solve the problem by adding another layer of indirection.

When you receive an event, put the file name into a job queue. The queue starts a new job processing the file when resources are reasonable free; this scheme guarantees that the event will eventually be acted upon, just not all of them immediately.