Question

I have a Perl program that I’ve written that parses SQL-like statements and creates a job to run on a mainframe that extracts fields and records based on the criteria. The records are returned to the PC and then output in various formats (e.g. csv, XML, etc.). I co-opted SQL’s INTO clause to list the files and outputs.

The issue is that writing records to a SQLite database (one of the output formats) takes a relatively long time and I don’t want that to hold up the processing of the next SQL command (in case there are multiple SQL queries passed in). So, I thought I would use fork() to make this happen.

The original bit of code is:

foreach (@into) {
    dointo($_,$recs);
}

@into is a list of files and formats (e.g. ‘File1.csv’ is a comma-delimited format sent to File1.csv, ‘File1.xml’ is an XML format written to File1.xml, etc.) to be processed. The subroutine dointo handles each of these. $recs is a sort of iterator that returns records in a variety of formats (flat, anonhash, anonarray, etc.)

So, I changed the code to:

foreach (@into) {
    unless (fork()) {
        dointo($_,$recs);
        exit 0;
    }
}

but now when the code runs, it seems to work, but it pulls a run-time error every time.

I didn’t capture the return from fork() because I really don’t care about waiting for the forked process to finish. Could this be the error? Does the parent process NEED to wait for the child processes to finish before it can safely exit?

Any help would be appreciated.

(BTW, Windows XP is the OS, Activestate Perl 5.10.0 is the Perl version)

Answer 1

Don’t write to a SQLite database in parallel.