Question

I am a newbie in Perl and trying to write output of parallel processes into a XML file using XML::Writer and see that some process(s) output is missing in generated XML.

Please help me to make $xmlWriter is thread safe so that, there will not be any problem while writing the output.

I’m Using ForkManager for spawning parallel processing

  my $xmlWriter = XML::Writer->new( OUTPUT => output.xml )
    ...
    $pm = new ForkManager(50);
    $xmlWriter->startTag("report");
    foreach $cmd(@cmdList) {
        $pid = $pm->start($cmd) and next;
        timeout 300 => sub {
            $status= system($cmd);
            ....
            $xmlWriter->startTag("task","command"=>"$cmd");
            $xmlWriter->startTag("status");
            $xmlWriter->characters("$status");
            $xmlWriter->endTag("status");
            $xmlWriter->endTag("task");
        }
    }
    $xmlWriter->endTag("report");
...

I have also tried making $xmlWriter as thread::shared variable and lock($xmlWriter) before writing into XML, but facing same issue.

Appreciate your help on this!

Answer 1

Don’t know were to start.

1. You got multiple independent copies of an XML::Writer object, all building the same XML document. That’s not giong to work.
2. You got multiple processes all using the same file handle.
3. Sharing an unshared variable so you can lock it makes absolutely no sense. If it’s not shared, how can locking help?

You need to limit yourself to one XML::Writer object. Have all the workers send back their output to the parent, and have the parent create the XML from the data sent.

Sending back the output would be done as per the example in the “Data structure retrieval” of the docs if you use Parallel::ForkManager.