Question

I’m to trying to automate a data analysis program using gnuplot, essentially what I’m doing is running a curve fitting program in gnuplot and then reading the log file to take out the desired values for further analysis.

Here is the section of code:

#Open curvefit log file to gather the needed coefficients
open (FILE_CURVE, 'fit.log') or die;
while (<FILE_CURVE>)
{   
    push(@log, $_);
    print "Im here\n";
}
close (FILE_CURVE);

My problem is it is not entering the while loop as I’m not seeing the print "Im here\n"; line of code.

Also at the beginning of the program I delete the log file so that it does not run away. The curve fit program re-creates it.

This is what the log file looks like. Note: there are two blank lines at the beginning of the file.

*******************************************************************************
Tue May 17 11:28:59 2011


FIT:    data read from 'temp_norm.txt' using 1:2
        #datapoints = 2000
        residuals are weighted equally (unit weight)

function used for fitting: g(x)
fitted parameters initialized with current variable values



 Iteration 0
 WSSR        : 566.797           delta(WSSR)/WSSR   : 0
 delta(WSSR) : 0                 limit for stopping : 1e-05
 lambda   : 1.49986

initial set of free parameter values

cc              = 100
dd              = 9.3

After 31 iterations the fit converged.
final sum of squares of residuals : 24.1325
rel. change during last iteration : 0

degrees of freedom    (FIT_NDF)                        : 1998
rms of residuals      (FIT_STDFIT) = sqrt(WSSR/ndf)    : 0.109901
variance of residuals (reduced chisquare) = WSSR/ndf   : 0.0120783

Final set of parameters            Asymptotic Standard Error
=======================            ==========================

cc              = 108.497          +/- 3.189        (2.939%)
dd              = 8.8375           +/- 0.0001571    (0.001777%)


correlation matrix of the fit parameters:

               cc     dd     
cc              1.000 
dd              0.246  1.000 

Answer 1

A couple of issues here:

1. It is strongly recommended to use the three-arg form of open and using a local var rather than FILEHANDLE, which will pollute your global namespace, so instead of

open (FILE_CURVE, 'fit.log') or die;

use

open(my $fh, '<', 'fit.log') or die; 
while(<$fh>){
}
close($fh);

2. Deleting a file while it has an open filehandle doesn’t “really” delete it and if you read from a filehandle on a “deleted” file, you’ll simply be reading from the old file. Files are truly deleted when all links to it, including open filehandles, are removed which can take some time. One strategy you might consider is:

sleep 1 while ! -f $path_to_logfile; # sleep until the file exists

3. There’s a lib for what you want.

Check out File::Tail
and Filesys::Notify::Simple

–EDIT–

Based on your comments, I’d guess something like this is occurring:

Assuming two agents:

a producer (gnuplot) that writes to the log file 

a consumer (your script) that reads from the log file

1. producer starts

   1.1 writes to log file

2. consumer starts

   2.1 deletes log file

   2.2 opens log file for read

   2.3 finds no lines to read

   2.4 exits

If this is the case, then by deleting the file at 2.1 to which the producer is still writing log messages, you are creating a situation where the producer is writing to a ‘dangling’ file, one that has been removed from the file system but on which there is still on open handle.

Then when you open the log at 2.2, the file is being recreated as an empty file which means your while loop will not find any lines and your print will not occur.

I’d recommend either:

1. start the consumer before you start the producer and have it wait until the log file exists to try opening it (using the sleep command above), or

2. start the producer first but don’t delete the log file in the consumer.