Question

I need to open a log file for writing. Trouble is, many things may do this at the same time, and I don’t want conflicts. Each write will be a single line, generally about 150 bytes (and always less than 1K), and getting things in chronological order is not strictly required.

I think what I want is to attempt to flock(), and if it fails, keep trying for a few seconds. If a lock can’t be established after a number of tries, then give up.

$fh=fopen($logfile, "a");

if (flock($fh, LOCK_EX|LOCK_NB)) {
  $locked=TRUE;
} else {
  $locked=FALSE;
  // Retry lock every 0.1 seconds for 3 seconds...
  $x=0; while($x++ < 30) {
    usleep(100000);
    if (flock($fh, LOCK_EX|LOCK_NB)) {
      $locked=TRUE;
      break;
    }
  }
}

if ($locked) {
  if (fwrite($fh, strftime("[%Y-%m-%d %T] ") . $logdata . "\n")) {
    print "Success.\n";
  } else {
    print "Fail.\n";
  }
  flock($fh, LOCK_UN)
} else {
  print "Lock failed.\n";
}

I have two questions, one general and one specific. First, aside from implementing the same solution in different ways (do...while, etc), is there a better general strategy for handling this kind of problem, that runs solely in PHP? Second, is there a better way of implementing this in PHP? (Yes, I separated these because I’m really interested in the strategy part.)

One alternative I’ve considered is to use syslog(), but the PHP code may need to run on platforms where system-level administration (i.e. adding things to /etc/syslog.conf) may not be available as an option.

UPDATE: added |LOCK_NB to the code above, per randy‘s suggestion.

Answer 1

In my long experience in PHP making logs (under linux!) I’ve never experienced problems of conflicts (even with hundreds of simultaneus and concurrent writes). So simple skip lock maagement:

$fh=fopen($logfile, "a");
if (fwrite($fh, strftime("[%Y-%m-%d %T] ") . $logdata . "\n")) {
    print "Success.\n";
  } else {
    print "Fail.\n";
  }
fclose($fh);

About this strategy the file logging (with or without lock) is not the best solution, because every fopen with the “a” imply a seek syscall to set the cursor at the end of the file. syslog keeping the file open avoid this overhead.

Of course the overhead become significative (in performance) with “big” files, an easy solution is create log file with date (or date-time) in the name.

ADD

apache package include a tester program: ab, allowing doing query in concurrency, you can test my thesis stressing your server with 1000000 of query done by 10to1000 threads.

ADD – following the comment

No, it’not an impossible task.

I found a note from http://php.net/manual/en/function.fwrite.php

If handle was fopen()ed in append mode, fwrite()s are atomic (unless
the size of string exceeds the filesystem’s block size, on some
platforms, and as long as the file is on a local filesystem). That is,
there is no need to flock() a resource before calling fwrite(); all of
the data will be written without interruption.

to know how big is a block in bytes (usual 4k):

dumpe2fs /dev/sd_your_disk_partition |less -i

The “atomicity” of write is realized blocking other “agents” to write (when you see process in “D” status in “ps ax”), BUT PHP stream facility can solve this problem, see: *stream_set_blocking*.
This approach can introduce partial writes, so you will have to validate the integrity of your records.

In any case a fwrite (network or file) is susceptible to block/failure regardless of use of flock. IMHO flock introduce only overhead.

About your original questions AND your goal (trying to implement corporate policy in a highly risk-averse environment), where even a fwrite can be problematic, I can only imagine a simple solution: use a DB

• most of complexity is out of PHP and written in C!
• total control of the flow of the operation
• hi level of concurrency