Question

I have the following data in a plain text file:

1.  Value
Location :  Value
Owner:  Value
Architect:  Value

2.  Value
Location :  Value
Owner:  Value
Architect:  Value

... upto 200+ ...

The numbering and the word Value changes for each segment.

Now I need to insert this data in to a MySQL database.

Do you have a suggestion on how can I traverse and scrape it so I can get the value of the text beside the number, and the value of “location”, “owner”, “architect” ?

Seems hard to do with DOM scraping class since there is no HTML tags present.

Answer 1

That will work with a very simple stateful line-oriented parser. Every line you cumulate parsed data into an array(). When something tells you’re on a new record, you dump what you parsed and proceed again.

Line-oriented parsers have a great property : they require little memory and what’s most important, constant memory. They can proceed with gigabytes of data without any sweat. I’m managing a bunch of production servers and there’s nothing worse than those scripts slurping whole files into memory (then stuffing arrays with parsed content which requires more than twice the original file size as memory).

This works and is mostly unbreakable :

<?php
$in_name = 'in.txt';
$in = fopen($in_name, 'r') or die();

function dump_record($r) {
    print_r($r);
}

$current = array();
while ($line = fgets($in)) {
    /* Skip empty lines (any number of whitespaces is 'empty' */
    if (preg_match('/^\s*$/', $line)) continue;

    /* Search for '123. <value> ' stanzas */
    if (preg_match('/^(\d+)\.\s+(.*)\s*$/', $line, $start)) {
        /* If we already parsed a record, this is the time to dump it */
        if (!empty($current)) dump_record($current);

        /* Let's start the new record */
        $current = array( 'id' => $start[1] );
    }
    else if (preg_match('/^(.*):\s+(.*)\s*/', $line, $keyval)) {
        /* Otherwise parse a plain 'key: value' stanza */
        $current[ $keyval[1] ] = $keyval[2];
    }
    else {
        error_log("parsing error: '$line'");
    }
}

/* Don't forget to dump the last parsed record, situation
 * we only detect at EOF (end of file) */
if (!empty($current)) dump_record($current);

fclose($in);
?>

Obvously you’ll need something suited to your taste in function dump_record, like printing a correctly formated INSERT SQL statement.