Question

I have a time dependent script I am working on and used microtime() to find the bottle neck. I determined the time increase is caused by doing a check on 300+ values to see if they exist in a database one at a time at 0.04 seconds a query.

The background of the script is it is a caching script. I need to see if it exists in the DB so I need a true/false (obtained by a rowCount) but i also need a way to relate a false to a value so I can update it. I know using a WHERE tag IN (:ARRAY) would work faster than the individual calls, but I cant think of a way to apply an association of true/false to value in this method.

My current code is below:

//loop through all our values!
    //prepare out reusuable statement
$stmt = $db->prepare("SELECT * from cache WHERE value=?");

foreach($values as $tempVal)
{
    //find if its in the database
    try
    {
        $stmt->execute(array($tempVal));
        $valCount = $stmt->rowCount();
    } catch(PDOException $ex) {
        echo "PDO error send this to us: " . $ex->getMessage();
    }

    //update flag
    $addToUpdate = 1;

    //if its in the database
    if($valCount > 0)
    {
        //get the tag data
        $valRes= $stmt->fetch();

        //check if cache expired
                    $addToUpdate = 0;
    } 

    //add to update list
    if($addToUpdate)
    {
                    //needs updating
        $updateList[] = $tempVal;

        //add to not in DB list to minimize queries
        if($tagTCount == 0)
        {
            $notInDB[$tempVal] = $tempVal;
        }
    }   

Any suggestions? I can explain more if anything is not clear.

Thank you,
Nick

Answer 1

So you just issue your query with the complete array, using the IN (?,?,?,?,?...) list:

// abstract, use a PDO wrapper of your choosing
$query = db("SELECT * FROM cache WHERE value IN (??)", $values);

Then iterate over the result list. Only matched $values will return. So build your first list from that:

foreach ($query as $row) {
     $updateList[] = $row["value"];
}

To get the list of absent entries, just diff that against your original array:

$notInDB = array_diff($values, $updateList);

You could of course use a second NOT IN query. But doing that differentiation in PHP is simpler.