Question

I have about 30K records in an XML file and this file is updated all the time.

I’m trying to insert and if exists update a MySQL db.

This is the code I’d like to use, but it runs very slowly, does anyone have any ideas for improving its performance?

// getting xml file
$dom = new DOMDocument();
$dom->load('products.xml');

// getting xml nodes using xpath
$xpath = new DOMXPath($dom);
$productid = $xpath->query('//NewDataSet/Product/ProductId');
$price = $xpath->query('//NewDataSet/Product/Price');

// Reading all nodes and if mach found in db update price, else insert as new record**
for($i=0;$i<$allNodes->length;$i++){
    $testproductid = $productid->item($i)->nodeValue;
    $testprice = $price->item($i)->nodeValue;
    if(mysql_num_rows(mysql_query("Select productid from test where productid ='$testproductid'"))){
        mysql_query("UPDATE test SET price = '$testprice' WHERE productid = '$testproductid'");
    }else{
        mysql_query("INSERT INTO test (price, productid) VALUES ('$testprice','$testproductid')");
    }
}

Answer 1

First, this line can result in bad behaviors:

if(mysql_num_rows(mysql_query("Select productid from test where productid ='$testproductid'")))

What happens if mysql_query() fails? Do something like that instead:

$res = mysql_query("Select productid from test where productid ='$testproductid'");
if ($res) {
... CODE HERE ...
}

Is productid an index? Also, you can formulate your query as:

Select productid from test where productid ='$testproductid' LIMIT 1

In this case, MySQL won’t look for more records. Also, try to insert more than one record in on single INSERT statement. See this:

http://dev.mysql.com/doc/refman/5.0/en/insert-speed.html

Take a look at the REPLACE command. That would replace the SELECT/UPDATE/INSERT conditions, but it might not be a great improvement of performance though.

http://dev.mysql.com/doc/refman/5.0/en/replace.html