Question

I have an XML file that is approximately 12mb which has about 16000 product’s. I need to process it into a database; however, at about 6000 rows it dies with a 500 error.
I’m using the Kohana framework (version 3) just in case that has anything to do with it.

Here’s my code that I have inside the controller:

$xml = new XMLReader();
$xml->open("path/to/file.xml");

$doc = new DOMDocument;

// Skip ahead to the first <product>
while ($xml->read() && $xml->name !== 'product');

// Loop through <product>'s
while ($xml->name == 'product')
{
   $node = simplexml_import_dom($doc->importNode($xml->expand(), true));
   // 2 queries to database put here
   $xml->next('product');
}

The XML is a a bunch of items for a store, so the two queries are a) insert ignore the store itself and b) insert the product

Any insight would be greatly appreciated.

Answer 1

Probably you are running out of memory. Try to increase your memory limit

ini_set('memory_limit','128M');

or whatever the amount of memory is neccesary (it depends on your server).
I leave you here some links with other ways of increasing the memory limit of your server:

PHP: Increase memory limit

PHP: Increase memory limit 2