Question

Im using a xml data feed to get information using simplexml and then generate a page using that data.

for this im getting the xml feed using

$xml = simplexml_load_file

Am i right in thinking that to parse the xml data the server has to download it all before it can work with it ?

Obviously this is no such problem with a 2kb file, but some files are nearing 100kb, so for every page load that has to be downloaded first before the php can start generating the page.

On some of the pages were only looking for a 1 attribute of an xml array so parseing the whole document seems unessarcery, normally i would look into caching the feed, but these feeds relate to live makets that are changing frequently so that not ideal as i would always have the up to the minute data.

Is there a better way to make more efficient calls of the xml feed ?

Answer 1

One of the first tactics to optimize XML parsing is by parsing on-the-fly – meaning, don’t wait until the entire data arrives, and start parsing immediately when you have something to parse.

This is much more efficient, since the bottleneck is often network connection and not CPU, so if we can find our answer without waiting for all network info, we’ve optimized quite a bit.

You should google the term XML push parser or XML pull parser

In the article Pull parsing XML in PHP – Create memory-efficient stream processing you can find a tutorial that shows some code on how to do it with PHP using the XMLReader library that is bundled with PHP5

Here’s a quote from this page which says basically what I just did in nicer words:

PHP 5 introduced XMLReader, a new class for reading Extensible Markup Language (XML). Unlike SimpleXML or the Document Object Model (DOM), XMLReader operates in streaming mode. That is, it reads the document from start to finish. You can begin to work with the content at the beginning before you see the content at the end. This makes it very fast, very efficient, and very parsimonious with memory. The larger the documents you need to process, the more important this is.

Parsing in streaming mode is a bit different from procedural parsing. Keep in mind that all the data isn’t already there. What you usually have to do is supply event handlers that implement some sort of state-machine. If you see tag A, do this, if you see tag B, do that.

Regarding the difference between push parsing and pull parsing take a look at this article. Long story short, both are stream-based parsers. You will probably need a push parser since you want to parse whenever data arrives over the network from your XML feed.

Push parsing in PHP can also be done with xml_parse() (libexpat with a libxml compatibility layer). You can see a code example xml_parse PHP manual page.