Dropping my lurker status to finally ask a question…
I need to know how I can improve on the performance of a PHP script that draws its data from XML files.
Some background:
-
I’ve already mapped the bottleneck to CPU – but want to optimize the script’s performance before taking a hit on processor costs. Specifically, the most CPU-consuming part of the script is the XML loading.
-
The reason I’m using XML to store object data because the data needs to be accessible via a browser Flash interface, and we want to provide fast user access in that area. The project is still in early stages though, so if best practice would be to abandon XML altogether, that would be a good answer too.
-
Lots of data: Currently plotting for roughly 100k objects, albeit usually small ones – and they must ALL be taken up into the script, with perhaps a few rare exceptions. The data set will only grow with time.
-
Frequent runs: Ideally, we’d run the script ~50k times an hour; realistically, we’d settle for ~1k/h runs. This coupled with data size makes performance optimization completely imperative.
-
Already taken an optimization step of making several runs on the same data rather than loading it for each run, but it’s still taking too long. The runs should generally use “fresh” data with the modifications done by users.
Just to clarify: is the data you’re loading coming from XML files for processing in its current state and is it being modified before being sent to the Flash application?
It looks like you’d be better off using a database to store your data and pushing out XML as needed rather than reading it in XML first; if building the XML files gets slow you could cache files as they’re generated in order to avoid redundant generation of the same file.