Apple strongly recommends using the binary plist format when reading large XML-based data sets into iPhone apps. Among their reasoning is the fact that XML parsing is very taxing on the iPhone. However, this requires that files residing on the remote web server be converted first.
For frequently-changing content, it is not acceptable to do this manually. If at all possible, I’d like to avoid having a web based app call the command line to perform the conversion (i.e., plutil).
Are there publicly available algorithms to perform this conversion?
Yes. All the plist code is part of CoreFoundation, which is opensource. CoreFoundation can be directly built and run on Linux and Windows, so you can write a CF tool using the normal APIs you would use on Mac OS X, but build and run it on other platforms.
The particular API you want to be looking at is CFPropertyListWriteToStream(). The code for CoreFoundation is available from Apple (tarball), among other places.
Finally depending on how often you update the file, how much processor you have to spare on the server, and how much repetition there is your data there may be one significant enhancement left that you can do. By default certain elements in binary plists are uniqued (such as strings). Other elements are not (such as arrays and dictionarts). The binary plist format allows them to be uniqued, the issue is that it is expensive to actually walk through and unique arrays and dictionaries. If you have a lot of identical arrays or dicts in your content you may see a significant size reduction by uniquing them. You can enable that by hacking up _flattenPlist() in CFBinaryPlist.c.
If you do that make sure to test it very thoroughly, and do not do on any files you cannot update over the network, just in case a future release makes any optimizations that break that. Also, make sure you are ready to turn it off at a moments notice.