I am writing a program in Java which tracks data about baseball cards. I am trying to decide how to store the data persistently. I have been leaning towards storing the data in an XML file, but I am unfamiliar with XML APIs. (I have read some online tutorials and started experimenting with the classes in the javax.xml hierarchy.)
The software has to major use cases: the user will be able to add cards and search for cards.
When the user adds a card, I would like to immediately commit the data to the persistant storage. Does the standard API allow me to insert data in a random-access way (or even appending might be okay).
When the user searches for cards (for example, by a player’s name), I would like to load a list from the storage without necessarily loading the whole file.
My biggest concern is that I need to store data for a large number of unique cards (in the neighborhood of thousands, possibly more). I don’t want to store a list of all the cards in memory while the program is open. I haven’t run any tests, but I believe that I could easily hit memory constraints.
XML might not be the best solution. However, I want to make it as simple as possible to install, so I am trying to avoid a full-blown database with JDBC or any third-party libraries.
So I guess I’m asking if I’m heading in the right direction and if so, where can I look to learn more about using XML in the way I want. If not, does anyone have suggestions about what other types of storage I could use to accomplish this task?
While I would certainly not discourage the use of XML, it does have some draw backs in your context.
“Does the standard API allow me to insert data in a random-access way”
Yes, in memory. You will have to save the entire model back to file though.
“When the user searches for cards (for example, by a player’s name), I would like to load a list from the storage without necessarily loading the whole file”
Unless you’re expected multiple users to be reading/writing the file, I’d probably pull the entire file/model into memory at load and keep it there until you want to save (doing periodical writes the background is still a good idea)
I don’t want to store a list of all the cards in memory while the program is open. I haven’t run any tests, but I believe that I could easily hit memory constraints
That would be my concern to. However, you could use a SAX parser to read the file into a custom model. This would reduce the memory overhead (as DOM parsers can be a little greedy with memory)
“However, I want to make it as simple as possible to install, so I am trying to avoid a full-blown database with JDBC”
I’d do some more research in this area. I (personally) use H2 and HSQLDB a lot for storage of large amount of data. These are small, personal database systems that don’t require any additional installation (a Jar file linked to the program) or special server/services.
They make it really easy to build complex searches across the datastore that you would otherwise need to create yourself.
If you were to use XML, I would probably do one of three things
1 – If you’re going to maintain the XML document in memory, I’d get familiar with XPath
(simple tutorial & Java’s API) for searching.
2 – I’d create a “model” of the data using Objects to represent the various nodes, reading it in using a SAX. Writing may be a little more tricky.
3 – Use a simple SQL DB (and Object model) – it will simply the overall process (IMHO)
Additional
As if I hadn’t dumped enough on you 😉
If you really want to XML (and again, I wouldn’t discourage you from it), you might consider having a look a XML database style solution
Apache Xindice (apparently retired)
Or you could have a look at some other people think
For example 😉