Question

I am stuck at a problem with strings. I have many (lets say more 100) two dimensional arrays with strings. I will need to search through them for occurrence of a match string (entered through a search bar). If any occurrence, the row containing the matched string have to be displayed in a table view. With each entered character in the search bar, the results must be refined and displayed in a table view.

All the strings data must be saved and must be opened for later use. Could any one please suggest on the following things?

1. Is it fine saving all the two dimensional arrays of strings in a database.(SQLite)? or each array in an xml file? or any other better idea?

2. How to search efficiently through the all the strings? I am thinking to create a trie data structure for better searching.

Thanks!

Answer 1

Given that you have up to 20,000,000 strings, I wouldn’t advise using XML for your file format. It will add a lot of overhead both in disk space and parsing time. A database definitely sounds like a better way to go to me. If the strings were all the max length, that would be 5GB of data just for the strings, so you might even want to compress them in some way, depending on the end user’s machine.

A trie also sounds like a good choice of data structure for searching through them. Though, it will be pretty big. It might not need to be the whole 5GB in memory, but with the various pointers and stuff needed to implement it, it will likely still be quite large. How large probably depends on the amount of overlap between the strings’ prefixes. You could use a compact prefix tree if the data proves too large.

Another option would be to sort them all and do a binary search as the user types. It would probably not be efficient to sort them on the fly, though, so you’d probably want to save a pre-sorted index of the strings. (Not sure if you can do that, or if the data changes while running the app.)