Question

We run a webserver that has to serve files from a rather large directory, so finding the file via a simple wildcard search like “abcd*jklp*” has serious performance issues.

Is there a way (a trick or a library) to speed up file search in Java? If not, is there a simple caching solution, such that each search is done only once unless the application explicitly empties the cache?

Answer 1

When your web application starts (and every x minutes after that) cache every file you intend to serve in a static variable. When users search for a specific file search your static cache rather than running a search on the actual file server.

Caching everything works great if all you allow is searching on a file name. You could store every file path in a List/Array. If the list/array is already sorted you can use a binary search for user search queries. If there are wild card(s) generate the proper RegEx.

For full text searching of file contents, storing it all statically would not be feasible. Do something lazy like back your files with a database or buy a search appliance like GSA http://www.google.com/enterprise/search/gsa.html