Question

I’m developing a small web-based (javascript) ‘application’ for an art project. The thing is called ‘Poetry Generator’, and it’s a script that generates random poems based on user input.

The script has to display a random word to the user every 1/10th of a second. The wordlist used, counts 109.582 words.

I’ve already tried different solutions:

1. put all the words in a text file, and get a random line of the textfile -> too slow (and the user has to download a 3MB text-file before being able to use the application)
2. put all the words in an array in the Javascript. -> javascript arrays apparently can’t handle 109.585 items
3. pull the words from a database using jQuery’s Ajax function with a Javascript interval function -> this solution worked perfectly when testing on my localhost, but once uploaded to a real web-environment, this method proved to be too slow. (And I could imagine that my hosting provider wouldn’t be so happy if I executed 10 query’s to their server every second.)

So.. Does anybody knows a different approach that I could use to show a random word on a webpage every 1/10th of a second? It doesn’t necessarily has to use php or javascript, as long as it runs in a browses, I’m happy!

Thanks in advance

Teis

Answer 1

There’s no reason you have to pull the entire dataset every tenth of a second. Pull a reasonable amount from a database every minute (which would be about 600 words), load it into a local javascript object, and iterate through it.

When either the array index becomes high enough or the timer hits one minute, poll for another set of 600.

When dealing with times as low as a tenth of a second, you don’t want to have to invoke the server EVERY single time! You could even load the entire data set into memcached and poll for random words, thus skipping costly database calls, as the entire data set is loaded into memory.