On the Stackoverflow podcast this week, Jeff mentioned that in 2004 he wrote a script which queried Google with 110,000 English words and collected a database containing the number of hits for each word. They use this on Stackoverflow e.g. for the ‘Related’ list on the right-hand side of each question page.
Since creating one of these today with a similar script would be difficult (as Joel mentioned, ‘at 30,000 words you get a knock at your door’), I was wondering if anyone knows of a more up-to-date, free database of Google word frequencies (e.g. for IT words which have surely changed since then such as jquery, ruby, azure, etc.).
A quick Google search(!) turns up a few hits. This link looks promising:
But it’s not targeted at IT words.