Question

I am looking for a database which could help me group thousands of English-language keywords to few general disciplines.

For example:

I HAVE THIS => I WANT TO HAVE THIS
cat => animal
chair => household
wine => drink
deer => animal
beer => drink
glass => household, drink
total 50 000 keywords => total <100 disciplines

I guess that organizing keywords into disciplines can be very tricky. I don’t really care about what will be the disciplines. I only want to have my 50 000 keywords grouped into less than 100 disciplines.

Do you know a dictionary database that can do that?

Answer 1

In terms of associating the keywords with dictionary entries:

• One general method of getting what you need would be to look at this as a multi-class classification problem:
  http://en.wikipedia.org/wiki/Multiclass_classification
• One implementation that I believe should work for your particular need would be SVM Light:
  http://svmlight.joachims.org/svm_struct.html
• You will still need to train the classifier with sample data, of course, so you will need to have a representative training set.

In terms of actually organizing the results in a database, it seems relatively simple:

• create a dictionary table (with the < 100 terms)
• create a keyword table, with foreign keys to the dictionary table
• the natural (logical) primary key of the keyword table would be (keyword, dictionary_table_id)