Home/ Questions/Q 8357211
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-09T10:24:34+00:00

I would like to use a supervised machine learning algorithm to predict a binary

  • 0

I would like to use a supervised machine learning algorithm to predict a binary function (true or false) for a set of sentences based on the presence or absence of words in the sentences.

Ideally, I would like to avoid having to hardcode the set of words used to decide on the output so that the algorithm automatically learns which words are (together ?) most likely to trigger specific outputs.

http://shop.oreilly.com/product/9780596529321.do (Programming Collective Intelligence) has a nice section in chapter 4 titled “Learning From Clicks” which describes how to do this by using 1 layer of hiden nodes in a neural network with one new hidden node for each new combination of input words.

Similarly, it is possible to create a feature for each word in the training data set and train pretty much any classic machine learning algorithm using these features. Adding new training data will generate new features which will require me to re-train the algorithm from scratch.

Which brings me to my questions:

  • is it actually a problem if I have to retrain everything from scratch whenever the training data set is extended ?
  • what kind of algorithm would more experience machine learning users recommend to use for this kind of problem ?
  • what criteria should I use in picking an algorithm versus another ? (other than actually trying them all and see which perform better with precision/recall metrics)
  • if you have worked on similar problems, what about extending the features with 2-grams (1 if a specific 2-gram is present, 0 if not) ? 3-grams ?
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You could look into the general area of topic modelling if you want to find words which are generally found together.

    The most simple approach would be to use latent semantic analysis ( http://en.wikipedia.org/wiki/Latent_semantic_analysis ), which is just applying SVD to a term document matrix. You’d then need to do some additional post hoc analysis to fit this to your particular outcome.

    A more involved, and much more complex approach would be to use latent dirichlet allocation ( http://en.wikipedia.org/wiki/Latent_Dirichlet_allocation )

    In terms of just adding new features (words) that is fine as long as you are going to retrain. You can also use TF/IDF to give that particular word a value when representing the matrix (Instead of just a 1 or 0).

    I don’t know what programming language you are trying to do this in, but I know there are libraries out there in Java and Pythont hat do all of the above.

    • 0