Question

I have tags on my website, and I input them one by one when I create a blog post. I love gmail’s new feature, that ask you if you want to include X in a mail, if you type Y’s name and that you often include both of them in the same messages.

I’d like to do something similar on my website, but I don’t know how to represent the tags “related-ness” in an object or database … thoughts ?

Answer 1

It all boils down to create associations between certain characteristics of your posts and certain tags, and then – when you press the “publish” button – to analyse the new post and propose all tags matched with your post characteristics.

This can be done in several ways from a “totally hard-coded” association to some sort of “learning AI”… and everything in-between.

Hard-coded solutions

This are the simplest algorithms to implement. You should first decide what characteristics of your post are relevant for tagging (e.g.: it’s length if you tag them “short” or “long”, the presence of photos or videos if you tag them “multimedia-content”, etc…). The most obvious is however to focus on which words are used in posts. For example you could build a mapping like this:

tag_hint_words = {'code-development' : ['programming', 
                                        'language', 'python', 'function', 
                                        'object', 'method'],
                  'family' : ['Theresa', 'kids', 
                              'uncle Ben', 'holidays']}

Then you would check your post for the presence of the words in the list (the code between [ and ] ) and propose the tag (the word before :) as a possible candidate.

A common approach is to give “scores”, or in other word to put a number that indicates the probability a given tag is the right one. For example: if your post would contain the sentence…

After months of programming, we finally left for the summer holidays at uncle Ben’s cottage. Theresa and the kids were ecstatic!

…despite the presence of the word “programming” the program should indicate family as the most likely tag to use, as there are many more words hinting.

Learning AI’s

One of the obvious limitations of the above method is that – say one day you pick up java beside python – you would probably need to change your code and include words like “java” or “oracle” too. The same applies if you create new tags.

To circumvent this limitation (and have some fun!!) you could try to implement a learning algorithm. Learning algorithms are those who refine their outcome the more you use them (so they indeed… learn!). Some algorithm requires initial training (many spam filters and voice recognition programs need this initial “primer”). Some don’t.

I am absolutely no expert on the subject, but two common AI’s are: the Naive Bayes Classifier and some flavour of Neural network.

Although the WP pages might look scary, they are surprisingly easy to implement (at least in Python). Here’s the recording of a lecture at PyCon 2009 on the subject “Easy AI with Python“. I found it very informative and even somehow inspiring! 🙂

HTH!