Question

I’m currently doing a project to collect and classify news articles, and I’m only interested in a small subset (for example sports-related news) of all the articles collected.

I’m new to Machine Learning and Text Classification. Should I classify the articles based on their titles or actual contents? A human being can usually tell with fair amount of confidence if the news article is relevant by just looking at the title. Hence I’m wondering if titles, instead of content, would give similar or better accuracy in automatic text classification?

The reason for this problem is that overall performance will improve a lot if the program analyses titles first when it finds a link, instead of retrieving every page from the urls and then analyses the contents.

Answer 1

The title is unlikely to provide enough information to classify an article. You can however analyse the title and, if your confident enough that you’ve got an accurate classification, you can classify it, otherwise look at the content.

Take something like Manchester in trouble. If you don’t know that Manchester is a sports team, the article could be economic or political or probably one of few other categories too. I suspect a lot of titles can only easily be classified by people because they’re familiar with the Proper Nouns relating to that category, and it could be difficult to get proper training data to train an agent to do this well.