The German website nandoo.net offers the possibility to shorten a news article. If you change the percentage value with a slider, the text changes and some sentences are left out.
You can see that in action here:
The news article is on the left side and tags are marked. The slider is on the top of the second column. The more you move the slider to the left, the shorter the text becomes.
How can you offer something like that? Are there any algorithms which you can use to achieve that?
My idea was that their algorithm counts the number of tags and nouns in a sentence. Then the sentences with fewest number of tags/nouns are left out.
Could that be true? Or do you have another idea?
I hope you can help me. Thanks in advance!
Usually you want to keep the sentences that have words that are more unique to that article.
That is, the more ‘generic’ the sentence is, the less it describes this particular article.
The normal way to do this is Bayesian analysis much like a spam-filter. First determine which words in the entire article appear more often than you’d expect, then find the sentences that feature those words.