Question

I’m wondering how to write this in python and connect it to django app. I mean extracting values from meta tags and img url having only url of website. The same as facebook do when user is pasting the link.

Getting title and meta tags from external website

Answer 1

Personally, I would choose to tackle this with the very nice Requests, BeautifulSoup, and LXML libraries.

Assuming we have the following model in models.py, we could override the save() method to populate the title, description, and keywords attributes:

from bs4 import BeautifulSoup
import requests

from django.db import models

class Link(models.Model):
    url = models.URLField(blank=True)
    title = models.CharField(max_length=20, blank=True)
    description = models.TextField(blank=True)
    keywords = models.TextField(blank=True)

    def save(self, *args, **kwargs):
        if self.url and not (self.title or self.keywords or self.description):
            # optionally, use 'html' instead of 'lxml' if you don't have lxml installed
            soup = BeatifulSoup(requests.get(self.url).content, "lxml")
            self.title = soup.title.string
            meta = soup.find_all('meta')
            for tag in meta:
                if 'name' in tag.attrs and tag.attrs['name'].lower() in ['description', 'keywords']:
                    setattr(self, tag.attrs['name'].lower(), tag.attrs['content'])

        super(Link, self).save(*args, **kwargs)

The logic from the overridden save() method could very well live in a view or utility function or even in another method on the Link model that could be called conditionally.

The above works with Django 1.4. No guarantees, but it should work on earlier versions as well.

Edit: Fix syntax error and mention alternative parser, thanks @jinesh and @stonefury.