Question

I’m creating a blog (and the rest of a website) using Python and Flask. Blog posts are written in Markdown and converted to HTML using the creatively named Markdown in Python. Both the Markdown (for future editing) and the HTML (for display) are stored in the database.

I want to be able to automatically get the first 300 characters of text (or 500, or 200 — I haven’t worked out the number) to use on pages when I don’t want to display the full blog post (like on the front page). However, the problem is that any simple way of doing it will potentially leave me with invalid HTML or Markdown:

HTML:

<p><em>Here</em> is <strong>formatted</strong> text.</p>

If I get the first ten characters of this, it will leave me halfway through formatted, and I would somehow need to close the <strong> and <p> tag.

Markdown:

*Here* is **formatted** text.

Likewise, getting the first ten characters will leave me needing to close the ** for bold.

Is there any way I can do this without needing to write a HTML or Markdown parser? Or, would I be better off just converting the HTML into plain text?

Answer 1

If you’re okay with summaries just being plain text, then Adam’s answer is certainly the best — convert to plain text first, and then truncate.

If you want to maintain formatting, then here’s another idea:

• Convert from Markdown to HTML.
• Run through the HTML with a parser of the sort that will give you a token stream (e.g. Perl’s HTML::TokeParser::Simple, but I’m sure there’s something comparable for Python — or you can turn any event-based parser into one of these).
• When you get element tokens, copy them to the output, while maintaining a stack of unclosed tags.
• When you get text tokens, copy them to the output, while maintaining a count of the amount of text you’ve outputted.
• When you get to a text token that would put you over the limit, copy only enough characters to reach the limit, generate closing tags for any unclosed tags on your stack, and stop processing.

If you were doing this with arbitrary HTML then you would have a lot of weird things to worry about, but since you’re coming from markdown it should actually work pretty well. Any decent markdown converter should generate well-formed HTML with a fairly small number of tags in it.