Question

i have a huge database of scraped forum posts that i am inserting into a website. however alot of people try to use html in their forum posts and often times do it wrong. because of this, there are always stray <strike> <b> </strike> </div> </b> tags in the posts which will end up messing up the webpage format when i add say 15 forum posts.

for now i have just been appending all possible end tags to the post just so that it might catch any open tag…is there a better way to do this short of parsing through the text and trying to manually remove each open tag. for loooooong forum posts this is an expensive transaction for a web app.

Answer 1

Have a look at HTML Tidy

There is a also a Python wrapper lib: µTidylib

Alternatively there is HTML Purifier