One mistake I see people making over and over again is trying to parse XML or HTML with a regex. Here are a few of the reasons parsing XML and HTML is hard:
People want to treat a file as a sequence of lines, but this is valid:
<tag attr='5' />
People want to treat < or <tag as the start of a tag, but stuff like this exists in the wild:
<img src='imgtag.gif' alt='<img>' />
People often want to match starting tags to ending tags, but XML and HTML allow tags to contain themselves (which traditional regexes cannot handle at all):
<span id='outer'><span id='inner'>foo</span></span>
People often want to match against the content of a document (such as the famous ‘find all phone numbers on a given page’ problem), but the data may be marked up (even if it appears to be normal when viewed):
<span class='phonenum'>(<span class='area code'>703</span>) <span class='prefix'>348</span>-<span class='linenum'>3020</span></span>
Comments may contain poorly formatted or incomplete tags:
<a href='foo'>foo</a> <!-- FIXME: <a href=' --> <a href='bar'>bar</a>
What other gotchas are you aware of?
Here’s some fun valid XML for you:
And this little bundle of joy is valid HTML:
Not to mention all the browser-specific parsing for invalid constructs.
Good luck pitting regex against that!
EDIT (Jörg W Mittag): Here is another nice piece of well-formed, valid HTML 4.01: