Question

I expected to find this in SO already… but haven’t so far

I’m talking about a regex which looks at an HTML ENCODED string: e.g. something like

blip ♦ trout’s mouth

Have I covered all the bases with &\w+; and &#[0-9]+;?

$encoded_string = htmlspecialchars($_GET["searchterms"]);
echo "<b>Search results for submitted string: \"$encoded_string\"</b><br><br>";
$html_special_chars_pattern = "!(&\\w+;|&#[0-9]+;)!";
$non_html_tokens = preg_split( $html_special_chars_pattern, $encoded_string, -1, PREG_SPLIT_DELIM_CAPTURE );

Answer 1

You are missing the &#xH; or &#XH; numeric character references.

5.3.1 Numeric character references

Numeric character references specify the code position of a character in the document character set. Numeric character references may take two forms:

• The syntax "&#D;", where D is a decimal number, refers to the ISO 10646 decimal character number D.

• The syntax "&#xH;" or "&#XH;", where H is a hexadecimal number, refers to the ISO 10646 hexadecimal character number H. Hexadecimal numbers in numeric character references are case-insensitive.

That is, &#[xX][a-fA-F0-9]+; in regular expression.