Question

I’m dealing with a replacement character inside a MySQL database… and it’s fine if it stays there but I’m trying to edit it. My form displays the character as a diamond shape with a question mark in it (�). So I submit the form, I compare the data between the one on the form to the one in the data to see if it has changed. The problem here is that when I submit the form it turns the replacement character into � which is the html entity equivalent so when this happens it fails the comparison and the code thinks the string has changed– which it has, but not really. I’ve tried to employ different methods of either turning the replacement character into the html entity equivalent from the database when it’s being compared –it starts to turn another seemingly normal characters into another replacement character html entity equivalent– and turning the html entities into the replacement character –which simply does not work for this– but they both fail. And yes, I have tried html_entity_decode() and htmlspecialchars_decode()

My questions is: How can I keep the replacement character from turning into an html entity?

Answer 1

For some reason, the webbrowser is submitting the � REPLACEMENT CHARACTER (U+FFFD) as it’s decimal, numeric HTML Entitiy: �. Probably you’re already outputting it that way to the browser?

However, if you expect the input to contain HTML entities, you need to decode them if you don’t want to store them into your database as HTML. To decode numeric entities within an incomming UTF-8 encoded string $str:

$convmap = array (0, 0x10FFFF, 0, 0xFFFFFF);
$output = mb_decode_numericentity($str, $convmap, 'UTF-8');

This code does actually do the conversion you’re looking for (Demo), however you should clarify first why a numeric HTML entity is submitted.

As you prefer unicode, I suggest you make use of UTF-8 for the webpage:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

and for the form:

<form action="" method="post" accept-charset="utf-8">

good luck.