I am returning some data from the database which is entered using many different methods, usually copied and pasted from word.
Rather than attempt to manually HtmlEncode every last character I had the bright idea to use the content-type on the page to load a different character set. Currently I am not doing anything to overwrite any default content-type. If I ran locally, the content-type was shown as utf-8 and the “foreign” characters appeared. When uploaded, the content-type was coming back as ISO-8859-1 and “foreign” characters were appearing as question marks (of two varieties…).
I then overwrote the character set returned using Response.Charset = "utf-8"; and despite the response headers correctly showing utf-8, I am still getting question marks in place of the correct characters. Please see the images below:
Local Page
Local Page Header
External Page (notice two types of question mark)
External Page Header
I’ve followed http://htmlpurifier.org/docs/enduser-utf8.html but apart from needing UTF-8 I couldn’t find any solution.
Solved. Still not sure entirely how everything’s working but I’ll explain what I did.
Despite headers saying the content-type was UTF-8, this didn’t solve the issue. As hinted at by Oded, I found in IIS > .Net Globalisation two settings (Requests and Responses) which were set to ISO-8859-1.
Changing these to UTF-8 solved the trick.
As for why, I’m not sure, as the headers seem to be the same, so if anyone could explain what’s happening that would be brilliant.