I have an ASP Classic app that allows people to copy and paste Word documents into a regular form field. I then post that document via jQuery Ajax to SQL Server, where the information is saved.
My problem is that the curly quotes and other word characters turn into strange characters when they come back out.
I’m trying to filter them on my save routines (classic asp stored procedure), but I still can’t quite eliminate the problems.
The ASP pages have this header with the ISO-8859-1 charset. Characters look fine when pasted into the text input fields.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
My jQuery code builds the following JSON in the ASP Page:
var jsonToSend = { serial: serial, critiqueText: escape(critiqueText) };
The database collation is set to SQL_Latin1_General_CP1_CI_AS
I use TEXT and VARCHAR fields to hold the text (yes, I know the Text field type is not preferred, but it’s what I have right now).
What must I do at each point to ensure that (1) the Word characters are stripped out, and (2) the encoding is consistent from front to back so I don’t get any odd characters displaying?
Oh- ASP Classic 3 running in 32-bit mode on Windows Server 2003 against SQL Server 2005.
Quick and dirty solution would be using nvarchar and ntext in your backend database. Strange chars you mention is problem of encoding. For example see below example.
You use ISO-8859-1 encoding in web page. This means that you are only able to save only ASCII characters that is only first 256 bit of full unicode. See this answer.
You use Latin1 in database. Approximately this three characters sets are equal. Latin1-General = Win 1252 = IEC_8859-1.
This means that whatever character you entered to database first 256 bits values are safe. If you know your client’s default encodings.
I suggest to try this default encoding to see if you can recover some information.
I gave example in Turkey, I know that most client’s use Win1254 therefore I will try to change values to that encoding and see I can recover anything.
Second part of your answer is that you can safely change from varchar to nvarchar without loss of information.
Here this without loss of information would be first part hex value (first 256 value).
Your strange chars would remain but other characters stays.
This answer and linked article gives more information.