Question

For some reason, when I open a CSV file, that appears totally fine in Excel and Notepad, in Microsoft Works, there are some strange characters appearing as the first thing in the file. By first thing I mean they appear in cell A1, before the text that appears in that cell.

Here is an example file

Illustration carrier,Net death benefit,Issue date,Issue state,Policy type,Member lives,Policy date,Termination of coverage,Termination of coverage action required,Accumulated value,AV,SV,Illustrated Annual Level Premiums,LP to NDB ratio,Premium financed,Name of program,Primary insured age,Primary insured gender,Secondary insured age,Secondary insured gender,Current bid,Time left,Bid1,Bid2,Bid3,Bid4,Bid5,Bid6,Bid7,Bid8,Bid9,Bid10,Bid11,Bid12,Bid13,Bid14,Bid15,Bid16,Bid17,Bid18,Bid19,Bid20
srwer,$0.00,14/05/2012,us state 2,policy Type 1,member life 4,16/05/2012,-,,$0.00,-,-,,Not yet implemented,no,,0y 0m,Demale,,Female,$0.00,"2 days, 7 hours, 51 minutes, 36 seconds".

I create these files in my application using this method

protected ActionResult ExportToCSV(string csvExport)
{
   var output = new MemoryStream();
   var writer = new StreamWriter(output, Encoding.UTF8);

   writer.Write(csvExport);

   writer.Flush();
   output.Position = 0;

   return File(output, "text/comma-seperated-values", "export.csv");
}

I have a few questions

1. Is there a way to change this code to save the files in ANSI format?
2. Will I be able to view the files perfectly in Notepad, Works and Excel?

Answer 1

I suspect it’s the UTF-8 byte order mark:



By convention, the presence of these characters at the beginning of a file indicate that the file is encoded using UTF-8. Modern applications look for these characters and automatically remove them if they exist. But Microsoft Works is an old application that probably doesn’t support UTF-8.

To remove the characters, you can open the file in Notepad, choose File > Save As, and select ANSI in the Encoding drop-down list.

UPDATE: If you need to support old non-Unicode applications like Microsoft Works, then you can specify Encoding.ASCII or Encoding.Default when creating a text file in .NET. But MSDN Library cautions:

Different computers can use different encodings as the default, and the default encoding can even change on a single computer. Therefore, data streamed from one computer to another or even retrieved at different times on the same computer might be translated incorrectly. In addition, the encoding returned by the Default property uses best-fit fallback to map unsupported characters to characters supported by the code page. For these two reasons, using the default encoding is generally not recommended. To ensure that encoded bytes are decoded properly, your application should use a Unicode encoding, such as UTF8Encoding or UnicodeEncoding, with a preamble. Another option is to use a higher-level protocol to ensure that the same format is used for encoding and decoding.