Question

Please consider the following problem.

I’m writing a quick Manipulate[] program to display a ton of information, but am running into problem with the unicode. Here is what I currently have as input and output:

Manipulate[
 request = filenumber <> "*";
 filenames = FileNames[request];
 display = Import[type, "List"];
 Short[display, 25]
 , {filenumber, "001", InputField}, {type, filenames, PopupMenu}]

The problem is that the French-language accents are showing up oddly. The quick workaround I thought of was to change my code to Import[type,"Plaintext"]; which works, but then displays the information in list form, like so:

What would you suggest as a way to get the clarity of the second example with the straightforward list format of the former? So that it wraps on the line rather than having a line break after each entry.

As an aside – probably just as important as the actual question itself – could anybody explain the rationale behind why importing as a “List” distorts the unicode? I’ve had a lot of trouble working around this, and understanding the underlying behaviour might help me move forward quicker.

Answer 1

Although Import does not have options associated with itself, it takes options relevant to the format being imported. Specifically see the Options section of ref/Format/List for the list of options.

In the case at hand, you can indicate the file encoding with CharacterEncoding->"UTF8":

Import[filename, "List", CharacterEncoding -> "UTF8"]