Question

I am making an xml document with japanese strings in it. I have tried different encodings but everytime when I save the file, it replaces the japanese strings with “??????”.. I have a very brief sample code here.. Please advise me towards what encoding should I use for the xml doc.

<?xml version="1.0" encoding="utf-8"?>
<config>
        <start_text>転送</start_text>
</config>

Please advise me regarding what encoding should I use so that the japanese characters are retained even after saving and closing the xml doc.
Thanks all

Answer 1

There are two Japanese characters in your XML file, and you get six question marks. This smells like UTF-8 encoding, as mentioned in your XML file. There are several possibilities:

The program that reads the XML file reads it with encoding ASCII, and therefore replaces every byte outside the ASCII range with a question mark.

The program reads the XML file properly, but then prints the UTF-8 encoded bytes to some output channel using the ASCII encoding, replacing each out-of-range byte with a question mark.

(Side note: If the output were 転é?? it would be UTF-8 printed to an ISO-8859-1 output channel. But it seems to be six question marks.)

You need to find out at which places the following transformations happen:

• A byte sequence is transformed to a character sequence. (Probably in the XML reader.)
• A character sequence is transformed to a byte sequence.
• A byte (or character) sequence is transformed into a glyph sequence. (For displaying them; this may be an editor or a console.)