Question

We’ve noticed that UTF8 characters don’t come out correctly when using UIDevice.CurrentDevice.Name in MonoTouch.

It comes out as “iPad 2 ??”, if you use some of the special characters like holding down the apostrophe key on the iPad keyboard. (Sorry don’t know the equivalent to show these characters in windows)

Is there a recommended workaround to get the correct text? We don’t mind to convert to UTF8 ourselves. I also tried simulating this from a UITextField and it worked fine–no UTF8 problems.

The reason this is causing problems is we are sending this text off to a web service, and it’s causing XML parsing issues.

Here is a snipped of the XmlWriter code (_parser.WriteRequest):

            using (XmlWriter xmlWriter = XmlWriter.Create(textWriter, new XmlWriterSettings 
                { 
#if DEBUG
                    Indent = true,
#else
                    Indent = false, NewLineHandling = NewLineHandling.None, 
#endif
                    OmitXmlDeclaration = true 
                }))
            {
                xmlWriter.WriteStartDocument();
                xmlWriter.WriteStartElement("REQUEST");
                xmlWriter.WriteAttributeString("TYPE", "EXAMPLE");
                xmlWriter.WriteEndElement();
                xmlWriter.WriteEndDocument();
            }

The TextWriter is passed in from:

public Response MakeRequest(Request request)
{
    var httpRequest = CreateRequest(request);

    WriteRequest(httpRequest.GetRequestStream(), request);

    using (var httpResponse = httpRequest.GetResponse() as HttpWebResponse)
    {
        using (var responseStream = httpResponse.GetResponseStream())
        {
            var response = new Response();
            ReadResponse(response, responseStream);
            return response;
        }
    }
}

private void WriteRequest(Stream requestStream, Request request)
{
    if (request.Type == null)
    {
        throw new InvalidOperationException("Request Type was null!");
    }

    if (_logger.Enabled)
    {
        var builder = new StringBuilder();
        using (var writer = new StringWriter(builder, CultureInfo.InvariantCulture))
        {
            _parser.WriteRequest(writer, request);
        }
        _logger.Log("REQUEST: " + builder.ToString());

        using (requestStream)
        {
            using (StreamWriter writer = new StreamWriter(requestStream))
            {
                writer.Write(builder.ToString());
            }
        }
    }
    else
    {
        using (requestStream)
        {
            using (StreamWriter writer = new StreamWriter(requestStream))
            {
                _parser.WriteRequest(writer, request);
            }
        }
    }
}

_logger writes to Console.WriteLine, it is enabled in #if DEBUG mode. Request is just a storage class with properties, sorry easy to confuse with HttpWebRequest.

I’m seeing ?? in both XCode’s console and MonoDevelop’s console. I’m also assuming the server is receiving them strangely as well, as I get an error. Using UITextField.Text with the same strange characters instead of the device description works fine with no issues. It makes me think the device description is the culprit.

EDIT: this fixed it –

Encoding.UTF8.GetString (Encoding.ASCII.GetBytes(UIDevice.CurrentDevice.Name));

Answer 1

Okay, I think I know the problem. You’re creating a StringWriter, which always reports its encoding as UTF-16 (unless you override the Encoding property). You’re then taking the string from that StringWriter (which will start with <?xml version="1.0" encoding="UTF-16" ?>) and writing it to a StreamWriter which will default to UTF-8. That mixture of encodings is causing the problem.

The simplest approach would be to change your code to pass a Stream directly to the XmlWriter – a MemoryStream if you really want, or just requestStream. That way the XmlWriter can declare that it’s using the exact encoding that it’s actually writing the binary data in – you haven’t got an intermediate step to mess things up.

Alternatively, you could create a subclass of StringWriter which allows you to specify the encoding. See this answer for some sample code.