Question

I’m trying to convert string using

Var
 encode:ansistring;
begin
  encode:=UTF8Encode('اختبار');
  showmessage(encode);
end;

It’s working fine in Delphi 7

but in Delphi XE2 it’s send Text as question marks

Any suggestions?

Answer 1

In your Delphi 7 code you probably wrote something like this:

var
  UTF8: string;
  InputString: WideString;//I guess that you used WideString
.....
UTF8 := UTF8Encode(InputString);

This was fine in Delphi 7 where string is an alias for AnsiString. In XE2 the generic string type is now an alias for UnicodeString which is UTF-16 encoded. That means that when the code above is compiled by XE2, the UTF-8 encoded buffer returned by UTF8Encode is interpreted as UTF-16 encoded text. And that mismatch is what leads to your string full of question marks.

So, if you just wrote

var
  UTF8: AnsiString;
  InputString: string;//aliased to UnicodeString
.....
UTF8 := UTF8Encode(InputString);

then you would have the same behaviour as for your Delphi 7 code.

However, this is not the way to do it in Unicode Delphi. Instead you should use the UTF8String type. This is defined as AnsiString(65001) which means a string of 8 bit character units with code page 65001, i.e. the UTF-8 codepage. When you do this you don’t need to call UTF8Encode at all since the encoding attached to the string type means that the compiler can generated code to convert the string. Now you would simply write:

var
  UTF8: UTF8String;
  InputString: string;//aliased to UnicodeString
.....
UTF8 := InputString;

The principal reference for the Unicode aspects of Delphi 2009 and later is Marco Cantù’s white paper: Delphi and Unicode which I recommend that you read before proceeding.