Question

I am having a problem with some characters in 2 strings that my program uses.

String #1 is filled using VB code that gets data from a 3rd party application.
String #2 gets similar data from the same 3rd party application, but it gets it with a C++ dll and sends it to VB.

The data has some weird symbols in it.

I don’t know a whole lot about encoding and different character sets, but I’ll try to explain it the best I can.

I will use “Т” as my example character.

“Т” (note this isnt a normal capital t) it is unicode decimal value 1058
http://www.unicodemap.org/details/0x0422/index.html

When this character appears in String #1 during runtime it appears as “?”, which I believe is just what VB6 does to show some unicode characters. When I use AscW on the character it returns the correct value of 1058.

When I output the string to a text file, it appears as “?”.

The same character in String #2 from the C++ DLL appears as 2 characters “Т”

When I output that string to a text file, the character appears properly as “Т”.

I was only outputting things to text files for testing purposes. I only need the 2 strings to be encoded / appear the same during run time.

Any idea whats going on here? Any way for me to get weird characters to appear the same in both strings?

Thanks

edit: also the C++ dll is in multi character set and sends the data in a BSTR string

CODE IN C++ DLL

allChat is a CString

BSTR Message;
int len = allChat.GetLength();
Message = SysAllocStringByteLen ((LPCTSTR)allChat,len+1);

Message is returned to the VB app.. and nothing happens to the string after that.

String #1 is just a regular VB string

Answer 1

From the way Cyrillic “T” becomes “Т”, you get your string as a UTF8 encoded string (I verified that with Notepad++ by switching encodings). You need to convert it to Unicode before sending it to your VB app. Note that your VB app needs to be Unicode, not ASCII.

You can convert UTF8 to std::wstring with this function:

std::wstring utf8to16( const char* src )
{
    vector<wchar_t> buffer;
    buffer.resize(MultiByteToWideChar(CP_UTF8, 0, src, -1, 0, 0));
    MultiByteToWideChar(CP_UTF8, 0, src, -1, &buffer[0], buffer.size());
    return &buffer[0];
}