Question

We have a program which runs on Windows and Linux. It writes out std::wstrings in binary to a file. We need to be able to read in files written from linux into windows. We write out strings as a list of wchar_t. On linux each wchar_t occupies 4 bytes. On Windows each wchar_t occupies 2 bytes.

When reading the file written by linux into Windows how can one take the four byte wchar_t and put it into the 2 byte wchar_t?

Thanks,
Adam

Answer 1

You can use UTF8-CPP to easily convert the file from UTF-32 to UTF-16:

#include <fstream>
#include <iterator>
#include <utf8.h>

int main(int argc, char** argv) {

    std::ifstream file("source.txt");
    std::string   intermediate;
    std::wstring  result;

    utf8::utf32to8(std::istreambuf_iterator<char>(file),
                   std::istreambuf_iterator<char>(),
                   std::back_inserter(intermediate));

    utf8::utf8to16(intermediate.begin(),
                   intermediate.end(),
                   std::back_inserter(result));

}

Unfortunately there is no utf8::utf32to16, though perhaps there should be.