In C I need to write a program that parses a CSV file encoded in UTF-16 take that information process it, and use that processed information to generate a new UTF-16 csv file.
As of right now all my program does is loads the contents of the CSV file (UTF-16) into a chunk of memory, I now need to somehow work with this UTF-16 chunk of memory and I’m not exactly sure how I’m going to go about doing that without converting it since I can’t even use strlen or similar on it because of all the NULLs it has.
How can I convert UTF-16 to UTF-8 and then back to UTF-16 ? I assume I’m going to need to get the total file size of the original CSV (UTF-16) file by using fseek and what not. But even after that how do I do the conversion?
The reason I’m doing this by the way is because I’m currently working with some “dump” files from WonderWare that are CSV formatted (UTF-16).
If you are on a *nix system you can use iconv
Example:
iconv -f UTF16 -t UTF8 file.csv