Home/ Questions/Q 529125
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-13T09:03:31+00:00

I’ve copied certain files from a Windows machine to a Linux machine. All the

  • 0

I’ve copied certain files from a Windows machine to a Linux machine.
All the files encoded with Windows-1252 need to be converted to
UTF-8.
The files which are already in UTF-8 should not be changed.

I’m planning to use the recode
utility for that.
How can I specify that the recode utility should only convert
windows-1252 encoded files and not the UTF-8 files?

Example usage of recode:

recode windows-1252.. myfile.txt

This would convert myfile.txt from windows-1252 to UTF-8.
Before doing this, I would like to know that myfile.txt is actually
windows-1252 encoded and not UTF-8 encoded.
Otherwise, I believe this would corrupt the file.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    How would you expect recode to know that a file is Windows-1252? In theory, I believe any file is a valid Windows-1252 file, as it maps every possible byte to a character.

    Now there are certainly characteristics which would strongly suggest that it’s UTF-8 – if it starts with the UTF-8 BOM, for example – but they wouldn’t be definitive.

    One option would be to detect whether it’s actually a completely valid UTF-8 file first, I suppose… again, that would only be suggestive.

    I’m not familiar with the recode tool itself, but you might want to see whether it’s capable of recoding a file from and to the same encoding – if you do this with an invalid file (i.e. one which contains invalid UTF-8 byte sequences) it may well convert the invalid sequences into question marks or something similar. At that point you could detect that a file is valid UTF-8 by recoding it to UTF-8 and seeing whether the input and output are identical.

    Alternatively, do this programmatically rather than using the recode utility – it would be quite straightforward in C#, for example.

    Just to reiterate though: all of this is heuristic. If you really don’t know the encoding of a file, nothing is going to tell you it with 100% accuracy.

    • 0