Question

I want to process a txt file using perl. I have succeeded in using this perl code to process wikipedia txt file before, so it is correct. But when I run the perl code, there appears an error:

utf8 "\xA1" does not map to Unicode at xxx

I check the txt file. There is \xA1 characters in it. I tried to remove those “\xA1” with RegEx s/(\xa1)+//g; but failed.

So what is the problem and how can I solve this?

Answer 1

At last, I revise my code. I use

binmode( IN,  ':stdio' );

instead of

binmode( IN,  ':utf8' );

and it works.