Question

I just got knocked down after our server has been updated from Debian 4 to 5.
We switched to UTF-8 environment and now we have problems getting the text printed correctly on the browser, because all files are in non-utf8 encodings like iso-8859-1, ascii, etc.

I tried many different scripts.

The first one I tried is "iconv". That one doesn’t work, it changes the content, but the file’s encoding is still non-utf8.

Same problem with enca, encamv, convmv and some other tools I installed via apt-get.

Then I found a python code, which uses chardet Universal Detector module, to detect encoding of a file (which works fine), but using the unicode class or the codec class to save it as utf-8 doesn’t work, without any errors.

The only way I found to get the file and its content converted to UTF-8, is vi.

These are the steps I do for one file:

vi filename.php
:set bomb
:set fileencoding=utf-8
:wq

That’s it. That one works perfect. But how can I get this running via a script?
I would like to write a script (Linux shell) which traverses a directory taking all php files, then converting them using vi with the commands above.
As I need to start the vi app, I do not know how to do something like this:

"vi --run-command=':set bomb, :set fileencoding=utf-8' filename.php"

Hope someone can help me.

Answer 1

This is the simplest way I know of to do this easily from the command line:

vim +"argdo se bomb | se fileencoding=utf-8 | w" $(find . -type f -name *.php)

Or better yet if the number of files is expected to be pretty large:

find . -type f -name *.php | xargs vim +"argdo se bomb | se fileencoding=utf-8 | w"