Question

How would you go about translating a document that contains the following character references to their actual readable characters in a bash script?

ā á ǎ à ē é ě è ī í ǐ ì ǖ ǘ ǚ ǜ ü ǖ ǘ ǚ ǜ ü 

These change in order to ā á ǎ à ē é ě è ī í ǐ ì ǖ ǘ ǚ ǜ ü ǖ ǘ ǚ ǜ ü

Answer 1

If you have access to Perl then it’s relatively simple:

perl -ne 'binmode STDOUT,':utf8';s/&#([0-9]*);/pack('U',$1)/eg;print' \   document.html 

Example:

#!/bin/bash html2utf8() {   perl -ne 'binmode STDOUT, ':utf8'; s/&#([0-9]*);/pack('U',$1)/eg; print' } echo 'testing 1 ā 2 Ĭ 3 ē' | html2utf8 

Produces:

testing 1 ā 2 Ĭ 3 ē