Question

A recent problem* left me wondering whether there is a text editor out there that lets you see every single character of the file, even if they are invisible? Specifically, I’m not looking for hex editing capabilities, I am interested in a text editor that’ll show me all of the invisible characters (not just the common whitespace / line break characters). The BOM marker is just one example, others are e.g. mathematical invisibles or possibly unsupported characters.

I’m not looking for a text editor that simply supports a large variety of text encoding / translations between encodings. All text editors I’ve come across treat the invisible characters correctly i.e. leave them invisible (or simply get removed in the translation as in the case of the BOM marker).

I’m asking this mostly out of academic interests, so I’m not particular about any specific OS. I can easily test Linux and OSX solutions, but if you recommend a Windows editor, I would appreciate if you include descriptions of how the editor handles invisibles other than whitespace / line breaks.

EDIT: I’m beginning to be sure that the behavior I want can be implemented in emacs/vim via either custom highlighting or by messing around with the font itself. A solution of this type would also be acceptable.

EDIT2: After looking at several options I found TextMate which at least shows a blank space where an invisible UTF-8 character is in the file. Slightly disappointed with SO’s ability to answer my question. Bounty goes to VIM, because that is the direction in which the solution most likely lies.

---

*The incident that lead me to this question: I wrote a perl script using TextWrangler and managed to change the encoding to UTF8 BOM, which inserts the BOM marker at the start of the file. Perl (or rather the operating system) promptly misses the #! and mayhem ensues. It then took me the better part of an afternoon to figure this out since most text editors do not show the BOM marker even with various “show invisibles” options turned on. Now I’ve learned my lesson and will use less immediately :-).

Answer 1

vim (in either textual or graphic mode) can show all control characters if you :set list. The BOM is a special case, controlled by the :set bomb or :set nobomb commands.