Question

PHP’s str_replace() was intended only for ANSI strings and as such can mangle UTF-8 strings. However, given that it’s binary-safe would it work properly if it was only given valid UTF-8 strings as arguments?

Edit: I’m not looking for a replacement function, I would just like to know if this hypothesis is correct.

Answer 1

Yes. UTF-8 is deliberately designed to allow this and other similar non-Unicode-aware processing.

In UTF-8, any non-ASCII byte sequence representing a valid character always begins with a byte in the range \xC0-\xFF. This byte may not appear anywhere else in the sequence, so you can’t make a valid UTF-8 sequence that matches part of a character.

This is not the case for older multibyte encodings, where different parts of a byte sequence are indistinguishable. This caused a lot of problems, for example trying to replace an ASCII backslash in a Shift-JIS string (where byte \x5C might be the second byte of a character sequence representing something else).