Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I found this in the wikipedia article on utf-8:
Sorting of UTF-8 strings as arrays of unsigned bytes will produce the same results as sorting them based on Unicode code points.
That would lead me to believe that for comparison purposes (sorting, binary search, etc) that comparing two byte arrays (i.e. byte-by-byte like memcmp) of utf-8 encoded strings would give the same results as comparing the actual unicode strings.
Is this true?
It depends on what you mean by “comparing the actual Unicode strings”.
If you’re just going to compare the code points (as 32-bit numbers) instead of the UTF-8 encoded code points, then the answer is yes: that will give the same results. The mapping from code points to UTF-8 encoded bytes is one-to-one.
If you’re going to do a proper Unicode string comparison, instead of bytewise comparison of the UTF-8, the answer is no. In Unicode, there can be different ways to represent the same character. For example, é can be represented in (at least) two ways:
U+00e9 (LATIN SMALL LETTER E WITH ACUTE), orU+0065 (LATIN SMALL LETTER E)followed byU+0301 (COMBINING ACUTE ACCENT).A properly written Unicode comparison function will consider these two to be identical.