Question

I have the following string

an-ca
an-ca

If you will look it closely you will see that they are different!

To compare 2 string like this I found this solution:

if (String.Compare(str1, str2, StringComparison.InvariantCulture) == 0) ...

So I have 2 questions

1. Way W3C did characters so are the same in different languages – so make problems of comparision

2. How in C# I can convert the string to ‘InvariantCulture’

EDIT:

This link may explain the problem better

Thanks in advance

Answer 1

There are a lot of cases for mismatching especially with characters than have no visible differences. For example we assume that a space is always a 32 (0x20) byte. When the text comes from web a space may be a 160 (0xa0) which is the “ ” entity. And there are more spaces. The same may happen with hyphens, dots, commas etc. Check this page for more information and especially this one as it contains the most usual cases for this king of mismatches.
With the contained information you can write a function (preferably straight-forward for better performance – aka a long switch) to recreate a string that is already html decoded.

As a proof of concept I’m including the code I’m using for creating a friendly URL and at the same time keep a dictionary for tagging based on titles. It covers only Windows Western and Central European character sets including thorn and eth. As you see there are some tricky parts and of course a lot of space for customization. This spaghetti code makes only one pass to the initial string.

public static string Latin(this string s, ref Dictionary<string, int> dict)
{
    StringBuilder
        f = new StringBuilder(new string(' ', s.Length * 5)),
        w = new StringBuilder(50);
    string word = "";
    int index = -1, len = s.Length, position = -1, ws = 0;
    while (++index < len)
    {
        if (s[index] > '\x007f')
        {
            switch (s[index])
            {
                case 'á':
                case 'Á':
                case 'à':
                case 'À':
                case 'â':
                case 'Â':
                case 'ä':
                case 'Ä':
                case 'ă':
                case 'Ă':
                case 'ā':
                case 'Ā':
                case 'ã':
                case 'Ã':
                case 'å':
                case 'Å':
                case 'ą':
                case 'Ą':
                    f[++position] = 'a';
                    break;
                case 'æ':
                case 'Æ':
                    f[++position] = 'a';
                    f[++position] = 'e';
                    break;
                case 'ć':
                case 'Ć':
                case 'č':
                case 'Č':
                case 'ç':
                case 'Ç':
                    f[++position] = 'c';
                    break;
                case 'ď':
                case 'Ď':
                case 'đ':
                case 'Đ':
                    f[++position] = 'd';
                    break;
                case 'é':
                case 'É':
                case 'è':
                case 'È':
                case 'ė':
                case 'Ė':
                case 'ê':
                case 'Ê':
                case 'ë':
                case 'Ë':
                case 'ě':
                case 'Ě':
                case 'ē':
                case 'Ē':
                case 'ę':
                case 'Ę':
                    f[++position] = 'e';
                    break;
                case 'ğ':
                case 'Ğ':
                case 'ģ':
                case 'Ģ':
                    f[++position] = 'g';
                    break;
                case 'ı':
                case 'í':
                case 'Í':
                case 'ì':
                case 'Ì':
                case 'İ':
                case 'î':
                case 'Î':
                case 'ï':
                case 'Ï':
                case 'ī':
                case 'Ī':
                case 'į':
                case 'Į':
                    f[++position] = 'i';
                    break;
                case 'ķ':
                case 'Ķ':
                    f[++position] = 'k';
                    break;
                case 'ĺ':
                case 'Ĺ':
                case 'ľ':
                case 'Ľ':
                case 'ļ':
                case 'Ļ':
                case 'ł':
                case 'Ł':
                    f[++position] = 'l';
                    break;
                case 'ń':
                case 'Ń':
                case 'ň':
                case 'Ň':
                case 'ñ':
                case 'Ñ':
                case 'ņ':
                case 'Ņ':
                    f[++position] = 'n';
                    break;
                case 'ó':
                case 'Ó':
                case 'ò':
                case 'Ò':
                case 'ô':
                case 'Ô':
                case 'ö':
                case 'Ö':
                case 'ō':
                case 'Ō':
                case 'õ':
                case 'Õ':
                case 'ő':
                case 'Ő':
                case 'ø':
                case 'Ø':
                    f[++position] = 'o';
                    break;
                case 'œ':
                case 'Œ':
                    f[++position] = 'o';
                    f[++position] = 'e';
                    break;
                case 'ŕ':
                case 'Ŕ':
                case 'ř':
                case 'Ř':
                case 'ŗ':
                case 'Ŗ':
                    f[++position] = 'r';
                    break;
                case 'ś':
                case 'Ś':
                case 'š':
                case 'Š':
                case 'ş':
                case 'Ş':
                    f[++position] = 's';
                    break;
                case 'ß':
                    f[++position] = 's';
                    f[++position] = 's';
                    break;
                case 'ť':
                case 'Ť':
                case 'ţ':
                case 'Ţ':
                    f[++position] = 't';
                    break;
                case 'ð':
                case 'Ð':
                case 'þ':
                case 'Þ':
                    f[++position] = 't';
                    f[++position] = 'h';
                    break;
                case 'ú':
                case 'Ú':
                case 'ù':
                case 'Ù':
                case 'û':
                case 'Û':
                case 'ü':
                case 'Ü':
                case 'ū':
                case 'Ū':
                case 'ů':
                case 'Ů':
                case 'ų':
                case 'Ų':
                case 'ű':
                case 'Ű':
                    f[++position] = 'u';
                    break;
                case 'ý':
                case 'Ý':
                case 'ÿ':
                case 'Ÿ':
                    f[++position] = 'y';
                    break;
                case 'ź':
                case 'Ź':
                case 'ż':
                case 'Ż':
                case 'ž':
                case 'Ž':
                    f[++position] = 'z';
                    break;
                case '\x00a0': // no-break space
                case '\x2000': // other spaces
                case '\x2001':
                case '\x2002':
                case '\x2003':
                case '\x2004':
                case '\x2005':
                case '\x2006':
                case '\x2007':
                case '\x2008':
                case '\x2009':
                case '\x200a':
                case '\x200b':
                case '\x200c':
                case '\x200d':
                    if (position > -1 && f[position] != '-') f[++position] = '-';
                    break;
            }
        }
        else if ((s[index] > '\x002f' && s[index] < '\x003a') || (s[index] > '\x0060' && s[index] < '\x007b'))
        {
            f[++position] = s[index];
        }
        else if ((s[index] > '\x0040' && s[index] < '\x005b'))
        {
            f[++position] = (char)(s[index] | ' ');
        }
        else
        {
            switch (s[index])
            {
                case '#':
                    if (position > 0 && (f[position] == 'c' || f[position] == 'j' || f[position] == 'f'))
                    {
                        f[++position] = 's';
                        f[++position] = 'h';
                        f[++position] = 'a';
                        f[++position] = 'r';
                        f[++position] = 'p';
                    }
                    else if (index + 1 < len && ((s[index + 1] >= '0') && (s[index + 1] <= '9')))
                    {
                        f[++position] = 'n';
                        f[++position] = 'o';
                    }
                    else if (position > -1 && f[position] != '-') f[++position] = '-';
                    break;
                case '+':
                    f[++position] = 'p';
                    f[++position] = 'l';
                    f[++position] = 'u';
                    f[++position] = 's';
                    break;
                case '.':
                    //f[++position] = 'd';
                    //f[++position] = 'o';
                    //f[++position] = 't';
                    break;
                case '-':
                    break;
                default:
                    if (position > -1 && f[position] != '-') f[++position] = '-';
                    break;
            }
        }
        if (f[position] == '-' && ws < position)
        {
            for (var i = ws; i < position; i++) w.Append(f[i]);
            word = w.ToString();
            if (dict.ContainsKey(word)) dict[word] += 1;
            else dict.Add(word, 1);
            w = new StringBuilder(50);
            ws = position + 1;
        }
    }
    if (f[position] == '-') position--;
    return f.ToString(0, position + 1);
}

Use it as a guide for you initial requirement.
For the record it generates “c-cplusplus-articles” for “C/C++ articles” and not “c-c—articles” or “cc-articles”.