Question

We have recently upgraded all our projects from .NET 3.5 to .NET 4. I have come across a rather strange issue with respect to string.IndexOf().

My code obviously does something slightly different, but in the process of investigating the issue, I found that calling IndexOf() on a string with itself returned 1 instead of 0.
In other words:

string text = "\xAD\x2D";          // problem happens with "­-dely N.China", too;
int index = text.IndexOf(text);    // see update note below.

Gave me an index of 1, instead of 0. A couple of things to note about this problem:

• The problems seems related to these hyphens (the first character is the Unicode soft hyphen, the second is a regular hyphen).

• I have double checked, this does not happen in .NET 3.5 but does in .NET 4.

• Changing the IndexOf() to do an ordinal compare fixes the issue, so for some reason that first character is ignored with the default IndexOf.

Does anyone know why this happens?

EDIT

Sorry guys, made a bit of a stuff up on the original post and got the hidden dash in there twice. I have updated the string, this should return index of 1 instead of 2, as long as you paste it in the correct editor.

Update:

Changed the original problem string to one where every actual character is clearly visible (using escaping). This simplifies the question a bit.

Answer 1

Your string exists of two characters: a soft hyphen (Unicode code point 173) and a hyphen (Unicode code point 45).

Wiki: According to the Unicode standard, a soft hyphen is not displayed if the line is not broken at that point.

When using "\xAD\x2D".IndexOf("\xAD\x2D") in .NET 4, it seems to ignore that you’re looking for the soft hyphen, returning a starting index of 1 (the index of \x2D). In .NET 3.5, this returns 0.

More fun, if you run this code (so when only looking for the soft hyphen):

string text = "\xAD\x2D";
string shy = "\xAD";
int i1 = text.IndexOf(shy);

then i1 becomes 0, regardless of the .NET version used. The result of text.IndexOf(text); varies indeed, which at a glance looks like a bug to me.

As far as I can track back through the framework, older .NET versions use an InternalCall to IndexOfString() (I can’t figure out to which API call that goes), while from .NET 4 a QCall to InternalFindNLSStringEx() is made, which in turn calls FindNLSStringEx().

The issue (I really can’t figure out if this is intended behaviour) indeed occurs when calling FindNLSStringEx:

LPCWSTR lpStringSource = L"\xAD\x2D";
LPCWSTR lpStringValue = L"\xAD";

int length;

int i = FindNLSStringEx(
    LOCALE_NAME_SYSTEM_DEFAULT,
    FIND_FROMSTART,
    lpStringSource,
    -1,
    lpStringValue,
    -1,
    &length,
    NULL,
    NULL,
    1);

Console::WriteLine(i);

i = FindNLSStringEx(
    LOCALE_NAME_SYSTEM_DEFAULT,
    FIND_FROMSTART,
    lpStringSource,
    -1,
    lpStringSource,
    -1,
    &length,
    NULL,
    NULL,
    1);

Console::WriteLine(i);

Console::ReadLine();

Prints 0 and then 1. Note that length, an out parameter indicating the length of the found string, is 0 after the first call and 1 after the second; the soft hyphen is counted as having a length of 0.

The workaround is to use text.IndexOf(text, StringComparison.OrdinalIgnoreCase);, as you’ve noted. This makes a QCall to InternalCompareStringOrdinalIgnoreCase() which in turn calls FindStringOrdinal(), which returns 0 for both cases.