Question

I’m trying to split a CString in two whenever there is a "\t" (tab space) character. This is the code I’m using:

CString val = data->str;
CString valTok;
CString accTok;

int indx = 0;
valTok = val.Tokenize(_T("\t"), indx);
++indx;
accTok = val.Tokenize(_T("\t"), indx);

Its great and all, but the first character of accTok is missing. For example, when val = "String1\tString2", valTok = "String1" but accTok ="tring2" … how come?

EDIT:

So following hvd’s advice I removed the indx increment and I have the following code:

CString val = ((CItemData*)lpDrawItemStruct->itemData)->str; // Was this value causing the problem in the end
CString valTok;
CString accTok;

int indx = 0;
valTok = val.Tokenize(_T("\t"), indx);
accTok = val.Tokenize(_T("\t"), indx); // ASSERT(iStart >= 0) fails

But now the first Tokenize returns -1 for some reason..! Any ideas?

SOLVED:

The problem was in the value passed to val:

((CItemData*)lpDrawItemStruct->itemData)->str 

which was sometimes NULL, causing an assertion when calling Tokenize a second time in some cases.

Answer 1

Lose the ++indx.

From the documentation of CStringT::Tokenize:

The value of iStart is updated to be the position following the ending delimiter character, or -1 if the end of the string was reached.

After the first call to Tokenize, indx already points past the '\t', it already points to the 'S' of “String2”. By incrementing indx, it will point to the t.

Here’s a simple sanity check for Tokenize:

CString val;
CString str1, str2, str3;
int indx;

val  = _T("");
indx = 0;
str1 = val.Tokenize(_T("\t"), indx);
assert(str1 == _T("") && indx == -1);

val  = _T("String1");
indx = 0;
str1 = val.Tokenize(_T("\t"), indx);
assert(str1 == _T("String1") && indx >= 0);
str2 = val.Tokenize(_T("\t"), indx);
assert(str2 == _T("") && indx == -1);

val  = _T("String1\tString2");
indx = 0;
str1 = val.Tokenize(_T("\t"), indx);
assert(str1 == _T("String1") && indx >= 0);
str2 = val.Tokenize(_T("\t"), indx);
assert(str2 == _T("String2") && indx >= 0);
str3 = val.Tokenize(_T("\t"), indx);
assert(str3 == _T("") && indx == -1);

Do you get different results?