Question

I’m using CString::Tokenize method in order to tokenize a string using a delimiter, but I noticed something strange, I call that method on my string inside a loop because I want to retrieve all the tokens inside the string, here’s my code:

CString strToken;
for(int nTokenPos = 0; nTokenPos < dialog->myValue.GetLength(); nTokenPos++)
{
   //TRACE( "The Size of the string is %d\n", dialog->myValue.GetLength());
   TRACE( "Iteration No %d\n",nTokenPos);
   strToken = dialog->myValue.Tokenize(_T("X"), nTokenPos);
   strToken+="\n";
   OutputDebugString(strToken);
}

note:dialog->myValue is the string that I want to tokenize. When I test that code on ’99X1596′ (for example) the output is:

Iteration No 0
99
Iteration No 4
596

another example: ‘4568X6547’
output:

Iteration No 0
4568
Iteration No 6
547

I don’t know why it ignores the first character after the delimiter ‘X’ also it skips one iteration!

Answer 1

You increase nTokenPos in the for loop. That’s why second token is broken. CString::Tokenize updates nTokenPos and uses it in following iterations.

Correct usage is like this:

CString str = "99X1596";
int curPos = 0;
CString resToken = str.Tokenize(_T("X"), curPos);
while(!resToken.IsEmpty())
{
   // Process resToken here - print, store etc
   OutputDebugString(resToken);

   // Obtain next token
   resToken = str.Tokenize(_T("X"), curPos);
}