Question

Since I couldn’t find anything on this in the documentation, I thought I ask it here. I have the following program (C++11):

#include <iostream> 
#include <boost/algorithm/string.hpp>

using namespace std;
using namespace boost;

int main () {
    string tmp = " #tag #tag1#tag2  #tag3 ####tag4   ";
    list<iterator_range<string::iterator> > matches;
    split( matches, tmp, is_any_of("\t #"), token_compress_on );

    for( auto match: matches ) {
            cout << "'" << match << "'\n";
    }
}

The output is:

''
'tag'
'tag1'
'tag2'
'tag3'
'tag4'
''

I would have thought that the token_compress_on option removes all empty tokens.
The solution is, for example, to use boost::trim_if. Nevertheless I was wondering if this is the desired behaviour of boost::split, and why this is happening?

(g++ 4.6.3, boost 1.48)

Answer 1

The behavior is intentional, because you could recreate the string (complete with starting and trailing spaces) from the split version. Boost doesn’t know if that whitespace is significant or not to you (it might be, as some file formats, for example, might force leading spaces/specific space counts).

You should trim_if or trim as you are if you do need to remove leading/trailing spaces.