Question

I’ve written a wavefront Obj parser class, to import obj models into my OpenGL projects. I tested the class in debug mode, and found that it was unbearably slow.

The code works, and I made the obvious tweaks to ensure that it was as efficient as was reasonably practical.

Still, loading my test file, a 12mb obj file, which runs to about 330,000 lines of text, took over a minute to parse.

Frustrated, I had a Google, and sure enough, I wasn’t the first person to run into this problem.

This guy who posted his query on gamedev.net simply ran his algorithm in release mode, outside the Visual Studio IDE and whammo, acceptable performance. This also worked for me, my ~70 seconds was reduced to ~3 seconds.

I did some profiling of the algorithm, and the bottlenecks are in the calls to std::getline, and in the following:

sstream >> sToken;

Where sstream is an std::stringstream and sToken is an std::string (with space pre-reserved).

Question

Why is the IDE so unbelievably slow at running my parsing algorithm (even in release mode) – and is there anything I can do to speed this up when running the code through the IDE (F5 – run the project)? This is making debugging impossibly slow. Is the IDE injecting code / hooks into the executable for running through the IDE, or could this be put down to cache misses or something else?

Optimizations

I do two passes through the file, on pass one, I just count the token types – so that I can reserve space (rather than iteratively growing the vectors that store vertices, normals, texcoords, faces etc.)

sLineBuffer.reserve( 100 );
sToken.reserve(10);

while( sstream.good() )
{
    sstream >> sToken;
    getline( sstream, sLineBuffer );

    if( sToken.compare("f") == 0 )
        nFaces ++;

    else if( sToken.compare("v") == 0 )
        nVertices ++;

    else if( sToken.compare("vn") == 0 )
        nNormals ++;

    else if( sToken.compare("vt") == 0 )
        nTextures ++;

    else if( sToken.compare("g") == 0 )
        nGroups ++;
}

m_Vertices.reserve( nVertices );
m_Normals.reserve( nNormals );
m_TexCoords.reserve( nTextures );
m_Faces.reserve( nFaces );
m_Groups.reserve( nGroups );

This first pass costs little (~8 seconds in debug mode, or ~0.3 seconds in release mode outside the IDE) and the efficiency saving is huge (reduces parse time from ~180 seconds in debug mode to ~60 seconds).

I also read the entire file into a stringstream, so as to take disk access out of the equation:

// Read entire file from disk into memory
fstream stream;
stringstream sstream;
stream.open( m_sFilename.c_str(), std::ios::in );
sstream << stream.rdbuf();
stream.close();

Also, where possible, throughout the algorithm, I try to reserve space for std::strings ahead of time, so that they’re not being resized on a per character basis:

sLineBuffer.reserve( 100 );
sToken.reserve(10);  // etc

Answer 1

The STL is written in such a way so as to expect the compiler to do heavy inlining of many small functions. The debugger though allows you to step into all of the wonderful layers of abstraction, and you pay dearly for it during debug mode, because it can’t inline anything.

Normally I wouldn’t give the following advice but in the context of parsing OBJ files I suggest just throwing out the STL and relying on good old fashioned fscanf statements. You’ll find a significant gain during debugging, and even a noticeable improvement in speed during release modes.