Question

I have an Wavefront .obj file parser which parses data using getline and stringstream. At first when models were tiny, there was no problem, but now, when I try to load a model with ~207000 lines, only for the first pass where I count all elements it takes ridiculous amount of time (~4.7s) on high end PC and the second pass takes half a minute. Blender on the other hand loads whole model in mere 2 seconds or so. I use visual studio 2012, currently in debug mode.

My code for counting elements looks like this:

istringstream input(obj);
string line;
while (getline(input, line)) {
    if (line.find("# ") != string::npos) {
        // Comments.
    }
    else if (line.find("f ") != string::npos) {
        faces++;
    }
    else if (line.find("v ") != string::npos) {
        vertices += 3;
    }
    else if (line.find("vn ") != string::npos) {
        normals += 3;
    }
    else if (line.find("vt ") != string::npos) {
        uvCoordinates += 2;
    }
    else if (line.find("o ") != string::npos) {
        // Count here, if needed.
    }
}

Code for actually loading whole data that takes ~30s:

istringstream input(obj);
string line;
if (faces.capacity() > UINT_MAX / 3) {
    LOGE("Model cannot have more faces than: %d", UINT_MAX / 3);
    return false;
}
while (getline(input, line)) {
    vector<string> arr = stringSplit(line, ' ');
    string param = arr[0];
    int params = arr.size();
    if (line.length() == 0) {
        continue;
    }

    if (arr[0] == "v") { // Vertices.
        vertices.push_back(stringToFloat(arr[1].c_str()));
        vertices.push_back(stringToFloat(arr[2].c_str()));
        vertices.push_back(stringToFloat(arr[3].c_str()));
    }
    else if (arr[0] == "vn") { // Normals.
        normals.push_back(stringToFloat(arr[1].c_str()));
        normals.push_back(stringToFloat(arr[2].c_str()));
        normals.push_back(stringToFloat(arr[3].c_str()));
    }
    else if (arr[0] == "f") { // Faces.
        if (params < 4) {
            //LOGI("LINE: %s", line.c_str());
            continue;
        }
        else if (params > 4) {
            LOGI("Line: %s", line.c_str());
            LOGE("Obj models must only contain triangulated faces.");
            return false;
        }
        Face face;
        parseFace(face, line);
        faces.push_back(face);
    }
    else if (arr[0] == "vt") { // UV coordinates.
        uvCoordinates.push_back(stringToFloat(arr[1].c_str()));
        uvCoordinates.push_back(stringToFloat(arr[2].c_str()));
    }
    else if (arr[0] == "mtllib") { // Material.
        material = arr[1];
    }
    else if (arr[0] == "o") { // Sub-model.
        // Separate models here, if needed.
    }
}

obj variable is a string containing whole file content.
Removing everything from the inside of the first loop changes nothing for the time impact.
Any ideas on how to optimize this?

Answer 1

Zeroth, profile!

First, if you are using istringstream just to call getline() to get a line out of a string, instead, create your own function that will simply search forward for next '\n' and gives you the string. You will avoid a lot of overhead that way.

Second, avoid multi-pass algorithms. Why do you need to count the objects in advance?

Third, avoid unnecessary repeated memory allocation/construction and freeing/destruction.

Move the arr variable out of the loop. Rework stringSplit() to split into existing elements of the existing vector to avoid reallocation of the vector and the strings in it:

vector<string> arr = stringSplit(line, ' ');

Unless you are modifying the element of the vector and you do need a copy of the string here, avoid the copying, use reference to const string instead:

string param = arr[0];

Here, instead of variable, initialization, push_back(), resize the vector first and then call parseFace() on the last element of it:

Face face;
parseFace(face, line);
faces.push_back(face);

Avoid these long if/else if chains or at least sort them so that the most frequent entities are at the top of the chain. Better, make a switch using the first letter and full comparison only in switch-case block. Compiler can optimize switch statements into either balanced decision trees or jump tables.

if (arr[0] == "v") { // Vertices.
//...
}
else if (arr[0] == "vn") { // Normals.
//...
}
else if (arr[0] == "f") { // Faces.
//...
}
else if (arr[0] == "vt") { // UV coordinates.
//...
}
else if (arr[0] == "mtllib") { // Material.
//...
}
else if (arr[0] == "o") { // Sub-model.
//...
}

EDIT:

As for the first pass, how does it affect performance if you do not have it and the vectors are resizing on the fly?

If you reserve in advance space in your vectors for, say, 1000 faces, 1000 normals, 3000 vertices (assuming 1:1:3 is a typical ratio between these entities) etc. then your vectors will grow much faster and will avoid large part of the copy overhead on resize, compared to starting from empty vector.

As for the faces, I meant changing this:

Face face;
parseFace(face, line);
faces.push_back(face);

Into this (if you keep the push_back() style of apprach):

std::size_t const faces_size = faces.size();
faces.resize(faces_size + 1);
parseFace(faces.back());

In all cases make sure to

1. benchmark at least 3 runs
2. do a change that is supposed to improve things
3. benchmark again