Question

I’d like to analyze the constantly updating image feed that comes from an iPhone camera to determine a general “lightness coefficient”. Meaning: if the coefficient returns 0.0, the image is completely black, if it returns 1.0 the image is completely white. Of course all values in between are the ones that I care about the most (background info: I’m using this coefficient to calculate the intensity of some blending effects in my fragment shader).

So I’m wondering if I should run a for loop over my pixelbuffer and analyze the image every frame (30 fps) and send the coeff as a uniform to my fragment shader or is there a way to analyze my image in OpenGL. If so, how should I do that?

Answer 1

There are more answers, each one with its own strong and weak points.

On CPU, it’s fairly simple: cycle through the pixels, sum them up, divide, and that’s it. It’s a five minutes work. It will take a few milliseconds with a good implementation.

int i, sum = 0, count = width * height * channels;
for(i=0;i<count;i++)
    avg += buffer[i];
double avg = double(sum) / double(count);

On GPU, it is likely to be much faster, but there are a few drawbacks: First one is the amount of work needed just to put everything in place. The GPUImage framework will save you some work, but it will also add a lot of code. If all you want to do is to sum the pixels, it may be a waste. Second problem is that sending pixels to GPU may take more than summing them up in the CPU. Actually, the GPU will justify the work only if you really need serious processing.

A third option, to use the CPU with a library, has the drawback that you add a lot of code for what you can do in 10 lines. But the result will be beautiful. Again, it justify if you also use the lib for other tasks. Here is an example in OpenCV:

cv::Mat frame(buffer, width, height, channels, type);
double avgLuminance = cv::sum(frame)/(double(frame.total()*frame.channels()));