Question

I have float alpha texture that contains amplitude values. It is converted to decibels and displayed in grayscale.

Here is conversation code (C++) :

const float db_min = -100, db_max = 0;
float image[height][width];
for (int y = 0; y<height; ++y) {
  for (int x = 0; x<width; ++x) {
    image[y][x]= 20.f * log(a[i])/log(10.f);
    image[y][x] = (image[y][x]-db_min)/(db_max-db_min);
  }
}

Here is Fragment Shader (GLSL):

#version 120
precision highp float;
varying vec2 texcoord;
uniform sampler2D texture;
void main() {
  float value = texture2D(texture, texcoord).a;
  gl_FragColor = vec4(value, value, value, 0);
}

Here is a screenshot:

Looks perfect! Now, I want to write conversation in Fragment Shader itself, instead of C++:

#version 120
precision highp float;
varying vec2 texcoord;
uniform sampler2D texture;
const float db_min = -100., db_max = 0.;
void main() {
  float value = texture2D(texture, texcoord).a;
  value = 20. * log(value)/log(10.);
  value = (value-db_min)/(db_max-db_min);
  gl_FragColor = vec4(value, value, value, 0);
}

Here is a screenshot:

Why are results different? What am I doing wrong?

Answer 1

Limited texel precision – that could be the problem. I can guess you’re keeping your (very-high-range as I understand looking at log) values in a 8-bit per-channel texture(e.g. RGBA8). Then you could use a floating-point format or pack your high-precision/range values to you 4-bytes format(e.g. a fixed point).