Home/ Questions/Q 7516521
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-30T01:09:23+00:00

I’m using floats to specify texture coordinates, in the range 0-1. OpenGL likes things

  • 0

I’m using floats to specify texture coordinates, in the range 0-1. OpenGL likes things in this range, and I’m fine specifying coordinates this way, but I’m concerned when I start using larger textures (say up 4096 or 8192 pixels), that I may start losing precision. For example, if I want to specify a coordinate of (1,1) in a 8192x8192px texture, that would map to 1/8192=0.0001220703125. That seems to evaluate to 0.000122070313 as a float though… I’m concerned that my OpenGL shader won’t map that to the same pixel I intended.

I could keep the coordinates as integers in pixels for awhile, but sooner or later I have to convert it (perhaps as late as in the shader itself). Is there a workaround for this, or is this something I should even be concerned about?

Multiplying it back out, I get 1.000000004096 which I guess would still be interpreted as 1? Actually, OpenGL does blending if its not a whole number, doesn’t it? Perhaps not with “nearest neighbour”, but with “linear” it ought to.

1/4096f * 4096 =   1, error = 0
1/8192f * 8192 =   1.000000004096, error =  0.000000004096
1/16384f * 16384 = 1.0000000008192, error = 0.0000000008192
1/32768f * 32768 = 0.9999999991808, error = 0.0000000008192
...
1/1048576f * 1048576 = 0.9999999827968, error = 0.0000000172032

(I’m using Visual Studio’s debugger to compute the float, and then multiplying it back out with Calculator)

Is the lesson here that the error is negligible for any reasonably sized texture?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    That seems to evaluate to 0.000122070313 as a float though… I’m concerned that my OpenGL shader won’t map that to the same pixel I intended.

    You should not be concerned. Floating point is called floating point because the decimal floats. You get ~7 digits of precision for your mantissa, more or less regardless of how large or small the float is.

    The float isn’t stored as 0.000122070313; it’s stored as 1.22070313×10^-4. The mantissa is 1.22070313, the exponent is -4. If the exponent were -8 instead, you would have the same precision.

    Your exponent, with single-precision floats, can go down to + or – ~38. That is, you can have 38 zeros between the decimal and the first non-zero digit of the mantissa.

    So no, you shouldn’t be concerned.

    The only thing that should concern you would be the precision of the interpolated value and the precision in the texture fetching. But these have nothing to do with the precision of data you store your texture coordinates in.

    • 0