Question

When I use my shaders I get following results:

One problem is that specular light is sort of deformed and you could see sphere triangles, another is, that I can see specular where I shouldn’t (second image). One spehere light is done in vertex shader, other in fragment.

Here is how my vertex light shader looks like:
Vertex:

// Material data.
uniform vec3 uAmbient;
uniform vec3 uDiffuse;
uniform vec3 uSpecular;
uniform float uSpecIntensity;
uniform float uTransparency;

uniform mat4 uWVP;
uniform mat3 uN;
uniform vec3 uSunPos;
uniform vec3 uEyePos;
attribute vec4 attrPos;
attribute vec3 attrNorm;
varying vec4 varColor;

void main(void)
{
    vec3 N = uN * attrNorm;
    vec3 L = normalize(uSunPos);
    vec3 H = normalize(L + uEyePos);
    float df = max(0.0, dot(N, L));
    float sf = max(0.0, dot(N, H));
    sf = pow(sf, uSpecIntensity);

    vec3 col = uAmbient + uDiffuse * df + uSpecular * sf;
    varColor = vec4(col, uTransparency);
    gl_Position = uWVP * attrPos;
}

Fragment:

varying vec4 varColor;

void main(void)
{

    //vec4 col = texture2D(texture_0, varTexCoords);
    //col.r += uLightDir.x;
    //col.rgb = vec3(pow(gl_FragCoord.z, 64));
    gl_FragColor = varColor;

}

It is possible, that my supplied data from code is wrong.
uN is world matrix (not inverted and not transposed, even though doing that didn’t seem to do anything different).
UWVP – world view projection matrix.

Any ideas, as to where the problem might be, would be appreciated.

[EDIT] Here is my light calculations done in fragment:
Vertex shader file:

uniform mat4 uWVP;
uniform mat3 uN;
attribute vec4 attrPos;
attribute vec3 attrNorm;
varying vec3 varEyeNormal;

void main(void)
{
    varEyeNormal = uN * attrNorm;
    gl_Position = uWVP * attrPos;
}

Fragment shader file:

// Material data.
uniform vec3 uAmbient;
uniform vec3 uDiffuse;
uniform vec3 uSpecular;
uniform float uSpecIntensity;
uniform float uTransparency;

uniform vec3 uSunPos;
uniform vec3 uEyePos;
varying vec3 varEyeNormal;

void main(void)
{
    vec3 N = varEyeNormal;
    vec3 L = normalize(uSunPos);
    vec3 H = normalize(L + uEyePos);

    float df = max(0.0, dot(N, L));
    float sf = max(0.0, dot(N, H));
    sf = pow(sf, uSpecIntensity);

    vec3 col = uAmbient + uDiffuse * df + uSpecular * sf;

    gl_FragColor = vec4(col, uTransparency);
}

[EDIT2] As Joakim pointed out, I wasn’t normalizing varEyeNormal in fragment shader. After fixing that, the result is much better in fragment shader. I also used normalize function on uEyePos, so specular no longer goes to dark side. Thanks for all the help.

Answer 1

Short answer: You need to normalize varEyeNormal in the fragment shader, instead of in the vertex shader.

Longer answer:
In order to get smooth lighting you will need to calculate the normals per pixel instead of per vertex. Varyings computed in the vertex shader are linearily interpolated before passed to the fragment shader, which works great as a shortcut in some cases but worse in others.

The reason you are seeing the triangle edges is because of the interpolation between normals result in shorter than 1.0 normals in all pixels between the vertices.

To correct this you will need to normalize the normals in the fragment shader instead of in the vertex shader.

The Normal matrix should be upper 3×3 of the transpose of the inverse of the Modelview matrix, which is equivalent to the upper 3×3 of of the Modelview matrix if the Modelview contains only rotations and translations (no scaling)

For more about the Normal matrix, see: http://www.lighthouse3d.com/tutorials/glsl-tutorial/the-normal-matrix/

(Edited for correctness according to comment below.)