Question

I am trying to make an app which use displacement map for 3d effect. In Flash it can be done using displacement filter like shown here: http://flashflex.com/pixel-bender-displacement-map-fakes-3d-effect-update/. There are few apps who do that on iPhone like one here: http://www.youtube.com/watch?v=NvCHHUN8nnE

I am wondering how do they perform this.

I am trying to use displacement map based technique describe here: http://www.cocos2d-iphone.org/forum/topic/11659 but it seems to be slow to be done on the fly.

Any pointer would be highly appreciated.

Thanks

Answer 1

Perhaps the simplest way to achieve displacement mapping in OpenGL ES 2.0 is also probably the fastest. You need two textures, one to contain color (a regular one), and a displacement map. When sampling the regular texture normally, your fragment shader would look like:

uniform sampler2D myTexture;
varying vec2 texcoords;

void main()
{
    gl_FragColor = texture2D(myTexture, texcoords);
}

This is pretty simple stuff. Now, to achieve displacement mapping, you need to:

uniform sampler2D myTexture;
uniform sampler2D myDisplacementMap;
varying vec2 texcoords;

void main()
{
    vec2 displacement = 0.1 * texture2D(myDisplacementMap, texcoords).rg;
    gl_FragColor = texture2D(myTexture, texcoords + displacement);
}

This is also simple. You have another texture where red and green are used as x and y for the displacement. This way, however, you only get static, view-independent displacement. To get real displacement, one would need to generate texture tangents and bitangents which contain the directions of texture axes in object space (a tricky concept, read the article). Once you have them, all you need is object-space eye position (can be calculated by multiplying vertex position by modelview-projection inverse in vertex shader), and by projecting this on the tangent/bitangent vectors, you can modulate the displacement in order to be view dependent:

attribute vec3 position, tangent, bitangent;
attribute vec2 texcoord;
varying vec2 texcoords;
varying vec3 eye_dir, tan, bitan;
uniform matrix4f modelviewProjectionMatrix;
uniform matrix4f modelviewProjectionMatrixInverse;

void main()
{
    gl_Position = modelviewProjectionMatrix * vec4(position, 1.0);
    tan = tangent;
    bitan = bitangent;
    texcoords = texcoord;
}

(that was vertex shader), and fragment shader here:

uniform sampler2D myTexture;
uniform sampler2D myDisplacementMap;
varying vec2 texcoords;
varying vec3 eye_dir, tan, bitan;

void main()
{
    vec2 eyen = normalize(eye_dir);
    vec2 modulation = vec2(dot(eyen, normalize(tan)), dot(eyen, normalize(bitan)));
    vec2 displacement = 0.1 * texture2D(myDisplacementMap, texcoords).rg * modulation;
    gl_FragColor = texture2D(myTexture, texcoords + displacement);
}

And that should do the trick … (note i wrote that from head so if there are any errors, don’t hesitate to comment)

EDIT: And there were errors, I swapped order of arguments to texture2D (sampler goes first).