Question

I’m trying to implement a delayed webcam viewer in javascript, using Three.js for WebGL capabilities.

I need to store frames grabbed from webcam, so they can be shown after some time (some milliseconds to some seconds). I’m able to do this without Three.js, using canvas and getImageData(). You can find an example on jsfidle.

I’m trying to find a way to do this without canvas, but using Three.js Texture or DataTexture object. Here an example of what I’m trying. The problem is that I cannot find how to copy the image from a Texture (image is of type HTMLVideoElement) to another.

In rotateFrames() function the older frame should be lost and newer should replace, like in a FIFO. But the line

frames[i].image = frames[i + 1].image;

is just copying the reference, not the texture data. I guess DataTexture should do this, but I’m not able to get a DataTexture out of a Texture or HTMLVideoElement.

Any idea?

Warning: you must have a camera and allow the browser to use it to get examples working. And obviously, you must have an updated browser.

Answer 1

Is this what your looking for??

http://fiddle.jshell.net/m4Bh7/10/

Edit:

function onFrame(dt) {
    if (video.readyState === video.HAVE_ENOUGH_DATA) { /* new frame available from webcam */
        context.drawImage(video, 0, 0,videoWidth,videoHeight);
        frames[framesNum - 1].image.data = new Uint8Array(context.getImageData(0,0,videoWidth, videoHeight).data.buffer);
        frames[framesNum - 1].needsUpdate = true;
    }
}

This is the most important part: It makes a copy of the frame and saves it as a data in a dataTexture.

function rotateFrames() {
    for (var i = 0; i != framesNum - 1; ++i) {
/*
         * FIXME: this does not work!
         */
        frames[i].image.data = frames[i + 1].image.data;
        frames[i].needsUpdate = true;
    }
}

This copies then the data from texture to texture in the frames.

new version:http://fiddle.jshell.net/hWL2E/4/

function onFrame(dt) {
    if (video.readyState === video.HAVE_ENOUGH_DATA) { /* new frame available from webcam */
        context.drawImage(video, 0, 0,videoWidth,videoHeight);
         var frame = new THREE.DataTexture(new Uint8Array(context.getImageData(0,0,videoWidth, videoHeight).data.buffer) ,videoWidth,videoHeight);
        frames[framesNum - 1] = frame;
        frames[framesNum - 1].needsUpdate = true;
        sprites[framesNum - 1].map = frames[framesNum - 1];
    }
}

It generates a new texture every frame for the video images.

function rotateFrames() {
    for (var i = 0; i != framesNum - 1; ++i) {
/*
         * FIXME: this does not work!
         */
        frames[i] = frames[i + 1];
        sprites[i].map = frames[i];

    }
}

this is what makes it better. Becuase it reuses the textures that were used, means that it doesn’t need to be resent to the GPU.