Question

I’m having a video copying pixels to a canvas and applying a real time bitmap filter using a web worker.

When using the web worker I can’t see the resulting change on the canvas (should be black ‘n white).

This is my code:
http://www.dev.createdbychristian.com/filter-test/

I can solve this by using an intermediate canvas, but I don’t really want to do that.

My code is loosely based on this example, same as mine just not using a web worker for the image filter processing.
http://techslides.com/demos/html5-video-canvas.html

What is wrong with my code?

NB: Requires a modern browser with support for Web Workers and HTML5 Video / canvas

Answer 1

This happens because draw() doesn’t wait for the return value of your WebWorker and uses requestAnimationFrame. If you want continuously filtered images you have to call draw after the filter has been applied:

filterWorker.addEventListener('message', function(e) {
    ctx.putImageData(e.data, 0, 0);
    requestAnimationFrame(draw);
}, false);

However, if the filter takes to long it is very likely that draw will fill the canvas with colored data to early to notice the change (I was able to see the black ‘n white version with FF17 in your current version).

What is wrong with my code?

You’re using a facility for parallel computation in a scenario where the order of operations is important: you want to apply the filter on each image before it is shown. Let me quote MDN:

About thread safety

The Worker interface spawns real OS-level threads, and mindful programmers may be concerned that concurrency can cause “interesting” effects in your code if you aren’t careful.

However, since web workers have carefully controlled communication points with other threads, it’s actually very hard to cause concurrency problems. There’s no access to non-threadsafe components or the DOM. And you have to pass specific data in and out of a thread through serialized objects. So you have to work really hard to cause problems in your code.

And as it is with real OS-level threads no assumptions in the order of execution can be made unless you use some blocking mechanism, but this isn’t included in JavaScript. So you’ll need to draw and manipulate the image in the same thread.

Further explanation

Sequential case

Have a look at the following picture:

This is what basically happens in the techdemo. draw() copies the video and immediately filters the image:

function draw(v,c,w,h,filter) {
    if(v.paused || v.ended) return false;
    c.drawImage(v,0,0,w,h);
    if (typeof filter === 'undefined') {
        // variable is undefined
    } else {
        var idata = c.getImageData(0,0,w,h);
        newdata = filterdata(idata,filter);
        c.putImageData(newdata,0,0);
    }
    setTimeout(draw,20,v,c,w,h,filter);
}

As you can see there is no gap between the the call of drawImage the following filter. The resulting picture is shown for approximately 20 milliseconds. Even if the filter would take 1ms the user likely won’t notice this.

Parallel case

Now it gets a little bit more complicated. Blackened text boxes are methods which modify the canvas directly, white boxes indicate a triggered event or registered timeout, and black boxes indicate the execution of the timeout or handling of the event:

Now, what happens? draw is still called approximately every 20ms and copies the video. Then it posts a message to the worker. However, it isn’t stated when this message is going to be sent. It could be placed in the worker’s event loop immediately (as shown in the picture), or in the event loop of "main" and dispatched later (which is more likely).

This process takes very, very long in comparison to the sequential method. Keep in mind that you also need to send the processed data back and that both filter and filterWorker‘s event listener may be triggered at a later time than you originally expected. But even if you actually process the data and display it, a following call to draw will destroy all your hard work.

Solution

Either don’t use a web worker, or use a hidden canvas as source for the worker.