Question

I’m trying to write an Xvfb-to-HTML5-canvas tool that will need to know when an X11 window changes so it can send a screen update to the client. Think of it like a web-based VNC or RDP but just for X11 windows (why send the whole desktop? =).

I thought there would be a straightforward way to do this via Xlib or xcb (xpyb) but in my experiments the best I’ve been able to do is detect when a window is created, destroyed, or moved. That’s great and all but I need to know when the contents of windows change as well (imagine sending a keystroke to an xterm and having it appear frozen until you move the window).

If someone knows of a way to tell when the contents of an X11 window have changed I’d love to hear it! I’m open to creative solutions. For example, I tried using ffmpeg to stream x11grab through a fifo with regular checks to see if anything changed but it turned out to be extremely inefficient in terms of CPU utilization (it also seems to slow the whole system down even if nothing is going on).

I also tried just grabbing 15fps worth of screenshots in a loop while checking for changes in the most efficient way I could (e.g. does this cStringIO buffer match the last one?). That also was very CPU intensive.

The ideal solution would be for me to be able to watch the file descriptor of a socket and call a handler when there’s a change in the X11 window. I’m willing to settle for detecting when the whole X11 screen has a change… That’d still be better than what I’ve got.

Any and all help with this is appreciated!

Answer 1

First of all, you can actually use vnc to track changes in just one window, not whole desktop. From x11vnc documentation:

-id windowid           Show the X window corresponding to "windowid" not
                       the entire display.  New windows like popup menus,
                       transient toplevels, etc, may not be seen or may be
                       clipped.  Disabling SaveUnders or BackingStore in the
                       X server may help show them.  x11vnc may crash if the
                       window is initially partially obscured, changes size,
                       is iconified, etc.  Some steps are taken to avoid this
                       and the -xrandr mechanism is used to track resizes.  Use
                       xwininfo(1) to get the window id, or use "-id pick"
                       to have x11vnc run xwininfo(1) for you and extract
                       the id.  The -id option is useful for exporting very
                       simple applications (e.g. the current view on a webcam).
-sid windowid          As -id, but instead of using the window directly it
                       shifts a root view to it: this shows SaveUnders menus,
                       etc, although they will be clipped if they extend beyond
                       the window.


-appshare              Simple application sharing based on the -id/-sid
                       mechanism.  Every new toplevel window that the
                       application creates induces a new viewer window via
                       a reverse connection.  The -id/-sid and -connect
                       options are required.  Run 'x11vnc -appshare -help'
                       for more info.

If you want to code similar functionality manually you need to use damage extension.

Here is simple example in javascript using node-x11 (sorry, I’m not sure about damage extension support in python)

var x11 = require('x11');

var X = x11.createClient(function(err, display) {
    X.require('damage', function(Damage) {
        var damage = X.AllocID();
        Damage.Create(damage, parseInt(process.argv[2]), Damage.ReportLevel.NonEmpty);
        X.on('event', function(ev) {
          Damage.Subtract(damage, 0, 0);
          console.log("window content changed!");
        });
    });
});

start it with window id as command line argument and you’ll be notified whenever window content is changed.