Question

I am currently using generators as a quick way to get the progress of long processes and I’m wondering how is it done usually as I find it not very elegant…

Let me explain first, I have a engine.py module that do some video processing (segmentation, bg/fg subtraction, etc) which takes a lot of time (from seconds to several minutes).

I use this module from a GUI written in wxpython and a console script.
When I looked at how to implement progress dialogs in wxpython, I saw that I must get somehow a progress value to update my dialog, which is pure logic you’ll admit…
So I decided to use the number of frame processed in my engine functions, yield the current frame number every 33 frames and yield None when the processing is finished.

by doing that here’s what it looks like:

dlg = wx.ProgressDialog("Movie processing", "Movie is being written...",
                           maximum = self.engine.endProcessingFrame,self.engine.startProcessingFrame,
                           parent=self,
                           style = wx.PD_APP_MODAL | wx.PD_ELAPSED_TIME | wx.PD_SMOOTH | wx.PD_CAN_ABORT)
state = self.engine.processMovie()
f = state.next()
while f != None:
    c, s = dlg.Update(f, "Processing frame %d"%f)
    if not c:break
    f = state.next()
dlg.Destroy()

That works very well, there is absolutely no noticeable speed loss, but I would like to be able to call processMovie() function without having to deal with generators if I don’t want to.

For instance my console script which uses the engine module doesn’t care of the progress, I could use it but it is destined to be executed in an environment where there is no display so I really don’t care about the progress…

Anyone with another design that the one I came up with? (using threads, globals, processes, etc)

There must be a design somewhere that does this job cleany I think 🙂

Answer 1

Using a generator is fine for this, but the whole point of using generators is so you can builtin syntax:

for f in self.engine.processMovie():
    c, s = dlg.Update(f, "Processing frame %d"%f)
    if not c: break

If you don’t care about that, then you can either say:

for f in self.engine.processMovie(): pass

or expose a function (eg. engine.processMovieFull) to do that for you.

You could also use a plain callback:

def update_state(f):
    c, s = dlg.Update(f, "Processing frame %d"%f)
    return c
self.engine.processMovie(progress=update_state)

… but that’s not as nice if you want to process the data piecemeal; callback models prefer to do all the work at once–that’s the real benefit of generators.