Question

Is there a way to write a callback function for pycurl’s READFUNCTION that does not return a string? I am planning on sending blocks of binary data via pycurl. i tried writing a callback function that does this:

def read_callback(self, size):
  for block in data:
    yield block

but pycurl exits with an error that says the return type must be a string.

Answer 1

The “read function” must return a string of bytes — remember, libcurl is a wrapper on an underlying C library, so of course it’s type-picky!-). However, it can perfectly be a binary string of bytes (in Python 2.* at least — I don’t think pycurl works with Python 3 anyway), so of course it can return “blocks of binary data” — as long as they’re encoded into strings of bytes and respect the size constraint.

What you just encoded, when called, returns a “generator function” — obviously no good. Assuming data is a list of non-empty byte strings, you need to dice and slice it appropriately, returning each time a string of bytes, with return, definitely not yield (!).

Not sure where that mysterious data of yours comes from — let’s assume you mean self.data instead. Then you need to keep track, in other instance variables, of the current index into self.data and possibly — if the items are longer than size — of the “yet unsent part” of the current piece, too.

E.g., it could be:

class MySender(object):
  def __init__(self, data):
    self.data = data
    self.i = 0
    self.rest = 0
  def readfunction(self, size):
    if self.i >= len(self.data):
      return ''
    result = self.data[self.i][self.rest:self.rest+size]
    self.rest += size
    if self.rest >= len(self.data[self.i]):
      self.i += 1
      self.rest = 0

If self.data’s items are other kinds of binary data (not already encoded as byte strings), you can turn them into byte strings e.g. with help from Python library modules such as struct and array.