Question

I have a function that inserts a chunk of data into oracle database. I’m trying to achieve this by using executemany.

My function looks like this:

  def InsertChunk(self):
    try:
      if len(self.list_dict_values) >= self.chunksize:
        self.db.cursor.executemany(
          str(self.insert_sql),
          self.list_dict_values
         )
        self.list_dict_values = []
    except cx_Oracle.Error, e:
      print e

This function is used by many tables and works fine if those tables don’t have a CLOB column in them. It works on tables with CLOB columns only when chunksize is set to 1 or 2. Sometimes it works on 3, but most of the time it doesn’t. I even got it working once when the chunksize was 4. I’m using this function to set the chunk size to something around 1000 to speed up the process.

When the chunksize is set to 3, sometimes it returns the following error:

ORA-24813: cannot send or receive an unsupported LOB.

And sometimes it says aborted and stops the script.

Any idea why this script has a different behavior every time it is run with the same parameters?

Answer 1

I had the same problem. In my case it was caused by using the cx_Oracle variable types incorrectly. When filling out my equivalent of list_dict_values I was doing something like this:

for row in list_dict_values:
  for key, val in row.iteritems():
     v = cursor.var(cx_Oracle.CLOB)
     v.setvalue(0, val)
     row[key] = v
..
InsertChunk()

Instead of many small variables, you need to create a single variable with an arraysize, then reference it in every row of your dict.

lobdict = {}
for k in list_dict_vals[0].keys():
   lobdict[k] = cursor.var(cx_Oracle.CLOB, arraysize=len(list_dict_vals))
for rownum, row in enumerate(list_dict_values):
  for key, val in row.iteritems():
     lob = lobdict[key]
     lob.setvalue(rownum, val)
     row[key] = lob
...
InsertChunk()

It seems weird to set every row to the same value, but it works – internally the oracle code wants to iterate through a list of pointers, so that’s what you need to make.