Question

I’m writing a bulk insert script using Django’s ORM + custom raw SQL. The code has the following outline:

import sys, os
from django.core.management import setup_environ
from my_project import settings
from my_project.my_app.models import Model1, Model2
setup_environ(settings)
from django.db import transaction
from django.db import connection

@transaction.commit_manually
def process_file(relevant_file):

    data_file = open(relevant_file,'r')

    cursor = connection.cursor()

    while 1:
        line = data_file.readline()
        if line == '':
            break

        if not(input_row_i%1000):
            transaction.commit()

        if ([some rare condition]):
            model_1 = Model1([Some assignments based on line])
            model_1.save()

        values = [Some values based on line]
        cursor.execute("INSERT INTO `table_1` ('field_1', 'field_2', 'field_3') VALUES (%i, %f, %s)", values)

    data_file.close()
    transaction.commit()

I keep getting the following error:

django.db.transaction.TransactionManagementError: Transaction managed block ended with pending COMMIT/ROLLBACK

How can I solve this?

Answer 1

You could try a workaround – place a transaction.commit() right after the model_1.save(). I think you need to isolate raw and ORM transactions.