Question

I’m looking for a way in SQLAlchemy to do a bulk INSERT whose rows are the result of a query. I know the session has the function add which can be used to add an individual object, but I can’t seem to find how how it works with a subquery.

I know I could iterate over the results of the subquery and add them individually, but this would seem to be somewhat inefficient. In my case I am dealing with a potentially very large set of data that needs insertion.

Answer 1

I see following options:

1. using SA Model: create underlying objects with data loaded from the database, add them to session and commit.
   • Pros: if you have any AS Model level validation, you are able to use it; also you can insert into multiple tables if you model objects are mapped to multiple tables (Joined-Table Inheritance); is RDBMS independent
   • Cons: most expensive
2. using Insert statements: load data from the database into python and execute using Insert Expressions
   • Pros: somewhat faster when compared to 1.
   • Cons: still expensive as python structures are created; cannot directly handle Joined-Table Inheritance
3. create data using solely RDBMS: bulk insert using RDBMS only bypassing SA and python altogether.
   • Pros: fastest
   • Cons: no business object validation performed; potentially RDBMS-specific implementation required

I would suggest either option 1) or 3).
In fact, if you do not have any object validation and you use only one RDBMS, I would stick to option 3).