Question

I have a problem with saving a huge amount of records to database using CFWheels. Here is an example:

<cfloop from="1" to="10000" index="i">
 <cfset var newUser = model("user").new()>
 <cfset newUser.name = "Test"&i>
 <cfset newUser.save()>
</cfloop>

This causes java.lang.OutOfMemoryError

Please help me how to solve this problem.

Answer 1

There are a couple fairly inefficient things going on here. First, it’s generating 1,000 user objects, which isn’t really a good idea to do in a single request in ColdFusion. Second, it’s running 1,000 database queries, which isn’t really a good idea to do in any programming language.

I would stop using model objects for a case like this and figure out how to condense the logic into a single database query. The ORM stuff in Wheels is generally very useful, but it has its limits in situations like this one.

For example, if you’re using SQL Server 2008, you can do this inside your user model to keep everything under a single call to cfquery:

<cffunction name="batchCreate">
    <cfquery datasource="#get('dataSourceName')#">
        INSERT INTO
            #this.tableName()# (#this.columnNameForProperty("name")#)
        VALUES
            <cfloop from="1" to="10000" index="i">
                (<cfqueryparam cfsqltype="cf_sql_varchar" value="Test#i#">)
                <cfif i lt 10000>,</cfif>
            </cfloop>
    </cfquery>
</cffunction>

Of course, the query will look different if you’re using MySQL or another database engine.