Question

I need some hibernate/SQL help, please. I’m trying to generate a report against an accounting database. A commission order can have multiple account entries against it.

        class CommissionOrderDAO {
            int id
            String purchaseOrder
            double bookedAmount
            Date customerInvoicedDate
            String state
            static hasMany = [accountEntries: AccountEntryDAO]
            SortedSet accountEntries

            static mapping = {
                version false
                cache usage: 'read-only'
                table 'commission_order'
                id column:'id', type:'integer'
                purchaseOrder column: 'externalId'
                bookedAmount column: 'bookedAmount'
                customerInvoicedDate column: 'customerInvoicedDate'
                state column : 'state'
                accountEntries sort : 'id', order : 'desc'
            }
            ...
        }

        class AccountEntryDAO implements Comparable<AccountEntryDAO> {
            int id
            Date eventDate
            CommissionOrderDAO commissionOrder
            String entryType
            String description
            double remainingPotentialCommission

            static belongsTo = [commissionOrder : CommissionOrderDAO]

            static mapping = {
                version false
                cache usage: 'read-only'
                table 'account_entry'
                id column:'id', type:'integer'
                eventDate column: 'eventDate'
                commissionOrder column: 'commissionOrder'
                entryType column: 'entryType'
                description column: 'description'
                remainingPotentialCommission formula : SQLFormulaUtils.AccountEntrySQL.REMAININGPOTENTIALCOMMISSION_FORMULA
            }

            ....
        }   

The criteria for the report is that the commissionOrder.state==open and the commissionOrder.customerInvoicedDate is not null. And the account entries in the report should be between the startDate and the endDate and with remainingPotentialCommission > 0.

I’m looking to display information on the CommissionOrder mainly (and to display account entries on that commission order between the dates), but when I use the following projection:

        def results = accountEntryCriteria.list {
            projections {
                like ("entryType", "comm%")
                ge("eventDate", beginDate)
                le("eventDate", endDate)
                gt("remainingPotentialCommission", 0.0099d)
                and {
                  commissionOrder {
                    eq("state", "open") 
                    isNotNull("customerInvoicedDate")
                  }
                }
             }
            order("id", "asc")
        }   

I get the correct accountEntries with the proper commissionOrders, but I’m going in backwards: I have loads of accountEntries which can reference the same commissionOrder. Aut when I look at the commissionOrders that I’ve retrieved, each one has ALL its accountEntries not just the accountEntries between the dates.

I then loop through the results, get the commissionOrder from the accountEntriesList, and remove accountEntries on that commissionOrder after the end date to get the “snapshot” in time that I need.

def getCommissionOrderListByRemainingPotentialCommissionFromResults(results, endDate) {
    log.debug("begin getCommissionOrderListByRemainingPotentialCommissionFromResults")
    int count = 0;
    List<CommissionOrderDAO> commissionOrderList = new ArrayList<CommissionOrderDAO>()
    if (results) {
        CommissionOrderDAO[] commissionOrderArray = new CommissionOrderDAO[results?.size()];
        Set<CommissionOrderDAO> coDuplicateCheck = new TreeSet<CommissionOrderDAO>()
        for (ae in results) {
            if (!coDuplicateCheck.contains(ae?.commissionOrder?.purchaseOrder) && ae?.remainingPotentialCommission > 0.0099d) {
                CommissionOrderDAO co = ae?.commissionOrder
                CommissionOrderDAO culledCO = removeAccountEntriesPastDate(co, endDate)
                def lastAccountEntry = culledCO?.accountEntries?.last()
                if (lastAccountEntry?.remainingPotentialCommission > 0.0099d) {
                    commissionOrderArray[count++] = culledCO
                }
                coDuplicateCheck.add(ae?.commissionOrder?.purchaseOrder)
            }
        }
        log.debug("Count after clean is ${count}")
        if (count > 0) {
            commissionOrderList = Arrays.asList(ArrayUtils.subarray(commissionOrderArray, 0, count))
            log.debug("commissionOrderList size = ${commissionOrderList?.size()}")
        }

    }
    log.debug("end getCommissionOrderListByRemainingPotentialCommissionFromResults")
    return commissionOrderList
}

Please don’t think I’m under the impression that this isn’t a Charlie Foxtrot. The query itself doesn’t take very long, but the cull process takes over 35 minutes. Right now, it’s “manageable” because I only have to run the report once a month.

I need to let the database handle this processing (I think), but I couldn’t figure out how to manipulate hibernate to get the results I want. How can I change my criteria?

Answer 1

Try to narrow down the bottle neck of that process. If you have a lot of data, then maybe this check could be time expensive.

coDuplicateCheck.contains(ae?.commissionOrder?.purchaseOrder)

in Set contains have O(n) complexity. You can use i.e. Map to store keys that you would check and then search for “ae?.commissionOrder?.purchaseOrder” as key in the map.

The second thought is that maybe when you’re getting ae?.commissionOrder?.purchaseOrder it is always loaded from db by lazy mechanism. Try to turn on query logging and check that you don’t have dozens of queries inside this processing function.

Finally and again I would suggest to narrow down where is the most expensive part and time waste.

This plugin maybe helpful.