Question

I try to execute several queries in one DAO-method. Test is FAILED (the data are not updated). Logs without exceptions.

public List<Domain> getNewDomains(final int maxAllowedItems, final Date timestamp) {

    return getJpaTemplate().execute(new JpaCallback<List<Domain>>() {
        @SuppressWarnings("unchecked")
        public List<Domain> doInJpa(EntityManager entityManager) throws PersistenceException {
            Calendar dayBefore = Calendar.getInstance();
            dayBefore.setTime(timestamp);
            dayBefore.add(Calendar.HOUR, -24);

            List ids = entityManager.createQuery("SELECT d.id FROM domain d WHERE d.crawlDate IS NULL and (d.lastRead IS NULL OR d.lastRead <= :dayBefore ) ")
            .setParameter("dayBefore", dayBefore.getTime())
            .setMaxResults(maxAllowedItems)
            .getResultList();

            LOG.debug("new domain IDS : " + ids.toString());

            if(ids.isEmpty()) {
                return new ArrayList<Domain>();
            }

            int result = entityManager.createQuery("UPDATE domain d SET d.lastRead = :timestamp WHERE d.id IN (:ids)")
            .setParameter("timestamp", timestamp)
            .setParameter("ids", ids).executeUpdate();

            LOG.debug("update result : " + result);

            return entityManager.createQuery("SELECT d FROM domain d WHERE d.id IN (:ids) ")
            .setParameter("ids", ids)
            .setMaxResults(maxAllowedItems)
            .getResultList();
        }
    });
}

First select processed correctly. But on update “lastRead” field state the same.

Test:

@Test
public void testGetNewItems() {
    List<Domain> items = domainDAO.getNewDomains(2, new Date());
    Assert.assertNotNull(items);
    Assert.assertTrue(items.isEmpty());

    String domainName = "example.com";
    Domain domain1 = new Domain(domainName);
    domainDAO.save(domain1);

    String domainName2 = "example2.com";
    Domain domain2 = new Domain(domainName2);
    domainDAO.save(domain2);
    Domain domain2FromDB = domainDAO.getByName(domainName2);
    Assert.assertEquals(domain2, domain2FromDB);
    Assert.assertNull(domain2FromDB.getCrawlDate());

    domain2FromDB.setCrawlDate(new Date());
    domainDAO.update(domain2FromDB);

    String domainName3 = "example3.com";
    Domain domain3 = new Domain(domainName3);
    domainDAO.save(domain3);

    items = domainDAO.getNewDomains(2, new Date());
    Assert.assertNotNull(items);
    Assert.assertEquals(2, items.size());
    Assert.assertTrue(items.contains(domain1));
    Assert.assertTrue(items.contains(domain3));
    Assert.assertFalse(items.contains(domain2FromDB));

    for (Domain item : items) {
        Assert.assertNotNull(item.getLastRead()); // FAILED assert
    }
}

Should I flash after update ?
What is the correct way to process several queries ?

Answer 1

Update & delete queries are considered Bulk Updates in JPA, and have different rules. Bulk updates execute directly on the database the persistence context (EntityManager) will not be updated with these changes. So when you query the persistence context it finds a matching entity – unknowingly returning stale data.

The JPA specification puts it like this :

Caution should be used when executing bulk update or delete operations because they may result in
inconsistencies between the database and the entities in the active persistence context. In general, bulk
update and delete operations should only be performed within a transaction in a new persistence con-
text or before fetching or accessing entities whose state might be affected by such operations.

You have a couple of options on how to fix the problem.

Rewrite the first query to return the entity instead of just the id. and modify the entity. Something like this should work

public List<Domain> doInJpa(EntityManager entityManager) throws PersistenceException {
    Calendar dayBefore = Calendar.getInstance();
    dayBefore.setTime(timestamp);
    dayBefore.add(Calendar.HOUR, -24);

    List<Domain> domains = entityManager.createQuery("SELECT d FROM domain d WHERE d.crawlDate IS NULL and (d.lastRead IS NULL OR d.lastRead <= :dayBefore ) ")
        .setParameter("dayBefore", dayBefore.getTime())
        .setMaxResults(maxAllowedItems)
        .getResultList();

    if(domains.isEmpty()) {
        return new ArrayList<Domain>();
    }

    for(Domain d : domains) { 
        d.setLastRead(timestamp);
    }

    return domains;     
}

Now you’re down to one query, the entities will be synchronized with the persistence context and your test should pass.

Failing that you can work around the issue with bulk updates by calling entityManager.refresh() on each domain – the refresh method will update the entity with the latest state from the database.