Question

I’ve been working on a web application, deployed on Tomcat 7, which use EclipseLink JPA to handle the persistence layer.

Everything works fine in a test environment but we’re having serious issues in the production environment due to a firewall cutting killing inactive connections. Basically if a connection is inactive for a while a firewall the sits between the Tomcat server and the DB server kill it, with the result of leaving “stale” connections in the pool.

The next time that connection is used the code never returns, until it gets a “Connection timed out” SQLException (full ex.getMessage() below).

EL Fine]: 2012-07-13
18:24:39.479–ServerSession(309463268)–Connection(69352859)–Thread(Thread[http-bio-8080-exec-5,5,main])–
MY QUERY REPLACED TO POST IT TO SO [EL Config]: 2012-07-13
18:40:10.229–ServerSession(309463268)–Connection(69352859)–Thread(Thread[http-bio-8080-exec-5,5,main])–disconnect
[EL Info]: 2012-07-13
18:40:10.23–UnitOfWork(1062365884)–Thread(Thread[http-bio-8080-exec-5,5,main])–Communication
failure detected when attempting to perform read query outside of a
transaction. Attempting to retry query. Error was: Exception
[EclipseLink-4002] (Eclipse Persistence Services –
2.3.0.v20110604-r9504): org.eclipse.persistence.exceptions.DatabaseException Internal
Exception: java.sql.SQLException: Eccezione IO: Connection timed out

I already tried several configuration in the persistence.xml, but since I have no access to the firewall configuration I had no luck with these methods. I also tried to use setCheckConnections()

ConnectionPool cp = ((JpaEntityManager)em).getServerSession().getDefaultConnectionPool();
        cp.setCheckConnections();
        cp.releaseConnection(cp.acquireConnection());

I managed to solve the issue in a test script using testOnBorrow, testWhileIdle and other features that are avalaible from DBCP Apache Commons. I’d like to know how to override the EclipseLink internal connection pool to use a custom connection pool so that I can provide an already configured pool, based on DBCP rather than just configuring the internal one using persistence.xml.

I know I should provide a SessionCustomizer, I’m uncertain which one is the correct pattern to use. Basically I would like to preserve the performance of DBCP in a JPA-like way.

I’m deploying on Tomcat 7, I know that if I switch to GF I won’t have this problem, but for a matter of consistency with other webapp on the same server I’d prefere to stay on Tomcat.

Answer 1

What you want is definitely possible, but you might be hitting the limits of the “do it yourself” approach.

This is one of the more difficult things to explain, but there are effectively two ways to configure your EntityManagerFactory. The “do it yourself” approach and the “container” approach.

When you call Persistence.createEntityManagerFactory it eventually delegates to this method of the PersistenceProvider interface implemented by EclipseLink:

EntityManagerFactory    createEntityManagerFactory(String emName, Map map) 

The deal here is EclipseLink will then take it upon itself to do all the work, including its own connection creation and handling. This is the “do it yourself” approach. I don’t know EclipseLink well enough to know if there is a way to feed it connections using this approach. After two days on Stackoverflow it doesn’t seem like anyone else has that info either.

So here is why this “works in GF”. When you let the container create the EntityManagerFactory for you by having it injected or looking it up, the container uses a different method on the PersistenceProvider interface implemented by EclipseLink:

EntityManagerFactory createContainerEntityManagerFactory(PersistenceUnitInfo info, Map map) 

The long and short of it is that this PersistenceUnitInfo is an interface that the container implements and has these two very key methods on it:

public DataSource getJtaDataSource();
public DataSource getNonJtaDataSource();

With this mode EclipseLink will not try to do its own connection handling and will simply call these methods to get the DataSource from the container. This is really what you need.

There are two possible approaches you could take to solving this:

• You could attempt to instantiate the EclipseLink PersistenceProvider implementation yourself and call the createContainerEntityManagerFactory method passing in your own implementation of the PersistenceUnitInfo interface and feed the DBCP configured DataSource instances into EclipseLink that way. You would need to parse the persistence.xml file yourself and feed that data in through the PersistenceUnitInfo. As well EclipseLink might also expect a TransactionManager, in which case you’ll be stuck unless you hunt down a TransactionManager you can add to Tomcat.

• You could use the Java EE 6 certified version of Tomcat, TomEE. DataSources are configured in the tomee.xml, created using DBCP with full support for all the options you need, and passed to the PersistenceProvider using the described createContainerEntityManagerFactory call. You then get the EntityManagerFactory injected via @PersistenceUnit or look it up.

If you do attempt to use TomEE, make sure your persistence.xml is updated to explicitly set transaction-type="RESOURCE_LOCAL" because the default is JTA. Even though it’s non-compliant to use JTA with the Persistence.createEntityManagerFactory approach, there aren’t any persistence providers that will complain and let you know you’re doing something wrong, they treat it as RESOURCE_LOCAL ignoring the schema. So when you go to port your app to an actual certified server, it blows up.

Another note on TomEE is that in the current release, you’ll have to put your EclipseLink libs in the <tomcat>/lib/ directory. This is fixed in trunk, just not released yet.

I’m not sure how useful these slides will be without the explanation that goes along with them, but the second part of this presentation is a deep dive into how container-managed EntityManager’s work, specifically with regards to connection handling and transactions. You can ignore the transaction part as you aren’t using them and already have an in production you’re not likely to dramatically change, but it might be interesting for future development.

Best of luck!