Question

I have an application in which a client component is run in a “managed” environment (as an Adobe InDesign plugin). In this “managed” environment, it doesn’t seem possible to create threads during execution, but only at application startup.

This client component connects to a Glassfish server through JNDI and RMI/IIOP to perform various EJB method calls that all work perfectly (except for some cumbersome CORBA serialization issues that are not important here). I would like to have it also connect using JMS, to perform some topic reading. The code I use to perform those topic reading is already under unit tests, making sure it can start, read the topic, and perform required integration (this is more an integration test than an unit test, but I have to be sure of my application communication layer, no ?). This code, when run in a unit test, perfectly works. Unfortunatly, when run from my client component (embedded in Indesign, and having no ability to create threads), it fails with a NullPointerException the following way :

29 mai 2012 18:07:22 com.sun.enterprise.connectors.ActiveRAFactory getActiveRA
INFO: Deployed RAR [ jmsra ] has inbound artifacts, but the runtime does not support it. Providing only outbound support 
29 mai 2012 18:07:22 org.hibernate.validator.util.Version <clinit>
INFO: Hibernate Validator 4.2.0.Final
29 mai 2012 18:07:24 com.sun.messaging.jms.ra.ResourceAdapter start
INFO: MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter: Version:  4.5  (Build 29-b) Compile:  Wed Feb  9 22:53:30 PST 2011
29 mai 2012 18:07:24 com.sun.messaging.jms.ra.ResourceAdapter start
INFO: MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter starting: broker is EMBEDDED, connection mode is Direct
29 mai 2012 18:12:58 com.sun.messaging.jms.blc.LifecycleManagedBroker start
GRAVE: MQJMSRA_RA4001: start:Aborting:Exception starting EMBEDDED broker=java.lang.NullPointerException
29 mai 2012 18:12:58 com.sun.messaging.jms.blc.LifecycleManagedBroker start
INFO: SJSMQ LifecycleManagedBroker configuration=
    brokerInstanceName       =imqbroker
    brokerBindAddress        =null
    brokerPort               =7676
    brokerHomeDir            =C:\Java-ext\glassfish3\mq
    brokerLibDir             =C:\Java-ext\glassfish3\mq\lib
    brokerVarDir             =C:\Java-ext\glassfish3\glassfish\domains\autocat\imq
    brokerJavaDir            =C:\Program Files\Java\jdk1.6.0_30\jre
    brokerArgs               =null
    MasterBroker             =null
    brokerId                 =null
    adminUsername            =admin
    adminPassword            =<default>
    adminPassFile            =null
    ConnectionURL            =mq://localhost:7676/
    dbType                   =null
    dbProps                  ={}
    dsProps                  ={}
    useJNDIRmiServiceURL     =true
    useSSLJMXConnector       =true
    brokerEnableHA           =false
    clusterId                =null
    rmiRegistryPort          =7776
    startRmiRegistry         =true
    brokerStartTimeout       =  jmxServiceURL            =null
60000

29 mai 2012 18:12:58 com.sun.enterprise.connectors.ActiveOutboundResourceAdapter init
GRAVE: RAR6035 : Resource adapter start failed. 
javax.resource.spi.ResourceAdapterInternalException: MQJMSRA_RA4001: start:Aborting:Exception starting EMBEDDED broker=java.lang.NullPointerException
    at com.sun.messaging.jms.blc.LifecycleManagedBroker.start(LifecycleManagedBroker.java:458)
    at com.sun.messaging.jms.ra.ResourceAdapter.start(ResourceAdapter.java:380)
    at com.sun.enterprise.connectors.ActiveOutboundResourceAdapter.startResourceAdapter(ActiveOutboundResourceAdapter.java:182)
    at com.sun.enterprise.connectors.ActiveOutboundResourceAdapter.init(ActiveOutboundResourceAdapter.java:129)
    at com.sun.enterprise.connectors.ActiveRAFactory.instantiateActiveResourceAdapter(ActiveRAFactory.java:135)
    at com.sun.enterprise.connectors.ActiveRAFactory.createActiveResourceAdapter(ActiveRAFactory.java:106)
    at com.sun.enterprise.connectors.service.ResourceAdapterAdminServiceImpl.createActiveResourceAdapter(ResourceAdapterAdminServiceImpl.java:212)
    at com.sun.enterprise.connectors.ConnectorRuntime.createActiveResourceAdapter(ConnectorRuntime.java:379)
    at com.sun.enterprise.resource.naming.AdministeredObjectFactory.getObjectInstance(AdministeredObjectFactory.java:107)
    at javax.naming.spi.NamingManager.getObjectInstance(Unknown Source)
    at com.sun.enterprise.naming.impl.SerialContext.getObjectInstance(SerialContext.java:556)
    at com.sun.enterprise.naming.impl.SerialContext.lookup(SerialContext.java:514)
    at com.sun.enterprise.naming.impl.SerialContext.lookup(SerialContext.java:455)
    at javax.naming.InitialContext.lookup(Unknown Source)
    at fr.perigee.autocat.indesign.remoting.jndi.jms.AdapterJMSListener.getConsumer(AdapterJMSListener.java:261)
    at fr.perigee.autocat.indesign.remoting.jndi.jms.AdapterJMSListener$AdapterJMSListenerRunnable.run(AdapterJMSListener.java:50)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
    at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
    at com.sun.messaging.jmq.jmsclient.runtime.impl.BrokerInstanceImpl.start(BrokerInstanceImpl.java:211)
    at com.sun.messaging.jms.blc.EmbeddedBrokerRunner.start(EmbeddedBrokerRunner.java:331)
    at com.sun.messaging.jms.blc.LifecycleManagedBroker.start(LifecycleManagedBroker.java:454)
    ... 24 more
Caused by: java.lang.NullPointerException
    at com.sun.messaging.jmq.jmsserver.data.TransactionList.<init>(TransactionList.java:182)
    at com.sun.messaging.jmq.jmsserver.Broker._start(Broker.java:1170)
    at com.sun.messaging.jmq.jmsserver.Broker.start(Broker.java:456)
    at com.sun.messaging.jmq.jmsserver.BrokerProcess.start(BrokerProcess.java:164)
    at com.sun.messaging.jmq.jmsserver.DirectBrokerProcess.start(DirectBrokerProcess.java:92)
    at com.sun.messaging.jmq.jmsclient.runtime.impl.BrokerInstanceImpl.start(BrokerInstanceImpl.java:206)
    ... 26 more

From what I have discovered so far, client code obtains connection to JMS server (connection to mq://localhost://7676), but then fail to start a TransactionList. Uncompiled source code (thanks Jad !) shows that it fails there :

    /* 182*/Globals.getClusterManager().addEventListener(((ClusterListener) (this)));

Trying to debug that, I also found that my client can’t start the org.glassfish.enterprise.iiop.util.S1ASThreadPoolManager due to the same impossibility to create threads.

I guess there is something wrong somewhere, but what ? I don’t know.

So, how can I tell the glassfish client environment that threads can only obtained from my personnal thread pool (created at application startup) instead of trying to create its own and failing miserably ?

Answer 1

Short version

I deployed my aplication using maven forgotting the jms-core.jar of Glassfish, as it is a runtime dependency of gf-client-module.jar, and my maven build only copied compile dependencies. Fixing the maven build fixed the issue.

Long version

Long version is more a story of fear, tears, and blood.

How did that clusterManager got null ?

As I initially had problems with that TransactionList associated ClusterManager, I tried to look into com.sun.messaging.jmq.jmsserver.Globals code how it was initialized. I foun d it was in a method aptly called Globals#initCLusterManager(MQAddress) which, using various class parameters (including a Properties object), loaded a ClusterManager that was to be used by my application. What ? A ClusterManager ? In client code ? What the hell is that (more on that later) ?

Anyway, that ClusterManager was loaded, but set to null afterwards (mainly due to a a use of loopback address in cluster config – once again, incredibly weird in a JMS client) … So looking how it was initialized was apparently not the correct path to bug solving, which left me puzzled.

What to do when test pass, but production code stills bugs ?

As I said in question, I have a test, which does (to my understanding) exactly the same thing. Test always succeed, and production always fail. So there must be something different there, no ?

So i did something new to me : I put a breakpoint in that Globals#initClusterManager code, and tried to reach it using both code : production and test. As one may guess, breakpoint was reached in prod, but not in test ! So I moved my breakpoint at some place where I already knew it would be reached : ActiveRAFactory#getActiveRA(ConnectorDescriptor, String) … (oh wait, no, I didn’t put it there, but the place where I put that breakpoint told me the point where execution path became different was that method call, so I finally put my breakpoint there).

With that breakpoint, I immediatly noticed something strange : the call to

    Collection<ActiveResourceAdapter> activeRAs =  activeRAHabitat.getAllByContract(ActiveResourceAdapter.class);

returned different results depending upon the fact I was in test or in prod : I got 4 elements in my list in test (including the ActiveJmsResourceAdapter), but only 3 in production.

Different ResourceAdapter, but why ?

Then I was puzzled : I then tended to blame Glassfish HK2 for no other reason than being an IoC container I didn’t yet know … Turned out I was wrong, oh so wrong !

Revelation came from an expression eval in Eclipse debugger : I typed as an expression

com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter.class

And all i got was a ClassNotFoundException … wait … a ClassNotFoundException ? And this class is in jms-core.jar ? I’m pretty sure that jar is in my production folder, no ? … No ? NO ?

God damn it !

Turned out the maven-dependency-plugin I use to copy jars in that folder uses copy-dependencies with an includeScope configured to “compile”.

Crazy bug with simple solution

So I added an other execution to copy-dependencies, configured to incldueScope “runtime”, and all worked OK.