I’m trying to get the new HornetQ 2.3.0.alpha replication running, on two different hosts. When I try it on my local machine it does work more or less (separated hornetq with different ports). I will explain it in more detail shortly.
I use the plain vanilla HornetQ 2.3.0.alpha on latest 64bit Xubuntu (local tests) and CentOs (distributed), Oracle JDK 6u33 (but also tried other versions).
Now when testing localy, I use a different port for the backup (5446). I start the live, then the backup server. The backup syncs with the live. I kill/stop the live. The backup will continue to work instead, and requests against the backup work (which now operates on 5445). Now I want to start the live again, which does not work, since the backup uses the 5445. Therefore my next test was to start the live/backup on separate machines.
On separated hosts, the live-server starts as expected. When I start the backup, it hangs when he copies some data directores. After that he gets a timeout. There is not firewall or anything else between them, I already checked that twice and with my local admin. Telnet’ing from the backup-host to the live-host works fine.
Here is the exception I get on the backup, I append my configuration below:
***********************************************************************************
java -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -Xms512M -Xmx1024M -Dhornetq.config.dir=../config/stand-alone/non-clustered -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dlogging.configuration=../config/stand-alone/non-clustered/logging.properties -Djava.library.path=. -classpath ../lib/netty.jar:../lib/jnpserver.jar:../lib/jnp-client.jar:../lib/jboss-mc.jar:../lib/jboss-jms-api.jar:../lib/hornetq-twitter-integration.jar:../lib/hornetq-spring-integration.jar:../lib/hornetq-service-sar.jar:../lib/hornetq-rest.jar:../lib/hornetq-journal.jar:../lib/hornetq-jms.jar:../lib/hornetq-jms-client.jar:../lib/hornetq-jboss-as-integration.jar:../lib/hornetq-core.jar:../lib/hornetq-core-client.jar:../lib/hornetq-commons.jar:../lib/hornetq-bootstrap.jar:../config/stand-alone/non-clustered:../schemas/ org.hornetq.integration.bootstrap.HornetQBootstrapServer hornetq-beans.xml
***********************************************************************************
Unable to read the logging configuration from '../config/stand-alone/non-clustered/logging.properties' (java.net.MalformedURLException: no protocol: ../config/stand-alone/non-clustered/logging.properties)
11:50:37,611 INFO [org.hornetq.integration.bootstrap] HQ101001: Starting HornetQ Server
11:50:38,932 INFO [org.hornetq.core.server] HQ111001: backup server is starting with configuration HornetQ Configuration (clustered=true,backup=true,sharedStore=false,journalDirectory=../data/journal,bindingsDirectory=../data/bindings,largeMessagesDirectory=../data/large-messages,pagingDirectory=../data/paging)
11:50:38,950 WARN [org.hornetq.core.server] HQ112216: Moving data directory ../data/bindings to ../data/bindings22
11:50:38,952 WARN [org.hornetq.core.server] HQ112216: Moving data directory ../data/journal to ../data/journal22
11:50:38,953 WARN [org.hornetq.core.server] HQ112216: Moving data directory ../data/paging to ../data/paging22
11:50:38,953 WARN [org.hornetq.core.server] HQ112216: Moving data directory ../data/large-messages to ../data/large-messages22
11:51:39,106 ERROR [org.hornetq.core.server] HQ114002: Failure in initialisation: java.lang.RuntimeException: Could not estabilish the connection
at org.hornetq.core.server.impl.HornetQServerImpl$SharedNothingBackupActivation.run(HornetQServerImpl.java:2152) [hornetq-core.jar:]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_33]
java.lang.RuntimeException: Could not estabilish the connection
at org.hornetq.core.server.impl.HornetQServerImpl$SharedNothingBackupActivation.run(HornetQServerImpl.java:2152)
at java.lang.Thread.run(Thread.java:662)
Configuration files
I used the default configuration files as base (config/stand-alone/non-clustered)
Live Configuration files
hornetq-configuration.xml
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
<paging-directory>${data.dir:../data}/paging</paging-directory>
<bindings-directory>${data.dir:../data}/bindings</bindings-directory>
<journal-directory>${data.dir:../data}/journal</journal-directory>
<journal-min-files>10</journal-min-files>
<large-messages-directory>${data.dir:../data}/large-messages</large-messages-directory>
<connectors>
<connector name="netty">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
<param key="host" value="${hornetq.remoting.netty.host:localhost}"/>
<param key="port" value="${hornetq.remoting.netty.port:5445}"/>
</connector>
<connector name="remote-connector">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
<param key="host" value="192.168.40.155"/>
<param key="port" value="5445"/>
</connector>
</connectors>
<acceptors>
<acceptor name="netty">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
<param key="host" value="0.0.0.0"/>
<param key="port" value="${hornetq.remoting.netty.port:5445}"/>
</acceptor>
</acceptors>
<security-settings>
<security-setting match="#">
<permission type="createNonDurableQueue" roles="guest"/>
<permission type="deleteNonDurableQueue" roles="guest"/>
<permission type="consume" roles="guest"/>
<permission type="send" roles="guest"/>
</security-setting>
</security-settings>
<address-settings>
<!--default for catch all-->
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<max-size-bytes>10485760</max-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
<address-full-policy>BLOCK</address-full-policy>
</address-setting>
</address-settings>
<shared-store>false</shared-store>
<clustered>true</clustered>
<failover-on-shutdown>false</failover-on-shutdown>
<cluster-user>cluster-user</cluster-user>
<cluster-password>cluster123</cluster-password>
<cluster-connections>
<cluster-connection name="test-cluster">
<address>jms</address>
<connector-ref>netty</connector-ref>
<retry-interval>2000</retry-interval>
<use-duplicate-detection>true</use-duplicate-detection>
<forward-when-no-consumers>true</forward-when-no-consumers>
<max-hops>1</max-hops>
<static-connectors allow-direct-connections-only="true">
<connector-ref>remote-connector</connector-ref>
</static-connectors>
</cluster-connection>
</cluster-connections>
<backup>false</backup>
</configuration>
hornetq-jms.xml
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
<connection-factory name="NettyConnectionFactory">
<xa>false</xa>
<ha>true</ha>
<reconnect-attempts>-1</reconnect-attempts>
<retry-interval>1000</retry-interval>
<connectors>
<connector-ref connector-name="netty"/>
</connectors>
<entries>
<entry name="/ConnectionFactory"/>
</entries>
</connection-factory>
<queue name="DLQ">
<entry name="/queue/DLQ"/>
</queue>
<queue name="ExpiryQueue">
<entry name="/queue/ExpiryQueue"/>
</queue>
<queue name="testing">
<entry name="/queue/testing" />
<durable>true</durable>
</queue>
</configuration>
Backup Configuration files
hornetq-configuration.xml
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
<paging-directory>${data.dir:../data}/paging</paging-directory>
<bindings-directory>${data.dir:../data}/bindings</bindings-directory>
<journal-directory>${data.dir:../data}/journal</journal-directory>
<journal-min-files>10</journal-min-files>
<large-messages-directory>${data.dir:../data}/large-messages</large-messages-directory>
<connectors>
<connector name="netty">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
<param key="host" value="${hornetq.remoting.netty.host:localhost}"/>
<param key="port" value="${hornetq.remoting.netty.port:5445}"/>
</connector>
<connector name="remote-connector">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
<param key="host" value="192.168.40.180"/>
<param key="port" value="5445"/>
</connector>
</connectors>
<acceptors>
<acceptor name="netty">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
<param key="host" value="0.0.0.0"/>
<param key="port" value="5445"/>
</acceptor>
</acceptors>
<security-settings>
<security-setting match="#">
<permission type="createNonDurableQueue" roles="guest"/>
<permission type="deleteNonDurableQueue" roles="guest"/>
<permission type="consume" roles="guest"/>
<permission type="send" roles="guest"/>
</security-setting>
</security-settings>
<address-settings>
<!--default for catch all-->
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<max-size-bytes>10485760</max-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
<address-full-policy>BLOCK</address-full-policy>
</address-setting>
</address-settings>
<shared-store>false</shared-store>
<clustered>true</clustered>
<failover-on-shutdown>false</failover-on-shutdown>
<cluster-user>cluster-user</cluster-user>
<cluster-password>cluster123</cluster-password>
<live-connector-ref connector-name="remote-connector"/>
<cluster-connections>
<cluster-connection name="test-cluster">
<address>jms</address>
<connector-ref>netty</connector-ref>
<retry-interval>2000</retry-interval>
<use-duplicate-detection>true</use-duplicate-detection>
<forward-when-no-consumers>true</forward-when-no-consumers>
<max-hops>1</max-hops>
<static-connectors allow-direct-connections-only="true">
<connector-ref>remote-connector</connector-ref>
</static-connectors>
</cluster-connection>
</cluster-connections>
<backup>true</backup>
</configuration>
hornetq-jms.xml
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
<connection-factory name="NettyConnectionFactory">
<xa>false</xa>
<ha>true</ha>
<reconnect-attempts>-1</reconnect-attempts>
<retry-interval>1000</retry-interval>
<connectors>
<connector-ref connector-name="netty"/>
</connectors>
<entries>
<entry name="/ConnectionFactory"/>
</entries>
</connection-factory>
<queue name="DLQ">
<entry name="/queue/DLQ"/>
</queue>
<queue name="ExpiryQueue">
<entry name="/queue/ExpiryQueue"/>
</queue>
<queue name="testing">
<entry name="/queue/testing" />
<durable>true</durable>
</queue>
</configuration>
When I created the sandbox locally, it uses the same configuration, except that for backup the hornetq-beans.xml is changed to move the ports from 1099/1098 to 1199/1198.
I’m working on this now for days, and tried already various combinations, so please excuse me If I missed something. Thanks a lot!
HornetQ 2.3 is under active development. I would recommend you to ask this on the User’s forum instead of SOF where all the developers work.
Anyways we are about to release a Beta where we changed how the connection happens between the nodes.. we even removed a paramter and changed a few things. So I would definitely recommend you to talk to us on the HOrnetQ user’s forum.. and maybe give it a try on hornetq/master on github.