I am currently in the process of setting up a tomcat 6 clustered environment, liferay 6.0.6.
4 nodes with session replication. No sticky session.
So following the guide supplied on this site i did the following :
webapps/conf/context.xml added :
webapps/ROOT/WEB-INF/web.xml added : to the top of the file, after the first bracket.
I also added distributable to all my custom portlets web.xml.
In setenv.sh : -Djava.net.preferIPv6Addresses=false -Djava.net.preferIPv4Stack=true
In webapps/conf/server.xml
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcatA" />
tomcatA/B/C/D accross the nodes.
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false" notifyListenersOnReplication="true" />
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.10" port="45564" frequency="500" dropTime="3000" />
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto" port="4000" autoBind="100" selectorTimeout="5000"
maxThreads="6" />
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" timeout="30000" />
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" />
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor" />
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.css;.*\.txt;" />
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener" />
</Cluster>
At startup each node detects each other and it seems to work. However , when someone tries to modify the webcontent we get an error:
SEVERE: Manager [localhost#/]: Unable to receive message through TCP channel
java.lang.IllegalStateException: setAttribute: Session already invalidated
at org.apache.catalina.session.StandardSession.setAttribute(StandardSession.java:1326)
at org.apache.catalina.ha.session.DeltaSession.setAttribute(DeltaSession.java:594)
at org.apache.catalina.ha.session.DeltaRequest.execute(DeltaRequest.java:164)
at org.apache.catalina.ha.session.DeltaManager.handleSESSION_DELTA(DeltaManager.java:1487)
at org.apache.catalina.ha.session.DeltaManager.messageReceived(DeltaManager.java:1437)
at org.apache.catalina.ha.session.DeltaManager.messageDataReceived(DeltaManager.java:1171)
at org.apache.catalina.ha.session.ClusterSessionListener.messageReceived(ClusterSessionListener.java:92)
at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:901)
at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:882)
at org.apache.catalina.tribes.group.GroupChannel.messageReceived(GroupChannel.java:269)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79)
at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.messageReceived(TcpFailureDetector.java:110)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79)
at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.java:241)
at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:225)
at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTask.java:188)
at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:91)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
my liferay-ext.properties :
cluster.link.autodetect.address=www.google.com:80
lucene.replicate.write=true
cluster.link.enabled=true
net.sf.ehcache.configurationResourceName=/cache/hibernate-clustered.xml
ehcache.multi.vm.config.location=/cache/liferay-multi-vm-clustered.xml
Ive also created a jsp on all nodes
<td>
Session ID</td>
<td><%= session.getId() %></td>
<% session.setAttribute("abc","abc"); %>
</tr>
<tr>
<td>
Created on</td>
<td><%= new java.util.Date(session.getCreationTime()).toString() %></td>
</tr>
</table>
</body>
</html>
With the same user i can switch servers and the session seems to be replicated without problems. However i still get the stacktrace when i go modify something in liferay.
Ive been stuck for a while now.
We have checked that all servers and JVM times are synched correctly with the NTP server.
No ports are blocked. The server does not have access to the internet.
They are all running on VM.
Anybody has any ideas what im doing wrong ??
thank you.
A few notes…
1.) The problem that you are reporting may have to do with the value being used for channelSendOptions.
Your current configuration is setting channelSendOptions to a value of 8. This means that messages are sent between your nodes are sent asynchronously. This is great for speed, but, it means that data can arrive out of order [1].
The error messages is indicating that it received a message to update a session attribute, however the session that is to be updated has already been invalidated (i.e. deleted). A common reason for getting this error is because the messages have been received out of order.
In most cases, you can correct this problem by setting channelSendOptions to a value of 6. This will send messages synchronously, which guarantees the order.
[1] – https://tomcat.apache.org/tomcat-6.0-doc/config/cluster.html#Attributes
2.) Another possibility, although less likely, is that you have hit a bug in Tomcat. You haven’t listed the specific version of Tomcat that you are using, but upgrading to the latest Tomcat 6.0.x version would also be a good idea.
3.) You indicated that “sessions cannot be replicated without sticky session”. This is incorrect. Session replication and session stickyness are two separate process. While they are often used together, session replication does not require sticky sessions and sticky sessions do not require session replication.
Session replication (called “clustering” by Tomcat) is the process of replicating session data across multiple Tomcat servers. When session data is replicated across multiple nodes, it does not matter which node a user’s request is sent to because they all have the same session data.
Session “stickyness” is performed by a load balancer and it is the process of the load balancer ensuring that one session will always be directed to the same backend Tomcat server.
For example, the load balancer directs a request to Tomcat Server A and that request results in a session being created. With session stickyness enabled, the load balancer will send every additional request with the same session id to Tomcat Server A. Other requests, without sessions or with different sessions, will continue to be load balanced normally.
Like chocolate and peanut butter, sticky sessions and session replication are often used together, but it is not a requirement. Basic load balancing can be performed with only sticky sessions, however users will lose session data in the event of a failure on a backend Tomcat node. With both session replication and sticky sessions, users will not be affected by a failure with one of the backend Tomcat nodes. They will automatically be routed by the load balancer to a different node and that node will have a copy of the user’s session data.