Chapter 7. Clustering

About Platform clustering
When should you consider clustering ?
Shared file system
Cluster setup
advanced configuration
Cluster management
Cluster profile
Initial Startup
Startup and shutdown
Clustering FAQ
How do I migrate from local to cluster mode ?
Why is startup failing with a Port value out of range error ?
How to solve the "failed sending message to null" error?

Installing eXo platform in cluster mode should be considered in the following cases:

These characteristics should be handled by the overall architecture of your system. Load Balancing is typically achieved by a front server or device that distributes the request to the cluster nodes. Also, high availability on the data layer can be typically achieved using the native replication implemented by RDBMS.

In this chapter, we will cover only the changes needed by eXo to work in a cluster.

In eXo Platform, the persistence mostly relies on JCR, which is a middleware between the eXo applications (including the portal) and the database. Hence this component must be configured to work in cluster.

The embedded JCR server requires a portion of its state to be shared on a file system shared among cluster nodes :

All nodes must have a read/write access on the shared file system.

Note

We strongly advise the use of a mount point on a SAN.

The switch to a cluster configuration is done in configuration.properties. This configuration.properties file must be set in the same way on all the cluster nodes.

First, switch the JCR to cluster mode.

gatein.jcr.config.type=cluster
gatein.jcr.index.changefilterclass=org.exoplatform.services.jcr.impl.core.query.jbosscache.JBossCacheIndexChangesFilter

This will tell the JCR to enable automatic network replication and discovery between other cluster nodes.

Next, configure the path for the shared filesystem :

gatein.jcr.storage.data.dir=/PATH/TO/SHARED/FS/values
gatein.jcr.index.data.dir=/PATH/TO/SHARED/FS/index

The path is shared, so all nodes will need read/write access to this path.

The cluster mode is preconfigured to work out of the box. It relies on the JBoss Cache configuration.

# JCR cache configuration
gatein.jcr.cache.config=classpath:/conf/jcr/jbosscache/${gatein.jcr.config.type}/config.xml

# JCR Locks configuration
gatein.jcr.lock.cache.config=classpath:/conf/jcr/jbosscache/${gatein.jcr.config.type}/lock-config.xml

# JCR Index configuration
gatein.jcr.index.cache.config=classpath:/conf/jcr/jbosscache/cluster/indexer-config.xml
gatein.jcr.jgroups.config=classpath:/conf/jcr/jbosscache/cluster/udp-mux.xml

You need to indicate the cluster kernel profile to eXo Platform. This can be done by editing gatein.sh like this: 

EXO_PROFILES="-Dexo.profiles=default,cluster"

or using starteXo script : 

./start_eXo.sh default,cluster

For the very first startup of your JCR cluster, you should only start a single node. This node will initialise the internal JCR database and create the system workspace. Once this first node is definitely started, you can start the other nodes.

Note

This contraint is only for the very first start. Once the initialization has been done, you can start nodes in any order

Nodes of the cluster will automatically try to join others at startup. Once they discover each other, they will synchronize their state. During the synchronization the node is not ready to serve requests.

If you intend to migrate your production system from local (non cluster) mode to cluster, follow these steps :

On Linux Platforms, if you encounter an error at startup like this :

[INFO] Caused by: java.lang.IllegalArgumentException: Port value out of range: 65536

This problem arise under specific circumstances when JGroups, the networking library behind the clustering, attempts to detect the IP to use for communication with other nodes. Verify that :

When starting up in the cluster mode under Linux and you encourter the following error:

Dec 15, 2010 6:11:31 PM org.jgroups.protocols.TP down
SEVERE: failed sending message to null (44 bytes)
java.lang.Exception: dest=/228.10.10.10:45588 (47 bytes)

You must remember that clustering on Linux only works with IPv4.

 -Djava.net.preferIPv4Stack=true

is mandatory for running in the cluster mode under Linux.