Spring and Infinispan Integration on Wildfly Cluster

Spring framework provides a caching abstraction API that makes it very easy and consistent to use a lot of embeddable caching frameworks like, EHCache, Infinispan, Hazelcast etc. But sometimes it is also necessary to integrate with the underlying application server’s caching framework, especially in a clustered, distributed environment. In this post I will show how we can integrate a spring-boot application, deployed on clustered Wildfly application servers (version 10.1.0.Final), to integrate with the distributed Infinispan cache. Below is an illustration:


Wildfly already packs Infinispan caching framework as a subsystem, which the applications can leverage from in order to cache shared/private objects. So lets first create a custom replicated-cache container which we can use to share cached objects across a cluster of Wildfly servers. In order to use replicated cache, each instance of Wildfly needs to be started with full-ha profile via standalone-full-ha.xml configuration file. Below is what we need to add in this configuration file in order to create a replicated cache container:

We can lookup this cache container using the following JNDI name:

In the Spring Boot Application we need the following dependency added to lookup and use the Infinispan cache.

Also because the dependency is “provided” by the application server, we need to add manifest entries so that we have access to all the required classes on runtime. In order to achieve this we’ll add the following plugin to the pom file.

Now the application is all setup to lookup and use the distributed Infinispan cache, below is how we can do this:

The complete Spring Boot application is available on Github.

This is just a simple usecase that shows how we can use Wildfly’s distributed cache in applications. In the next blog entry I will demonstrate how this distributed Infinispan cache can be used to integrate CAS Single Sign On in order to effectively achieve SSO/SLO in a clustered/distributed Wildfly environment.

Clustering and Load balancing Tomcat servers

Tomcat is by far the most popular open source Java servlet container that can easily be scaled to ensure high availability. In this post I will show how easy it is to distribute and scale java web applications using tomcat clusters. In this example we will also use Apache web server to load balance the cluster.

Below is a simple setup of a cluster with two Tomcat 7 nodes with Apache web server 2.2 as a load balancer.



Load balanced clustering on tomcat is done in four simple steps:

1- Uniquely naming each node
2- Configure clustering in server.xml
3- Configure the session persistence in context.xml (Optional)
4- Enable distribution in the Java web application

The following configuration is done on each tomcat instance either running on the same machine or on separate virtual or physical servers. If multiple instances are running on the same machine then it should be ensured that each instance is listening on different ports.

Naming the tomcat nodes

The server node can be named by adding the jvmRoute element to the tomcat Engine as shown below. This node name is appended to the session id.

Configuring the cluster

Clustering can easily be enabled in tomcat by adding the following to the server.xml file

[turbo_widget widget-prefix=text&obj-class=WP_Widget_Text&widget-text–title=&widget-text–text=%3C!–+Post+Ad+–%3E%0A%3Cins+class%3D%22adsbygoogle%22%0A+++++style%3D%22display%3Ablock%22%0A+++++data-ad-client%3D%22ca-pub-4433761869744390%22%0A+++++data-ad-slot%3D%229020480262%22%0A+++++data-ad-format%3D%22auto%22%3E%3C%2Fins%3E%0A%3Cscript%3E%0A(adsbygoogle+%3D+window.adsbygoogle+%7C%7C+%5B%5D).push(%7B%7D)%3B%0A%3C%2Fscript%3E&widget-text–filter=false]

This will also enable session replication on all tomcat nodes. Below is a detailed example that shows more options, including multicast and deployer configuration. See the tomcat documentation for more info.

Configuring session persistence (Optional)

Session persistence in configured in the context.xml file by defining a persistence manager. By default the session is persisted in-memory but optionally a disk or database storage can be also be configured. The example below shows how the session is persisted on the local disk using a session store.

Making the application distributable

The java web applications can be configured to run in a cluster by adding the following to application’s web.xml file.

Once all this configuration is done on each node, the servers will be running in a cluster with session replication enabled.

[turbo_widget widget-prefix=text&obj-class=WP_Widget_Text&widget-text–title=&widget-text–text=%3C!–+Post+Ad+–%3E%0A%3Cins+class%3D%22adsbygoogle%22%0A+++++style%3D%22display%3Ablock%22%0A+++++data-ad-client%3D%22ca-pub-4433761869744390%22%0A+++++data-ad-slot%3D%229020480262%22%0A+++++data-ad-format%3D%22auto%22%3E%3C%2Fins%3E%0A%3Cscript%3E%0A(adsbygoogle+%3D+window.adsbygoogle+%7C%7C+%5B%5D).push(%7B%7D)%3B%0A%3C%2Fscript%3E&widget-text–filter=false]

Load balancing the cluster

Now the next step is to configure load balancing to effectively use the scalable tomcat cluster and to ensure high availability. Meaning if one of the nodes, in the cluster, goes down, the application will still be available on the other node. And with session replication enabled the service and user experience will not be impacted.

In this example I am using Apache’s mod_proxy_ajp to load balance the two tomcat instances. The configuration is fairly simple and is added to the httpd.conf file. The example also shows how session sticky-ness is enabled. The loadfactor defines the weighted load that is applied on each node. The lbmethod configures the load balancing strategy, which can either be byrequests or bytraffic.

Note: mod_proxy_ajp is not the only way apache is used to load balance tomcat instances, the other commonly used methods are mod_jk and mod_proxy.