Note: In a future article titled 'WSO2 Carbon Cluster Configuration Language'[1], more specific details of the WSO2 Carbon clustering configuration language will be provided.
Abstract
Clustering for high availability and scalability is one of the main requirements of any enterprise deployment. This is also true for WSO2's open source SOA middleware products. However, configuring a cluster can be tricky. WSO2 Carbon, which is the name of the base framework and the architecture of the newer generation of WSO2's open source SOA middleware products, has extensive support for clustering. WSO2 Carbon inherits clustering capabilities from Apache Axis2. State replication amongst members in the same group as well as cluster management are available in WSO2 Carbon.Table of Contents
Introduction
The Apache Axis2[2] Web services engine is the base of WSO2 Carbon[3], which is the foundation of the newer generation of Java middleware products from WSO2. WSO2 Carbon inherits clustering features from Apache Axis2. Axis2 clustering is designed in a way that all functionality is abstracted out by a set of interfaces. By implementing these interfaces and providing the implementation classes in the clustering configuration section, any clustering implementation can be plugged in. One may require to use a group management framework of his choice, in which case he can implement these interfaces and then plug them using the clustering configuration section in the axis2.xml file. We will be taking a look at the usages of these interfaces later. The default clustering implementation in Axis2 is based on Apache Tribes[4], the popular group management framework used by Apache Tomcat[5].The Axis2 clustering APIs address 2 concerns: state replication and management. State replication refers to the synchronizing of states related to different members in a cluster group. Management can again be divided into two aspects; Group Management and Node Management.
The 4 interfaces that may be used by an Axis2 clustering implementation are:
- org.apache.axis2.clustering.ClusteringAgent
- org.apache.axis2.clustering.state.StateManager
- org.apache.axis2.clustering.management.NodeManager
- org.apache.axis2.clustering.management.GroupManagementAgent
org.apache.axis2.clustering.ClusteringAgent
This is the main interface in the Axis2 clustering implementation. In order to plug-in a new clustering implementation, this interface has to be implemented.The ClusteringAgent is responsible for initializing all clustering related functionalities of a Carbon member or node. Generally, the initilization of a node in a cluster is handled here. It is also responsible for getting this node to join the cluster. This node should not process any Web services requests until it successfully joins the cluster. Generally, this node will also need to obtain state information and/or configuration information from a neighboring node. State information needs to be obtained as prior to joining the group this new member needs to be in sync with the other members with respect to the state. It is also possible that configuration changes have taken place. For example, new service deployments or service undeployments may have taken place after nodes in a group were initialized. Hence, it is essential that all configuration changes are kept in sync across a group. This interface is also responsible for properly initializing the org.apache.axis2.clustering.state.StateManager, org.apache.axis2.clustering.management.NodeManager and org.apache.axis2.clustering.management.GroupManagementAgent implementations. In the case of a static membership scheme, members are read from the axis2.xml file and added to the ClusteringAgent. Later, we will take a look at different membership schemes supported by Axis2.
In the axis2.xml, the instance of this interface is specified using the "clustering" class attribute.
There can also be several "parameter" elements, which are children of the "clustering" element in the axis2.xml file. Generally, these parameters will be specific to the ClusteringAgent implementation. In the default Apache Tribes based implementation, there are several parameters that are required to initialize the Tribes Channel properly.
org.apache.axis2.clustering.state.StateManager
This interface is responsible for handling state replication. The word 'state' here means the serializable values stored in the Axis2 context hierarchy which need to be kept in sync across all the members of a cluster group. Property changes (state changes) in the Axis2 context hierarchy in the node which runs this StateManager will be propagated to all other nodes in its group. In the Tribes based clustering implementation, we have only enabled replication of serializable objects stored in the Apache Axis2 ConfigurationContext, ServiceGroupContext and ServiceContext. Hence, if a user requires state replication, it is this user's responsibility to store these values in the proper Axis2 contexts. Generally, Web services authors do not have to handle state replication, as it is handled by the clustering implementation just before request completion. This is done at the Axis2 MessageReceivers. However, if the developer writes his own MessageReceiver, he will need to call the org.apache.axis2.clustering.state.Replicator#replicate() method. Also note that at any point org.apache.axis2.clustering.state.Replicator#replicate() can be called, if the developer wishes to force state replication.It is not mandatory to have a StateManager in a node. If we are not interested in high availability, we may disable state replication. In such a scenario, in general, the purpose of a clustered deployment is to achieve scalability. In addition, one may also enable clustering without state replication simply to utilize the group communication capabilities of the underlying Group Communication Framework (GCF). In such a case, the purpose of a clustered deployment may also be management of the cluster using the underlying GCF. WSO2 Carbon provides such a component which allows cluster management.
The implementation of this interface is set by reading the "stateManager" element in the axis2.xml clustering section.
org.apache.axis2.clustering.management.NodeManager
This interface is responsible for handling management of a particular member. It is not mandatory to have a NodeManager in a node. Node management is generally used for deploying or undeploying services across a cluster group. However, this interface is not used at the moment in Carbon. The implementation of this interface is set by reading the "nodeManager" element in the axis2.xml.org.apache.axis2.clustering.management.GroupManagementAgent
This is the interface through which group management events are notified and messages are sent to members in a particular group. This will only be used when a member is running in group management mode. In group management mode, a member is special and belongs to all groups which it is managing. Hence, any membership changes in the groups it manages will be notified by the underlying group management framework to the group management agent. A group management agent should be specified for each group that has to be managed by the member. In WSO2 Carbon cluster management, the cluster manager node needs to define group management agents for each group.The WSO2 Enterprise Service Bus (ESB) is capable of dynamic load balancing. Here, new members can join and leave the application group, and this will be reflected in the membership aware dynamic load balancer. The clustering configuration for the dynamic load balancer needs to be configured with group management to allow group membership discovery.
Membership Schemes
The Axis2 and WSO2 Carbon clustering implementations support several membership schemes.Static Membership
In this scheme, only a defined set of members can be in a group. The Group Membership Service(GMS) will detect members joining or leaving the group. External members cannot join the group. Each node may obtain group member details from a central repository or configuration file.Dynamic Membership
In this scheme, membership is not predefined. Members can join a group by specifying the appropriate group name, and also leave the group. The Group Management Service (GMS) will detect new members joining or leaving. Group membership information may be obtained from the GMS.Hybrid Membership
This scheme is also called Well-Known Addressed (WKA) based membership. In this scheme, there are a set of well-known members. We can consider these members as belonging to a static group. External members can join this group by notifying one of the well known members. These external members can then get the current group membership from this well-known member. When new members join the group, well-known members will notify all other members. When members leave the group, the GMS can detect this event.Conclusion
In this article, we had a detailed look at some of the core clustering concepts in WSO2 Carbon and Apache Axis2. These concepts will be essential when it comes to understanding different cluster configuration options, as well as developing different clustering implementations for Apache Axis2 and WSO2 Carbon. In the next article in this series, WSO2 Carbon Cluster Configuration Language, we will see how the concepts we discussed in this article can be realized.References
- WSO2 Carbon Cluster Configuration Language
- Apache Axis2, http://ws.apache.org/axis2/
- WSO2 Carbon, https://wso2.org/projects/carbon/
- Apache Tribes, http://tomcat.apache.org/tomcat-6.0-doc/tribes/introduction.html
- Apache Tomcat, http://tomcat.apache.org/
No comments:
Post a Comment