How to Enable Registry Data Replication among WSO2 ESB Cluster Nodes

As an example for the cluster, we are using WSO2 ESB. You can use the given steps for any product and configurations will be identical for them too. Let's assume we are going to have a setup as below table with the back-end database with MySQL. If you are going to use any other database please change the database connection URL accordingly.

Instance Name		Database Name -  Mounted path
Master  (10.3.24.97)	local	master
	config	greg - /_system/esbConfig
	governance	greg - /_system/governance
	user-manager	usermgt
Slave (10.3.24.98)	local	slave
	config	greg - /_system/esbConfig
	governance	greg - /_system/governance
	user-manager	usermgt
GREG (10.3.24.99)	All	greg

1. If you look at the above table, there are 4 separate databases with the following names 

        greg

        usermgt

        slave

        master

 

        So the first step is to create the above four databases in your setup by using our database scripts in CARBON_HOME/dbscripts.

    

        1.1 If you are having your database remotely, please configure mysql instance to allow accept remote connections.

http://www.debianhelp.co.uk/remotemysql.htm

 

       1.2 Create databases in your mysql server with the above names.

 

2. Let's configure the GREG. And if you look at the table above, you can see that we are using a single database for all the things in GREG instance, so we need to change the configuration in registry.xml and user-mgt.xml.

 

    2.1 Open the GREG_HOME/repository/conf/registry.xml file remove the original dbConfig element and add the database configuration below. (IP addresses and database urls have to change acccording to your setup).

       <currentDBConfig>wso2registry</currentDBConfig>

       <readOnly>false</readOnly>

       <registryRoot>/</registryRoot>

       <dbConfig name="wso2registry">

            <url>jdbc:mysql://**:3306/greg?autoReconnect=true</url>

            <userName>root</userName>

            <password>root123</password>

            <driverName>com.mysql.jdbc.Driver</driverName>

            <maxActive>50</maxActive>

            <maxWait>60000</maxWait>

            <minIdle>5</minIdle>

        </dbConfig>

 

    2.2 Open the GREG_HOME/repository/conf/user-mgt.xml file and change the database configuration as below (IP addresses and urls have to change according to your setup).

 

            <Property name="url">jdbc:mysql://10.3.24.96:3306/greg?autoReconnect=true</Property>

            <Property name="userName">root</Property>

            <Property name="password">root123</Property>

            <Property name="driverName">com.mysql.jdbc.Driver</Property>

            <Property name="maxActive">50</Property>

            <Property name="maxWait">60000</Property>

            <Property name="minIdle">5</Property>

 

   2.3 Copy mysql jdbc driver library to GREG_HOME/repository/components/lib direcotry, becasue by default we do not ship the mysql drivers in GREG.

 

    2.4 If we examine the above configuration you can see that we are pointing the database in to same database 'greg', and now start the GREG instance by running wso2server.sh or wso2server.bat, based on your platform.

          (This will create tables for user manager database and registry database(pick the database configuration from currentDBConfig element in registry.xml since there might be number of database configurations in your registry.xml which we put 

           to perform mounting from other databases but only one database configuration is configured as currentDBConfig)).

 

    2.5 After successfully starting GREG instance create following directory in GREG which will be mounted by ESB  instances running in the cluster.

 

           /_system/esbConfig

 

3. Now let's configure ESB master node, here we are changing the default database configuration, to point it's local registry to 'master', point user manager database to 'usermgt' and mount it's 

    configuration and governance registries from GREG database.

 

    3.1 Let's add local registry configuration and user-manager configuration.

        

            3.1.1. Open the ESB_HOME/repository/conf/registry.xml file remove the original dbConfig element and add the database configuration below. (IP addresses and database urls have to change acccording to your setup).

      

            <currentDBConfig>wso2registry</currentDBConfig>

                <readOnly>false</readOnly>

                <registryRoot>/</registryRoot>

                <dbConfig name="wso2registry">

                <url>jdbc:mysql://10.3.24.96:3306/master?autoReconnect=true</url>

                <userName>root</userName>

                <password>root123</password>

                <driverName>com.mysql.jdbc.Driver</driverName>

                <maxActive>50</maxActive>

                <maxWait>60000</maxWait>

                <minIdle>5</minIdle>

            </dbConfig>

             

            3.1.2 Open the ESB_HOME/repository/conf/user-mgt.xml file and change the database configuration as below(IP addresses and urls have to change according to your setup).

 

                <Property name="url">jdbc:mysql://10.3.24.96:3306/usermgt?autoReconnect=true</Property>

                <Property name="userName">root</Property>

                <Property name="password">root123</Property>

                <Property name="driverName">com.mysql.jdbc.Driver</Property>

                <Property name="maxActive">50</Property>

                <Property name="maxWait">60000</Property>

                <Property name="minIdle">5</Property>

 

    3.2 Now let's configure the mounting configuration, for that we need to add more parameters to registry.xml file.

    

            3.2.1 Open the ESB_HOME/repository/conf/registry.xml file remove the original dbConfig element and add the database configuration below. (IP addresses and database urls have to change acccording to your setup).

     

             <dbConfig name="esbMountRegistry">

                <url>jdbc:mysql://10.3.24.96:3306/greg?autoReconnect=true</url>

                <userName>root</userName>

                <password>root123</password>

                <driverName>com.mysql.jdbc.Driver</driverName>

                <maxActive>50</maxActive>

                <maxWait>60000</maxWait>

                <minIdle>5</minIdle>

            </dbConfig>

    

 

            <remoteInstance url="https://10.3.24.99:9443/registry">

                <id>Mount1</id>

                <dbConfig>esbMountRegistry</dbConfig>

                <readOnly>false</readOnly>

                <registryRoot>/</registryRoot>

            </remoteInstance>

 

            <mount path="/_system/config" overwrite="true">

                 <instanceId>Mount1</instanceId>

                <targetPath>/_system/esbConfig</targetPath>

            </mount>

 

            <mount path="/_system/governance" overwrite="true">

                <instanceId>Mount1</instanceId>

                <targetPath>/_system/governance</targetPath>

            </mount>

 

            Please change the remoteInstance URL according to the configuration of the GREG running machine. Please note that "InstanceIds", "id" and "dbConfig" elements should be mapped properly if you are using different

            names for them.

 

    3.3 Copy mysql jdbc driver library to ESB_HOME/repository/components/lib direcotry since by default we do not ship the mysql drivers in ESB.

 

4. Now let's configure ESB slave node, here we are changing the default database configuration, to point it's local registry to slave,point user manager database to usermgt and mount it's 

    configuration and governance registries from GREG database.

 

    4.1 Let's add local registry configuration and user-manager configuration.

        

            4.1.1. Open the ESB_SLAVE_HOME/repository/conf/registry.xml file remove the original dbConfig element and add the database configuration below. (IP addresses and database urls have to change acccording to your setup).

      

            <currentDBConfig>wso2registry</currentDBConfig>

                <readOnly>false</readOnly>

                <registryRoot>/</registryRoot>

                <dbConfig name="wso2registry">

                <url>jdbc:mysql://10.3.24.96:3306/slave?autoReconnect=true</url>

                <userName>root</userName>

                <password>root123</password>

                <driverName>com.mysql.jdbc.Driver</driverName>

                <maxActive>50</maxActive>

                <maxWait>60000</maxWait>

                <minIdle>5</minIdle>

            </dbConfig>

             

            4.1.2 Open the ESB_SLAVE_HOME/repository/conf/user-mgt.xml file and change the database configuration as below(IP addresses and urls have to change according to your setup).

                     Note : Here we are changing the mode to readOnly in database configuration.

                <ReadOnly>true</ReadOnly>

                <Property name="url">jdbc:mysql://10.3.24.96:3306/usermgt?autoReconnect=true</Property>

                <Property name="userName">root</Property>

                <Property name="password">root123</Property>

                <Property name="driverName">com.mysql.jdbc.Driver</Property>

                <Property name="maxActive">50</Property>

                <Property name="maxWait">60000</Property>

                <Property name="minIdle">5</Property>

 

    4.2 Now let's configure the mounting configuration. For that we need to add more parameters to registry.xml file.

    

    4.2.1 Open the ESB_SLAVE_HOME/repository/conf/registry.xml file remove the original dbConfig element and add the database configuration below. (IP addresses and database urls have to change acccording to your setup).

                     Note: Here we are changing the mode to readOnly in database mounting configuration.

 

             <dbConfig name="esbMountRegistry">

                <url>jdbc:mysql://10.3.24.96:3306/greg?autoReconnect=true</url>

                <userName>root</userName>

                <password>root123</password>

                <driverName>com.mysql.jdbc.Driver</driverName>

                <maxActive>50</maxActive>

                <maxWait>60000</maxWait>

                <minIdle>5</minIdle>

            </dbConfig>

    

            <remoteInstance url="https://10.3.24.99:9443/registry">

                <id>Mount1</id>

                <dbConfig>esbMountRegistry</dbConfig>

                <readOnly>true</readOnly>

                <registryRoot>/</registryRoot>

            </remoteInstance>

 

            <mount path="/_system/config" overwrite="true">

                 <instanceId>Mount1</instanceId>

                <targetPath>/_system/esbConfig</targetPath>

            </mount>

 

            <mount path="/_system/governance" overwrite="true">

              <instanceId>Mount1</instanceId>

                <targetPath>/_system/governance</targetPath>

            </mount>

 

            Please change the remoteInstance URL according to the configuration of the GREG running machine. Please note that "InstanceIds", "id" and "dbConfig" elements should be mapped properly if you are using different

            names for them.

           

            Note:  If you compare the configuration with master node, we are changing only the configuration of local registry and all the other configurations are same as the master node. The registry access mode in mounting is readonly but the local registry configuration should be read-write since it is specific for each node.

 

 4.3 Copy mysql jdbc driver library to ESB_SLAVE_HOME/repository/components/lib direcotry since by default we do not ship the mysql drivers in ESB.

 

5. Now the cluster is successfully setup to replicate the data in Governance Registry among cluster nodes. If you are going to enable Axis2 Clustering, please refer the articles written for Axis2 Clustering. (If you are having a state for each instance, as an example when you use Caching and Throttling in WSO2 ESB, you need to enable Axis2 Clustering).

 

6. You can start Master and Slave nodes and create a sequence in Master node, save it to config registry and double check whether those sequences are replicating on slave node. 
Author : Lahiru Gunathilaka, Senior Software Engineer, WSO2 Inc.