Friday, September 9, 2011

Sharing Registry Space across Multiple Product Instances


Contents

Applies To

WSO2 Carbon3.0.0 and later
Environmentall supported

Introduction

WSO2 Carbon is the base platform for all WSO2 Java products. Built on OSGi, Carbon encapsulates major SOA functionality along with a set of core services available to all WSO2 Java products in common. The Registry kernel of Carbon provides the basic registry and repository functionality. Products based on Carbon uses the services provided by the Registry kernel to establish its own registry space, which is utilized for storing data and persisting configuration.

The Registry space provided to each product contains three major partitions.
  • The Local Data Repository
  • The Configuration Registry
  • The Governance Registry

The local data repository contains system configuration as well as runtime data that is local to a single instance of a product. For example, the local data repository is used to store mount configurations that are local to each instance. This partition of the registry space is not intended to be shared among multiple servers. The configuration registry, which is the most widely used partition of the registry space, contains product specific configuration. These configuration can be shared across multiple instances of the same product (a cluster of ESB nodes for example). The third and final partition of the registry space is the governance registry, which contains data and configuration shared across the platform. The WSO2 Governance Registry makes use of this partition of the registry space to store services and related metadata such as WSDLs, schemas, policies and endpoints. The governance registry partition has been designed in a way that it can be made use of by multiple instances of various Carbon based products.

Deployment Strategies

Two of the three partitions mentioned above can be shared across multiple product instances. Therefore, we can identify four main deployment strategies.

Strategy A
Strategy A
The entire registry space is local to the single server instance. The registry space is not shared in this deployment strategy. This is the recommended deployment strategy for a single instance (stand-alone deployment) of a product. You also could use this model, if you have two or more instances of a product and do not require sharing data or configuration among them.

Strategy B
Strategy B
The configuration and governance registry is shared among instances of a group/cluster. For example, a set of WSO2 Enterprise Service Businstances that have been configured to operate in a clustered environment can have a single configuration and governance registry which is shared across each node of the cluster. A separate instance of the WSO2 Governance Registry is used to provide the space used in common.

Strategy C
Strategy C
The configuration registry remains local to each node while the governance registry is shared among two or more nodes. This strategy is ideal for a deployment involving instances of multiple products. Such as the WSO2 Web Services Application ServerWSO2 Enterprise Service Bus and the WSO2 Governance Registry. The governance registry space of each product instance will be a shared space provided by the WSO2 Governance Registry instance.

Strategy D
Strategy D
Both the configuration registry and the governance registry will be shared among nodes in this model. However, the sharing happens at two levels. The configuration registries of multiple instances of a single product group/cluster will be shared among the member nodes. The governance registries of all instances (belonging to one or more products) will be shared among each other. This strategy is ideal when there are clusters of multiple products involved. An example is a deployment involving a cluster of WSO2 Web Services Application Server nodes and a cluster of WSO2 Enterprise Service Bus nodes.

In here, three instances of WSO2 Governance Registry can be used to provide the spaces shared at various levels. The first instance provides the space for the shared configuration registry for the WSO2 Web Services Application Server cluster. The second provides the space for the shared configuration registry for the WSO2 Enterprise Service Bus cluster. And, the third provides the space for the governance registry shared among the members of both clusters.

This strategy can also be thought of as a combination of the strategy B and strategy C.

Configuration Options

In order to separate the registry space into partitions that can be shared across multiple nodes, we first need to configure the Registry Kernel of each instance. This can be done by editing the corresponding registry.xml file found in the CARBON_HOME/repository/conf directory. TheWSO2 Carbon 3.0.0 release includes two types of configuration models that can be used.
  • JDBC-based configuration model
  • Atom-based configuration model

The difference between the two models is that the JDBC-based configuration model uses database calls to communicate with remote instances providing the shared space for a given partition, while to Atom-based configuration model uses Atom Publishing Protocol (AtomPub) over HTTP for the remote calls.

The JDBC-based configuration model

This model involves three main types of configuration elements.
  • The database configuration
  • The remote instance configuration
  • The mount configuration

The database configuration should include JDBC connection details (or the data source information) required to connect to each database (one for the configuration registry, and another for the governance registry), as applicable. The remote instance configuration defines the database configuration to use, whether instance is used in READ-ONLY mode, and the root of the registry that is being mounted. Finally, the mount configuration defines the mount path on the local instance along with the target path on the remote instance. The configurations for the four main strategies described above are as follows.

Please note that we have used the MySQL DBMS as an example. Details on how to configure other supported DBMS are found in here. Also, it is important to note that we have assumed that you will use the existing configuration for the Registry Kernel as it is, and add the following configuration in addition to what's already on the registry.xml file.

Strategy A
Strategy A
This strategy requires no additional configuration.

Strategy B
Strategy B
In here, we will be considering a cluster WSO2 Enterprise Service Bus nodes sharing the configuration and governance registry space provided by the WSO2 Governance Registry. To start with, you will have to create a collection named /_system/nodes on the WSO2 Governance Registryinstance. Also, let's assume that the WSO2 Governance Registry instance has a database configuration as follows:
    <currentDBConfig>wso2registry</currentDBConfig>

    <dbConfig name="wso2registry">
        <url>jdbc:mysql://10.20.30.40:3306/registryDB</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>
Then, the corresponding configuration that is required to be added to the registry.xml file of the WSO2 Enterprise Service Bus master node will be:
    <dbConfig name="sharedRegistry">
        <url>jdbc:mysql://10.20.30.40:3306/registryDB</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.40:9443/registry">
        <id>sharedRegistryInstance</id>
        <dbConfig>sharedRegistry</dbConfig>
        <readOnly>false</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
and, the corresponding configuration that is required to be added to the registry.xml files of each WSO2 Enterprise Service Bus slave node will be:
    <dbConfig name="sharedRegistry">
        <url>jdbc:mysql://10.20.30.40:3306/registryDB</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.40:9443/registry">
        <id>sharedRegistryInstance</id>
        <dbConfig>sharedRegistry</dbConfig>
        <readOnly>true</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>


Strategy C
Strategy C
In here, we will be considering a single WSO2 Enterprise Service Bus node and a single WSO2 Web Services Application Server node sharing the governance registry space provided by the WSO2 Governance Registry. Let's assume that the WSO2 Governance Registry instance has a database configuration as follows:
    <currentDBConfig>wso2registry</currentDBConfig>

    <dbConfig name="wso2registry">
        <url>jdbc:mysql://10.20.30.40:3306/registryDB</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>
Then, the corresponding configuration that is required to be added to the registry.xml files of the WSO2 Enterprise Service Bus node and theWSO2 Web Services Application Server node will be:
    <dbConfig name="governanceRegistry">
        <url>jdbc:mysql://10.20.30.40:3306/registryDB</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.40:9443/registry">
        <id>governanceRegistryInstance</id>
        <dbConfig>governanceRegistry</dbConfig>
        <readOnly>false</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>


Strategy D
Strategy D
In here, let's assume that we have a cluster of WSO2 Enterprise Service Bus nodes and a cluster of WSO2 Web Services Application Servernodes sharing a governance registry space provided by a WSO2 Governance Registry node named G-Reg 1. Let's also assume that the cluster ofWSO2 Enterprise Service Bus share a configuration registry space provided by a WSO2 Governance Registry node named G-Reg 2 and the cluster of WSO2 Web Services Application Server share a configuration registry space provided by a WSO2 Governance Registry node named G-Reg 3.

To start with, you will have to create a collection named /_system/nodes on the instances G-Reg 2 and G-Reg 3. Also, let's assume that the G-Reg 1 instance has a database configuration as follows:
    <currentDBConfig>wso2registry</currentDBConfig>

    <dbConfig name="wso2registry">
        <url>jdbc:mysql://10.20.30.41:3306/registryDB1</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>
and, the G-Reg 2 instance has a database configuration as follows:
    <currentDBConfig>wso2registry</currentDBConfig>

    <dbConfig name="wso2registry">
        <url>jdbc:mysql://10.20.30.42:3306/registryDB2</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>
and, the G-Reg 3 instance has a database configuration as follows:
    <currentDBConfig>wso2registry</currentDBConfig>

    <dbConfig name="wso2registry">
        <url>jdbc:mysql://10.20.30.43:3306/registryDB3</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>
Then, the corresponding configuration that is required to be added to the registry.xml file of the WSO2 Enterprise Service Bus master node will be:
    <dbConfig name="configRegistry">
        <url>jdbc:mysql://10.20.30.42:3306/registryDB2</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <dbConfig name="governanceRegistry">
        <url>jdbc:mysql://10.20.30.41:3306/registryDB1</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.42:9443/registry">
        <id>configRegistryInstance</id>
        <dbConfig>configRegistry</dbConfig>
        <readOnly>false</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <dbConfig>governanceRegistry</dbConfig>
        <readOnly>false</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
and, the corresponding configuration that is required to be added to the registry.xml files of each WSO2 Enterprise Service Bus slave node will be:
    <dbConfig name="configRegistry">
        <url>jdbc:mysql://10.20.30.42:3306/registryDB2</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <dbConfig name="governanceRegistry">
        <url>jdbc:mysql://10.20.30.41:3306/registryDB1</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.42:9443/registry">
        <id>configRegistryInstance</id>
        <dbConfig>configRegistry</dbConfig>
        <readOnly>true</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <dbConfig>governanceRegistry</dbConfig>
        <readOnly>true</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
Also, the corresponding configuration that is required to be added to the registry.xml file of the WSO2 Web Services Application Server master node will be:
    <dbConfig name="configRegistry">
        <url>jdbc:mysql://10.20.30.43:3306/registryDB3</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <dbConfig name="governanceRegistry">
        <url>jdbc:mysql://10.20.30.41:3306/registryDB1</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.43:9443/registry">
        <id>configRegistryInstance</id>
        <dbConfig>configRegistry</dbConfig>
        <readOnly>false</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <dbConfig>governanceRegistry</dbConfig>
        <readOnly>false</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
and, the corresponding configuration that is required to be added to the registry.xml files of each WSO2 Web Services Application Server slave node will be:
    <dbConfig name="configRegistry">
        <url>jdbc:mysql://10.20.30.43:3306/registryDB3</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <dbConfig name="governanceRegistry">
        <url>jdbc:mysql://10.20.30.41:3306/registryDB1</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.43:9443/registry">
        <id>configRegistryInstance</id>
        <dbConfig>configRegistry</dbConfig>
        <readOnly>true</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <dbConfig>governanceRegistry</dbConfig>
        <readOnly>true</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
Optionally, you can add the following configuration to the registry.xml files of G-Reg 2 and G-Reg 3, so that the governance registry found at each node in the deployment would point to the same location.
    <dbConfig name="governanceRegistry">
        <url>jdbc:mysql://10.20.30.41:3306/registryDB1</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>80</maxActive>
        <maxWait>6000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <dbConfig>governanceRegistry</dbConfig>
        <readOnly>false</readOnly>
        <registryRoot>/</registryRoot>
    </remoteInstance>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
Another variation to the original strategy is to use a single WSO2 Governance Registry instead of having three distinct nodes. In order to do this, we need to create two collections named /_system/nodes1 and /_system/nodes2. In this scenario, the target path of the governance registry mount of the nodes of the WSO2 Enterprise Service Bus cluster would be /_system/nodes1 and the target path of the governance registry mount of the nodes of the WSO2 Web Services Application Server cluster would be /_system/nodes2. The JDBC URL of all configurations mentioned above would then be something similar to jdbc:mysql://10.20.30.40:3306/registryDB and the remote instance URL of all configurations mentioned above would then be something similar to https://10.20.30.40:9443/registry.

The Atom-based configuration model

This model involves two main types of configuration elements.
  • The remote instance configuration
  • The mount configuration

Unlike the JDBC-based configuration model, the Atom-based configuration model only requires two configuration elements. The remote instance configuration is a bit different and defines the URL of the remote registry along with valid credentials that can connect to it. The mount configuration is the same as in the JDBC-based configuration model which defines the mount path on the local instance along with the target path on the remote instance. The configurations for the four main strategies described above are as follows.

Once again (as in the JDBC-based configuration model), it is important to note that we have assumed that you will use the existing configuration for the Registry Kernel as it is, and add the following configuration in addition to what's already on the registry.xml file.

Strategy A
Strategy A
This strategy requires no additional configuration.

Strategy B
Strategy B
In here, we will be considering a cluster WSO2 Enterprise Service Bus nodes sharing the configuration and governance registry space provided by the WSO2 Governance Registry. To start with, you will have to create a collection named /_system/nodes on the WSO2 Governance Registryinstance. Also, let's assume that the URL of WSO2 Governance Registry instance is https://10.20.30.40:9443/registry. Then, the corresponding configuration that is required to be added to the registry.xml file of the WSO2 Enterprise Service Bus master node will be:
    <remoteInstance url="https://10.20.30.40:9443/registry">
        <id>sharedRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
and, the corresponding configuration that is required to be added to the registry.xml files of each WSO2 Enterprise Service Bus slave node will be:
    <remoteInstance url="https://10.20.30.40:9443/registry">
        <id>sharedRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>sharedRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>


Strategy C
Strategy C
In here, we will be considering a single WSO2 Enterprise Service Bus node and a single WSO2 Web Services Application Server node sharing the governance registry space provided by the WSO2 Governance Registry. Let's assume that the URL of WSO2 Governance Registry instance ishttps://10.20.30.40:9443/registry. Then, the corresponding configuration that is required to be added to the registry.xml files of theWSO2 Enterprise Service Bus node and the WSO2 Web Services Application Server node will be:
    <remoteInstance url="https://10.20.30.40:9443/registry">
        <id>governanceRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>


Strategy D
Strategy D
In here, let's assume that we have a cluster of WSO2 Enterprise Service Bus nodes and a cluster of WSO2 Web Services Application Servernodes sharing a governance registry space provided by a WSO2 Governance Registry node named G-Reg 1. Let's also assume that the cluster ofWSO2 Enterprise Service Bus share a configuration registry space provided by a WSO2 Governance Registry node named G-Reg 2 and the cluster of WSO2 Web Services Application Server share a configuration registry space provided by a WSO2 Governance Registry node named G-Reg 3.

To start with, you will have to create a collection named /_system/nodes on the instances G-Reg 2 and G-Reg 3. Let's assume that the URLs of the instances G-Reg 1G-Reg 2 and G-Reg 3 are https://10.20.30.41:9443/registryhttps://10.20.30.42:9443/registry, andhttps://10.20.30.43:9443/registry.

Then, the corresponding configuration that is required to be added to the registry.xml file of the WSO2 Enterprise Service Bus master node will be:
    <remoteInstance url="https://10.20.30.42:9443/registry">
        <id>configRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
and, the corresponding configuration that is required to be added to the registry.xml files of each WSO2 Enterprise Service Bus slave node will be:
    <remoteInstance url="https://10.20.30.42:9443/registry">
        <id>configRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
Also, the corresponding configuration that is required to be added to the registry.xml file of the WSO2 Web Services Application Server master node will be:
    <remoteInstance url="https://10.20.30.43:9443/registry">
        <id>configRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
and, the corresponding configuration that is required to be added to the registry.xml files of each WSO2 Web Services Application Server slave node will be:
    <remoteInstance url="https://10.20.30.43:9443/registry">
        <id>configRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>configRegistryInstance</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>


    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
Optionally, you can add the following configuration to the registry.xml files of G-Reg 2 and G-Reg 3, so that the governance registry found at each node in the deployment would point to the same location.
    <remoteInstance url="https://10.20.30.41:9443/registry">
        <id>governanceRegistryInstance</id>
        <username>username</username>
        <password>password</password>
    </remoteInstance>

    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>
Another variation to the original strategy is to use a single WSO2 Governance Registry instead of having three distinct nodes. In order to do this, we need to create two collections named /_system/nodes1 and /_system/nodes2. In this scenario, the target path of the governance registry mount of the nodes of the WSO2 Enterprise Service Bus cluster would be /_system/nodes1 and the target path of the governance registry mount of the nodes of the WSO2 Web Services Application Server cluster would be /_system/nodes2. The remote instance URL of all configurations mentioned above would then be something similar to https://10.20.30.40:9443/registry.

Summary

Having read through this tutorial, we have understood the three main partitions of the Registry space of the WSO2 Carbon Registry Kernel. All the products based on the WSO2 Carbon Framework contains these three partitions allowing resource sharing and configuration replication. We have understood the various strategies that we can use to share these registry spaces among multiple product instances. Having understood the various strategies to share the registry space, we then went through some sample configurations (based around the MySQL database for the JDBC-based configuration model).

Author :

Senaka Fernando, Software Engineer, WSO2 Inc., senaka at wso2 dot com




































































AttachmentSize
patterns1.png15.7 KB
atom-patterns4.png32.58 KB
patterns4.png30.17 KB
jdbc-patterns4.png32.53 KB
atom-patterns2.png24.84 KB
atom-patterns3.png24.54 KB
jdbc-patterns2.png24.81 KB
jdbc-patterns3.png24.5 KB
patterns2.png23.69 KB
patterns3.png23.39 KB
sameera's picture

Hi Senaka, Very informative

Hi Senaka,
Very informative article. Nice work
In strategy B of the Atom-based registry configuration, you have presented two configurations for the ESB master node and for the slave node. But those two configurations looks very similar. Is that intentional or is there something else that we need to do mark instances as slave nodes?
Thanks,
Sameera
senakaf's picture

Re: Hi Senaka, Very informative

Hi Sameera,
Thanks and very good question. In Atom-based configuration, the credential (username/password) will determine the level of access you get, read-only/read-write, unlike in the JDBC-based configuration. So, essentially you can use two pairs of username/password for the two types of nodes.
Thanks,
Senaka.
harshana.wso2.com's picture

A Great Article

Hi Senaka,
This is a great article and covers number of useful scenarios. Keep up the good work!!
Thanks and Regards,
Harshana
preben's picture

Error while following strategy B

Good article, but I get an error when following strategy B with a JDBC based configuration:
ESB-registry.xml
configRegistry
false
/
jdbc:mysql://localhost:3306/regdb
regadmin
regadmin
com.mysql.jdbc.Driver
80
60000
5
configRegistryInstance
configRegistry
false
/
configRegistryInstance
/_system/nodes
Why is it using the embeddedregistry - shouldn't it be remote ?
Error:
[2010-05-02 15:53:36,921] ERROR - EmbeddedRegistry Failed to close transaction.
java.sql.SQLException: Total number of available connections are less than the total number of closed connections
at org.wso2.carbon.registry.core.jdbc.utils.Transaction$ManagedRegistryConnection.close(Transaction.java:550)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.endTransaction(EmbeddedRegistry.java:421)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.commitTransaction(EmbeddedRegistry.java:411)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.put(EmbeddedRegistry.java:645)
at org.wso2.carbon.registry.core.utils.RegistryUtils.addBaseCollectionStructure(RegistryUtils.java:566)
at org.wso2.carbon.registry.core.session.UserRegistry.addSystemCollections(UserRegistry.java:265)
at org.wso2.carbon.registry.core.session.UserRegistry.init(UserRegistry.java:242)
at org.wso2.carbon.registry.core.session.UserRegistry.(UserRegistry.java:187)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService.getUserRegistry(EmbeddedRegistryService.java:396)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService.getRegistry(EmbeddedRegistryService.java:422)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService.getRegistry(EmbeddedRegistryService.java:439)
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService.getRegistry(EmbeddedRegistryService.java:435)
at org.wso2.carbon.registry.core.internal.RegistryCoreServiceComponent.registerBuiltInHandlers(RegistryCoreServiceComponent.java:273)
at org.wso2.carbon.registry.core.internal.RegistryCoreServiceComponent.buildRegistryService(RegistryCoreServiceComponent.java:389)
at org.wso2.carbon.registry.core.internal.RegistryCoreServiceComponent.activate(RegistryCoreServiceComponent.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:230)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:140)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:330)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:560)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:182)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:292)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:226)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:124)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:930)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:220)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:149)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:757)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:712)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:129)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:206)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:506)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:524)
at org.wso2.carbon.user.core.internal.Activator.startDeploy(Activator.java:29)
at org.wso2.carbon.user.core.internal.BundleCheckActivator.start(BundleCheckActivator.java:50)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:782)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:773)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:754)
at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:352)
at org.eclipse.osgi.framework.internal.core.AbstractBundle.resume(AbstractBundle.java:370)
at org.eclipse.osgi.framework.internal.core.Framework.resumeBundle(Framework.java:1068)
at org.eclipse.osgi.framework.internal.core.StartLevelManager.resumeBundles(StartLevelManager.java:557)
at org.eclipse.osgi.framework.internal.core.StartLevelManager.incFWSL(StartLevelManager.java:464)
at org.eclipse.osgi.framework.internal.core.StartLevelManager.doSetStartLevel(StartLevelManager.java:248)
at org.eclipse.osgi.framework.internal.core.StartLevelManager.dispatchEvent(StartLevelManager.java:445)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:220)
at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:330)
senakafdo's picture

Re: Error while following strategy B

Hi Preben,
Thanks for the compliments.
From your exception, it seems that it might rather be an issue with transaction handling rather than the configuration of the registry. What version (beta/RC) of G-Reg did you try? And, can you paste the contents of the log file to this thread or drop a mail to the dev list (as it seems that you can't attach a file to this thread)?
Thanks and Best Regards,
Senaka.
preben's picture

Hi Senaka Send you a update

Hi Senaka
Send you a update on this by email with log attachments.
Best regards
Preben
senakafdo's picture

Re: Hi Senaka Send you a update

Hi Preben,
The issue seems to be in your registry.xml file. You cannot use "configRegistry" as your current DB configuration. I have assumed that you will keep the default registry.xml configuration and add the provided configurations in addition to the existing configuration.
"Also, it is important to note that we have assumed that you will use the existing configuration for the Registry Kernel as it is, and add the following configuration in addition to what's already on the registry.xml file."
This means, that your registry.xml should look something similar to:
<wso2registry>
<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<registryRoot>/</registryRoot>
<dbConfig name="wso2registry">
<url>jdbc:mysql://localhost:3306/regdbBase</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>80</maxActive>
<maxWait>60000</maxWait>
<minIdle>5</minIdle>
</dbConfig>
<dbConfig name="configRegistry">
<url>jdbc:mysql://localhost:3306/regdb</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>80</maxActive>
<maxWait>60000</maxWait>
<minIdle>5</minIdle>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>configRegistryInstance</id>
<dbConfig>configRegistry</dbConfig>
<readOnly>false</readOnly>
<registryRoot>/</registryRoot>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>configRegistryInstance</instanceId>
<targetPath>/_system/nodes</targetPath>
</mount>
<versionResourcesOnChange>true</versionResourcesOnChange>
<staticConfiguration>
<versioningProperties>true</versioningProperties>
<versioningComments>true</versioningComments>
<versioningTags>true</versioningTags>
<versioningRatings>true</versioningRatings>
</staticConfiguration>
</wso2registry>
Thanks and Best Regards,
Senaka.
preben's picture

Hi Senaka Thanks for

Hi Senaka
Thanks for clarifying - my mistake.
works perfect.
Hope that we will be ready to go into poc of registry soon.
Best Regards
Preben
rajika's picture

Good article Senaka. Rajika

Good article Senaka.
Rajika
senakafdo's picture

Re: Good article Senaka. Rajika

Thanks Rajika.
Regards,
Senaka.

No comments:

Post a Comment