Rediscover SOA with WSO2 Carbon and WS-Discovery

As the number of services and resources grow in an SOA deployment, the task of locating which service resides where is going to become tedious. Wrong or outdated endpoint configurations can lead to application faults, service outages and business revenue losses. WS-Discovery is a standard protocol for publishing and discovering service endpoints in a complex SOA. Using WS-Discovery, systems can automatically discover the services and endpoints they are interested in without any human intervention. In this article, Hiranya Jayathilaka provides a brief introduction to WS-Discovery and how this standard is supported by WSO2 Carbon based products such as WSO2 AS, WSO2 DSS, WSO2 GReg and WSO2 ESB. Date: Mon, 10th Oct, 2011 Level: Intermediate Reads: 169 Comments: 5 | Login or register to post comments Printer-friendly version Send to friend PDF version  Post to twitter

Hiranya Jayathilaka WSO2 Inc

Contents

• Prerequisites
• Need for Dynamic Discovery in SOA
• Introducing WS-Discovery
  • WS-Discovery Concepts
  • WS-Discovery Messages
• WS-Discovery Support in WSO2 Carbon
• Implementing an Inventory Management System - Mission Briefing
  
  • A Word on Documentation Conventions
• Iteration 1 - Running the Pilot Project
  • Planning for Inevitable Change
• Iteration 2 - Discovery Enabled Inventory Management System
  • Reviewing the Code
• Iteration 3 - Going Public
  • Getting Rid of Hard Coded Addresses
• Room for Improvement
• Summary and Conclusion

Prerequisites

This tutorial contains a number of sample services and applications. If you wish to try them out, you will need following software. Please have them all installed on a single machine.

• JDK 1.6 (version 1.6.0_23 recommended)
• WSO2 Data Services Server 2.6.0 or higher
• WSO2 Application Server 4.1.0 or higher
• WSO2 Enterprise Service Bus 4.0.0 or higher
• WSO2 Governance Registry 4.0.0 or higher
• MySQL
• MySQL JDBC driver 5.1 or higher (version 5.1.8 recommended)

You can read through and follow this article even if you don't want to try out the samples. But it is highly recommended that you do at least some of the hands on work described in the tutorial and get familiar with practical WS-Discovery.

Need for Dynamic Discovery in SOA

Services are the fundamental building blocks of service oriented architecture. They are reusable, functional entities that can be consumed by both human users and applications. In addition to the services, a typical SOA deployment may also consist of an enterprise service bus, a registry/repository, a business process engine and various other tools for master data management, identity management and data visualization. Despite all the benefits offered by SOA, connecting all these diverse components together is never an easy task. The developers should be aware of the endpoints of all the services and applications in order to be able to piece them together. Needless to mention when there are dozens of services to be interconnected this becomes a cumbersome task as it requires the developers to keep track of many host names, IP addresses, port numbers and URLs. To make things worse sometimes different services are managed by different teams, departments or organizations. In such situations, service endpoints may change time to time without prior notice thus breaking other services and applications that depend on them. Such practical problems make it absolutely necessary to have a mechanism in place for discovering the target services at runtime.
Another problem that many SOA developers and architects face is moving applications across different environments. Most SOA development teams have to deal with at least three different environments, namely development, QA and production. When an application is moved from one environment to another all the components with address URLs embedded within them would break. A similar problem arises when building a service oriented architecture in a dynamic environment such as an enterprise cloud. In cloud deployments it is almost impossible to know the exact hosts and ports on which a particular service will be deployed as they change rather frequently. Therefore some mechanism for dynamically discovering the services is required. With dynamic discovery capabilities, developers no longer have to hard code URLs and endpoints into the services and configuration artifacts they develop. Rather they would configure the services and artifacts with some generic, well known keywords and identifiers and let the applications resolve them to the actual endpoints at runtime. This way applications will continue to work even if a particular URL is changed while the system is running.

Introducing WS-Discovery

WS-Discovery is a standard protocol for dynamically discovering service endpoints. Using this protocol, service provider applications (target services) can advertise their endpoints to interested parties. On the other hand service consumer applications (clients) can use WS-Discovery to find the required endpoints by either listening for the notifications published by the target services or querying a central repository that contains endpoint details. WS-Discovery standard has two versions as of 2011 with the version 1.1 published in 2009 being the latest. This version of the protocol defines two main operational modes.

1. Ad hoc mode
2. Managed mode

In the ad hoc mode applications use a multicast communication protocol based on UDP to publish and discover service endpoints. Target services multicast endpoint update notifications. Clients can listen for these notifications to find the latest endpoint details. In the managed mode, applications publish and discover endpoints through a central party known as the discovery proxy. Target services notify the discovery proxy with the endpoint information. Similarly clients can query the discovery proxy to find the required endpoints. This operational mode is more suitable for corporate networks where multicast traffic is usually not preferred.
WS-Discovery also allows room for a hybrid operational mode where ad hoc mode and managed mode can coexist side by side. In such an environment, target services will multicast the endpoint update notifications as if they were running in ad hoc discovery mode. A discovery proxy will receive these notifications and maintain a directory of endpoints. Clients can then either query the discovery proxy or directly receive the endpoint updates from the services over multicast. The protocol also allows room for clients to seamlessly switch between ad hoc mode and managed mode.

WS-Discovery Concepts

WS-Discovery protocol associates a unique identifier with each service. This is referred to as the stable identifier of the service as it does not change over time or along with other variable parameters of the service. This ID can be used across the SOA deployment to uniquely identify a particular service instance. WS-Discovery may also associate a given service with a set of scopes, types and transport addresses (endpoints). A scope is a user defined category label (a tag) attached to the service. Any valid URI can be used as a scope. A service may belong to zero, one or more scopes as configured by the user. Scopes are used mainly to group and categorize services so they can be indexed and searched. Types are also used to further categorize and search for services. Any qualified XML name (QName) can be used as a type. In most scenarios it makes sense to treat the WSDL port types of a service as its WS-Discovery types. Transport addresses represent the endpoint references over which the service can be reached. A service may have zero or more transport addresses depending on its availability level. A service which does not have any transport addresses is considered to be down or unavailable.
When WS-Discovery is used to publish and discover endpoints, target services should always publish the endpoints along with their stable ID values, scopes and types. Clients can find the necessary service endpoints if they know the exact ID of the service they wish to communicate with. If not they can perform a search by scopes or types to find a service ID that matches with the provided service metadata. Once a matching service ID is found, more look ups can be performed to resolve the service ID into a set of endpoints.

WS-Discovery Messages

WS-Discovery protocol defines the messages that need to be exchanged in order to successfully publish or discover service endpoints. Altogether there are six types of messages defined in the standard.

1. Hello messages
2. Bye messages
3. Probe messages
4. Probe match messages
5. Resolve messages
6. Resolve match messages

WS-Discovery requires Hello messages to be sent out whenever the endpoints are modified in a service. Hello messages contain the unique service ID, scopes, types and transport addresses of the services. Clients or the discovery proxies should update their configurations accordingly as and when they receive Hello notifications from the target services. Target services will send out Bye messages when a service is deactivated or shutdown. Bye messages indicate that a particular service is no longer available over one or more endpoints. When deployed in managed mode, Hello messages and Bye messages should be sent using the in-only message exchange pattern. Therefore the target service should not expect a response from the discovery proxy. If HTTP is used as the transport, the proxy should simply respond back with a 202 Accepted response for Hello and Bye messages. A sample Hello message and a Bye message is given below.

      <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
         <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <wsa:To>http://localhost:8080/services/DiscoveryProxy</wsa:To>
            <wsa:ReplyTo>
               <wsa:Address>http://www.w3.org/2005/08/addressing/none</wsa:Address>
            </wsa:ReplyTo>
            <wsa:MessageID>urn:uuid:d5c85841-abff-45d1-85ee-933cb824b2d5</wsa:MessageID>
            <wsa:Action>http://docs.oasis-open.org/ws-dd/ns/discovery/2009/01/Hello</wsa:Action>
         </soapenv:Header>
         <soapenv:Body>
            <wsd:Hello xmlns:wsd="http://docs.oasis-open.org/ws-dd/ns/discovery/2009/01">
               <wsa:EndpointReference xmlns:wsa="http://www.w3.org/2005/08/addressing">
                  <wsa:Address>urn:uuid:50bccd63-8c1f-4f9d-a0db-8b0f290306e3</wsa:Address>
               </wsa:EndpointReference>
               <wsd:Types xmlns:axis2ns2="http://echo.services.core.carbon.wso2.org">axis2ns2:echoPortType</wsd:Types>
               <wsd:Scopes>http://docs.oasis-open.org/ws-dd/ns/discovery/2009/01/DefaultScope</wsd:Scopes>
               <wsd:XAddrs>https://localhost:8243/services/echo http://localhost:8280/services/echo</wsd:XAddrs>
               <wsd:MetadataVersion>1</wsd:MetadataVersion>
            </wsd:Hello>
         </soapenv:Body>
      </soapenv:Envelope>

      <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
         <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <wsa:To>http://localhost:8080/services/DiscoveryProxy</wsa:To>
            <wsa:ReplyTo>
               <wsa:Address>http://www.w3.org/2005/08/addressing/none</wsa:Address>
            </wsa:ReplyTo>
            <wsa:MessageID>urn:uuid:98861fea-da54-47c3-8253-802f29419d51</wsa:MessageID>
            <wsa:Action>http://docs.oasis-open.org/ws-dd/ns/discovery/2009/01/Bye</wsa:Action>
         </soapenv:Header>
         <soapenv:Body>
            <wsd:Bye xmlns:wsd="http://docs.oasis-open.org/ws-dd/ns/discovery/2009/01">
               <wsa:EndpointReference xmlns:wsa="http://www.w3.org/2005/08/addressing">
                  <wsa:Address>urn:uuid:50bccd63-8c1f-4f9d-a0db-8b0f290306e3</wsa:Address>
               </wsa:EndpointReference>
               <wsd:Types xmlns:axis2ns6="http://echo.services.core.carbon.wso2.org">axis2ns6:echoPortType</wsd:Types>
               <wsd:Scopes>http://docs.oasis-open.org/ws-dd/ns/discovery/2009/01/DefaultScope</wsd:Scopes>
               <wsd:XAddrs>https://localhost:8243/services/echo http://localhost:8280/services/echo</wsd:XAddrs>
               <wsd:MetadataVersion>1</wsd:MetadataVersion>
            </wsd:Bye>
         </soapenv:Body>
      </soapenv:Envelope>

Probe messages are sent by clients in managed mode to query a discovery proxy. A probe may contain the scopes and types for which a matching service ID is required. Discovery proxy should respond to a Probe request by sending a Probe Match message. A Probe Match will contain a list of services that match the search criterion specified in the Probe. If no matching services can be found, discovery proxy should send back an empty Probe Match. Clients running in the ad hoc mode may send Probe messages over multicast. In this case Probe messages are sent out as in-only requests. Any target services or discovery proxies listening for such multicast probes may respond to the client with a Probe Match over a unicast channel.
Clients running in the managed mode can send Resolve messages to the discovery proxy to resolve a service ID into one or more endpoints. Resolve messages contain the service ID which needs to be translated. Discovery proxy should respond with a Resolve Match message with a list of endpoints upon receiving a Resolve message. If a service does not exist by the specified ID, an empty Resolve Match should be returned. Similar to the case of Probe messages, clients running in ad hoc mode may multicast Resolve messages.

WS-Discovery Support in WSO2 Carbon

WSO2 Carbon comes with a complete implementation of WS-Discovery 1.1 managed mode. Using this feature, service hosting products like Application Server, Data Services Server and Mashup Server can publish service endpoints to a user defined WS-Discovery proxy. Message routing and brokering products like WSO2 ESB can use the discovery protocol to find the service endpoints in the network and create proxy services and mediation sequences on top of them. WSO2 Governance Registry ships with the basic WS-Discovery proxy components and hence can be used as a discovery proxy. Also WS-Discovery libraries of Carbon can be used in other Java applications to construct WS-Discovery aware tools.
WS-Discovery support of Carbon is implemented as a set of OSGi bundles:

1. org.wso2.carbon.discovery.core - Contains the core WS-Discovery message definitions, management utilities, service observers and notification system
2. org.wso2.carbon.discovery.proxy - Contains the WS-Discovery proxy implementation
3. org.wso2.carbon.discovery.ui - UI components for connecting to a remote discovery proxy and browsing/searching the discovered services
4. org.wso2.carbon.discovery.admin.ui - UI components for enabling and disabling WS-Discovery service notifications
5. org.wso2.carbon.discovery.mediation.ext - Contains a WS-Discovery based mediation registry extension for the ESB
6. org.wso2.carbon.discovery.module - A WS-Discovery based Axis2 client module

The org.wso2.carbon.discovery.core bundle is a dependency for all other WS-Discovery components. Products like WSO2 Application Server and WSO2 Data Services Server ship with the core discovery bundle and hence are capable of firing discovery notifications. WSO2 Governance Registry is armed with the core component as well as the org.wso2.carbon.discovery.proxy component and therefore can function as a discovery proxy. WSO2 ESB ships with the discovery core bundle and the org.wso2.carbon.discovery.ui. With those components in place, ESB management console can be used to connect to remote discovery proxy servers and search for services. Starting from Carbon 3.2.0 release this UI component is also integrated with WSO2 Identity Server and WSO2 Business Process Server to facilitate locating of remote endpoints.
The org.wso2.carbon.discovery.mediation.ext component can be installed on the ESB through the Carbon Feature Manager to support proxy services with dynamically discovered endpoints (this is discussed in detail towards the latter part of the article). The org.wso2.carbon.discovery.admin.ui can be installed on any service hosting product based on Carbon. This component will become a part of the standard product distributions in the near future. It is also part of the WSO2 Stratos based cloud platform. The org.wso2.carbon.discovery.module is currently not used in any Carbon based WSO2 product. It is intended to be used in client side applications as an Axis2 client extension.
Note that even though these components are shipped only with certain WSO2 products, there is nothing preventing a user from deploying these bundles in other Carbon based products and using them. Thanks to the highly modular and componentized architecture provided by Carbon, a user can simply use the Carbon Feature Manager UI/API to get them installed to any Carbon server. All these components and associated features are available in the WSO2 P2 feature repository.
The rest of this article describes how to build a loosely coupled system based on SOA using the WS-Discovery support provided by WSO2 Carbon. We will use WSO2 Application Server for hosting the core business services of the system. WSO2 Governance Registry will be used as the central discovery proxy of the overall architecture. We will also use WSO2 ESB as the integration framework which enables consumers to communicate with the business services. WS-Discovery protocol will be used to discover all the available web services, data services and proxy services in the system.

Implementing an Inventory Management System - Mission Briefing

We are going to create a simple inventory management system for "Trader Paul's", your friendly neighborhood department store. Users will interact with the system through a web interface to add items, browse catalogs and modify items available in the inventory. Most of the core system features will be implemented as web services so they can be easily consumed by other applications of the store. Some of the services will also be exposed to the public Internet over a secure channel so that customers, partners and suppliers can integrate their own applications with the "Trader Paul's" inventory manager.
The goal of this effort is to see how WS-Discovery can be used to construct complex distributed systems. Therefore we will not pay too much attention to proper management information system design. We will also keep concerns such as identity management aside for the moment and keep the system absolutely simple.
It's clear that we are going to need some kind of a persistent store for this system. We will be using a RDBMS such as MySQL for this purpose. WSO2 Data Services Server will be used to service enable the data. The web front end will be developed as a Java Web application and deployed on WSO2 Application Server along with any other Java Web Services we may have to develop. We can use WSO2 ESB to expose services to the Internet securely. The centerpiece of the solution is going to be WSO2 Governance Registry which will be used by the ESB and other client applications to discover the necessary service endpoints.

A Word on Documentation Conventions

The rest of this article describes a sequence of steps that you can carry out using a set of WSO2 products. Most of these action items involve signing into the web based management console of a WSO2 server and performing certain tasks. The menu of the WSO2 management console consists of four tabs:

1. Main
2. Configure
3. Monitor
4. Tools

Most of the tasks described here require you to interact with the 'Main' tab. Therefore if a particular action item does not specify a tab name, you should select the 'Main' tab and proceed with the options available there. When it is required to switch to a different tab, the document will explicitly mention that.
Also the management console contains menus and submenus. Following notation is used to refer various options available in these menus.
Manage > Web Services > List (Select the 'List' option under 'Manage' menu and 'Web Services' submenu)

Manage > Applications > Add (Select the 'Add' option under 'Manage' menu and 'Applications' submenu)

Iteration 1 - Running the Pilot Project

A possible first cut version of the overall solution is available in the solution1.zip archive. This solution is made of following components:

1. A relational database based on MySQL
2. A single data service (InventoryDataService) which enables applications to access the database
3. A couple of Java web services (ItemKeyGenerator and PriceFinder) that play supporting roles in the overall business logic
4. The web front end implemented as a Java webapp

I have deliberately left WSO2 ESB, WSO2 Governance Registry and discovery concerns out of this solution. Let's first try to deploy and run this basic implementation. In the process let's also try to understand the limitations of this solution and why discovery is important in a system such as this one.
Here are the steps to deploy the initial solution on a single box:
1. Install WSO2 Data Services Server (DSS) by extracting the downloaded archive.
2. Install WSO2 Application Server (AS) by extracting the downloaded archive.
3. Copy the MySQL JDBC driver to the repository/components/lib directory of DSS.
4. Open the repository/conf/carbon.xml file of the Application Server and set the port offset to '1'. This is required to avoid any port conflicts between the DSS and AS.

<Ports>
   <Offset>1</Offset>
   ...
</Ports>

5. Launch both servers. Go to the bin directory of each server and execute the relevant start up script.

Unix/Linux: sh wso2server.sh
Windows: wso2server.bat

6. Login to MySQL server and create a new database named 'inventory'. If needed create a separate user account in MySQL and grant the necessary permissions so the user can read and write to the database.

mysql> create database inventory;

7. Download and extract the solution1.zip file
8. Run the SQL script file named inventory.sql against the inventory database. That will create a set of tables and populate them with some test data.
9. Open the InventoryDataService.dbs file in a text or XML editor. Locate and modify the database connection settings section so that it points to the inventory database you just created. You will have to modify the JDBC connection URL and the user credentials.

   <config id="mysql">
      <property name="org.wso2.ws.dataservice.driver">com.mysql.jdbc.Driver</property>
      <property name="org.wso2.ws.dataservice.protocol">jdbc:mysql://localhost:3306/inventory</property>
      <property name="org.wso2.ws.dataservice.user">root</property>
      <property name="org.wso2.ws.dataservice.password">password</property>
   </config>

10. Sign in to the management console of DSS (https://localhost:9443/carbon). Click on 'Manage > Web Services > Add > Data Service > Upload'. Browse for the modified InventoryDataService.dbs file and upload it. The InventoryData service will get deployed on the server in a few seconds. Click on the 'Manage > Web Services > List' option and verify the service has been deployed successfully.
11. Sign in to the management console of AS (https://localhost:9444/carbon). Click on 'Manage > Applications > Add'. Browse for the TraderPaulsIMS-1.0.0.car file and upload it. It will take a few seconds for the services to get deployed. Once done, two new web services will be listed under 'Manage > Web Services > List'. The services are called ItemKeyGenerator and PriceFinder. In addition, a new web application named inventory will be listed under 'Manage > Web Applications > List'.

[2011-08-28 11:30:04,759]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Successfully Deployed Carbon Application : TraderPaulsIMS {super-tenant}
[2011-08-28 11:30:14,125]  INFO {org.wso2.carbon.core.deployment.DeploymentInterceptor} -  Deploying Axis2 service: PriceFinder {super-tenant}
[2011-08-28 11:30:14,815]  INFO {org.apache.axis2.deployment.DeploymentEngine} -  Deploying Web service: PriceFinder.aar - file:/home/hiranya/Desktop/discovery/wso2as-4.1.0/repository/deployment/server/axis2services/PriceFinder.aar
[2011-08-28 11:30:14,982]  INFO {org.wso2.carbon.core.deployment.DeploymentInterceptor} -  Deploying Axis2 service: ItemKeyGenerator {super-tenant}
[2011-08-28 11:30:15,232]  INFO {org.apache.axis2.deployment.DeploymentEngine} -  Deploying Web service: ItemKeyGenerator.aar - file:/home/hiranya/Desktop/discovery/wso2as-4.1.0/repository/deployment/server/axis2services/ItemKeyGenerator.aar
[2011-08-28 11:30:17,782]  INFO {org.wso2.carbon.server.TomcatGenericWebappsDeployer} -  Deployed webapp: StandardEngine[Tomcat].StandardHost[defaulthost].StandardContext[/inventory].File[/home/hiranya/Desktop/discovery/wso2as-4.1.0/repository/deployment/server/webapps/inventory.war]

Now we are all set for a pilot run of the Trader Paul's inventory management system. Launch a web browser and enter the URL http://localhost:9764/inventory. That should take you to the home page of the inventory manager which looks similar to the following:

Click on some of the available options and play around with it. Add a couple of items to the inventory and make sure everything works fine.
Most of the time the web application communicates with the InventoryData service. The webapp contacts the ItemKeyGenerator when adding a new item to the inventory. This service is responsible for generating a unique key for the item being inserted. PriceFinder service is invoked when you try to view the details of an item. The marked price value and the discount rate is calculated by the PriceFinder.

Planning for Inevitable Change

So far so good. We have the inventory management system up and running. What's even more impressive is that we have implemented the entire system using web services. As a result, the overall solution architecture is quite flexible and the individual components (services, data access and webapp) are not tightly coupled with each other. Moving to SOA seems to be the best thing Trader Paul's IT department has ever done. But then comes the inevitable change.
Let's assume the Trader Paul's system administrators wanted to do a major rearrangement of their network infrastructure. As a result of this, the InventoryData service was moved to a new server which has a different host name and a port. How will the web application cope up with this change?
To see what will happen simply shutdown the data services server. Open up its repository/conf/carbon.xml file and change the port offset value to 2.

<Ports>
   <Offset>2</Offset>
   ...
</Ports>

Basically we are incrementing the port number of DSS by 2. So the data service which was previously exposed on port 9763 will now be exposed on port 9765. Start the DSS instance with the above change and now try to browse the inventory web application. You should be able to access the home page without any issues, but pretty much everything else will fail with an error page like the following.

An exception similar to the following will be logged on the Application Server logs.

[2011-08-28 11:14:54,651]  INFO {org.apache.axis2.transport.http.HTTPSender} -  Unable to sendViaPost to url[http://localhost:9763/services/InventoryData]
java.net.ConnectException: Connection refused
   at java.net.PlainSocketImpl.socketConnect(Native Method)
   at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
   at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
   at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
   at java.net.Socket.connect(Socket.java:525)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:140)
   at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
   at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)

Ouch! The inventory manager is not so loosely coupled after all. The webapp has assumed that the service endpoints will not change over time. While the system is impervious to any changes in the actual service implementations, it is vulnerable to changes in service endpoints. The root cause of the problem is the following 3 classes of the web application:

1. InventoryDataClient
2. ItemKeyGeneratorClient
3. PriceFinderClient

These client implementations have hard coded service endpoints within them. For instance here's the constructor of the InventoryDataClient which contains the data service endpoint as a hard coded string.

    public InventoryDataClient() throws InventoryException {
        try {
            stub = new InventoryDataStub("http://localhost:9763/services/InventoryData");
        } catch (AxisFault axisFault) {
            handleException("Error while initializing the data service stub", axisFault);
        }
    }

So this means, every time the endpoint of a service changes, we have to modify the source code of the web application, recompile and redeploy it. Needless to say this is a not an acceptable solution since it requires a lot of manual intervention and can cause lot of unnecessary system downtime. So what choices do we have? How can we make the system impervious to service endpoint changes? Here's a couple of common solutions developers tend to implement when faced with a problem of this nature.

1. Specify service endpoints as init parameters of the web applications
2. Read service endpoints dynamically from a custom configuration file

While these solutions are better than hard coding the URL values, both approaches are not without limitations. The first option requires the webapp to be redeployed every time a service endpoint has been updated (init parameters are loaded only during webapp initialization). So in the long run it's going to affect the system availability a number of times. The second one requires you to write a bunch of additional code and maintain a custom configuration file which is not governed by any J2EE or WS standards. However the biggest issue is both solutions need some work to be done manually which means somebody has to keep track of all the endpoints and update the web.xml file or the custom configuration file accordingly. In addition to that both solutions become difficult to maintain when the number of service endpoints increases over time. If the system communicates with several hundred services provided by various organizations and departments, keeping track of all the endpoint changes is not going to be an easy task. So the bottom line is none of these approaches really scale up to the expectations.
This is where WS-Discovery comes into the picture. WS-Discovery provides a fully automated, fully dynamic approach for managing and discovering service endpoints. It doesn't require any manual intervention at all and as a developer or a system administrator you only need to worry about the URL of the WS-Discovery proxy.
Now let's try to redo the existing inventory management solution by introducing WS-Discovery.

Iteration 2 - Discovery Enabled Inventory Management System

WS-Discovery enabled services and the web application can be found in solution2.zip archive. Let's deploy this new version of the inventory management system and see what are the changes we have made to support WS-Discovery and how adding dynamic discovery support is going to help us. We are not quite read to being ESB into the picture yet. But in this iteration we are going to setup WSO2 Governance Registry which will act as our discovery proxy.
As described in a previous section, WS-Discovery enabled systems identify services based on their scopes and types. So the first thing we have to do is decide on the appropriate scopes for our three web services. A scope is nothing more than a label that developer decides to stick on a service. Later we can use these labels to index and search the services. The listing below shows a possible scope allocation for the web services used in the inventory manager.

• InventoryData - http://ims.traderpauls.com/data/main
• ItemKeyGenerator - http://ims.traderpauls.com/data/keygen
• PriceFinder - http://ims.traderpauls.com/sales/pricing

In reality it is possible to assign more than one scope for a service. But in this scenario we will maintain a one-to-one mapping between services and scopes and keep our discovery logic simple.
In WSO2 Carbon, the scope of a service is defined by adding a parameter named 'wsDiscoveryParams' to the service artifact. This can be done via the management console after a service has been deployed. Alternatively this can be mentioned in the service meta descriptor file (services.xml file) at development time. The service artifacts found in the solution2.zip archive are already updated with the above parameter. Other than that no changes have been made to the service implementations.
Follow these steps carefully to deploy the new version of the inventory manager. This assumes you have the first cut version of the system already deployed on Application Server and Data Services Server.
1. To start with, login to the management console of Application Server and delete the old TraderPaulsIMS Carbon application we deployed earlier. This can be done by navigating to 'Manage > Applications > List' and clicking on the 'Delete' option next to the application name. It will remove the ItemKeyGenerator, PriceFinder and the inventory webapp from the system.
2. Shutdown the Application Server. We will restart this server shortly with some minor configuration changes.
3. Shutdown the Data Services Server. Do not remove the InventoryData service from the server. We can use it later as it is.
4. Install WSO2 Governance Registry (GReg)
5. Open the repository/conf/carbon.xml file of GReg and set the port offset to 10 (We don't want GReg port number to be anywhere near the other servers)

<Ports>
   <Offset>10</Offset>
   ...
</Ports>

6. Start WSO2 GReg by executing the relevant start up script in the bin directory.
7. Open the repository/conf/axis2.xml file of the Data Services Server and add the following parameter. There are already some parameters defined in the axis2.xml. You can add this one somewhere around the same region.

<parameter name="DiscoveryProxy">http://localhost:9773/services/DiscoveryProxy</parameter>

This parameter tells the server that a discovery proxy is available in the above URL to which service metadata can be published by sending discovery notifications (HELLO/BYE messages).
8. Open the repository/conf/axis2.xml file of the Application Server and add the same 'DiscoveryProxy' parameter we added to DSS.
9. Download and extract the solution2.zip archive.
10. Copy the InventoryData_services.xml file to the repository/deployment/server/dataservices directory of DSS.
11. Launch Data Services Server and Application Server.
12. Login to the Application Server and upload the TraderPaulsIMS-2.0.0.car file as a Carbon application (Manage > Applications > Add). This will redeploy the ItemKeyGenerator, PriceFinder and the inventory webapp.
We have finished deploying the new version of the solution. At first glimpse it looks like a long process. But if you carefully study the steps we followed, you will realize that we have done only four changes to the setup we already had.

• Installing WSO2 GReg
• Adding the 'DiscoveryProxy' parameter to the axis2.xml file of AS and DSS
• Adding the wsDiscoveryParams parameter to all the services that need to be discovered
• Deploying a new and improved inventory webapp

While we were busy deploying services and changing configurations, Application Server and Data Services Server was also busy trying to publish service metadata to the discovery proxy. In fact by now all three services should be discovered by GReg. To make sure everything has gone according to the plan, launch a web browser and login to the GReg management console (https://localhost:9453/carbon). Click on the 'Metadata > List > Services' option and you should see a bunch of discovered services.

If you get a page similar to this then rest assured that all the services have been published to the discovery proxy. You see more than three entries here because AS and DSS publish all the services deployed on them. That includes some of the sample services such as Hello service and Version service.
So far everything is good. Now it's time to check up on the new version of the web application we just deployed. Navigate to the home page of the inventory manager and click on a few links to see whether everything is working fine as before. If all is well, shutdown the Data Services Server. Open its repository/conf/carbon.xml file and change the port offset value to something like 5. (Avoid the values 1 and 10 because they are already used by AS and GReg)

<Ports>
   <Offset>5</Offset>
   ...
</Ports>

Now restart the Data Services Server and go back to the web application. Click on a few links. Unlike the previous solution, the new system will deal with the endpoint change gracefully. The web application will automatically discovery and bind with the new endpoint reference of the InventoryData service. The user (you) will not see any errors nor notice any differences in the UI.

Reviewing the Code

So it seems we finally have a truly loosely coupled system at hand. Let's go through the changes we have done to make this possible and try to understand how the new system works.
First of all we added a parameter named DiscoveryProxy to the axis2.xml file of AS and DSS.

<parameter name="DiscoveryProxy">http://localhost:9773/services/DiscoveryProxy</parameter>

This parameter is configured with a URL which points to the discovery proxy service that runs on GReg (this service is available on GReg by default). The parameter tells the servers that a discovery proxy is available to which service metadata can be published. When configured with this parameter, AS and DSS send notifications to the discovery proxy whenever a service is deployed, updated or undeployed. The same parameter can be used in several other WSO2 products like ESB and Mashup Server to trigger discovery notifications.
Then we explicitly defined a scope for each of the services by setting a service level parameter named 'wsDiscoveryParams'.

<parameter name="wsDiscoveryParams">
    <Scopes>http://ims.traderpauls.com/data/keygen</Scopes>
</parameter>

When DSS or AS fires discovery notifications for a service, it looks for this parameter to find the set of scopes to which the service belongs. This piece of information will be included in the HELLO notifications sent to GReg where it is stored as a resource.
The last important change we did was the following utility method we added to the webapp. The InventoryDataClient, ItemKeyGeneratorClient and PriceFinderClient no longer have hard coded endpoints within them. Instead they all invoke this method to find the service endpoints. This method executes a WS-Discovery probe on GReg to retrieve the most up-to-date endpoint details.

        public static String findEndpoint(String scope) throws DiscoveryException {
  // Create a DiscoveryClient instance
  InventoryConfigHolder configHolder = InventoryConfigHolder.getInstance();
  DiscoveryClient client = new DiscoveryClient(configHolder.getClientConfigurationContext(), 
    configHolder.getDiscoveryProxyURL());
  
  // Execute a probe
  URI[] scopes = new URI[] { URI.create(scope) };
  TargetService[] services = client.probe(null, scopes);
  if (services != null && services.length > 0) {
   URI[] endpoints = services[0].getXAddresses();
   // The service may have multiple endpoints
   // We pick the HTTP endpoint
   for (URI endpoint : endpoints) {
    if (InventoryConstants.HTTP_PROTOCOL.equals(endpoint.getScheme())) {
     return endpoint.toString();
    }
   }
  }
  throw new DiscoveryException("Unable to locate a service for the scope: " + scope);
 }

Note that the method accepts a scope value as an argument. This is passed in by each of the client implementations. The scope string is included in the probe request sent to GReg which performs a search based on that. It will return all the services that match the given scope. Since we have a one-to-one mapping between scopes and services, each scope will bring back exactly one service. Each service in the Probe Match may have more than one endpoint. So we simply pick the http endpoint and ignore the rest.
The end result of this exercise is that we no longer have to be concerned about the changes that may occur in data service or business service endpoints. Given that there can be hundreds of services in a complex system, it indeed is a relief. We only need to worry about the changes that may occur in the discovery proxy endpoint. In this example implementation we're reading the discovery proxy URL as an init-parameter of the webapp.

Iteration 3 - Going Public

Time to expose some of our services to the public Internet. But we don't want external users to directly hit our service interfaces. And we definitely don't want to expose all our services and operations to external parties. Therefore let's define the scope of this development iteration as follows.

1. Expose the PriceFinder service and all the read operations of the InventoryData service to the Internet.
2. All interfaces exposed to the Internet must be secured with UsernameToken security.

WSO2 ESB is an ideal tool for exposing your existing services to other parties. We can create some proxy services in the ESB to expose the necessary endpoints and then secure the proxy services with UsernameToken security. However adding an ESB into the picture creates new opportunities in the discovery space. When creating proxy services, we would ideally want to use WS-Discovery to locate the target service endpoints. Also once a proxy service has been created we would want to use the WS-Discovery protocol to discover the proxy service endpoints. And finally it would be great if ESB used the WS-Discovery protocol to contact the back end services when mediating requests. That way we don't have to change the proxy service settings even if the back end service endpoints were changed. So now lets see how these requirements can be implemented with WSO2 ESB.
To start with we are going to install WSO2 ESB and create a proxy service for the PriceFinder service. We will use WS-Discovery to find the endpoint of the PriceFinder.
1. Install WSO2 ESB by extracting the zip file.
2. Open the repository/conf/carbon.xml file of the ESB and set the port offset to 20.

<Ports>
   <Offset>20</Offset>
   ...
</Ports>

3. Open the repository/conf/axis2.xml file and add the DiscoveryProxy parameter so that any services we create in the ESB will be discovered by GReg.

<parameter name="DiscoveryProxy">http://localhost:9773/services/DiscoveryProxy</parameter>

4. Start WSO2 ESB by executing the start up script in the bin directory and sign in to the ESB management console (https://localhost:9463/carbon).
5. Download the TraderPaulsPublicIMS-1.0.0.car file and upload it to the ESB by using the 'Manage > Applications > Add' option. This will upload two WSDL files named PriceFinder.wsdl and InventoryData.wsdl to the embedded registry of the ESB. Files will be uploaded to the /_system/config/tutorial collection. These files contain the interfaces that we intend to expose to the Internet from the ESB. The PriceFinder.wsdl is the WSDL of the PriceFinder service without any modifications. The InventoryData.wsdl is the WSDL of the InventoryData service with all the write operations taken out.
6. Select the 'Configure' tab and click on the 'Configure > WS-Discovery' option in the menu. You will be taken to the WS-Discovery Control Panel.
7. Click on the 'Add Discovery Proxy' option. The 'Configure WS-Discovery Proxy' page will be displayed.

8. Enter any name for the discovery proxy (eg: GRegOnLocalhost) and enter the URL http://localhost:9773/services/DiscoveryProxy. Click 'Save' to apply the settings. You will be taken back to the WS-Discovery Control Panel which will list the discovery proxy you just configured. If GReg is already running, server status will be shown as On-line.

9. Click on the 'View' option under the available Actions for the discovery proxy. A list of all discovered services will be displayed.

10. You can use the search box at the top to execute probes on the discovery proxy and reduce the number of services listed. Try searching for following scopes.

• http://testme.org - Doesn't bring back any results
• http://ims.traderpauls.com - Returns all 3 services used by the inventory manager
• http://ims.traderpauls.com/data - Returns the InventoryData service and the ItemKeyGenerator
• http://ims.traderpauls.com/sales - Returns the PriceFinder service

Note that the discovery proxy maps the scopes to the services in a hierarchical manner. This is why searching for http://ims.traderpauls.com/data returns both the InventoryData and ItemKeyGenerator services. This search behavior is defined in RFC3616. WS-Discovery specifications states that this strategy should be followed by default when mapping scope URIs to target services.
11. Once you have located the PriceFinder service on the list, click on its UUID. You will be taken to a page where you can see more specific data related to the PriceFinder and its endpoints.
12. Expand the 'Create Proxy Services' panel. Enter the name PriceFinder for the proxy and select the HTTP endpoint of the service as the Target Address. Leave the 'Dynamic Address' box unchecked for the moment and click 'Save' to create the proxy service.
13. Select the Main tab and go to the 'Manage > Web Services > List'. You should see the PriceFinder proxy already listed there. Also click on 'Manage > Service Bus > Source View'. You should be able to see the actual configuration of the PriceFinder service as follows.

    <proxy name="PriceFinder" startOnLoad="true" trace="disable">
        <target>
            <endpoint>
                <address uri="http://127.0.0.1:9764/services/PriceFinder"/>
            </endpoint>
            <outSequence>
                <send/>
            </outSequence>
        </target>
    </proxy>

This still has the PriceFinder endpoint hard coded. We will fix that later. For now let's continue by making a few adjustments to the proxy service. We should secure the proxy service and link it with the PriceFinder.wsdl we have uploaded to the registry.
14. Go to 'Manage > Web Services > List' and click on the PriceFinder service. You will be taken to the Service Dashboard.
15. Click on 'Security' under the 'Quality of Service Configuration'.
16. Select 'Yes' for 'Enable Security?' option.
17. Select 'UsernameToken' from the list of choices and click 'Next'.
18. Select the admin role from the list of roles. This means only the users in admin role can invoke the service. Click 'Finish' to apply security.
19. Finally go to 'Manage > Service Bus > Source View'. Add a publishWSDL element and the wsDiscoveryParams parameter to the proxy service. Modify the proxy configuration as follows in the source view and click 'Update'.

    <proxy name="PriceFinder" startOnLoad="true" trace="disable">
        <target>
            <endpoint>
                <address uri="http://127.0.0.1:9764/services/PriceFinder"/>
            </endpoint>
            <outSequence>
                <send/>
            </outSequence>
        </target>
        <policy key="conf:/repository/axis2/service-groups/PriceFinder/services/PriceFinder/policies/UTOverTransport"/>
        <enableSec/>

        <publishWSDL key="conf:tutorial/PriceFinder.wsdl"/>
        <parameter name="wsDiscoveryParams">
            <Scopes>http://public.ims.traderpauls.com/sales/pricing</Scopes>
        </parameter>
    </proxy>

We are done configuring the PriceFinder proxy service with security. In the process we also specified a scope for the service. Head over to the WS-Discovery Control Panel in the 'Configure' tab and search for the scope http://public.ims.traderpauls.com/sales/pricing. The PriceFinder proxy will be listed as the only hit which means the proxy service has been successfully discovered by GReg. Note that the service has only a HTTPS endpoint. When we applied UsernameToken security on the proxy service, the ESB disabled the HTTP endpoint of the service. ESB does not expose UsernameToken secured services over HTTP.
This is another opportunity to experiment with the hierarchical search capability of the discovery proxy. Perform a search for the scope http://ims.traderpauls.com. This will return all the services deployed in DSS and AS. Now perform a search for the scope http://public.ims.traderpauls.com. This will return the PriceFinder proxy we just created in the ESB. Note how the search results vary depending on the search string we provide. By specifying proper scopes for the services we can improve our capacity to manage and look up services in a WS-Discovery enabled environment.
By following a similar process as above we can create a proxy service for the InventoryData service. That is left as an exercise to the reader. You will end up with a proxy configuration similar to the following.

    <proxy name="InventoryData" startOnLoad="true" trace="disable">
        <target>
            <endpoint>
                <address uri="http://127.0.0.1:9768/services/InventoryData"/>
            </endpoint>
            <outSequence>
                <send/>
            </outSequence>
        </target>
        <policy key="conf:/repository/axis2/service-groups/PriceFinder/services/PriceFinder/policies/UTOverTransport"/>
        <enableSec/>
        <publishWSDL key="conf:tutorial/InventoryData.wsdl"/>
        <parameter name="wsDiscoveryParams">
            <Scopes>http://public.ims.traderpauls.com/data/main</Scopes>
        </parameter>
    </proxy>

Getting Rid of Hard Coded Addresses

One of the limitations in the proxy services we created above is that they have hard coded address endpoints. Therefore if the back end service endpoints change, the proxy services will not work. Ideally the proxy services should use WS-Discovery to dynamically look up the service endpoints. This can be achieved by installing an additional plug-in called WS-Discovery Mediation Extensions Kit. Follow these steps to install the plug-in and configure the proxy services with dynamic endpoint look up capability.
1. Sign in to the ESB management console.
2. Select the 'Configure' tab and click on 'Configure > Features'. You will be taken to the Carbon Feature Manager which consists of multiple horizontal tabs.
3. Select the 'Repository Management' horizontal tab and click on 'Add Repository'.

4. Enter any name for the repository (eg: WSO2MainRepo) and provide the URL http://dist.wso2.org/p2/carbon/releases/3.2.0. Then click the Add button to apply the settings.

5. Now select the Available Features tab and click on Find Features (note that the repository you created is already selected). ESB will connect to the plug-in repository and retrieve a list of all available features which will be displayed on the management console.

6. Scroll down to locate the WS-Discovery Mediation Extensions plug-in. Check the box next to it and click Install. This will start the plug-in installation wizard.

7. Simply proceed through the wizard by clicking the Next button. Accept the license agreement when prompted. This will start the plug-in installation. This could take a few minutes (depending on the speed of your network connection).
8. Once the installation is complete, you will be asked to restart the server. Click on the Finish button and simply logout from the management console. Now restart the ESB.
9. Now the required plug-in is installed. Sign in to the ESB management console again. Select 'Manage > Service Bus > Source View'.
10. Notice the mediation registry declaration at the top of the configuration. We should add a couple of new parameters to this configuration before we can setup proxy services with dynamic endpoint look up. The modified registry declaration should look like this.

    <registry provider="org.wso2.carbon.mediation.registry.WSO2Registry">
        <parameter name="cachableDuration">15000</parameter>
        <parameter name="extensions">org.wso2.carbon.discovery.mediation.ext.WSDiscoveryRegistryExtension</parameter>
        <parameter name="discoveryProxy">http://localhost:9773/services/DiscoveryProxy</parameter>
    </registry>

11. Now we are all set to create proxy services with dynamic endpoint look up feature. Select the Configure tab and click on WS-Discovery. Search for the PriceFinder service and click on the service ID in the result list.
12. Click to expand the Create Proxy Services panel. Enter DynamicPriceFinder as the service name and check the Dynamic Address check box. Select the HTTP endpoint as the Target Address and click Save to create the proxy service. What we did differently here was checking the Dynamic Address check box.
13. Now select the Main tab and click on Source View. You will see a proxy configuration similar to the following.

    <proxy name="DynamicPriceFinder" startOnLoad="true" trace="disable">
        <target endpoint="wsdd://urn:uuid:23a79ddc-f524-421e-9770-32931d381c82/http">
            <outSequence>
                <send/>
            </outSequence>
        </target>
    </proxy>

Note that it does not contain any hard coded URLs. Instead it has the target service ID prefixed by wsdd:// as the endpoint. This will be used by the ESB mediation registry to execute a WS-Discovery resolve operation. Also note that the endpoint contains the suffix /http. This is because we selected the HTTP endpoint when creating the proxy service. This tells the ESB to use the HTTP endpoint if the resolve operation returns more than one endpoint.
14. Let's add a WSDL and a discovery scope for this proxy service (I'm not going describe how to secure a service again. But if needed it can be done by following the same exact steps we went through earlier). Modify the configuration as follows and click 'Update' on 'Manage > Service Bus > Source View'.

    <proxy name="DynamicPriceFinder" startOnLoad="true" trace="disable">
        <target endpoint="wsdd://urn:uuid:23a79ddc-f524-421e-9770-32931d381c82/http">
            <outSequence>
                <send/>
            </outSequence>
        </target>
        <publishWSDL key="conf:tutotial/PriceFinder.wsdl"/>
        <parameter name="wsDiscoveryParams">
            <Scopes>http://public.ims.traderpauls.com/sales/dynamic_pricing</Scopes>
        </parameter>
    </proxy>

Now you can try this out using a client tool such as SOAP UI. The WSDL of the proxy service can be found at http://localhost:8280/services/DynamicPriceFinder?wsdl. Send a few messages and make sure it works. Restart the Application Server on a different port and see if the proxy service continues to respond as expected. Please note that whenever the ESB performs a WS-Discovery resolve, it caches the results for a certain period. This is done in order to reduce the number of WS-Discovery look ups performed, thereby improving the response time of the calls. By default the cache duration is set to 15 seconds. This is configured by the following parameter in the mediation registry declaration.

        <parameter name="cachableDuration">15000</parameter>

Therefore it will take about 15 seconds for the endpoint changes to be reflected on the proxy service. In production deployments a higher value can be set for the cachableDuration parameter depending on the frequency of the endpoint changes.

Room for Improvement

We have reached the end of our little inventory manager project. We have all the pieces of the system implemented as web services and all of them are dynamically wired using the WS-Discovery protocol. An architecture such as this is highly flexible and virtually invulnerable to any kind of change in the individual components. However that doesn't mean it's perfect for all situations. There are still certain limitations and plenty of room for improvement.
1. Single point of failure at GReg -
All the components in the system rely on WSO2 GReg to discover the service endpoints at runtime. Therefore a failure in GReg would cause all other components to fail. We can avoid this by implementing a high availability plan for GReg. WSO2 GReg supports clustering. So we can run multiple instances of GReg pointed to the same database (database should also be high available).
In a clustered GReg setup there will be multiple discovery proxy endpoints. So we will have to implement some load balancing or fail over routing on the endpoints. WSO2 ESB can be used to implement such requirements. Individual components of the system will use the ESB as the discovery proxy which will simply forward the messages to an actual discovery proxy endpoint on one of the available GReg instances.
2. Performance hit -
In the final version of the inventory manager, every time the webapp need to invoke a back end service, it has to actually perform two service calls. The first call is against the discovery proxy to find the service endpoint. The second call is against the actual service endpoint. This in turns increases the response time of the system. If GReg takes time to respond to the discovery probes, the webapp user will get a sluggish UI experience.
To avoid this we need to make sure GReg is easily accessible from the Application Server. It would be a good idea to maintain a fast network connection between AS and GReg. Also if GReg is getting a lot of discovery messages, it would be a good idea to run several instances of GReg with some load balancing mechanism. Also at application level we can improve performance by caching discovery probe results for a certain period of time. WSO2 ESB already does this when wiring to back end services using WS-Discovery.

Summary and Conclusion

That brings us to the end of this introductory tutorial on WS-Discovery. We started by looking at why endpoint discovery is important for SOA and enterprise application development. We studied some of the issues present in applications that do not have dynamic endpoint discovery capabilities. After that we went through the basics of WS-Discovery protocol and explored its operational modes, message types and message flows.
Then we switched over to exploring the WS-Discovery features available in the WSO2 Carbon platform. We created a simple inventory management system using WSO2 Application Server and WSO2 Data Services Server. We practically experienced the problems that can manifest within a system which does not have endpoint discovery capabilities. Then we fixed all those issues by introducing WS-Discovery into our solution. We used WSO2 Governance Registry as a discovery proxy and configured all the services and the web application to use the discovery protocol to publish and discover service endpoints.
Finally we looked at some of the discovery capabilities available in WSO2 ESB. We learned how to use the WS-Discovery control panel of the ESB to connect to a remote discovery proxy and browse discovered services and endpoints. We also learned how easy it is to create proxy services using a discovered service endpoint. We also looked at how to install the WS-Discovery mediation extensions kit on the ESB and how to use that plug-in to construct smart proxy services with 100% dynamic endpoint discovery capabilities.
As of now WSO2 Carbon only supports the managed mode of WS-Discovery. But please note that we are currently working on implementing the ad hoc mode of WS-Discovery as well. That will enable Carbon based servers to publish and discover services using a UDP multicast based communication protocol.
Compared to most WS-* standards, WS-Discovery is a very simple protocol. Anybody can learn the protocol end-to-end by reading the protocol specification once. But as we witnessed in our examples, it is a very powerful standard and if applied properly, can solve a plethora of endpoint management problems in the enterprise. What’s really impressive is that this protocol can enable different SOA stacks to work together seamlessly. For an example of this please refer the article titled “Integrating Mule with WSO2 Registry Using WS-Discovery” by Jos Dirksen.
If you are wondering who are the other vendors that support WS-Discovery, please be aware that Microsoft already has included WS-Discovery support in their .NET 4.0platform.
I hope you found the content in this article useful. Please do try out the samples as described to get the full experience of WSO2 Carbon and WS-Discovery. Please feel free to use our discovery libraries in your own applications and send in your valuable feedback to our mailing lists.

Author

Hiranya Jayathilaka, Technical Lead, WSO2 Inc.

Search and Replace in all files within a directory recursively on Linux

I have a directory on my computer (I run linux for my desktop) that has a bunch of directories in it. I needed to change one string in each of the files to be something else.

I did this like 2 months ago but this time I had a really hard time finding the solution. I also remember that last time I tried like 6 different commands and none of them worked.

So, to search recursively through directories, looking in all the files for a particular string, and to replace that string with something else (on linux), this command should work:

find ./ -type f -exec sed -i ‘s/string1/string2/’ {} \;

Where string1 is the search string and string2 is the replace string.

Just for future reference

(also, it looks like my wordpress theme is stylizing those single quotes to look like something else. When I copied and pasted that onto the command line I got this error: sed: -e expression #1, char 1: unknown command: `
I fixed it by replacing those ` (backtick) characters with single quotes)

Note: Be sure to replace the ` (backtick) characters in all these commands with single quotes.

[tags]search and replace, linux search and replace, recursive search and replace, sed, xargs[/tags]

Developing WS-BPEL Processes using WSO2 Carbon Studio

Business Process Execution Language or WS-BPEL lets people orchestrate tasks and activities exposed as Web services. WSO2 Business Process Server can execute process descriptions written in WS-BPEL. Since BPEL is structured and based on XML, it is now far easier for business users to model and execute business processes using open source graphical tools like Eclipse BPEL Designer. This tutorial describes how to Develop a WS-BPEL Processes using WSO2 Carbon Studio. Date: Tue, 7th Jun, 2011 Level: Intermediate Reads: 1657 Comments: 3 | Login or register to post comments Printer-friendly version Send to friend PDF version  Post to twitter

Hasitha Aravinda WSO2

Introduction

The purpose of this tutorial is to demonstrate how to use WSO2 Carbon Studio to model a WS-BPEL Process.

Content

• Scenario : Invoke single Web service using BPEL process.
• Developing WS-BPEL Processes using WSO2 Carbon Studio
  • Pre-Requirements
  • Step 1: Deploying AdderService.aar
  • Step 2: Create a Carbon Application Project
  • Step 3: Creating the BPEL process
  • Step 4: Creating a Binding and Service to the AdderProcessArtifacts.wsdl
  • Step 5: Modify the service description file AdderProcessArtifacts.wsd
  • STEP 6: BPEL process Designing
  • Step 7: Set the Partner Links for Invoke activities
  • Step 8: Assigning Varaibles
  • Step 9: Creating Deploy.xml
  • Step 10: Creating WSO2 BPS server and Deploying the Carbon Application in WSO2 BPS.

Scenario: Invoke single Web service using BPEL process.

In this tutorial I am going to create a BPEL process that returns addition of two integer numbers. For the number addition operation the BPEL process invokes an existing service and gets the result from the web service. This web service takes two integers and returns addition of the two integer values. I’m going to host this web service on local Axis2 server. You can download AdderService.aar from the Attachments below.

Following diagram will give abstract view of our BPEL process.

Developing WS-BPEL Processes using WSO2 Carbon Studio.

Pre-Requirements:

 For deploying the WS-BPEL process explained in this tutorial, it is required to have the following applications installed in your machine.

• Java Runtime Environment 1.5 or above
• Eclipse SDK 3.5 (Eclipse Galileo) with Carbon Studio 1.0.X.
• Apache Axis2 server 1.6.
• WSO2 Business Process Server 2.0.X

 Tested Environment

• Ubuntu 10.10
• WSO2 BPS-2.0.2
• Apache-Axis2 1.6
• Eclipse SDK 3.5 (Eclipse Galileo) with Carbon Studio 1.0.3
   

Let’s start to build our BPEL process.

Step 1: Deploying AdderService.aar
 

1.1  First Deploy the AdderService.aar file in Apache Axis2 server.
1.2  Then run the axis2server. You can find the AdderService.wsdl from http://localhost:8080/axis2/services/AdderService?wsdl. We will need this wsdl during our BPEL process development. So save the AdderService.wsdl.

Step 2: Create a Carbon Application Project

2.1  Open Eclipse IDE. Go to File → New → Other  (see Figure 2.1)


Figure 2.1

2.2  From the dialog window select Carbon Application Project and click Next (see Figure 2.2).


Figure 2.2

2.3  Give a name to the Carbon Application Project. In this example,  I’m giving the Carbon Application Project name asWS_NumberAdderCarbon. Then click Finish. (See Figure 2.3)


Figure 2.3

Note:

Eclipse will prompt you to “root-artifact.xml” automatically. (If not, open it from Project Explorer window) 


Figure 2.4

2.4  Let’s create a new BPEL project. Click on “Create New Artifact” (figure 2.4) and select “BPEL WorkFlow” (Figure 2.5). In the opened dialog (Figure 2.6) click on “New BPEL Project”.

             
Figure 2.5   and    Figure 2.6

2.5  It will show dialog box (Figure 2.7). Give a name to our BPEL project. Here I am giving the project name as BPELNumberAdder.
2.6  Then click Finish to close the wizard and again click Finish to close the New BPEL Workflow dialog. Now you can see the newly created BPEL project appears in Project Explorer window.


Figure 2.7

Step 3: Creating the BPEL process.

Now we have already created BPEL project. Now-onwards we are going to create our BPEL process.
3.1  Let’s create a BPEL process file. For that right-click on BPEL project “BPELNumberAdder” form the “Project Explorer”.

Note:

If you expand the WS_NumberAdderCarbon project folder in the Project Explorer window, you can see another folder calledBPELNumberAdder. This is an instance of our BPELNumberAdder Project. Both are linked in together. In Other words all changes which are done to the BPELNumberAdder project, automatically applied to the BPELNumberAdder folder of the WS_NumberAdderCarbon. Here I am using the BPELNumberAdder project for BPEL process development.

3.2  Then go to New → Other. It will show a Wizard (Figure 3.1). From the wizard, select “New BPEL process File” which is under the BPEL 2.0. Then click Next.


Figure 3.1

3.3  In the next dialog (Figure 3.2), it will ask for a name for BPEL process, a namespace, and a template. Here I am giving the Process name as “AdderProcess” and Namespace as “http://NumberAdder.com”. The process which we are going to create is a synchronous process. So select template as Synchronous BPEL process. Then click Finish.


Figure 3.2

Note:

It will automatically generate AdderProcessArtifacts.wsdl and AdderProcess.bpel. If we look at the folder structure of the BPEL process, we can easily figure out these two files. (Figure 3.3)


Figure 3.3


AdderProcessArtifacts.wsdl is used to expose our BPEL process as a service. If you open the wsdl, you can see that, it's only define the port type, messages etc. In-order to deploy our BPEL process in WSO2-Business Process Server we need to define a Binding and a Service. In next section I will demonstrate how to create a binding and a service to our BPEL process.

Step 4: Creating a Binding and Service to the AdderProcessArtifacts.wsdl

First let’s add a binding to the AdderProcessArtifacts.wsdl.
4.1  Write click on the Editor, select Add Binding. (Figure 4.1)


Figure 4.1

4.2  Select the Binding. From the Properties window you can set a name for the Binding. Change it, if you need. Then select “AdderProcess” for the port Type from the drop-down menu.
4.3  Then click on “Generate Binding Content” button. In the Binding Wizard dialog (Figure 4.2), select “SOAP” for protocal and click Finish.


Figure 4.2

Note:

There are Three SOAP Binding Options. They are Document Literal , rcp literal and rcp encoded. The WSO2 Business Process Server use an Axis2 integration layer. Axis2 does not support rcp encoded SOAP binding. So DO NOT select rcp encoded as a SOAP binding options.

4.4  Then we need to add a service. Again right-click on the editor and select “Add Service”. Then click on service and rename it asNumberAdderService.
4.5  Select the port and using property window rename the port as NumberAdderServicePort. Then select the binding that we have created from the drop-down menu.
After the Completing the step 4, your AdderProcessArtifacts.wsdl will look like this. See figure 4.3


Figure 4.3

Now we have finished the defining the Binging and Service for our BPEL process.

Variables in BPEL process hold the state of the running process instance. In our example, variables are needed to store the sent and received messages to/from partner service. So we need to initialize the variables to invoke external service. Here we use WSDL Message type variables. (there are other type of variables: XML Schema type, XML Schema element, etc). WSDL Message type Variables correspond to web service message types that are defined in WSDL files imported by the process. Let’s look at the how to define the input and output variables for our BPEL process. So we need to Modify the service description file of the BPEL process.

Step 5: Modify the service description file AdderProcessArtifacts.wsd

Here I am going to define our input and output parameters of the porject. As I discussed earlier our BPEL process takes two integers values and returns an integer value that is the addition of input values. BPEL editor creates automatically inputs and output parameters. But we need to change those parameters such a way that they suits for our service.
5.1  Open the AdderProcessArtifacts.wsdl file. First set the input parameters. Click the arrow next to AdderProcessRequest.


Figure 5.1

5.2  It will open another Tab called “Inline Schema of AdderProcessArtifacts.wsdl”. Defaults it will have single variable called input with typestring. Rename it as x and set its type into int.
5.3  Then we need another element. To add another element right-click on AdderProcessRequestType and add an Element. Name it as y and select type as int. (see Figure 5.2)


Figure 5.2

5.4  Now we have done the input request parameters configurations. We can follow the same sequence to define the response parameters. In Response we need one parameter to return the integer value of addition of inputs. Here I name it as result and set its type as int.
(Tip: another way to add the elements is using the Outline view. See figure 5.3. In Outline view, go to the Elements and expands it. Then goto theAdderProcessRequest → local type → sequence. In here you can see the  elements that just created. )


Figure 5.3

Now we have configured the AdderProcessArtifacts.wsdl. The Next step is design part of our BPEL process.

STEP 6: BPEL process Designing.
 

6.1  Open the AdderProcess.bpel file from the Project Explorer window. Here you can see the template for our business process. The BPEL editor automatically generates receiveInput and replyOutput activites. (Figure 6.1) Also it will generate partnerLink and variables used in these two activities.


Figure 6.1

Note:

In our BPEL process we need to invoke an external service (Adder Service). To invoke this service we need to assign the our input variables into external service’s input and again assign the reply from the external service to our BPEL process output. So in our scenario we need two assign activities and one invoke activity.

6.2  Let’s add an assign activity in between receiveInput and replyOutput activities. To add assign activity drag it from the Action section of thePalette.(figure 6.2)


Figure 6.2

(Tip: another way to add activity element to the editor is right-click on element and use add-before option).

This assign responsible for mapping inputs between our Process and external service. Rename it as AssignInputVariables.

(Tip: Renaming can be done by double-clicking on it or using the Properties view).

6.3  Then we need to add an invoke activity. Before add an invoke activity, we need to add a Sequence. Add a Sequence after the Assign activity. Drag Sequence from the Control2 section and drop between Assign and replyOutput.
6.4  In the Sequence section we are going to add our invoke activity. Drag an Invoke activity from Action section and place it in Sequence. Rename the Invoke activity as InvokeAdderService.
6.5  Then we need another Assign activity to mapping variables between external service reply to our process reply. Add another Assign activity between sequence and replyoutput. Rename it as AssignOutputVariables.

After the end of step 6, your design should look like this.


Figure 6.3

Step 7: Set the Partner Links for Invoke activities .

 Before this step we need to import Saved AdderService.wsdl to our BPEL project folder.
7.1  To import that, right-click on BPEL project folder and select Import. Under the General section select “File System” and click Next.
7.2  In next window Click Browse button and select the folder which is AdderService.wsdl file is located. Then select AdderService.wsdl file and click Finish. (Figure 7.1)


Figure 7.1

7.3  Then click on Invoke activity and go to the Properties window. Then click on Details tab.
7.4  In the “Partner Link” drop-down menu select “Create Global Partner Link” and Give a Name For the partnerLink. Here i'm giving Partner-link name as AdderServricePL.
7.5  Then it view a dialog box to Choose Partner-link type dialog. Now I’m going to add Partner-link type. It is coming from the AdderService.wsdl. To add partner-link type in to dialog, click on Add WSDL. Then select the Resources from Import Source section and select the AdderService.wsdl from the project folder. Then Click Ok. (see Figure 7.2)


Figure 7.2

7.6  It will add AdderServicePortType to the list. Select it and click Ok.
7.7  Then it will ask Partner-link type name and give a name for it. Here I'm giving it as AdderServicePLType and Clck Next. (Figure 7.3)

Figure 7.3

7.8  Then it will ask Role name, give it as AdderRole and Select the PortType from the dialog box. Then Click Finish.
Now you have created a Partner-link for our invoke activity. In the Properties view you can see the partner-link that you have created.
7.9  Then select the add operation from the quick pick list which is right side of properties window. It will automatically complete the remaining parts of the properties window. (Figure 7.4)


Figure 7.4

Now you have completed the defining the partner-link for invoke activity. Now let’s look at assigning variables for activities.

Step 8: Assigning Varaibles.

8.1 Click on “AssignInputVaraiabes”. In properties view click on Details tab.
Now we are going to initialize the input variables. Click on New button. Select “payload: adderPorcessRequest” of input from “Form” list and  “parameters: add” of AdderServicePLRequest from “TO” list. (see Figure 8.1)

Note:

Wrong selections can cause error(s) during run time.

Important:

While you are configuring the variables, the editor will ask for initialize the variables. Click yes. It will automatically generate codes for Variable initialization.


Figure 8.1

Now you should have two assigns in your design. They are “Fixed value to variable” and “variable to variable”. Now let’s configurations them one by one.
8.2  First click on “variable to variable”. Then set the values of “From list” to the “x: int” of the “payload of the input”  and set the values of “To list” to the “args0: int” of the “parameters: add of the AdderServicePLRequest”. (See figure 8.2)


Figure 8.2

8.3  Then add another “Variable to Variable” assign for variables “y: int” to “args1:int”. Follow the same procedure which is used in above.
8.4  Then click on “Fixed value to Variable”.
Check whether our input args0 and args1 variables are initialized or not. If not initialize them.

Note:

Here is an example. Not initialized variable cause to “selection Failure” error in run time. 

Not initialized.

<ns:add xmlns:ns="http://ws.apache.org/axis2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> 
</ns:add>

Initialized statement

<ns:add xmlns:ns="http://ws.apache.org/axis2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> 
<ns:args0></ns:args0> 
<ns:args1></ns:args1>
</ns:add>

Now you should have one “Fixed Value to Variable” and two “Variable to Variable” initializations for  “AssignInputVaraiabes” assign activity.
8.5  Follow the same procedure for the “AssignOutputVariables” assign activity. Here you have to assing the “return:int” of the “parameter of AdderServicePLResponse”  to “result : int ” of the “payload of the output.”

Note:

Here we are using only one way to define variables. We can initialize variables in different ways.   If you click on drop-drown menus of From and To, you can see several other options are available like Expressions, Property of a Variable, Partner Link Reference etc. 

Tips: If you use Xpath Expressions for variable initializing, editor support for auto completion by ctr + space.

Now we have finished the BPEL process designing. In order to deploy our BPEL process in WSO2 Business Process Server (BPS), we need to have Deployment Descriptor (deploy.xml). The deploy.xml is used to configure one or several processes to use specific services by WSO2 BPS.

Step 9: Creating Deploy.xml
 

9.1  Right-Click on the BPLEAdderNumber project in Project Explorer winndow. Then Goto New->Other.
9.2  From the Wizard Dialog box, select “Apache ODE  Deployment Descriptor” which is under the BPEL 2.0. (Figure 9.1) Click Next.


Figure 9.1

9.3  In next Window it will ask for the file name and BPEL project related to the deploy.xml. DO not Change anything and Click Finish.

NOTE: 

Note that the file name cannot be changed in Deployment descriptor (deploy.xml). As disused earlier it is used by WSO2 BPS to configure one or several processes to use specific services. Also it is important to know the Structure of the process container (see figure 9.2). As you see in figure it uses a flat file structure (All files are in the same level). We can see the same structure of the process container in Apache ODE.


Figure 9.2

9.4  Open the deploy.xml in the browser.

Note:

In here You can set the options for BPEL process. For example you can specify the process state (whether it activated, deactivated or retired) after the deployment, can set the process executed only in memory, Inbound Interfaces (Service) and Outbound Interfaces (Invokes) etc.  (see figure 9.3)

Figure 9.3

Inbound interfaces means the services provided by the process. In this option you have to  specify the service, port and binding you want to use for each PartnerLink listed.
9.5  Let’s define the Inbound Interfaces. As you see in the diagram it will show you Client partner-link which is related to our BPEL process. Click on corresponding “Associated Port” in Inbound Interfaces. Here associated port is “NumberAdderServicePort” and select it from the drop-drown menu. (Figure 9.4). It fill automatically complete the other fields as well.

Figure 9.4

Let’s set the outbound interfaces (invokes). Outbound interfaces table contains interfaces the process invokes. In this example we use only one invoke activity. So we have to specify the service, port and binding for AdderServicePL Partner-link.
9.6  Click on Associated Port in AdderServicePL row in the Outbound Interface table. Then select “AdderServicehttpSOAP11Endpoint”. It will automatically complete the other fields.
9.7  Save the project.
Now we have completed implementing our BPEL process. Only remaining part is deploy our Carbon application in WSO2 Business Process Server. In next step I will discuss the how to deploy our Carbon application in WSO2 BPS.
 
Step 10: Creating WSO2 BPS server and Deploying the Carbon Application in WSO2 BPS.

10.1  Let’s create a Server first. For that, right-click on the Servers window. (If Server window not appearing in the Eclipse IDE, go to Window → Show View → Servers.). Then select New → Server.
10.2  It will show “New Server” Window. From the List expand the WSO2 folder and Select “WSO2 Carbon 3.1 based server” and (Figure 10.1). If you need, you can specify the Server name here. Here I rename it as WSO2 BPS.

Note:

The BPS 2.0.2 we are using is based on Carbon 3.1.0. That is why we select 3.1 based server.



Figure 10.1

10.3  Then we have to set the CARBON_HOME. Click browse button and select the WSO2 Business Process Server home folder. If you didn’t have the WSO2 BPS you can download it from http://wso2.com/products/business-process-server. After selecting the CARBON_HOME ClickFinish.

Note:

If you want to configure more setting of the Server like Service port, Web Console port etc. click Next and configure them.
After completing this step 10.3 the WSO2 BPS is added to Servers List. (See Figure 10.2)

 
Figure 10.2

10.4  Let’s deploy our carbon application in WSO2 BPS.  Click on WS_NumberAdderCarbon project in Project Explorer and Drag and drop it on Server. As you see in Figure 10.3 our carbon project is added to the WSO2 BPS server.


Figure 10.3

10.5  To run the server right-click on the server and select Start. In Console window, you can see the server Process’s log.
10.6  It will automatically open the WSO2 BPS login page in your default web browser. Login into the server using admin as username and againadmin as password.


Figure 10.4

10.7  After the login to the WSO2 BPS, you will redirect to the WSO2 BPS home page. Click on the “Processes” under the “Business Processes”. It will show the currently deployed processes.


Figure 10.5

Here you can see the AdderProcess is deployed in WSO2 BPS. The Process ID generated as {“Process NameSpace”} ProcessName-VersionNo
10.8  Click on the AdderProcess. It will show the details of the Process. (Figure 10.6)
To try the process click on the “Create Instance”. It will open an another Web browser window. (Figure 10.7) Let''s try our process. Type two numbers for x and y. Then click Process, then it will show the answer in the window.


Figure 10.6


Figure 10.7

Congratulation! We have finished our BPEL process.

Note:

We can use SOAP UI tool to test our process. Using SOAP UI application we can see the request/response soap messages. It will more helpful the error handling purposes. 

Author

 Hasitha Aravinda,  Intern WSO2.

Attachment	Size
AdderService.aar	1.23 KB