Thursday, October 6, 2011

JSON Support for Apache Axis2


This article by Isuru Suriarachchi introduces you to JSON and it's support for Apache Axis2/Java. He uses tests and samples to explain JSON support and outlines future improvements in this area in Axis2.
Date: Thu, 15th Feb, 2007
Level:
Reads: 24305 Comments: 0 | Login or register to post comments
Isuru Eranga Suriarachchi
Intern Student
WSO2 Inc.

What is JSON?

JSON (Java Script Object Notation) is another data exchangeable format like XML, but it is more lightweight and easily readable. It is based on a subset of Javascript language. Therefore, Javascript can understand JSON, and it can make Javascript objects by using JSON strings. JSON is based on key-value pairs and it uses colons to separate keys and values. JSON doesn't use end tags, and it uses braces (curly brackets) to enclose JSON Objects.
e.g. <root><test>json object</test></root> == {“root”:{“test”:”json object”}}
When it comes to converting XML to JSON and vice versa, there are two major conventions, one named "Badgerfish" and the other, “Mapped”. The main difference between these two conventions exists in the way they map XML namespaces into JSON.
<xsl:root xmlns:xsl="http://foo.com"><data>my json string</data></xsl:root>
This XML string can be converted into JSON as follows.
Using “Badgerfish”
{"xsl:root":{"@xmlns":{"xsl":"http://foo.com"},"data":{"$":"my json string"}}}
Using “Mapped”
If we use the namespace mapping as http://foo.com -> foo
{"foo.root":{"data":"my json string"}}
JSON support implementation is a new feature in Apache Axis2/Java. It will become a crucial improvement in the future with applications like Javascript Web services.

Why JSON Support for Axis2?

Apache Axis2 is a Web services stack that delivers incoming messages into target applications. In most cases, these messages are SOAP messages. In addition, it is also possible to send REST messages through Axis2. Both types of messages use XML as their data exchangeable format. So if we can use XML as a format, why not use JSON as another format?
There are many advantages of implementing JSON support in Axis2. Mainly, it helps the Javascript users (services and clients written in Javascript) to deal with Axis2. When the service or the client is in Javascript, it can use the JSON string and directly build Javascript objects to retrieve information, without having to build the object model (OMElement in Axis2). Also, Javascript services can return the response through Axis2, just as a JSON string can be shipped in a JSONDataSource. This architecture is explained in more detail in the next section of this article.
Other than for that, there are some extra advantages of using JSON in comparison to XML. Although the conversation “XML or JSON?” is still a hot topic [3], many people accept the fact that JSON can be passed and built easily by machines than in the case of XML.

Architecture of JSON Support Implementation in Axis2

As you know, AXIOM (AXIS Object Model) is the XML representation used in Axis2. Messages are written to the transport by serializing the OMElement using a StAX writer on the sending side. On the receiving side, the OMElement is built by parsing the input stream using a StAX parser.
According to this architecture, a JSON StAX parser and writer implementation should be used to read and write JSON messages. The JSON StAX writer should be able to write JSON to the output stream when the StAX events are fired, and the JSON StAX parser should be able to make a StAX reader by reading the input stream.
Jettison is the JSON StAX parser and writer implementation used in implementing JSON support for Axis2. It consists of two different parsers and writers for the “Badgerfish” and “Mapped” conventions. Using Jettison, both formats can be supported in Axis2, and the user can use the one which is more appropriate for the application.
As you might already know, there is a class in AXIOM extending the OMElement called OMSourcedElementImpl, which keeps an OMDataSource inside. This data source contains input data (e.g. an input stream) without consuming it until the application needs it. This data can be used in a way most appropriate to the application. For example, if the application can utilize the raw data inside this data source, it can access it without building the object model.
As I explained earlier, Javascript can understand JSON without building the OMElement. On the other hand, Java services build the OMElement in order to retrieve the data inside. So the advantage of using OMSourcedElementImpl instead of just using OMElementImpl is that OMSourcedElementImpl can directly provide the JSON string in its data source without having to build the object model when the service is a Javascript. When it comes to Java services, it can provide the necessary data by building the object model (just as a normal OMElementImpl).
According to the concept of making all the builders and senders pluggable in Axis2, OMBuilder has become the common interface for all the builders on the receiving side, and MessageFormatter has become the common interface for all the senders on the sending side. Here, the senders and builders are selected without the awareness of the Axis2 kernel. In this architecture, a JSONOMBuilder and a JSONMessageFormatter should be there in order to send and receive JSON messages. All relevant classes are in the “json” module in Axis2. Classes with JSONBadgerfishxxx format are used for the “Badgerfish” convention, and classes with JSONxxx format are used for the “Mapped” convention.
Note: “Mapped” formatted JSON with namespaces are not supported in Axis2, because it is not possible to know the namespace mappings used on one side of the transport to the other side (client or service). The reason to implement the “Mapped” support in Axis2 is because it is easy to be used in Javascript than “Badgerfish”. As there are no namspaces used in Javascript, the “Mapped” convention can be used in applications involving Javascript without any trouble.
Let's look at the receiving side first. JSONOMBuilder (which implements OMBuilder interface) is the builder which creates the OMSourcedElementImpl and returns it when the getDocumentElement() method is called. When creating this new OMSourcedElementImpl object, the correct local name and the namespace of the top element of the object model should be provided as parameters. Currently, the local name is provided by reading the input stream until the local name of the top element ends and an empty namespace is provided.
        public OMElement getDocumentElement() {
                OMFactory factory = OMAbstractFactory.getOMFactory();
                OMNamespace ns = new OMNamespaceImpl("", "");
                if (localName == null) {
                    localName = getLocalName();
                }
                JSONDataSource jsonDataSource = getDataSource();
                return new OMSourcedElementImpl(localName, ns, factory, jsonDataSource);
        }
JSONDataSource implements the OMDataSource interface. It consists of a getReader() method that gives the XMLStreamReader and three serialize methods. This getReader() method returns the JSON StAX reader when it is called in the OMSourcedElementImpl while building the object model.
e.g. getReader() method in JSONBadgerfishDataSource
        public javax.xml.stream.XMLStreamReader getReader() throws javax.xml.stream.XMLStreamException {
                BadgerFishXMLInputFactory inputFactory = new BadgerFishXMLInputFactory();
                return inputFactory.createXMLStreamReader(new JSONTokener("{" + localName + ":" + this.getJSONString()));
        }
Out of the three serialize methods in JSONDataSource, one writes XML by using the StAX reader provided by the getReader() method. The other two write JSON into the output stream directly by converting the input stream into a string. As explained above, this JSONDataSource should provide a method to get the JSON string directly in Javascript services if we really want to make use of it. The getCompleteJOSNString() method is provided exactly for this purpose. Therefore, the JSON string can be obtained in the following manner in Javascript applications.
        OMSourcedElementImpl omSourcedElement = (OMSourcedElementImpl)
                msgContext.getEnvelope().getBody().getFirstElement();
        JSONDataSource source = (JSONDataSource) omSourcedElement.getDataSource();
        String jsonString = source.getCompleteJOSNString();
So far, our main focus was on the receiving side of the JSON message. Now let's talk about the JSONMessageFormatter on the sending side. OMElement is serialized within the getBytes() and writeTo() methods using the JSON writer returned from the getJSONWriter() method.
e.g. getJSONWriter() method in JSONBadgerfishMessageFormatter
        protected XMLStreamWriter getJSONWriter(OutputStream outStream) {
                return new BadgerFishXMLStreamWriter(new OutputStreamWriter(outStream));
        }
In this serialization, there is something important to consider. If this OMElement is an OMSourcedElementImpl and it contains a JSONDataSource with the correct convention, we can directly access the JSON string and write to the output stream. This avoids the overhead of building the OM and serializing it to the output stream. Therefore, this check is done in both the above methods before the serialization of the OMElement.
        if (element instanceof OMSourcedElementImpl && getStringToWrite(((OMSourcedElementImpl)
                                                                        element).getDataSource()) != null) {
                    String jsonToWrite = getStringToWrite(((OMSourcedElementImpl) element).getDataSource());
                    return jsonToWrite.getBytes();
                } else {
                //serialize the OMElement
                }
        }

        protected String getStringToWrite(OMDataSource dataSource) {
                //checking whether the convention is correct
                if (dataSource instanceof JSONBadgerfishDataSource) {
                    return ((JSONDataSource) dataSource).getCompleteJOSNString();
                } else {
                    return null;
                }
        }
There are some applications in which we have to send requests using the HTTP GET method, expecting a JSON response (e.g. Yahoo search APIGoogle AJAX API ). In this case, we have to send the GET request using the JSONMessageFormatter. This facility is provided through thegetTargetAddress() method. Here, if the HTTP method is GET, all the required parameters are attached to the target URL and a new URL object is returned.
These Builders and MessageFormatters are selected using the MESSAGE_TYPE property in the MessageContext. The Builder and the MessageFormatter should be mapped to the relevant MESSAGE_TYPE in the axis2.xml file.

How to use JSON in Axis2

Still, JSON doesn't have a standard and unique content type. “application/json” (this is the content type which is approved in the JSON RFC ), “text/javascript” and “text/json” are some of the commonly used content types of JSON. Due to this problem, in Axis2, the user has been given the freedom of selecting the content type.

Step 1

Map the appropriate MessageFormatter and OMBuilder with the content type you are using in the axis2.xml file.
e.g.1: If you are using the “Mapped” convention with the content type “application/json”
        <messageFormatters>        
                <messageFormatter contentType="application/json"
                                 class="org.apache.axis2.json.JSONMessageFormatter"/>
                <!-- more message formatters -->
        </messageFormatters>   
    
        <messageBuilders>
                <messageBuilder contentType="application/json"
                                 class="org.apache.axis2.json.JSONOMBuilder"/>
                <!-- more message builders -->
        </messageBuilders>
e.g.2: If you are using the “Badgerfish” convention with the content type “text/javascript”
        <messageFormatters>        
                <messageFormatter contentType="text/javascript"
                                 class="org.apache.axis2.json.JSONBadgerfishMessageFormatter"/>
                <!-- more message formatters -->
        </messageFormatters> 

        <messageBuilders>
                <messageBuilder contentType="text/javascript"
                                 class="org.apache.axis2.json.JSONBadgerfishOMBuilder"/>
                <!-- more message builders -->
        </messageBuilders>

Step 2

On the client side, make the ConfigurationContext by reading the axis2.xml in which the correct mappings are given.
e.g.
        File configFile = new File("test-resources/axis2.xml");
        configurationContext = ConfigurationContextFactory
                        .createConfigurationContextFromFileSystem(null, configFile.getAbsolutePath());
        ..........        
        ServiceClient sender = new ServiceClient(configurationContext, null);

Step 3

Set the MESSAGE_TYPE option with exactly the same content type you used in the axis2.xml.
e.g. If you use the content type “application/json”,
        Options options = new Options();        
        options.setProperty(Constants.Configuration.MESSAGE_TYPE, “application/json”);
        //more options
        //...................        

        ServiceClient sender = new ServiceClient(configurationContext, null);        
        sender.setOptions(options);
If you are sending a request to a remote service, you have to know the exact JSON content type that is used by that service, and you have to use that content type in your client as well.

Tests and Samples for JSON

Integration Test

The JSON integration test is available under “test” in the “json” module of Axis2. It uses the SimpleHTTPServer to deploy the service. A simple echo service is used to return the incoming OMSourcedElementImpl object, which contains the JSONDataSource. There are two test cases for two different conventions in JSONIntegrationTest.

Yahoo-JSON Sample

This sample is available in the “samples” module of Axis2. It is a client which calls the Yahoo search API using the GET method, with the parameter “output=json”. The Yahoo search service sends the response as a “Mapped” formatted JSON string with the content type “text/javascript”. This content type is mapped with the JSONOMBuilder in the axis2.xml. All the results are shown in a GUI.
These two applications are good examples of using JSON support for Axis2. You can understand the architecture of JSON support implementation in Axis2 by looking at these samples.

Conclusion and Future Improvements

Currently there are two different parsers and two different builders used in Axis2 for “Badgerfish” and “Mapped” JSON conventions. Therefore, the user has to map the correct parser and builder with the content type. This is a bit inconvenient for the user. This can be avoided by implementing a parser and a builder which can parse and build both conventions. This should be a superset of both “Badgerfish” and “Mapped” which can understand both.
If there are better conventions of XML to JSON mapping in the future, they can be plugged into Axis2 by implementing StAX parsers and builders for them. You just have to write a MessageFormatter and an OMBuilder for a new convention.

Resources

[1] JSON Site
[2] JSON RFC
[3] http://www.json.org/xml.htmlhttp://www.tbray.org/ongoing/When/200x/2006/12/21/JSON http://ajaxian.com/archives/json-vs-xml-the-debate
[4] The JSON group on Yahoo

Author

Isuru Eranga Suriarachchi, Undergraduate Intern at WSO2 Inc. isuru at wso2 dot com

No comments:

Post a Comment