Ushani Balasooriya's Blog: ESB

Showing posts with label ESB. Show all posts

Tuesday, August 29, 2017

Salesforce Integration with WSO2 Enterprise Integrator

In enterprise Salesforce integration scenarios, most often than not there is a backend that expects Salesforce user account details in a JSON object format. We will demonstrate how easy it is to query Salesforce objects and construct a JSON object as expected by the backend, using WSO2 Enterprise Integrator and the Salesforce connector.
Have a look at this article to know how this can be done.

http://wso2.com/library/articles/2017/08/salesforce-integration-with-wso2-enterprise-integrator/

Thursday, August 3, 2017

WHY WSO2 ESB?

I have been writing lot of posts about WSO2 ESB. But have you ever thought why we should use WSO2 ESB over other competitors? Have a look at Samisa's article.

Below points are taken from his article.

WSO2 advantages over competitors

Ability to easily integrate any component framework. Support of Java based extensions and multiple scripting options. There is no need to have WSO2 specific code to integrate anything with WSO2 ESB
Numerous built-in message mediators, solution templates and connectors to third-party cloud systems to help cut down redundant engineering efforts and enable significant component reuse
Freedom for architects and developers to pick and choose message formats, transports, and style of services they want to expose using the ESB
Component oriented architecture and cloud and container support enables you to deploy the ESB using a topology of your choice based on your needs in a secure, scalable and adaptive manner
The ready-made scripts and tools help with rapid deployments, ensuring the ability to go to market quickly with your solution using the ESB
Continuous innovation that helps build future proof solutions on top of the ESB
Rigorous and frequent product update cycles and state-of-the-art tooling support for managing ESB deployments with DevOps practices. Using Docker descriptors and Puppet scripts
Proactive testing and tuning of performance and innovation around performance enhancements

Read http://wso2.com/library/articles/2017/07/what-is-wso2-esb/ for more information.

Thursday, February 16, 2017

How to include batch of test data in to Salesforce Dev accounts?

When you work with salesforce, you will need to have test data in salesforce dev account.

In WSO2 if you use salesforce connector, sometimes you will need to deal with queryMore function.
For more information, please check this link.

This is a sample on how to include test data in to Salesforce.

Salesforce it self provide an awesome tool called Data loader. You can go in to this document from this link.

Im going to use this in an open source /linux environment.

Pre Req : Need JDK 1.8

Step 1 : Install data loader.

1. Check out the code from git. (https://github.com/forcedotcom/dataloader)

git clone https://github.com/forcedotcom/dataloader.git

2. Build it

mvn clean package -DskipTests

3. To run the data loader

java -jar target/dataloader-39.0-uber.jar

Step 2 : Login to Data loader

Provide your username (email address), password along with your security token and login url E.g., (https://login.salesforce.com/services/Soap/u/39.0) I have explained how to find your api login url in one of my previous blog post.

Step 3 : Create your test data.

Click on "Export" and Next and select the salesforce object (In Here I have selected Account) where you need to have test data.

Then select the fields from the check boxes and click on Finish.

Existing data will be exported in to a csv file.

Open the extract CSV in an excel sheet and create any number of test data for just by dragging the last cell. It will increment the data in each cell. Note : You should delete the existing data in the Account from CSV before you upload. So newly incremented data will be there.

Step 3 : Import test data in to Data Loader

Next step is just just click on the "Import" -> Select the salesforce object (in here it is Account) -> Click Next -> Click on Create or Edit a Map -> Map the attributes with the coulmns in CSV as below.

Click Next -> Finish. Select a file location to save error files.

Then it will insert the bulk data and you will receive it once it is finished and success. You can also view errors if exists.

Now if you query salesforce from developer console, you will be able to see your data.

That's it! :) Happy coding!

Monday, April 11, 2016

How to hide credentials used in mediation configuration using Secure Vault in WSO2 ESB

Eventhough we use secure vault to encrypt password, it is not possible to use secure vault directly in the mediation configuration. As an example imagine you need to hide a password given in a proxy.

All you have to do is using Secure Vault Password Management screen in WSO2 ESB.

1. Run sh ciphertool.sh -Dconfigure and enable secure vault
2. Start the WSO2 ESB with
3. Go to Manage -> Secure Vault Tool and then click Manage Passwords
4. You will see the below screen.

5. Click on Add New Password to encrypt and store and specify values.

Vault Key - The alias for the password. E.g., sfpassword
Password - The actual password.
Re-enter password - Re enter the password.

6. Once you save you will be able to save the newly created vault key.

7. Now you can use it in your proxy or any other mediation configuration as below.

E.g.,

   <inSequence>  
      <property name="password" expression="wso2:vault-lookup('sfpassword')"/>  
      <log level="custom">  
       <property name="InMsg" value="Invoking the proxy"/>  
       <property name="pickPassword" expression="get-property('password')"/>  
      </log>  
      <property name="OUT_ONLY" value="true"/>  
      <send/>  
    </inSequence>

8. If you want to hide the salesforce password,

Salesforce Configuration :

  <salesforce.getUserInfo configKey="MySFConfig"/>   
    <salesforce.query>   
    <batchSize>1</batchSize>   
    <queryString>{$ctx:CompleteLeadQuery}</queryString>   
  </salesforce.query>

MySFConfig with salesforce connection details :

 <salesforce.init>  
   <username>xxxx@abc.com</username>  
   <password>{wso2:vault-lookup('sfpassword')}</password>  
   <loginUrl>https://login.salesforce.com/services/Soap/u/27.0</loginUrl>  
   <blocking>false</blocking>  
 </salesforce.init>

Tuesday, February 2, 2016

Simple WSO2 ESB API which Queries salesforce and build a json array using payloadFactory Mediator

1. Download WSO2 ESB and Salesforce connector.
2. Add the salesforce connector and enable it.

For more information please follow : https://docs.wso2.com/display/ESBCONNECTORS/Working+with+Salesforce+Connector+Operations

Then the below API can be used to query User object using profile id and then build a json object using payload factory mediator.

MySFConfig should have the required login information.
00e90000001aVwiAAE is the profile id of a user.

 <api xmlns="http://ws.apache.org/ns/synapse" name="leads1" context="/leads1">  
   <resource methods="GET">  
    <inSequence>  
      <property name="LeadQuery" value="Select u.Username, u.ProfileId, u.Name, u.LastName, u.Email From User u where ProfileId='" scope="default" type="STRING"/>  
      <property name="Apostrophe" value="'" scope="default" type="STRING"/>  
      <property name="ProfileId" value="00e90000001aVwiAAE" scope="default" type="STRING"/>  
      <property name="CompleteLeadQuery" expression="fn:concat($ctx:LeadQuery, $ctx:ProfileId, $ctx:Apostrophe)" scope="default" type="STRING"/>  
      <salesforce.getUserInfo configKey="MySFConfig"/>  
      <salesforce.query>  
       <batchSize>1</batchSize>  
       <queryString>{$ctx:CompleteLeadQuery}</queryString>  
      </salesforce.query>  
      <property xmlns:ns="urn:partner.soap.sforce.com" xmlns:sf="urn:sobject.partner.soap.sforce.com" name="Name" expression="//ns:queryResponse/ns:result/ns:records/sf:Name/text()" scope="default" type="STRING"/>  
      <property xmlns:ns="urn:partner.soap.sforce.com" xmlns:sf="urn:sobject.partner.soap.sforce.com" name="Username" expression="//ns:queryResponse/ns:result/ns:records/sf:Username/text()" scope="default" type="STRING"/>  
      <property xmlns:ns="urn:partner.soap.sforce.com" xmlns:sf="urn:sobject.partner.soap.sforce.com" name="LastName" expression="//ns:queryResponse/ns:result/ns:records/sf:LastName/text()" scope="default" type="STRING"/>  
      <property xmlns:ns="urn:partner.soap.sforce.com" xmlns:sf="urn:sobject.partner.soap.sforce.com" name="Email" expression="//ns:queryResponse/ns:result/ns:records/sf:Email/text()" scope="default" type="STRING"/>  
      <log level="full" separator=","/>  
      <payloadFactory media-type="json">  
       <format>{ "ProfileId": { "source": "SALESFORCE", "id": "$1" },  "Name": "$2", "Username": "$3", "LastName":  "$4", "Email": "$5"  }</format>  
       <args>  
         <arg evaluator="xml" expression="$ctx:ProfileId"/>  
         <arg evaluator="xml" expression="$ctx:Name"/>  
         <arg evaluator="xml" expression="$ctx:Username"/>  
         <arg evaluator="xml" expression="$ctx:LastName"/>  
         <arg evaluator="xml" expression="$ctx:Email"/>  
       </args>  
      </payloadFactory>  
      <property name="messageType" value="application/json" scope="axis2" type="STRING"/>  
      <respond/>  
    </inSequence>  
   </resource>  
 </api>

Monday, April 27, 2015

How to Enable and test Custom SSL Profiles in WSO2 ESB used for SSL communicating

For this I have used WSO2 ESB 4.8.1 and WSO2 Application Server 5.2.1.
WSO2 ESB uses the truststore for SSL communicating and keystore-truststore pair for Mutual SSL communicating.

In here I have used a trust store for SSL communicating.

Configure App Server as backend

Configure backend :

1. Use app server as backend

2. Create a new keystore in App server in <Appserver_Home>/repository/resources/security

keytool -genkey -alias appserver -keyalg RSA -keysize 1024 -keypass password -keystore appserver.jks -storepass password

3. Export in to a pem file by following command

keytool -export -alias appserver -keystore appserver.jks -storepass password -file appserver.pem

4. Edit the carbon.xml in appserver as below :

 <KeyStore>  
    <!-- Keystore file location-->  
      <Location>${carbon.home}/repository/resources/security/appserver.jks</Location>  
 <!-- Keystore type (JKS/PKCS12 etc.)-->  
     <Type>JKS</Type>  
 <!-- Keystore password-->  
     <Password>password</Password>  
 <!-- Private Key alias-->  
     <KeyAlias>appserver</KeyAlias>  
 <!-- Private Key password-->  
     <KeyPassword>password</KeyPassword>  
 </KeyStore>

Configure ESB :

1. Created a new keystore.

keytool -genkey -alias esb -keyalg RSA -keysize 1024 -keypass password -keystore esb.jks -storepass password

2. Copy and paste appserver.pem in to the <ESB_HOME>repository/resources/security folder Import appserver.pem in to esb.jks by following command.

keytool -import -alias appservernewesb -file appserver.pem -keystore esb.jks -storepass password

3. Configure esb for custom profile in axis2.xml as below.

 <parameter name="customSSLProfiles">  
  <profile>  
    <servers>10.100.0.31:9443</servers>  
    <TrustStore>  
     <Location>repository/resources/security/esb.jks</Location>  
     <Type>JKS</Type>  
     <Password>password</Password>   
    </TrustStore>  
  </profile>  
 </parameter>

Invoke and Test :

1. Restart Appserver (offset=0) and ESB (offset = 10) by the command :

"sh wso2server.sh" or "sh wso2server.sh -Djavax.net.debug=ssl:handshake " to view the detailed logs.

Following logs should be printed during restart.

[2015-04-27 18:33:19,397] INFO - ClientConnFactoryBuilder HTTPS Loading Identity Keystore from : repository/resources/security/wso2carbon.jks
[2015-04-27 18:33:19,400] INFO - ClientConnFactoryBuilder HTTPS Loading Trust Keystore from : repository/resources/security/client-truststore.jks
[2015-04-27 18:33:19,408] INFO - ClientConnFactoryBuilder HTTPS Loading custom SSL profiles for the HTTPS sender
[2015-04-27 18:33:19,408] INFO - ClientConnFactoryBuilder HTTPS Loading Trust Keystore from : repository/resources/security/esb.jks
[2015-04-27 18:33:19,409] INFO - ClientConnFactoryBuilder HTTPS Custom SSL profiles initialized for 1 servers

2. Create the below proxy in ESB.

 <?xml version="1.0" encoding="UTF-8"?>  
 <proxy xmlns="http://ws.apache.org/ns/synapse"  
     name="SecureHello"  
     transports="https,http"  
     statistics="disable"  
     trace="disable"  
     startOnLoad="true">  
   <target>  
    <outSequence>  
      <send/>  
    </outSequence>  
    <endpoint>  
      <address uri="https://localhost:9443/services/HelloService/"/>  
    </endpoint>  
   </target>  
   <publishWSDL uri="http://localhost:9763/services/HelloService?wsdl"/>  
   <description/>  
 </proxy>

3. Invoke the Proxy.

 <body>  
   <p:greet xmlns:p="http://www.wso2.org/types">  
    <!--0 to 1 occurrence-->  
    <name>ushani</name>  
   </p:greet>  
 </body>

Following response will be received.

 <ns:greetResponse xmlns:ns="http://www.wso2.org/types">  
   <return>Hello World, ushani !!!</return>  
 </ns:greetResponse>

Tuesday, March 17, 2015

How to monitor the created tcp connections for a particular proxy service in WSO2 ESB

As an example if we want to monitor the number of tcp connections created during a proxy invocation, the following steps can be performed.

1. Assume you need to monitor the number of tcp connections created for the following proxy service :

 <?xml version="1.0" encoding="UTF-8"?>  
 <proxy xmlns="http://ws.apache.org/ns/synapse"  
     name="Proxy1"  
     transports="https,http"  
     statistics="disable"  
     trace="disable"  
     startOnLoad="true">  
   <target>  
    <inSequence>  
      <property name="NO_KEEPALIVE" value="true" scope="axis2"/>  
      <clone>  
       <target>  
         <sequence>  
          <send>  
            <endpoint>  
             <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>  
            </endpoint>  
          </send>  
         </sequence>  
       </target>  
        <target>  
         <sequence>  
          <send>  
            <endpoint>  
             <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>  
            </endpoint>  
          </send>  
         </sequence>  
       </target>  
       <target>  
         <sequence>  
          <send>  
            <endpoint>  
             <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>  
            </endpoint>  
          </send>  
         </sequence>  
       </target>  
      </clone>  
    </inSequence>  
    <outSequence>  
      <aggregate>  
       <completeCondition>  
         <messageCount/>  
       </completeCondition>  
       <onComplete xmlns:m0="http://services.samples" expression="//m0:getQuoteResponse">  
         <send/>  
       </onComplete>  
      </aggregate>  
    </outSequence>  
   </target>  
   <description/>  
 </proxy>

You will see there are 5 clones. Therefore it should create only 5 tcp connections.

2. We have used SimpleStockQuoteService service as the backend.

3. Since you need to monitor the connections created, we should delay the response coming from the backend. Therefore we need to change the code slightly in the SimpleStockQuoteService.

We have include a Thread.sleep() for 10 seconds until we monitor the number of connections.

Therefore go to <ESB_HOME>/samples/axis2Server/src/SimpleStockQuoteService/src/samples/services/SimpleStockQuoteService.java

and add the Thread.sleep(10000); as below to hold the response for sometime.

 public GetQuoteResponse getQuote(GetQuote request) throws Exception {  
 Thread.sleep(10000);  
 if ("ERR".equals(request.getSymbol())) {  
 throw new Exception("Invalid stock symbol : ERR");  
 }  
 System.out.println(new Date() + " " + this.getClass().getName() +  
 " :: Generating quote for : " + request.getSymbol());  
 return new GetQuoteResponse(request.getSymbol());  
 }

4. Build the SimpleStockQuoteService once again by “ant” in here, <ESB_HOME>/samples/axis2Server/src/SimpleStockQuoteService

5. Now start the axis2server

6. Now you have to open a terminal and provide the following netstat command to get the process id.

sudo netstat --tcp --listening --programs

You will see the relevant SimpleStockQuoteService which is up in port 9000 like below.

tcp6 0 0 [::]:9000 [::]:* LISTEN 20664/java

So your process ID will be 20664.

7. Then open a terminal and provide the below command to view the tcp connections for the particular process id.

watch -n1 -d "netstat -n -tap | grep 20664"

8. Now open your soapui and send the following request to Proxy1

 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://services.samples" xmlns:xsd="http://services.samples/xsd">  
   <soapenv:Header/>  
   <soapenv:Body>  
    <ser:getQuote>  
      <!--Optional:-->  
      <ser:request>  
       <!--Optional:-->  
       <xsd:symbol>IBM</xsd:symbol>  
      </ser:request>  
    </ser:getQuote>  
   </soapenv:Body>  
 </soapenv:Envelope>

9. View the tcp connections created in the terminal which you have been monitoring as soon as you send the request. You should be able to view only 3 connections since we have configured like that in proxy using clone mediator.

tcp6 0 0 127.0.0.1:9000 127.0.0.1:44218 ESTABLISHED 20664/java

tcp6 0 0 127.0.0.1:9000 127.0.0.1:44219 ESTABLISHED 20664/java

tcp6 0 0 127.0.0.1:9000 127.0.0.1:44217 ESTABLISHED 20664/java

Tuesday, February 17, 2015

A sample on a WSO2 ESB proxy with a DBLookup mediator and a dedicated faultSequence to excute during an error

Databases :

DB customer should be created with the required tables.

DB : customerDetails
table : customers

Proxy :

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="TestProxyUsh"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence onError="testError">
         <dblookup>
            <connection>
               <pool>
                  <password>root</password>
                  <user>root</user>
                  <url>jdbc:mysql://host:3306/customerDetails</url>
                  <driver>com.mysql.jdbc.Driver</driver>
               </pool>
            </connection>
            <statement>
               <sql>SELECT * FROM customers WHERE name =?</sql>
               <parameter value="WSO2" type="INTEGER"/>
               <result name="company_id" column="id"/>
            </statement>
         </dblookup>
      </inSequence>
      <faultSequence>
         <log level="custom">
            <property name="text" value="An unexpected error occurred for the service"/>
         </log>
      </faultSequence>
   </target>
   <description/>
</proxy>


onError targeting sequence :

<sequence xmlns="http://ws.apache.org/ns/synapse" name="testError">
   <log>
      <property name="error" value="Error Occurred"/>
   </log>
</sequence>

Invoke :

curl -v http://host:8280/services/TestProxyUsh

Monday, February 16, 2015

A possible reason and solution for getting 400 Bad Request even when the synapse configuration in WSO2 ESB is correct

Sometimes you might spend lot of time in analyzing the reason for getting 400 bad request when you invoke an up and running backend service through WSO2 ESB.

When you configure and save the synapse configuration of your proxy, you will not see any issue. But sometimes there can a possible reason for this issue.

One good example is explained below :

Look at the below synapse configuration of WSO2 ESB proxy service.

<proxy name="statuscodeProxy"
          transports="https http"
          startOnLoad="true"
          trace="disable">
      <description/>
      <target>
         <inSequence>
            <log level="full"/>
            <send>
               <endpoint>
                  <address uri="http://10.100.0.31:8090"/>
               </endpoint>
            </send>
         </inSequence>
         <outSequence>
            <log level="full"/>
            <send/>
         </outSequence>
      </target>
   </proxy>

By the look of it, this is alright and you are good to configure it in WSO2 ESB. But the endpoint defined as : http://10.100.0.31:8090 is not exactly the endpoint the backend service expects.

Sometimes there can be missing parameters etc. But if you exactly know your backend service does not expect any parameters and still it throws a 400 bad reques, can you imagine the reason for this?

After spending hours, I accidently found a possible reason :

it was due to the backend expects a / at the end of the endpoint. So it expects :

http://10.100.0.31:8090/ except http://10.100.0.31:8090

If you configure a tcpmon in between ESB and the backend, you will see the difference as below :

http://10.100.0.31:8090 : GET HTTP/1.1 - 400 bad Request

http://10.100.0.31:8090/ : GET / HTTP/1.1 - 200 OK

So if you too come across this situation, please try out this option. Sometimes this might save your time! :)

Wednesday, February 11, 2015

Reason and solution for not loading WSDL in a clustered setup fronted by NGINX


	For more information about nginx http proxy module please refer : http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version
	Issue : Sometimes a WSDL in a clustered setup fronted by NGINX might not be loaded.

Reason :

By default, http version 1.0 is used in nginx. Version 1.1 is recommended for use with keepalive connections. Therefore sometimes the wsdl will not be loaded.

You can confirm it by doing a curl command to the wsdl ngingx url.

E.g., curl http://wrk.test.com:8280/services/test?wsdl
Then you will get half of the wsdl.

Solution :

To avoid this, you will have to configure the nginx configuration file to set it's http version.

       proxy_http_version 1.1;

Example :

As an example,

server{
      listen 8280;
      server_name wrk.test.com;

location / {
       proxy_set_header X-Forwarded-Host $host;
       proxy_set_header X-Forwarded-Server $host;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

       proxy_http_version 1.1;

        proxy_pass http://esbworkers;
        proxy_redirect http://esbworkers http://wrk.esb.com:8280/;

        }
}

This configuration is applicable for nginx versions above 1.1.4

Monday, December 1, 2014

WSO2 ESB Script Mediator Using JavaScript with a Json payload

Synapse supports Mediators implemented in a variety of scripting languages such as JavaScript, Groovy, or Ruby. For More information on script mediator please refer this link.

This blog post will explain a scenario of using script mediator with a Json payload.

Configure back-end
==============

1. Deploy this webapp in an appserver.
https://svn.wso2.org/repos/wso2/trunk/commons/qa/qa-artifacts/esb/esb481/JSONscenarios/withScriptMediator/pizzashop-rs_1.0.zip

Configure ESB
=============

2. Provide the following api synapse configuration.

<api name="pizzashop" context="/pizzashop">
 <resource methods="GET" uri-template="/api/menu/pizza*">
  <inSequence>
   <send>
    <endpoint>
     <http method="get"
uri-template="http://localhost:9765/pizzashop-rs_1.0/services/menu/pizza/all"/>
    </endpoint>
   </send>
  </inSequence>
  <outSequence>
   <log level="full"/>
    <script language="js">var payload = mc.getPayloadJSON();mc.setPayloadJSON(payload);    </script>
     <send/>
  </outSequence>
 </resource>
</api>

Invoking the service
==============

3. Invoke the service

curl -v "http://localhost:8280/pizzashop/api/menu/pizza/all"

You will get all the available pizza list.

Friday, July 12, 2013

How to test the disable HTTP chunking for outgoing messaging via WSO2 ESB and a REST request

There are few HTTP transport-specific properties in ESB.

Properties use in ESB access various types of information regarding a message that passes through the ESB. Also properties can be used to control the behavior of the ESB on a given message.

WSO2 ESB support transfer-encoding – chunked by default. To make it disabled, there is a property that would help to perform it.

      <property name="DISABLE_CHUNKING" value="true" scope="axis2"/>

http://docs.wso2.org/wiki/display/ESB470/HTTP+Transport+Properties

Transfer-encoding - chunked is a feature came along with HTTP 1.1. Therefore no matter what HTTP 1.0 will support only content type. Therefore if the message is forced to send as HTTP1.0 you will be able to see only the content type. Refer the previous blog post for more reference. Outgoing message can be seen in here.

By the following steps I will be explaining how to test this behavior if WSO2 ESB.

Download WSO2 ESB, http://wso2.com/products/enterprise-service-bus

To Setup :
----------------

Go to <ESB_Home>/samples/axis2Server/src/SimpleStockQuoteService and build it, ant
Start the Go to <ESB_Home>/samples/axis2Server and start the axis2Server by the following command, sh axis2server.sh.
Start the ESB Server, <ESB_Home>/bin by providing, sh wso2server.sh.
Run a tcpMon to view the outgoing messages.
Start the tcpmon in the <ESB_Home>/bin - sh tcpmon.sh

Create 2 listeners as below.

1. Listen Port 8281, Target host – localhost, Target port – 8280 – Before hits the ESB

2. Listen Port 9001, Target host – localhost, target port – 9000 – After passing through ESB and Before hits the backend

 <?xml version="1.0" encoding="UTF-8"?>  
 <definitions xmlns="http://ws.apache.org/ns/synapse">  
   <sequence name="fault">  
    <log level="full">  
      <property name="MESSAGE" value="Executing default &#34;fault&#34; sequence"/>  
      <property name="ERROR_CODE" expression="get-property('ERROR_CODE')"/>  
      <property name="ERROR_MESSAGE" expression="get-property('ERROR_MESSAGE')"/>  
    </log>  
    <drop/>  
   </sequence>  
   <sequence name="main">  
    <header name="Action" value="urn:getQuote"/>  
    <filter source="get-property('To')" regex=".*/StockQuote.*">  
     <property name="DISABLE_CHUNKING" value="true" scope="axis2"/>  
      <send>  
       <endpoint>  
         <address uri="http://localhost:9001/services/SimpleStockQuoteService"  
             format="soap11"/>  
       </endpoint>  
      </send>  
      <drop/>  
    </filter>  
    <send/>  
   </sequence>  
 </definitions>

To Send the Rest request :

---------------------------------------------

Go to axis2Client, <ESB_Home>/samples/axis2Client and send the following REST request.

 ant stockquote -Dtrpurl=http://localhost:8281/services/StockQuote -Drest=true

Observations :
-------------------------

On the server side, Standard :: Stock price = $70.54750695255485 will be generated.
Observe the tcpMon.

Rest request sent to ESB can be seen in port 8281,

Transfer-Encoding: chunked

Outgoing message from ESB can be seen in port 9001,

Content-Length: 275

How to test the Force HTTP 1.0 outgoing messages via WSO2 ESB and a REST request

There are few HTTP transport-specific properties in ESB.

Properties use in ESB access various types of information regarding a message that passes through the ESB. Also properties can be used to control the behavior of the ESB on a given message.

WSO2 ESB support HTTP 1.1 outgoing messages by default. To make it HTTP 1.0, there is a property that would help to perform it.

      <property name="FORCE_HTTP_1.0" value="true" scope="axis2"/>

http://docs.wso2.org/wiki/display/ESB470/HTTP+Transport+Properties

By the following steps I will be explaining how to test this behavior if WSO2 ESB.

Download WSO2 ESB, http://wso2.com/products/enterprise-service-bus

To Setup :
----------------

Go to <ESB_Home>/samples/axis2Server/src/SimpleStockQuoteService and build it, ant
Start the Go to <ESB_Home>/samples/axis2Server and start the axis2Server by the following command, sh axis2server.sh.
Start the ESB Server, <ESB_Home>/bin by providing, sh wso2server.sh.
Run a tcpMon to view the outgoing messages.
Start the tcpmon in the <ESB_Home>/bin - sh tcpmon.sh

Create 2 listeners as below.

1. Listen Port 8281, Target host – localhost, Target port – 8280 – Before hits the ESB

2. Listen Port 9001, Target host – localhost, target port – 9000 – After passing through ESB and Before hits the backend

 <?xml version="1.0" encoding="UTF-8"?>  
 <definitions xmlns="http://ws.apache.org/ns/synapse">  
   <sequence name="fault">  
    <log level="full">  
      <property name="MESSAGE" value="Executing default &#34;fault&#34; sequence"/>  
      <property name="ERROR_CODE" expression="get-property('ERROR_CODE')"/>  
      <property name="ERROR_MESSAGE" expression="get-property('ERROR_MESSAGE')"/>  
    </log>  
    <drop/>  
   </sequence>  
   <sequence name="main">  
    <header name="Action" value="urn:getQuote"/>  
    <filter source="get-property('To')" regex=".*/StockQuote.*">  
     <property name="FORCE_HTTP_1.0" value="true" scope="axis2"/>  
      <send>  
       <endpoint>  
         <address uri="http://localhost:9001/services/SimpleStockQuoteService"  
             format="soap11"/>  
       </endpoint>  
      </send>  
      <drop/>  
    </filter>  
    <send/>  
   </sequence>  
 </definitions>

To Send the Rest request :

---------------------------------------------

Go to axis2Client, <ESB_Home>/samples/axis2Client and send the following REST request.

 ant stockquote -Dtrpurl=http://localhost:8281/services/StockQuote -Drest=true

Observations :
-------------------------

On the server side, Standard :: Stock price = $157.06490695255485 will be generated.
Observe the tcpMon.

Rest request sent to ESB can be seen in port 8281,

POST /services/StockQuote HTTP/1.1

Outgoing message from ESB can be seen in port 9001,

POST /services/SimpleStockQuoteService HTTP/1.0

Saturday, June 29, 2013

How to test Multipart/form-data with attachments and json content type pass-through support for WSO2 ESB.

I hope the reader of this blog has got an idea about WSO2 ESB and the pass-through transport and the different type of content types support for the product.

For more information read on, http://wso2.com/products/enterprise-service-bus/

To test Multipart/form data, we need the below things ready.

A backend service.
tcpmon to view the pass through
A form to submit data
Attachments – text files, json file, an image
WSO2 ESB with the correct synapse configuration of the proxy

Backend Service

As the backend service, I will be setting the offset to 2 and run a WSO2 App Server, which can also be downloaded from http://wso2.com/products/application-server/. I am using 5.1.0.
I will be using the HelloService in unsecured mode.
I assume now that I have the HelloService up and runninn in, http://localhost:9765/services/HelloService

Set up tcp mon

1. Start the tcpmon in the <ESB_Home>/bin - sh tcpmon.sh

2. Create 2 listeners as below.

1. Listen Port 8281, Target host – localhost, Target port – 8280 – Before hits the ESB/ HelloProxyService proxy

2. Listen Port 9766, target host – localhost, target port – 9765 – After passing through ESB and Before hits the backend

A form to submit data

I will be using a html which contains a form to submit data and to upload attachments and which has the form action set to the our first tcpmon (Before hits the ESB/ HelloProxyService proxy),
form action="http://localhost:8281/services/HelloProxyService/greet"

 <html>  
  <head><title>multipart/form-data - Client</title></head>  
  <body>   
 <form action="http://localhost:8281/services/HelloProxyService/greet" method="POST" enctype="multipart/form-data">  
 User Name: <input type="text" name="name">  
 User id: <input type="text" name="id">  
 User Address: <input type="text" name="add">  
 AGE: <input type="text" name="age">  
  <br>   
 Upload :   
 <input type="file" name="datafile" size="40" multiple>  
 </p>  
  <input type="submit" value="Submit">  
  </form>  
  </body>  
 </html>

Attachments

As attachments, I have just a text file with some texts in it
A jpeg image
A Json file with a json object.

E.g., jsonfile.txt

 {  
 "name":"John Johnson",  
 "street":"Oslo West 555",  
 "age":33,  
 "phone":"555 1234567"  
 }

Synapse configurations

The below proxy will allow the user to view the data that has been sent.
<Port> = 9766 or a different port you provide

 <proxy name="HelloProxyService"  
      transports="https http"  
      startOnLoad="true"  
      trace="disable">  
    <description/>  
    <target>  
      <endpoint>  
       <address uri="http://localhost:<port>/services/HelloService"/>  
      </endpoint>  
      <outSequence>  
       <property name="OUT_ONLY" value="true"/>  
       <send/>  
      </outSequence>  
    </target>  
   </proxy>

2. If you want to test whether this form data saved in to a file correctly, you can enable VFS transport in axis2.xml and use the following synapse accordingly. We will be saving it in to a file in the given location.

 <proxy name="HelloProxyService"  
      transports="https http"  
      startOnLoad="true"  
      trace="disable">  
    <description/>  
    <target>  
      <endpoint>  
       <address uri="vfs:file:///home/ushani/Downloads/out"/>  
      </endpoint>  
      <inSequence>  
       <property name="OUT_ONLY" value="true"/>  
      </inSequence>  
      <outSequence>  
       <send/>  
      </outSequence>  
    </target>  
   </proxy>

To Test :

Open the html form in browser format and fill in the input values and attach the attachment (Json file or a text file or an image) submit and observe in tcpmon.

(Listen Port 8281, Target host – localhost, Target port – 8280 – Before hits the ESB/ HelloProxyService proxy)

(Listen Port 9766 or a different port, target host – localhost, target port – 9765 or different port which the backend service is run – After passing through ESB and Before hits the backend)

Wednesday, April 24, 2013

How to test SOAP Fault Data in WSO2 ESB and JMS message store, a message processor with a reply sequence and a fault sequence

In this post I am not going to explain about message store or message processor and their behavior. I will exactly explain how to test SOAP Fault Data in WSO2 ESB and JMS message store, a message processor with a reply sequence and a fault sequence.

Before you try out this sample, it is better to read the following articles and blog posts to get more information.

To read more on WSO ESB : http://wso2.com/products/enterprise-service-bus/
Product Documentation : http://docs.wso2.org/wiki/display/ESB460/Enterprise+Service+Bus+Documentation
Message Store : http://docs.wso2.org/wiki/display/ESB460/Message+Stores
JMS Message Stores : http://docs.wso2.org/wiki/display/ESB460/JMS+Message+Store
Message processor : http://docs.wso2.org/wiki/display/ESB460/Message+Processors
Mediators : http://docs.wso2.org/wiki/display/ESB460/Mediators
Samples on Forward Mesaging patterns with Message Stores and processors : http://docs.wso2.org/wiki/display/ESB460/Store+and+Forward+Messaging+Patterns+with+Message+Stores+and+Processors
Error Codes : http://docs.wso2.org/wiki/display/ESB460/Error+Handling+and+Error+Codes
Blogs : http://techfeast-hiranya.blogspot.com/2012/01/wso2-esb-tips-tricks-08-message-stores.html and http://maharachchi.blogspot.com/2012/09/now-you-can-send-soapfaults-to-fault.html
Behaviour : https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbxKydMUlsUcTyFfMyUqeMps2pWqBhVnt-NP6Tn7zWYsQGOv1Wc0kl5-7Ha64B5iaqtYGXP92t1yLGFFTCJrplWNesVJCEkBgxDShrSjd84XwC6eW1HEYu6ptgMzPMFLGcC82Y2C-nisTI/s1600/dlc.png

Applications and networks can be full of errors, Applications crash. Network routers and links get into states where they cannot pass messages through with the expected efficiency. These error conditions are very likely to cause a fault or trigger a runtime exception in the ESB.

When applications and networks can be full of errors. Applications crash. Network routers and links get into states where they cannot pass messages through with the expected efficiency. These error conditions are very likely to cause a fault or trigger a runtime exception in the ESB.

It is not mandatory to associate each sequence and proxy service with a fault sequence. In situations where a fault sequence is not specified explicitly, a default fault sequence will be used to handle errors.

Whenever an error occurs in WSO2 ESB, the mediation engine attempts to provide as much information as possible on the error to the user by initializing the following properties on the erroneous message:

ERROR_CODE
ERROR_MESSAGE
ERROR_DETAIL
ERROR_EXCEPTION etc

We can configure a message broker to save the failed requests in a message store for later delivery. When the client sends requests to a proxy service , the proxy service stores the messages to a JMS message store if it cannot be delivered and then the back-end service is invoked by a message forwarding processor, which picks stored messages in the JMS message store.

To test this scenario we need the following:

Message Broker – I will be using WSO2 Message Broker (MB)
WSO2 ESB
WSO2 Application Server to invoke a service – Axis2Service (You can use any other service which will generate an exception)

To configure WSO2 ESB, AS and MB, follow the steps below:

Follow the steps in the following location. http://docs.wso2.org/wiki/display/ESB460/Configure+with+WSO2+Message+Broker
Or 1 and 2 steps in http://sandapa.blogspot.com/2013/02/jms-messaging-with-wso2-esb-460-and.html
Now the ESB and Message broker should be ready.

Configurations :
---------------------------
Set the offset as 2 in carbon.xml (<AS_HOME/repository/conf>) and start the Application server deploying the Axis2service. Refer the following document. http://docs.wso2.org/wiki/display/AS510/Installation+Guide
Create a proxy. Provide the following synapse configuration.

  <proxy name="FaultTestProxy"  
      transports="https http"  
      startOnLoad="true"  
      trace="disable">  
    <description/>  
    <target>  
      <inSequence>  
       <log level="full"/>  
       <property name="target.endpoint" value="AxisEp"/>  
       <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>  
       <property name="FORCE_ERROR_ON_SOAP_FAULT" value="true"/>  
       <store messageStore="Queue1"/>  
      </inSequence>  
      <outSequence>  
       <send/>  
      </outSequence>  
      <faultSequence>  
       <log level="full" category="ERROR"/>  
      </faultSequence>  
    </target>  
   </proxy>

6. Set the endpoint as following to invoke Axis2Service. This can be any service which will help to produce an error.

  <endpoint name="AxisEp">  
    <address uri="http://localhost:9765/services/Axis2Service">  
      <suspendOnFailure>  
       <errorCodes>-1</errorCodes>  
       <progressionFactor>1.0</progressionFactor>  
      </suspendOnFailure>  
    </address>  
   </endpoint>

7. Create a message store as following.

 <messageStore class="org.wso2.carbon.message.store.persistence.jms.JMSMessageStore"  
          name="Queue1">  
    <parameter name="java.naming.factory.initial">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>  
    <parameter name="store.jms.cache.connection">false</parameter>  
    <parameter name="java.naming.provider.url">repository/conf/jndi.properties</parameter>  
    <parameter name="store.jms.JMSSpecVersion">1.1</parameter>  
    <parameter name="store.jms.destination">Queue1</parameter>  
   </messageStore>

8. Create a Scheduled Message Forwarding Processor as following.

 <messageProcessor class="org.apache.synapse.message.processors.forward.ScheduledMessageForwardingProcessor"  
            name="MyMsgProc"  
            messageStore="Queue1">  
    <parameter name="message.processor.reply.sequence">Axis_Ok</parameter>  
    <parameter name="max.delivery.attempts">4</parameter>  
    <parameter name="interval">5000</parameter>  
    <parameter name="message.processor.fault.sequence">Axis_Fault</parameter>  
   </messageProcessor>

9. Following fault sequence should be configured

 <sequence name="Axis_Fault">  
    <log level="custom" category="ERROR">  
      <property name="text" value="An error occured"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_MESSAGE"  
           expression="get-property('ERROR_MESSAGE')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_CODE"  
           expression="get-property('ERROR_CODE')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_DETAIL"  
           expression="get-property('ERROR_DETAIL')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_EXCEPTION"  
           expression="get-property('ERROR_EXCEPTION')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="FaultTo"  
           expression="get-property('FaultTo')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="FAULT"  
           expression="get-property('FAULT')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="SENDING_FAULT"  
           expression="get-property('SENDING_FAULT')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd" name="REPLY" expression="/"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="RESPONSE"  
           expression="get-property('RESPONSE ')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="HTTP_SC"  
           expression="get-property('HTTP_SC')"/>  
    </log>  
    <log level="full" category="ERROR"/>  
   </sequence>

The whole Synapse Configuration would be as follows :

 <?xml version="1.0" encoding="UTF-8"?>  
 <definitions xmlns="http://ws.apache.org/ns/synapse">  
   <registry provider="org.wso2.carbon.mediation.registry.WSO2Registry">  
    <parameter name="cachableDuration">15000</parameter>  
   </registry>  
 #######################  
 # Proxy  
 #######################   
   <proxy name="FaultTestProxy"  
      transports="https http"  
      startOnLoad="true"  
      trace="disable">  
    <description/>  
    <target>  
      <inSequence>  
       <log level="full"/>  
       <property name="target.endpoint" value="AxisEp"/>  
       <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>  
       <property name="FORCE_ERROR_ON_SOAP_FAULT" value="true"/>  
       <store messageStore="Queue1"/>  
      </inSequence>  
      <outSequence>  
       <send/>  
      </outSequence>  
      <faultSequence>  
       <log level="full" category="ERROR"/>  
      </faultSequence>  
    </target>  
   </proxy>  
 #######################  
 # Endpoint  
 #######################   
   <endpoint name="AxisEp">  
    <address uri="http://localhost:9765/services/Axis2Service">  
      <suspendOnFailure>  
       <errorCodes>-1</errorCodes>  
       <progressionFactor>1.0</progressionFactor>  
      </suspendOnFailure>  
    </address>  
   </endpoint>  
 #######################  
 # Fault Sequence  
 #######################   
   <sequence name="Axis_Fault">  
    <log level="custom" category="ERROR">  
      <property name="text" value="An error occured"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_MESSAGE"  
           expression="get-property('ERROR_MESSAGE')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_CODE"  
           expression="get-property('ERROR_CODE')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_DETAIL"  
           expression="get-property('ERROR_DETAIL')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="ERROR_EXCEPTION"  
           expression="get-property('ERROR_EXCEPTION')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="FaultTo"  
           expression="get-property('FaultTo')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="FAULT"  
           expression="get-property('FAULT')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="SENDING_FAULT"  
           expression="get-property('SENDING_FAULT')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd" name="REPLY" expression="/"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="RESPONSE"  
           expression="get-property('RESPONSE ')"/>  
      <property xmlns:ns="http://org.apache.synapse/xsd"  
           name="HTTP_SC"  
           expression="get-property('HTTP_SC')"/>  
    </log>  
    <log level="full" category="ERROR"/>  
   </sequence>  
    <sequence name="main">  
    <in>  
      <log level="full"/>  
      <filter source="get-property('To')" regex="http://localhost:9000.*">  
       <send/>  
      </filter>  
    </in>  
    <out>  
      <send/>  
    </out>  
    <description>The main sequence for the message mediation</description>  
   </sequence>  
 #######################  
 # Message Store  
 #######################   
   <messageStore class="org.wso2.carbon.message.store.persistence.jms.JMSMessageStore"  
          name="Queue1">  
    <parameter name="java.naming.factory.initial">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>  
    <parameter name="store.jms.cache.connection">false</parameter>  
    <parameter name="java.naming.provider.url">repository/conf/jndi.properties</parameter>  
    <parameter name="store.jms.JMSSpecVersion">1.1</parameter>  
    <parameter name="store.jms.destination">Queue1</parameter>  
   </messageStore>  
 #######################  
 # Message Processor  
 #######################   
   <messageProcessor class="org.apache.synapse.message.processors.forward.ScheduledMessageForwardingProcessor"  
            name="MyMsgProc"  
            messageStore="Queue1">  
    <parameter name="message.processor.reply.sequence">Axis_Ok</parameter>  
    <parameter name="max.delivery.attempts">4</parameter>  
    <parameter name="interval">5000</parameter>  
    <parameter name="message.processor.fault.sequence">Axis_Fault</parameter>  
   </messageProcessor>  
 </definitions>

To Test:
------------

1. Via SOAP create a new SOAP project and provide the following wsdl URL,
http://localhost:8281/services/FaultTestProxy?wsdl

2. Then stop the Axis2Service and send the following request in echoInt method. (We are sending a String value except an int value)

 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://service.carbon.wso2.org">  
   <soapenv:Header/>  
   <soapenv:Body>  
    <ser:echoInt>  
      <!--Optional:-->  
      <ser:x>String</ser:x>  
    </ser:echoInt>  
   </soapenv:Body>  
 </soapenv:Envelope>

3. Then the message store will store the message. Then start the Axis2Service. Then the Message Processor will pick the message and try to forward it. Due to the string value, following Error codes will be printed in the ESB Server console.

ERROR - LogMediator text = An error occured, ERROR_MESSAGE = Invalid value "String" for element x, ERROR_CODE = Client, ERROR_DETAIL = , ERROR_EXCEPTION = , FaultTo = , FAULT = TRUE, SENDING_FAULT = null, REPLY = <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header><ns:BAMEvent xmlns:ns="http://wso2.org/ns/2010/10/bam" activityID="13320046414250_138"/></soapenv:Header><soapenv:Body><soapenv:Fault xmlns:axis2ns26="http://schemas.xmlsoap.org/soap/envelope/"><faultcode>axis2ns26:Client</faultcode><faultstring>Invalid value "String" for element x</faultstring><detail/></soapenv:Fault></soapenv:Body></soapenv:Envelope>, RESPONSE = null, HTTP_SC = 500

This was done with the great help provided by Charitha Kankanamge. :)

Ushani Balasooriya's Blog