Thursday, September 14, 2017

How to enable tracing logs on PCF DEV

Most of the time PCF does not show much error logs. After going through their documentation I found a catch. Since our calls are mainly API based, we can enable tracing logs for these calls and get an idea of the errors we get.

E.g.,  I had a situation, no matter how correctly developed app it is, when I push it in to dev environment, it keeps on crashing.

Only error log I could see was below

2017-09-14T15:23:56.84+0530 [API/0] OUT Updated app with guid 0163061c-0120-4662-a0b1-930d7ce6505b ({"state"=>"STOPPED"})

The  below command actually helped me to get more details of the error log.

What you have to do is append -vin to your cf command.

E.g.,


sudo cf push --docker-image test:test test -u process -v
  
This gives more error logs as below :

REQUEST: [2017-09-14T18:02:32+05:30]
GET /v2/apps/abc44f1f-3072-4c85-a72a-d0d6738dfb97/instances HTTP/1.1
Host: api.local.pcfdev.io
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: go-cli 6.29.2+c66d0f3.2017-08-25 / linux



RESPONSE: [2017-09-14T18:02:32+05:30]
HTTP/1.1 200 OK
Connection: close
Content-Length: 96
Content-Type: application/json;charset=utf-8
Date: Thu, 14 Sep 2017 12:32:33 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 09c44df1-7f8b-4147-6cd8-fa033dd68477
X-Vcap-Request-Id: 09c44df1-7f8b-4147-6cd8-fa033dd68477::ba3471cb-55be-46ac-a119-db14696c8f62

{"0":{"state":"DOWN","uptime":32,"since":1505392320,"details":"insufficient resources: memory"}}

REQUEST: [2017-09-14T18:02:32+05:30]
GET /v2/apps/abc44f1f-3072-4c85-a72a-d0d6738dfb97/stats HTTP/1.1
Host: api.local.pcfdev.io
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: go-cli 6.29.2+c66d0f3.2017-08-25 / linux



RESPONSE: [2017-09-14T18:02:32+05:30]
HTTP/1.1 200 OK
Connection: close
Content-Length: 253
Content-Type: application/json;charset=utf-8
Date: Thu, 14 Sep 2017 12:32:33 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 95f66d78-59b3-4e69-4b18-508de8d2c102
X-Vcap-Request-Id: 95f66d78-59b3-4e69-4b18-508de8d2c102::801fb2ec-4bc1-45a7-8726-10b57b7b6c73

{"0":{"state":"DOWN","stats":{"name":"working","uris":["working.local.pcfdev.io"],"host":null,"port":null,"uptime":32,"mem_quota":2147483648,"disk_quota":1073741824,"fds_quota":16384,"usage":{"time":"2017-09-14 12:32:33 UTC","cpu":0,"mem":0,"disk":0}}}}
0 of 1 instances running, 1 down

How can I reconfigure PCF DEV VM with different size of memory?

Sometimes you might get an error message as Insufficient memory wen you try to deploy apps in to pcf environment. Then you need to reconfigure your VM with more memory as below.

1. First you need to stop or destroy your running CF DEV env if there is already running environment.

cf dev stop
cf dev destroy

PCF Dev VM has been destroyed.


2. Then uninstall the current env

sudo cf uninstall-plugin pcfdev

Uninstalling plugin pcfdev...
OK
Plugin pcfdev 0.27.0 successfully uninstalled.




3. Then re install the plugin by running the below command in your extracted zip folder. E.g., pcfdev-v0.26.0+PCF1.10.0-linux.zip -> pcfdev-v0.26.0+PCF1.10.0-linux

./pcfdev-v0.27.0+PCF1.11.0-linux

Plugin successfully installed. Current version: 0.27.0. For more info run: cf dev help




4. Now re start allocating the memory in mega bytes

To change the allocated memory, run the following command, replacing NEW-ALLOCATED-MEMORY with the amount of memory you want to allocate in megabytes:
 
$ cf dev start -m NEW-ALLOCATED-MEMORY

By default, PCF Dev tries to allocate half of the memory available on your host machine, with a minimum of 3GB and a maximum of 4GB.


cf dev start -m 4000



Reference : https://docs.pivotal.io/pcf-dev/faq.html

Tuesday, September 5, 2017

Do you get a "Got permission denied while trying to connect to the Docker daemon socket" even after successful docker login?

Do you get a warning during the login to docker hub via terminal even after providing correct credentials?

Is the warning looks like below?

docker login
Warning: failed to get default registry endpoint from daemon (Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.30/info: dial unix /var/run/docker.sock: connect: permission denied). Using system default: https://index.docker.io/v1/
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.

Username: ushanib
Password:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.30/auth: dial unix /var/run/docker.sock: connect: permission denied

This is because you have to run the command as the super user as below:

sudo docker login

Thursday, August 3, 2017

WHY WSO2 ESB?

I have been writing lot of posts about WSO2 ESB. But have you ever thought why we should use WSO2 ESB over other competitors? Have a look at Samisa's article.
Below points are taken from his article.

WSO2 advantages over competitors

  • Ability to easily integrate any component framework. Support of Java based extensions and multiple scripting options. There is no need to have WSO2 specific code to integrate anything with WSO2 ESB
  • Numerous built-in message mediators, solution templates and connectors to third-party cloud systems to help cut down redundant engineering efforts and enable significant component reuse
  • Freedom for architects and developers to pick and choose message formats, transports, and style of services they want to expose using the ESB
  • Component oriented architecture and cloud and container support enables you to deploy the ESB using a topology of your choice based on your needs in a secure, scalable and adaptive manner
  • The ready-made scripts and tools help with rapid deployments, ensuring the ability to go to market quickly with your solution using the ESB
  • Continuous innovation that helps build future proof solutions on top of the ESB
  • Rigorous and frequent product update cycles and state-of-the-art tooling support for managing ESB deployments with DevOps practices. Using Docker descriptors and Puppet scripts
  • Proactive testing and tuning of performance and innovation around performance enhancements

Friday, May 12, 2017

How to create a virtual IP address in Fedora

Sometimes you will face a situation like creating virtual IP addresses in your machine for testing purposes.

This simple blog post is on how to do it.

Step 1 :


Type ifconfig in console to list down the current  network configurations. It will display as below.

enp0s25: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 54:ee:75:83:7e:a6  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xf1200000-f1220000

enp0s25:1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.100.0.24  netmask 255.255.255.0  broadcast 10.100.0.255
        ether 54:ee:75:83:7e:a6  txqueuelen 1000  (Ethernet)
        device interrupt 20  memory 0xf1200000-f1220000

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 1295  bytes 168545 (164.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1295  bytes 168545 (164.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.124.1  netmask 255.255.255.0  broadcast 192.168.124.255
        ether 52:54:00:a3:65:c6  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.0.24  netmask 255.255.255.0  broadcast 10.100.0.255
        inet6 fe80::1a5e:fff:fed4:229a  prefixlen 64  scopeid 0x20<link>
        ether 18:5e:0f:d4:22:9a  txqueuelen 1000  (Ethernet)
        RX packets 307132  bytes 230860558 (220.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 187836  bytes 37165700 (35.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp4s0:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.2  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 18:5e:0f:d4:22:9a  txqueuelen 1000  (Ethernet)


Step 2 : 


Add your address in to your network interface using below command.

192.168.1.3 is my new IP and wlp4s0 is my network interface.

ip addr add 192.168.1.3/24 dev wlp4s0 label wlp4s0:1

Step 3 : 


Use ip addr show command to display the current addresses.

ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s25: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 54:ee:75:83:7e:a6 brd ff:ff:ff:ff:ff:ff
    inet 10.100.0.24/24 brd 10.100.0.255 scope global enp0s25:1
       valid_lft forever preferred_lft forever
3: wlp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 18:5e:0f:d4:22:9a brd ff:ff:ff:ff:ff:ff
    inet 10.100.0.24/24 brd 10.100.0.255 scope global dynamic wlp4s0
       valid_lft 2897sec preferred_lft 2897sec
    inet 192.168.1.2/24 scope global wlp4s0:0
       valid_lft forever preferred_lft forever
    inet 192.168.1.3/24 scope global secondary wlp4s0:1
       valid_lft forever preferred_lft forever
    inet6 fe80::1a5e:fff:fed4:229a/64 scope link 
       valid_lft forever preferred_lft forever
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 52:54:00:a3:65:c6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.124.1/24 brd 192.168.124.255 scope global virbr0
       valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 500
    link/ether 52:54:00:a3:65:c6 brd ff:ff:ff:ff:ff:ff


Step 4: 


If you want you can even add it in to your host entries. 

Add host entries to /etc/hosts

192.168.1.2 testhost1.com


Step 5 : 


Ping to your new IP address to make sure they respond

ping 192.168.1.3

PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data.
64 bytes from 192.168.1.3: icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from 192.168.1.3: icmp_seq=2 ttl=64 time=0.038 ms
64 bytes from 192.168.1.3: icmp_seq=3 ttl=64 time=0.038 ms
64 bytes from 192.168.1.3: icmp_seq=4 ttl=64 time=0.062 ms
64 bytes from 192.168.1.3: icmp_seq=5 ttl=64 time=0.082 ms

Salesforce Integration with Ballerina

Introduction



Ballerina is a programming language which is designed to do integrations with applications and services that can handle all integration scenarios. It is a language which is flexible which allows you to build your integrations from sequence diagrams as well. Its native support for REST, Swagger, JSON, XML and also the support for popular connectors like Facebook, twitter, salesforce makes Ballerina a powerful language. The beauty of Ballerina is exposed by its own composer which allows you to easily draw diagrams that generates the integration code for you and vise versa. Ballerina language syntax is modeled after the best practices of popular languages like Java.


Ballerina contains Salesforce SOAP and REST connectors that allows you to easily work  with records in Salesforce, a web-based service that allows organizations to manage Customer Relationship Management (CRM) data. You can use the Salesforce connector to create, query, retrieve, update, and delete records in your organization's Salesforce data. The connector uses the Salesforce APIs to interact with Salesforce.


Nowadays most companies use Salesforce to monitor, maintain and manage their customer  relationships in order to optimize revenue growth. Salesforce is known as the world’s first CRM platform which can be accessible from anywhere over the internet which also perfectly utilizes the Cloud computing concept. Salesforce App Cloud provides an integrated architecture that lets you focus on building solutions using Salesforce that can improve Enterprise Business generation. Developers can now assemble applications with a few clicks or code and then instantly launch them across any device to connect with customers, employees or partners.


Below are some of the advantages that you can gain by using Ballerina when it comes to Salesforce.
  • Easy to integrate with Salesforce APIs
  • Query/Insert/Update data within Ballerina
  • Enrich queried results into standard coding formats like JSON, XML
  • Connect to other third party backends where you can easily send the enriched data


User Story



In enterprise Salesforce integration scenarios, most often than not there is a backend which expects Salesforce user account details in a json object format. This post will demonstrate how easy it is to query Salesforce objects and construct a json object as expected by the backend, using Ballerina.


Used tools and techniques



  • Use Salesforce Soap connector provided by Ballerina to create the connection and query the record.
  • Retrieve the data by xpath.
  • Transform the record into a JSON object using particular transformation functions and build the json object.
  • Send it to a backend which expects the enriched JSON object


Resources

Ballerina 0.8.6 distribution


Prerequisites

User should have a salesforce Dev account with username, password and security token
Note : 39.0 is the API version
Soap version 1.1 will be used


Let's Code ..

Step 1 : Starting the Composer

You can simply use ballerina composer to code since it provides you both the designer and source view. In order to start the composer, go to <BALLERINA_HOME>/bin and type ‘sh composer’ for unix based systems or ‘composer’ for windows.


You will see a log as,


INFO  - Ballerina Composer URL: http://localhost:9091


You can open the Composer tool in your browser, using the following URL.


http://localhost:9091


Step 2 :Arguments when running the scenario



In this example, I will be using the main function to perform the required functionality. In Ballerina, the main() function takes an array of strings as its only argument and optionally returns an int.  This is also the potential entry point for command line execution. Any package may have such a function and the execution instruction decides which main() is executed. This documentation provides more information about functions in Ballerina.


I will be sending all the required information as arguments when running the sample.


I will be providing query [0], username [1], password+securityToken [2] ,  the query string [3], login url [4] and soap version [5] as arguments.  


query <userName> <password+securityToken> <queryString> <loginUrl> <soapVersion>


E.g.,
query ushanib@gmail.com password1@\!bE6zsu58ZxvxbNgVSmGXvVgG6 "Select u.Username, u.ProfileId, u.Name, u.LastName, u.Email From User u where Username = 'ushani1@wso2.com'" https://login.salesforce.com/services/Soap/u/39.0 1.1


If you want, you can also assign them in your code without passing them as arguments.


E.g.,
salesforcesoap:ClientConnector sales = create salesforcesoap:ClientConnector("ushanib@gmail.com", "password1@!bE6zsu58ZxvxbNgVSmGXvVgG6", "https://login.salesforce.com/services/Soap/u/39.0", "1.1");


Step 3 : Querying the salesforce object



Inside the main() function, we need to create the salesforce connector connection using the arguments passed in the command line.


Refer this documentation for more information on salesforce soap connector. You need to pass the below mentioned parameters to create the connection.


connector ClientConnector (string username, string password, string loginUrl, string soapVersion) {


function main(string[] args) {
     ..
xml soapResponse;
xml[] headers = [];
string username = args[1];
string password = args[2];
   string loginurl = args[4];
string soapver = args[5];


salesforcesoap:ClientConnector sales = create salesforcesoap:ClientConnector(username, password, loginurl, soapver);


I will be passing our connector details, headers and the query as arguments to retrieve the response in soap. You can see the information in our connector implementation here.


query (ClientConnector s, xml[] headers, string query) (xml) {


Create the header value to be passed as an argument.


xml queryOptions = `<urn:QueryOptions xmlns:urn="urn:partner.soap.sforce.com"><urn:batchSize>1</urn:batchSize></urn:QueryOptions>`;
headers = [queryOptions];
soapResponse = salesforcesoap:ClientConnector.query(sales, headers, args[3]);


I will be printing the queried response as below for our reference.


system:println(xmls:toString(soapResponse));


If you need to perform a different operation, you can extend this by adding another if statement.
E.g., update record, insert record. All the available functions can be found in here. I have used a query statement here in order to execute other operations just by adding an if statement.
Then you should pass any arguments to recognise the operation (E.g., updateRecord) as your 0th argument in command line and other arguments if you wish to pass them as command lines arguments.  


E.g.,
  } else if (args[0] == "updateRecord") {


Step 4 : Transform salesforce search result into a json payload



Use xpath to retrieve the value as a string variable as below :


string Sfusername = xmls:getString(soapResponse, "/*:queryResponse/*:result/*:records/*:Username/text()");


string SfProfileId = xmls:getString(soapResponse, "/*:queryResponse/*:result/*:records/*:ProfileId/text()");


string SfName = xmls:getString(soapResponse, "/*:queryResponse/*:result/*:records/*:Name/text()");


string SfLastName = xmls:getString(soapResponse, "/*:queryResponse/*:result/*:records/*:LastName/text()");


string SfEmail = xmls:getString(soapResponse, "/*:queryResponse/*:result/*:records/*:Email/text()");


Create a json payload with dummy values as below :


json jsPayload = `{"SFUser":{"Sfusername":"Sfusername", "SfProfileId":"SfProfileId", "SfName":"SfName", "SfEmail":"SfLastName", "SfEmail":"SfEmail"}}`;


Construct the json payload using json path as below. You can find the API specification for the json API in here.


Set the string value of the element that matches the given JSONPath.


function set(json j, string jsonPath, string value)


Parameter Name
Data Type
Description
j
A JSON object
jsonPath
The path of the JSON element
value
A string value


jsons:set(jsPayload, "$.SFUser.Sfusername", Sfusername);
jsons:set(jsPayload, "$.SFUser.SfProfileId", SfProfileId);
jsons:set(jsPayload, "$.SFUser.SfName", SfName);
jsons:set(jsPayload, "$.SFUser.SfLastName", SfLastName);
jsons:set(jsPayload, "$.SFUser.SfEmail", SfEmail);
system:println(strings:valueOf(jsPayload));


Step 5 : Send the payload to the particular backend



First I will be initialising a message.
message payloadBack = {};


Then I have set the message payload using the JSON payload object I have created.  Particular API documentation can be found in here.


function setJsonPayload(message m, json payload)


Parameter Name
Data Type
Description
m
message
The current message object
payload
json
The JSON payload object


messages:setJsonPayload(payloadBack, jsPayload);


Then I have used the POST action of HTTP connector as explained in this documentation.


I have created a connection of my backend at the beginning as below.


http:ClientConnector payloadEP = create http:ClientConnector("http://backendjson.com");


Payload will be sent to the configured backend which expects a JSON payload and it will print the response message once it is sent.


message response = http:ClientConnector.post(payloadEP, "/", payloadBack);
system:println("Payload sent to the backend");
system:println(response);


The entire sample is shown below (image 1) as displayed in composer design view :
Screenshot from 2017-04-24 12-26-10.png
(Image 1- Design View of the code)


Below is the representation in source view :
Screenshot from 2017-04-24 12-25-56.png
(Image 2 - Ballerina composer source view)

Running the Sample

You can run the sample from <BALLERINA_HOME>/bin by providing the below command in UNIX.

./ballerina run main <path to .bal file> query <userName> <password+securityToken> <queryString> <loginUrl> <soapVersion>


E.g.,


./ballerina run main ../../ballerina-tools-0.86/samples/sfquery.bal query ushanib@gmail.com password1@!bE6zsu58ZxvxbNgVSmGXvVgG6 "Select u.Username, u.ProfileId, u.Name, u.LastName, u.Email From User u where Username = 'ushani1@wso2.com'" https://login.salesforce.com/services/Soap/u/39.0 1.1


Results



<queryResponse xmlns="urn:partner.soap.sforce.com" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:sf="urn:sobject.partner.soap.sforce.com">
  <result xsi:type="QueryResult">
     <done>true</done>
     <queryLocator xsi:nil="true"/>
     <records xsi:type="sf:sObject">
        <sf:type>User</sf:type>
        <sf:Id xsi:nil="true"/>
        <sf:Username>ushani1@wso2.com</sf:Username>
        <sf:ProfileId>00e90000001aVwiAAE</sf:ProfileId>
        <sf:Name>Balasooriya</sf:Name>
        <sf:LastName>Balasooriya</sf:LastName>
        <sf:Email>ushani@wso2.com</sf:Email>
     </records>
     <size>1</size>
  </result>
</queryResponse>
{"SFUser":{"Sfusername":"ushani1@wso2.com","SfProfileId":"00e90000001aVwiAAE","SfName":"Balasooriya","SfEmail":"ushani@wso2.com"}}
Payload sent to the backend
{
   "success": true,
}

References