Data over web services

Original content

This document describes a convenient way to deliver data through a PDI transformation over a web service introduced in PDI 4.2.0-M2.

See also Carte Configuration and the calls to runJob / runTrans

Use-case

Here’s the situation: you need to supply a 3rd party data.  You don’t want to spend a lot of time coding web services and what not to deliver the data in a language neutral format.  Anybody can read data from a simple web service.  You can use any programming language and it’s easy to test in a browser.

The way that it works arguably couldn’t be easier…  Let’s take a look.

We start with a simple transformation that reads some customer data.  We take the “Read customer data” example from the samples:samples/transformations/CSV Input - Reading customer data.ktr
Next, we’re going to replace the dummy step with a “Text File Output” step (An “XML Output” or “JSON Output” step is also possible):

Save the transformation in the same samples folder under the “Servlet Data Example.ktr” filename.  In my case the full filename is: (from the transformation settings dialog)/home/matt/svn/kettle/trunk/samples/transformations/Servlet Data Example.ktr
Now you can use an instance of Carte that runs on your local host on port 8282 (execute carte.sh 127.0.0.1 8282) to get the data in your browser. 

The URL

For file (XML) based transformations:http://\* (http://*)username:password@hostname:port/kettle/executeTrans/?trans=PathTo{}File*
For transformations stored in a repository:http://\* (http://*)username:password@hostname:port/kettle/executeTrans/?rep=RepositoryName&user=RepositoryUser&pass=RepositoryPassword&trans=PathTo{}TransformationInRepository*
Please note that you need to replace forward slashes with %2F in your browser.  That leads to this URL in our example:

http://cluster:cluster@127.0.0.1:8282/kettle/executeTrans/?trans=%2Fhome%2Fmatt%2Fsvn%2Fkettle%2Ftrunk%2Fsamples%2Ftransformations%2FServlet Data Example.ktr

You can also pass transformation parameters and variables through the URL.  Simply add &PARAM=value and the appropriate parameter or variable in the transformation will be set prior to execution.

Easy testing

The result is can be tested with your browser:

JavaScript

If you need more power and flexibility as to what you are sending to the web service, it is possible to get a hold of the servlet print writer in JavaScript too:

    var out = _step_.getTrans().getServletPrintWriter();
    out.println("<H1>Hello, world!\n</H1>");