Executing a PDI transformation

When we execute a transformation we typically load the ETL metadata that describes the workload. This is handled by the TransMeta object. Then we execute this workload with a transformation engine handled by the Trans object. This means we first need to load a TransMeta object from XML (with a TransMeta constructor as described below where repository can be null) or from a repository

// Initialize, load settings, plugins, ...
//
KettleEnvironment.init();

...

TransMeta transMeta = new TransMeta("/foo/bar/trans.ktr", repository);
Trans Trans = new Trans(transMeta);

// The following will run the transformation in a separate thread.
//
trans.execute(arguments);

// If you want to wait until the transformation is finished...
//
trans.waitUntilFinished(); //

// If you want to know about the execution result.
//
Result result = trans.getResult();

The Result object contains all sorts of interesting results pertaining to the execution, including the number of errors, parsed files and much more as described over here.

Note: command line arguments are not to be confused with named parameters.

Want to add parameters, variables or arguments to your transformation execution?

try {
  for (String key : parameterMap.keySet()) {
    transMeta.setParameterValue(key, parameterMap.get(key));
  }
  for (String key : variableMap.keySet()) {
    transMeta.setVariable(key, variableMap.get(key));
  }
} catch (UnknownParamException e) {
    error(e.getMessage());
}

transMeta.setArguments(arguments);

Retrieving data from a step

To retrieve rows of data from any Kettle step copy you can attach a row listener to the step.

Trans Trans = new Trans(transMeta);

// prepare the execution of the transformation (instead of simply execute)
//
trans.prepareExecution(arguments);


// Find a step thread (ready to run but not yet started)
// You can also use method Trans.findBaseStep which gives back a list of all the step copies
//
StepInterface step = trans.findRunThread("Your Step Name");


// Attach a row listener to a step copy
//
step.addRowListener(new RowAdapter() {

    public void rowReadEvent(RowMetaInterface rowMeta, Object[] row) throws KettleStepException {
       // Here you get the rows as they are read by the step     
    }

    public void rowWrittenEvent(RowMetaInterface rowMeta, Object[] row) throws KettleStepException {
       // Here you get the rows as they are written by the step
    }
  }
);


// Now start the transformation threads...
//
trans.startThreads();

// If you want to wait until the transformation is finished...
//
trans.waitUntilFinished(); //

// If you want to know about the execution result.
//
Result result = trans.getResult();

Passing rows of data to a transformation

To pass rows of data to a Kettle transformation you can use an Inject step as a placeholder.

See the documentation for more information regarding Kettle API usage.