Executing a PDI transformation
When we execute a transformation we typically load the ETL metadata that describes the workload. This is handled by the TransMeta object. Then we execute this workload with a transformation engine handled by the Trans object. This means we first need to load a TransMeta object from XML (with a TransMeta constructor as described below where repository can be null) or from a repository
| Code Block |
|---|
// Initialize, load settings, plugins, ...
//
KettleEnvironment.init();
...
TransMeta transMeta = new TransMeta("/foo/bar/trans.ktr", repository);
Trans Trans = new Trans(transMeta);
// The following will run the transformation in a separate thread.
//
trans.execute(arguments);
// If you want to wait until the transformation is finished...
//
trans.waitUntilFinished(); //
// If you want to know about the execution result.
//
Result result = trans.getResult();
|
The Result object contains all sorts of interesting results pertaining to the execution, including the number of errors, parsed files and much more as described over here.
Note: command line arguments are not to be confused with named parameters.
Want to add parameters, variables or arguments to your transformation execution?
| Code Block |
|---|
try {
for (String key : parameterMap.keySet()) {
transMeta.setParameterValue(key, parameterMap.get(key));
}
for (String key : variableMap.keySet()) {
transMeta.setVariable(key, variableMap.get(key));
}
} catch (UnknownParamException e) {
error(e.getMessage());
}
transMeta.setArguments(arguments);
|
Retrieving data from a step
To retrieve rows of data from any Kettle step copy you can attach a row listener to the step.
| Code Block |
|---|
Trans Trans = new Trans(transMeta);
// prepare the execution of the transformation (instead of simply execute)
//
trans.prepareExecution(arguments);
// Find a step thread (ready to run but not yet started)
// You can also use method Trans.findBaseStep which gives back a list of all the step copies
//
StepInterface step = trans.findRunThread("Your Step Name");
// Attach a row listener to a step copy
//
step.addRowListener(new RowAdapter() {
public void rowReadEvent(RowMetaInterface rowMeta, Object[] row) throws KettleStepException {
// Here you get the rows as they are read by the step
}
public void rowWrittenEvent(RowMetaInterface rowMeta, Object[] row) throws KettleStepException {
// Here you get the rows as they are written by the step
}
}
);
// Now start the transformation threads...
//
trans.startThreads();
// If you want to wait until the transformation is finished...
//
trans.waitUntilFinished(); //
// If you want to know about the execution result.
//
Result result = trans.getResult();
|
Passing rows of data to a transformation
To pass rows of data to a Kettle transformation you can use an Inject step as a placeholder.
See the documentation for more information regarding Kettle API usage.