Exporting resources

What is it?

It's a method of exporting not only the loaded transformation or job, but all used sub-transformations and sub-jobs.

See also the concept: Lifecycle Management

What does it do?

It encapsulates the ETL metadata in such a way that it is runnable from inside a zip file.  That means that the source of the jobs or transformations is irrelevant.  It can be coming from a repository, from a local file, from a remote website, etc.  The end result is comprised of the following:

  • A single zip file
  • Containing all the used (or potentially used) jobs, transformations or sub-transformations (mappings) in XML format 

What nifty tricks are in there?

  • References to sub-jobs and sub-transformations are changed to use relative paths
  • References to data files are abstracted to use parameters
    • For example /path/to/foo/foo.csv is renamed to ${DATA_PATH_1}/foo.xml)
  • Parameters are added for all the different file locations in the various transformations and jobs. 
    • For example parameter DATA_PATH_1 has default value /path/to/foo/ and can be changed at runtime
    • This makes it easy to transport the package to a different location
  • A description of the package including :
    • the reference to parent job or transformation
    • the command to execute the job inside the zip file
  • The content of the zip file can be unzipped

Where can I find it?

In version 3.2.0-RC1 or later:

  1. Under the File menu in Spoon version 3.2.0-RC1 or later. Choose the "Export all linked resources to XML" entry.
  2. In the Kitchen command line tool, with the "export" option

What's next?

Next, we're going to send the zip files over to the slave servers during execution.  This will require a few new servlets on the Carte server and is slated for 3.3.