Pentaho Data IntegrationImport User Documentation

Import User Documentation

Owned by Former user (Deleted)
Last updated: Jan 25, 2012 by Former user (Deleted)Version comment

Import User Documentation

The "Import" tool provides a command line interface for the import of transformations and jobs into a repository.

This utility was introduced in PDI version 4.2.0.

See also the concept: Lifecycle Management

Options

  • -rep : The name of the repository to import into
  • -user : The user name for the repository
  • -pass : The password for the repository
  • -dir : The repository target directory
  • -limitdir : An optional list of comma separated source directories to limit the import to
  • -file : The path to the export file that needs to be imported (optional, you can also specify a list of files after the other options)
  • -filedir :The directory in which the to be imported files live.  A list of files can be specified after the options.  This directory is put in front of the filename path (optional)
  • -rules : The path to the rules file that you want to use during the import
  • -norules : Do not use a rules file (Y/N), default to N
  • -comment : The comment that will be set for the new revisions of the transformations and jobs caused by the import.
  • -replace : Set to Y to replace existing transformations and jobs in the repository, defaults to N
  • -coe : Continue on error.  Warning, use with caution as this will ignore validation errors!
  • -version : Show the version, revision and build date

As mentioned you can specify a list of filenames to be imported after the specification of these options.  The files can be any file type that has transformations or jobs in them.  In other words, any XML file with <transformation> and <job> entries in them.

Return codes

Import returns the following error codes:

  • 0 : The import ran without a problem.
  • 1 : a) Some needed arguments or repository information are missing or b) import was not able to connect to the repository connection or c) target repository directory is not existing
  • 2 : Unexpected error during repository import (see also PDI-7296)
  • 6 : When called with option -version
  • 7 : Exception occurred during rule loading
  • 9 : Command line usage printing

Examples

Import a single transformation into the repository:

sh import.sh -rep=RepName -user=joe -pass=password -dir=/ktr -norules "samples/transformations/Getting Started Transformation.ktr"

Import all transformations in a directory:

sh import.sh -rep=RepName -user=joe -pass=password -dir=/ktr -norules -filedir=/foo/ $( ls /foo/*.ktr )

Import a repository export with a rules file:

sh import.sh -rep=PRODUCTION -user=joe -pass=password -dir=/import -rules=prod-rules.xml \
             -file=UAT-export-20110616.xml -coe=false -replace=true \
             -comment="New version upload from UAT"

Rules

Beyond the general rules described above you have to provide an "import rules" file to the import utility.  The rules allow an administrator to make sure that the quality of the imported repository objects adhere to a certain standard.  Those among you that find the built-in rules to be insufficient can write and deploy new rules as plug-ins.  For more information see the PDI SDK.

To be valid for the Import utility, the XML described in the rules below need to be placed in an XML document with a parent <rules> element.

A transformation requires a description

Description: This rule verifies that all transformations have a description with a certain specified minimum length.

Rules file XML:

<rule>
        <id>TransformationHasDescription</id>
        <enabled>true</enabled>
        <min_length>20</min_length>
 </rule>

A job requires a description

Description: This rule verifies that all jobs have a description with a certain specified minimum length.

Rules file XML:

<rule>
        <id>JobHasDescription</id>
        <enabled>true</enabled>
        <min_length>20</min_length>
 </rule>

A transformation has to log to a transformation log table

Description: Verifies that each imported transformation has configured the transformation logging table appropriately.  It can also validate that the schema name, table name and connection name contain a certain standard value.

Rules file XML:

<rule>  
   <id>TransformationHasTransLogConfigured</id>
   <enabled>true</enabled>
   <schema_name/>
   <table_name/>
   <connection_name/>
 </rule>

A job has to log to a job log table

Description: Verifies that each imported job has configured the job logging table appropriately.  It can also validate that the schema name, table name and connection name contain a certain standard value.

Rules file XML:

<rule>
   <id>JobHasJobLogConfigured</id>
   <enabled>true</enabled>
   <schema_name/>
   <table_name/>
   <connection_name/>
 </rule>

Validate the configuration of the specified database

Description: Verifies that a database is configured according to the specification given.

Rules file XML: (copy the connection XML from the right click database connection menu in Spoon)

<rule>
    <id>DatabaseConfiguration</id>
    <enabled>true</enabled>
    <connection>
      <name>MySQL</name>
      <server>localhost</server>
      <type>MYSQL</type>
      <access>Native</access>
      <database>test</database>
      <username>foo</username>
      <password>Encrypted 2be98afc86aa7f2e4cb79ce10df90acde</password>
      <attributes>
        <attribute><code>PORT_NUMBER</code><attribute>3306</attribute></attribute>
      </attributes>
    </connection>
 </rule>

Transformation has no disabled hops

Description: Use this rule to make sure that there are no accidentally disabled hops in your transformations.

Rules file XML:

<rule>
    <id>TransformationHasNoDisabledHops</id>
    <enabled>true</enabled>
 </rule>

Job has no disabled hops

Description: Use this rule to make sure that there are no accidentally disabled hops in your jobs.

Rules file XML:

<rule>
    <id>JobHasNoDisabledHops</id>
    <enabled>true</enabled>
 </rule>

A transformation must have a note

Description: This rule makes it mandatory for a transformation to have at least one note for documentation purposes.

Rules file XML:

<rule>
    <id>TransformationHasANote<id>
    <enabled>true</enabled>
 </rule>

A job must have a note

Description: This rule makes it mandatory for a job to have at least one note for documentation purposes.

Rules file XML:

<rule>
    <id>JobHasANote</id>
    <enabled>true</enabled>
 </rule>