PostgreSQL Bulk Loader
Description
The PostgreSQL bulk loader is an experimental step in which we will to stream data from inside Kettle to the psql command using "COPY DATA FROM STDIN" into the database.
This way of loading data offers the best of both worlds : the performance of a bulk load and the flexibility of a Pentaho Data Integration transformation.
Make sure to check out the "#Set up authentication" section below!
Note: This step does not work with a JNDI defined connection, only JDBC is supported.
Note: This step does not support timestamps at the moment (5.3). Timestamps should be converted to Date before this step. Using timestamps results in null-values in the table.
Options
Option |
Description |
---|---|
Step name |
Name of the step.
|
Connection |
Name of the database connection on which the target table resides. |
Target schema |
The name of the Schema for the table to write data to. This is important for data sources that allow for table names with dots '.' in it. |
Target table |
Name of the target table. |
psql path |
Full path to the psql utility. |
Load action |
Insert, Truncate. Insert inserts, truncate first truncates the table.
|
|
|
Fields to load |
This table contains a list of fields to load data from, properties include:
|
Metadata Injection Support
All fields of this step support metadata injection. You can use this step with ETL Metadata Injection to pass metadata to your transformation at runtime.
Set Up Authentication
"psql" doesn't allow you to specify the password. Here is a part of the connection options: