...
Pentaho MapReduce can be configured through the pentaho-mapreduceplugin.properties
found in the plugin's base directory, or overridden per Pentaho MapReduce job entry if they are defined in the User Defined properties tab.
...
- Unzip pentaho-mapreduce-libraries.zip, it contains a single lib/ directory with the required Kettle dependencies
- Copy additional libraries to the lib/ directory
- Zip up the lib/ directory into pentaho-mapreduce-libraries-custom.zip so the archive contains the lib/ with all jars within it (you may create subdirectories within lib/. All jars found in lib/ and its subdirectories will be added to the classpath of the executing job.)
- Update
pentaho-mapreduceplugin.properties
and update the following properties:Code Block pmr.kettle.installation.id=custom pmr.libraries.archive.file=pentaho-mapreduce-libraries-custom.zip
The next time you execute Pentaho MapReduce the custom Kettle environment will be copied into HDFS at pmr.kettle.dfs.install.dir/custom
and used when executing the job. You can switch between Kettle environments by specifying the pmr.kettle.installation.id
property as a User Defined property per Pentaho MapReduce job entry or globally in the pentaho-mapreduceplugin.properties
file*.
*Note: Only if the installation referenced by pmr.kettle.installation.id
does not exist will the archive file and additional plugins currently configured will be used to "install" it into HDFS.
...