Additional Configuration for YARN Shims
Prerequisite: Make sure that your client has been configured so it can access any node in your cluster.
- Set the active Hadoop distribution.
- Configure the cluster settings.
- Navigate to the folder that contains the shim, then open the yarn-site.xml file in a text editor. Check the value for the yarn.application.classpath. The Adjust the following parameters as needed.
Parameter | Values |
|---|---|
yarn.application.classpath |
...
|
...
|
...
yarn. |
...
resourcemanger.hostname |
...
...
|
...
|
...
|
...
...
|
...
|
...
|
...
|
yarn.resourcemanager.address |
...
|
...
|
yarn.resourcemanager.admin. |
...
address |
|
CDH 5.x Shim
If you are using CDH 5.x, there are a few more things that you need to do:
- (CDH 5.1 and 5.2 only): * Navigate to the folder that contains the shim, then open the hive-site.xml file in a text editor. Modify the hive.metastore.uris property so that it points to the location of your hive metastore. Save and close the hive-site.xml file.
- Navigate to the folder that contains the shim, then open the mapred-site.xml file, in a text editor. Modify the Then make the changes in the table.
Parameter | Value |
|---|---|
mapreduce.jobhistory.address |
...
Set this to the place where |
...
job history logs are stored. | |
mapreduce.app-submission.cross-platform | Add this parameter to the mapred-site.xml file between the <property> tags, then set it to true, like this: |
Note: Not all shim properties can be set in the Spoon user interface, nor are instructions for modifying them listed here. If you need to set additional properties that are not addressed in these instructions, you will need to set them manually in the *-site.xml files that are in the shim directory. Consult your Hadoop distribution's vendor for details about the properties you want to set.