Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Additional Configuration for YARN Shims

Prerequisite: Make sure that your client has been configured so it can access any node in your cluster.

  1. Set the active Hadoop distribution.
  2. Configure the cluster settings.
  3. Navigate to the folder that contains the shim, then open the yarn-site.xml file in a text editor.  Adjust the following parameters as needed.

Parameter

Values

yarn.application.classpath

  • All shims: Classpaths needed to execute YARN applications.  Separate paths with a comma.

yarn.resourcemanger.hostname

  • CDH 5.x - Update the hostname in your environment or use the default: clouderamanager.cdh5.test
  • HDP 2.x - Update the hostname in your environment or use the default:  sandbox.hortonworks.com
  • All other shims: Hostname in your environment.

yarn.resourcemanager.address

  • All shims: Update hostname and port to match your environment.

yarn.resourcemanager.admin.address

  • All shims: Update hostname and port to match your environment.

CDH 5.x Shim

If you are using CDH 5.x, there are a few more things that you need to do:

  1. (CDH 5.1 and 5.2 only): * Navigate to the folder that contains the shim, then open the hive-site.xml file in a text editor. Modify the hive.metastore.uris property so that it points to the location of your hive metastore. Save and close the hive-site.xml file.
  2. Navigate to the folder that contains the shim, then open the mapred-site.xml file, in a text editor. Then make the changes in the table.

Parameter

Value

mapreduce.jobhistory.address

Set this to the place where job history logs are stored.

mapreduce.app-submission.cross-platform

Add this parameter to the mapred-site.xml file between the <property> tags, then set it to true, like this:

<name>mapreduce.app-submission.cross-platform</name>
<value>true</value>

This property allows mapreduce jobs to run on windows and linux platforms, and vice versa.

Note: Not all shim properties can be set in the Spoon user interface, nor are instructions for modifying them listed here. If you need to set additional properties that are not addressed in these instructions, you will need to set them manually in the *-site.xml files that are in the shim directory. Consult your Hadoop distribution's vendor for details about the properties you want to set.

  • No labels