Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Additional Configuration for YARN Shims

  1. Set the active Hadoop distribution.
  2. Configure the cluster settings.
  3. Navigate to the folder that contains the shim, then open the yarn-site.xml file in a text editor.
  4. Check the value for the yarn.application.classpath. The yarn.application.classpath specifies the classpath to use on the cluster for executing YARN applications. If the paths for your environment are not listed, add them. Paths are separated by a comma.
  5. Check the value for the yarn.resourcemanager.hostname.
    On CDH 5 the default is clouderamanager.cdh5.test. 
    For HDP 2, the default value is sandbox.hortonworks.com.
    Since your hostname will probably be different, make the change to the value.  <value>yarncluster.mycompany.com</value>
  6. Check the value for the yarn.resourcemanager.address and change the port to match your environment if necessary like this:  <value>${yarn.resourcemanager.hostname}:9000</value>
  7. If you are using CDH 5.x, there are a few more things that you need to do## Navigate to the folder that contains the shim, then open the hive-site.xml file in a text editor.## Modify the hive.metastore.uris property so that it points to the location of your hive metastore.## Save and close the hive-site.xml file.## Navigate to the folder that contains the shim, then open the mapred-site.xml file, in a text editor.## Modify the mapreduce.jobhistory.address property so that it points to the place where the job history logs are stored.

...