Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

Additional Configuration for YARN Shims

  1. Set the active Hadoop distribution.
  2. Configure the cluster settings.
  3. Navigate to the folder that contains the shim, then open the yarn-site.xml file in a text editor.
  4. Check the value for the yarn.application.classpath. The yarn.application.classpath specifies the classpath to use on the cluster for executing YARN applications. If the paths for your environment are not listed, add them. Paths are separated by a comma.
  5. Check the value for the yarn.resourcemanager.hostname.
    For CDH 5, the default is clouderamanager.cdh5.test. 
    For HDP 2, the default value is sandbox.hortonworks.com.
    * For all YARN shims: Since your hostname will probably be different, make the change to the value.  <value>yarncluster.mycompany.com</value>
  6. For HDP 2, check the value for the yarn.resourcemanager.address and change the port to match your environment if necessary like this:  <value>${yarn.resourcemanager.hostname}:9000</value>
  7. If you are using CDH 5.x, there are a few more things that you need to do## Navigate to the folder that contains the shim, then open the hive-site.xml file in a text editor.## Modify the hive.metastore.uris property so that it points to the location of your hive metastore.## Save and close the hive-site.xml file.## Navigate to the folder that contains the shim, then open the mapred-site.xml file, in a text editor.## Modify the mapreduce.jobhistory.address property so that it points to the place where the job history logs are stored.

Note: Not all shim properties can be set in the Spoon user interface, nor are instructions for modifying them listed here. If you need to set additional properties that are not addressed, you will need to set them manually in the *-site.xml files that are in the shim directory. Consult your Hadoop distribution's vendor for details about the properties you want to set.

  • No labels