Deploying Tez on Big Data Service HA Clusters

Deploy Apache Tez on Big Data Service HA clusters.

  1. Configure Tomcat:Spark-Ranger Plugin Extension
    1. Download Tomcat from following link: https://tomcat.apache.org/download-90.cgi.
    2. Deploy Tez web server on the node where timeline server is running. The default timeline server node is mn1.
      sudo tar xf /tmp/apache-tomcat-9.0.70.tar.gz -C /opt/
      sudo ln -s /opt/apache-tomcat-9.0.70 /opt/tomcat
  2. Setup the Tez-UI.war file:
    1. Download Tez-ui-0.10.1.war from https://repo1.maven.org/maven2/org/apache/tez/tez-ui/0.10.1/.
    2. Unzip the war file and copy it to /opt/tomcat/webapps.
      sudo cp /tmp/tez-ui-0.10.1.war /opt/tez/
      sudo unzip /opt/tez/tez-ui-0.10.1.war -d /opt/tez/tez-ui
      sudo cp -R /opt/tez/tez-ui /opt/tomcat/webapps/
    3. Update /opt/tomcat/webapps/tez-ui/config/config.js with the hostname for both the timeline server and the resource manager:
      ENV = {
        hosts: {
      	timeline: "http://testcluster-mn1.testsubnet.testvcn.oraclevcn.com:8188",
      	rm: "http://testcluster-mn0.testsubnet.testvcn.oraclevcn.com:8088",
      }
      };
  3. Kinit with ~/spnego.service.keytab by copying krb5.conf, spnego.service.keytab for UI access. Configure the timeline server and Tez from Ambari Ui:
    1. Access Apache Ambari.
    2. From the side toolbar, under Services select Tez.
    3. Select Configs.
    4. On the Advanced tab, update the following:
      Yarn-site.xml:
      yarn.timeline-service.generic-application-history.enabled : true
      
      yarn.timeline-service.hostname : testcluster-mn1.testsubnet.testvcn.oraclevcn.com
      
      yarn.timeline-service.http-cross-origin.allowed-origins : *
      yarn.timeline-service.http-authentication.proxyuser.ambari-server.hosts=*
      yarn.timeline-service.http-authentication.proxyuser.ambari-server.users=*
      yarn.timeline-service.http-authentication.proxyuser.ambari-server.groups=*
      Tez-site.xml:
      tez.allow.disabled.timeline-domains : true
      tez.am.history.logging.enabled : true
      tez.dag.history.logging.enabled : true
      tez.history.logging.service.class : org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService 
      
      tez.tez-ui.history-url.base : http://testcluster-mn1.testsubnet.testvcn.oraclevcn.com:8080/tez-ui/
      
      yarn.timeline-service.enabled :  true
      
      tez.am.view-acls *
  4. Restart Yarn, Tez, and Hive from the Ambari Server.
  5. Restart Tomcat.
    [root@tezui2un0 apache-tomcat-9.0.70]# ./bin/shutdown.sh
    
    [root@tezui2un0 apache-tomcat-9.0.70]# ./bin/start.sh
  6. Run Kinit.
  7. Verify the Tez UI (http://daily-cluster-ha11-mn1.bmbdcsad1.bmbdcs.oraclevcn.com:8080/tez-ui/.