Configurando o Apache Hue

Observação

Ignore esta etapa para clusters do Big Data Service com a versão 3.0.7 ou mais recente, uma vez que o Apache Hue já está instalado e configurado nos nós do cluster.

Esta seção detalhes do procedimento que você segue para configurar o Apache Hue para um cluster inseguro ou seguro.

Configurando o Apache Hue para um Cluster Não Seguro

  1. Crie um usuário hue para o Apache Hue executando os seguintes comandos:
    sudo useradd -u 54336 hue
    sudo chown -R hue:hue /usr/local/hue
    sudo su - hdfs -c "hdfs dfs -mkdir /user/hue"
    sudo su - hdfs -c "hdfs dfs -chown hue:hadoop /user/hue"
    sudo su - hdfs -c "hdfs dfs -chmod 755 /user/hue"
    sudo mysql -u root
  2. Execute as seguintes instruções na interface MySQL:
    create database desktop;
    grant all privileges on *.* to 'hue'@'localhost ';
    grant all on hue.* to 'hue'@'localhost';
    alter user 'hue'@'localhost' identified by 'secretpassword';
    flush privileges;
    exit
  3. Edite o arquivo hue.ini e atualize as seções dos serviços que você deseja usar com o Hue.
    sudo vim /usr/local/hue/desktop/conf/hue.ini
    • MySQL
      [[database]]
      # Database engine is typically one of:
      # postgresql_psycopg2, mysql, sqlite3 or oracle.
      
      engine=mysql
      host=localhost
      port=3306
      user=hue
      password=secretpassword
      name=desktop
      
    • HDFS
      [[hdfs_clusters]]
      	[[[default]]]
      	# Enter the filesystem uri
      	fs_defaultfs=hdfs://<master_node_fully_qualified_hostname>:8020
      
      	# Use WebHdfs/HttpFs as the communication mechanism.
      	# Domain should be the NameNode or HttpFs host.
      	# Default port is 14000 for HttpFs.
      
           webhdfs_url=http://<master_node_fully_qualified_hostname>:50070/webhdfs/v1
      
    • YARN
      [[yarn_clusters]]
      
          [[[default]]]
           # Enter the host on which you are running the ResourceManager
           resourcemanager_host=<master_node_fully_qualified_hostname>    
      
           # The port where the ResourceManager IPC listens on
           resourcemanager_port=8050
      
           # URL of the ResourceManager API
           resourcemanager_api_url=http://<master_node_fully_qualified_hostname>:8088
      
           # Whether to submit jobs to this cluster
           submit_to=True
      
           # URL of the ProxyServer API
           proxy_api_url=http://<master_node_fully_qualified_hostname>:8088
      
           # URL of the HistoryServer API
           history_server_api_url=http://<utility_node_fully_qualified_hostname>:19888
      
           # URL of the spark history server
           spark_history_server_url=http://<master_node_fully_qualified_hostname>:18081
      
    • Oozie
      [liboozie]
      # The URL where the Oozie service runs on.
      oozie_url=http://<utility_node_fully_qualified_hostname>:11000/oozie
      
    • Hive
      [beeswax]
      
      # Host where HiveServer2 is running.
      hive_server_host=<utility_node_fully_qualified_hostname>
      
      # Http thrift port for HiveServer2.
      hive_server_http_port=10002
      
      # Use SASL framework to establish connection to host.
      use_sasl=true
      
    • HBase
      [hbase]
      
      # Comma-separated list of HBase Thrift server 1 for clusters in the format of '(name|host:port)'.
      hbase_clusters=(Cluster|<master_node_fully_qualified_hostname>:9090)
      
    • A lista de serviços que não devem ser carregados na inicialização do servidor
      [desktop]
      
      # Comma separated list of apps to not load at server startup.
      # e.g.: pig,zookeeper
      app_blacklist=impala,security
      
  4. Abra um novo arquivo, nomeie-o como pseudo-distributed.ini e, em seguida, adicione as seguintes seções:
    sudo vim /usr/local/hue/desktop/conf/pseudo-distributed.ini
    [beeswax]
    
    max_number_of_sessions=-1
    [[hdfs_clusters]]
        # HA support by using HttpFs
    
        [[[default]]]
          # Enter the filesystem uri
          fs_defaultfs=hdfs://<master_node_fully_qualified _hostname>:8020
    
          webhdfs_url=http://<master_node_fully_qualified_hostname>:50070/webhdfs/v1
    [[yarn_clusters]]
    
    resourcemanager_api_url=http://<master_node_fully_qualified_hostname>:8088
    8088
  5. Atualize as permissões do Hive.
    sudo chmod -R 777 /etc/hive
    sudo su - hue
    cd /usr/local/hue/build/env/bin/
    ./hue syncdb --noinput
    ./hue migrate
    nohup ./hue runcpserver &
    

Configurar o Apache Hue para um cluster seguro

  1. Crie um usuário hue para o Apache Hue executando os seguintes comandos:
    sudo useradd -u 54336 hue
    sudo chown -R hue:hue /usr/local/hue
    sudo su - hdfs -c "hdfs dfs -mkdir /user/hue"
    sudo su - hdfs -c "hdfs dfs -chown hue:hadoop /user/hue"
    sudo su - hdfs -c "hdfs dfs -chmod 755 /user/hue"
    sudo mysql -u root
  2. Execute as seguintes instruções na interface MySQL:
    create database desktop;
    grant all privileges on *.* to 'hue'@'localhost ';
    grant all on hue.* to 'hue'@'localhost';
    alter user 'hue'@'localhost' identified by 'secretpassword';
    flush privileges;
    exit
  3. Estabeleça SSH no nó principal usando o IP público e a chave privada associados ao cluster.
    ssh opc@<masternode0_hostname/private_ip>
  4. Faça log-on em kadmin.
    sudo kadmin.local
  5. Crie um principal para o servidor Hue e substitua o FQDN pelo nome de domínio totalmente qualificado do nó do utilitário em que o Hue está instalado.
    addprinc -randkey -maxrenewlife 90day +allow_renewable hue/<FQDN>
    xst -k hue.service.keytab hue/<FQDN>@BDACLOUDSERVICE.ORACLE.COM
    exit
  6. Atualize as permissões para ativar a cópia do arquivo keytab no nó do utilitário.
    sudo chmod 777 hue.service.keytab
    exit
  7. Copie o arquivo keytab e defina as permissões corretas e a propriedade.
    sudo scp opc@<master_node0_hostname>:~/hue.service.keytab
    /etc/security/keytabs/hue.service.keytab
    sudo chown hue:hadoop /etc/security/keytabs/hue.service.keytab
    sudo chmod 600 /etc/security/keytabs/hue.service.keytab
    
  8. Certifique-se de que o keytab esteja acessível testando kinit e copie as credenciais armazenadas no cache para configurar o Kerberos na configuração do hue posteriormente.
    sudo su - hue
    kinit -k -t /etc/security/keytabs/hue.service.keytab hue/<FQDN>@BDACLOUDSERVICE.ORACLE.COM
    exit
    
  9. Edite o arquivo hue.ini e atualize as seções para os serviços que você deseja usar com Hue.
    sudo vim /usr/local/hue/desktop/conf/hue.ini
    • MySQL
      [[database]]
      # Database engine is typically one of:
      # postgresql_psycopg2, mysql, sqlite3 or oracle.
      
      engine=mysql
      host=localhost
      port=3306
      user=hue
      password=secretpassword
      name=desktop
      
    • HDFS
      [[hdfs_clusters]]
      	[[[default]]]
      	# Enter the filesystem uri
      	fs_defaultfs=hdfs://<master_node_0_fully_qualified_hostname>:8020
      
      	# Use WebHdfs/HttpFs as the communication mechanism.
      	# Domain should be the NameNode or HttpFs host.
      	# Default port is 14000 for HttpFs.
      
           webhdfs_url=http://<master_node_0_fully_qualified_hostname>:50070/webhdfs/v1
      
        # Change this if your YARN cluster is Kerberos-secured
        security_enabled=true
      
    • YARN
      [[yarn_clusters]]
      
          [[[default]]]
           # Enter the host on which you are running the ResourceManager
           resourcemanager_host=<master_node_0_fully_qualified_hostname>     
           # URL of the ResourceManager API
           resourcemanager_api_url=http://<master_node_0_fully_qualified_hostname>:8088
      
           # Whether to submit jobs to this cluster
           submit_to=True
      
           # URL of the ProxyServer API
           proxy_api_url=http://<master_node_0_fully_qualified_hostname>:8088
      
           # URL of the HistoryServer API
           history_server_api_url=http://<master_node_1_hostname>:19888
      
           # URL of the spark history server
           spark_history_server_url=http://<master_node_fully_qualified_hostname>:18081
      
      
      	   # Change this if your YARN cluster is Kerberos-secured
           security_enabled=true
      
    • Oozie
      [liboozie]
      # The URL where the Oozie service runs on.
      oozie_url=http://<utility_node_fully_qualified_hostname>:11000/oozie
      
      # Change this if your YARN cluster is Kerberos-secured
      security_enabled=true
      
    • Hive
      [beeswax]
      
      # Host where HiveServer2 is running.
      hive_server_host=<utility_node_fully_qualified_hostname>
      
      # Http thrift port for HiveServer2.
      hive_server_http_port=10002
      
      # Use SASL framework to establish connection to host.
      use_sasl=true
      
    • HBase
      [hbase]
      
      # Comma-separated list of HBase Thrift server 1 for clusters in the format of '(name|host:port)'.
      hbase_clusters=(Cluster|<master_node_0_fully_qualified_hostname>:9090)
      
    • A lista de serviços que não devem ser carregados na inicialização do servidor
      [desktop]
      
      # Comma separated list of apps to not load at server startup.
      # e.g.: pig,zookeeper
      app_blacklist=impala,security
      
    • Definições do Kerberos
      [[kerberos]]
        # Path to Hue's Kerberos keytab file
      	hue_keytab=/etc/security/keytabs/hue.service.keytab
      	# Kerberos principal name for Hue
      	hue_principal=hue/<utility_node_hostname>@BDACLOUDSERVICE.ORACLE.COM
        kinit_path=/usr/bin/kinit
        ccache_path=/tmp/krb5cc_54366
      
  10. Abra um novo arquivo, nomeie-o como pseudo-distributed.ini e, em seguida, adicione as seguintes seções:
    sudo vim /usr/local/hue/desktop/conf/pseudo-distributed.ini
    
    [beeswax]
    
    max_number_of_sessions=-1
    [[hdfs_clusters]]
        # HA support by using HttpFs
    
        [[[default]]]
          # Enter the filesystem uri
          fs_defaultfs=hdfs://<master_node_fully_qualified_hostname>:8020
    
          webhdfs_url=http://<master_node_fully_qualified_hostname>:50070/webhdfs/v1
    [[yarn_clusters]]
    
    resourcemanager_api_url=http://<master_node_fully_qualified_hostname>:8088
    
  11. Atualize as permissões do Hive.
  12. Na Interface do Usuário do Ambari, em HDFS, vá para Resumo. Verifique se o host mn0 está definido como Ativo na seção no lado direito da página. Caso contrário, reinicie o nó mn1 na console do cluster do Big Data Service.
  13. Execute os seguintes comandos para iniciar o Apache Hue:
    sudo chmod -R 777 /etc/hive
    sudo su - hue
    cd /usr/local/hue/build/env/bin/
    ./hue syncdb --noinput
    ./hue migrate
    nohup ./hue runcpserver &
    crontab -e
    0 */12 * * * cd /usr/local/hue/build/env/bin ; ./hue kt_renewer &
  14. Salve e saia com o comando :wq.