Oracle Cloud Infrastructure Documentation

Configuring Apache Hue

Note

Skip this step for Big Data Service clusters with version 3.0.7 or later as Apache Hue is already installed and configured in the cluster nodes.

This section details the procedure that you follow to configure Apache Hue for a nonsecure or a secure cluster.

Configuring Apache Hue for a Nonsecure Cluster

  1. Create a user hue for Apache Hue by running the following commands:
    sudo useradd -u 54336 hue
sudo chown -R hue:hue /usr/local/hue
sudo su - hdfs -c "hdfs dfs -mkdir /user/hue"
sudo su - hdfs -c "hdfs dfs -chown hue:hadoop /user/hue"
sudo su - hdfs -c "hdfs dfs -chmod 755 /user/hue"
sudo mysql -u root
  2. Run the following statements in the MySQL interface:
    create database desktop;
grant all privileges on *.* to 'hue'@'localhost ';
grant all on hue.* to 'hue'@'localhost';
alter user 'hue'@'localhost' identified by 'secretpassword';
flush privileges;
exit
  3. Edit the hue.ini file and update the sections for the services that you want to use with Hue.
    sudo vim /usr/local/hue/desktop/conf/hue.ini
    • MySQL
      [[database]]
# Database engine is typically one of:
# postgresql_psycopg2, mysql, sqlite3 or oracle.

engine=mysql
host=localhost
port=3306
user=hue
password=secretpassword
name=desktop
    • HDFS
      [[hdfs_clusters]]
	[[[default]]]
	# Enter the filesystem uri
	fs_defaultfs=hdfs://<master_node_fully_qualified_hostname>:8020

	# Use WebHdfs/HttpFs as the communication mechanism.
	# Domain should be the NameNode or HttpFs host.
	# Default port is 14000 for HttpFs.

     webhdfs_url=http://<master_node_fully_qualified_hostname>:50070/webhdfs/v1
    • YARN
      [[yarn_clusters]]

    [[[default]]]
     # Enter the host on which you are running the ResourceManager
     resourcemanager_host=<master_node_fully_qualified_hostname>    

     # The port where the ResourceManager IPC listens on
     resourcemanager_port=8050

     # URL of the ResourceManager API
     resourcemanager_api_url=http://<master_node_fully_qualified_hostname>:8088

     # Whether to submit jobs to this cluster
     submit_to=True

     # URL of the ProxyServer API
     proxy_api_url=http://<master_node_fully_qualified_hostname>:8088

     # URL of the HistoryServer API
     history_server_api_url=http://<utility_node_fully_qualified_hostname>:19888

     # URL of the spark history server
     spark_history_server_url=http://<master_node_fully_qualified_hostname>:18081
    • Oozie
      [liboozie]
# The URL where the Oozie service runs on.
oozie_url=http://<utility_node_fully_qualified_hostname>:11000/oozie
    • Hive
      [beeswax]

# Host where HiveServer2 is running.
hive_server_host=<utility_node_fully_qualified_hostname>

# Http thrift port for HiveServer2.
hive_server_http_port=10002

# Use SASL framework to establish connection to host.
use_sasl=true
    • HBase
      [hbase]

# Comma-separated list of HBase Thrift server 1 for clusters in the format of '(name|host:port)'.
hbase_clusters=(Cluster|<master_node_fully_qualified_hostname>:9090)
    • The list of services that must not be loaded at server startup
      [desktop]

# Comma separated list of apps to not load at server startup.
# e.g.: pig,zookeeper
app_blacklist=impala,security
  4. Open a new file, name it pseudo-distributed.ini, and then add the following sections:
    sudo vim /usr/local/hue/desktop/conf/pseudo-distributed.ini
[beeswax]

max_number_of_sessions=-1
[[hdfs_clusters]]
    # HA support by using HttpFs

    [[[default]]]
      # Enter the filesystem uri
      fs_defaultfs=hdfs://<master_node_fully_qualified _hostname>:8020

      webhdfs_url=http://<master_node_fully_qualified_hostname>:50070/webhdfs/v1
[[yarn_clusters]]

resourcemanager_api_url=http://<master_node_fully_qualified_hostname>:8088
8088
  5. Update the permissions for Hive. 
    sudo chmod -R 777 /etc/hive
sudo su - hue
cd /usr/local/hue/build/env/bin/
./hue syncdb --noinput
./hue migrate
nohup ./hue runcpserver &

Configure Apache Hue for a secure cluster

  1. Create a user hue for Apache Hue by running the following commands:
    sudo useradd -u 54336 hue
sudo chown -R hue:hue /usr/local/hue
sudo su - hdfs -c "hdfs dfs -mkdir /user/hue"
sudo su - hdfs -c "hdfs dfs -chown hue:hadoop /user/hue"
sudo su - hdfs -c "hdfs dfs -chmod 755 /user/hue"
sudo mysql -u root
  2. Run the following statements in the MySQL interface:
    create database desktop;
grant all privileges on *.* to 'hue'@'localhost ';
grant all on hue.* to 'hue'@'localhost';
alter user 'hue'@'localhost' identified by 'secretpassword';
flush privileges;
exit
  3. SSH to master node using the public IP and private key associated with the cluster. 
    ssh opc@<masternode0_hostname/private_ip>
  4. Log on to kadmin.
    sudo kadmin.local
  5. Create a principal for the Hue server and replace FQDN with the fully qualified domain name of the utility node where Hue is installed. 
    addprinc -randkey -maxrenewlife 90day +allow_renewable hue/<FQDN>
xst -k hue.service.keytab hue/<FQDN>@BDACLOUDSERVICE.ORACLE.COM
exit
  6. Update the permissions to enable copy keytab to utility node.
    sudo chmod 777 hue.service.keytab
exit
  7. Copy the keytab file and set proper permissions and ownership.
    sudo scp opc@<master_node0_hostname>:~/hue.service.keytab
/etc/security/keytabs/hue.service.keytab
sudo chown hue:hadoop /etc/security/keytabs/hue.service.keytab
sudo chmod 600 /etc/security/keytabs/hue.service.keytab
  8. Ensure that the keytab is accessible by testing kinit and copy the cached credentials to set up Kerberos in hue config later.
    sudo su - hue
kinit -k -t /etc/security/keytabs/hue.service.keytab hue/<FQDN>@BDACLOUDSERVICE.ORACLE.COM
exit
  9. Edit the hue.ini file and update the sections for the services that you want to use with Hue.
    sudo vim /usr/local/hue/desktop/conf/hue.ini
    • MySQL
      [[database]]
# Database engine is typically one of:
# postgresql_psycopg2, mysql, sqlite3 or oracle.

engine=mysql
host=localhost
port=3306
user=hue
password=secretpassword
name=desktop
    • HDFS
      [[hdfs_clusters]]
	[[[default]]]
	# Enter the filesystem uri
	fs_defaultfs=hdfs://<master_node_0_fully_qualified_hostname>:8020

	# Use WebHdfs/HttpFs as the communication mechanism.
	# Domain should be the NameNode or HttpFs host.
	# Default port is 14000 for HttpFs.

     webhdfs_url=http://<master_node_0_fully_qualified_hostname>:50070/webhdfs/v1

  # Change this if your YARN cluster is Kerberos-secured
  security_enabled=true
    • YARN
      [[yarn_clusters]]

    [[[default]]]
     # Enter the host on which you are running the ResourceManager
     resourcemanager_host=<master_node_0_fully_qualified_hostname>     
     # URL of the ResourceManager API
     resourcemanager_api_url=http://<master_node_0_fully_qualified_hostname>:8088

     # Whether to submit jobs to this cluster
     submit_to=True

     # URL of the ProxyServer API
     proxy_api_url=http://<master_node_0_fully_qualified_hostname>:8088

     # URL of the HistoryServer API
     history_server_api_url=http://<master_node_1_hostname>:19888

     # URL of the spark history server
     spark_history_server_url=http://<master_node_fully_qualified_hostname>:18081


	   # Change this if your YARN cluster is Kerberos-secured
     security_enabled=true
    • Oozie
      [liboozie]
# The URL where the Oozie service runs on.
oozie_url=http://<utility_node_fully_qualified_hostname>:11000/oozie

# Change this if your YARN cluster is Kerberos-secured
security_enabled=true
    • Hive
      [beeswax]

# Host where HiveServer2 is running.
hive_server_host=<utility_node_fully_qualified_hostname>

# Http thrift port for HiveServer2.
hive_server_http_port=10002

# Use SASL framework to establish connection to host.
use_sasl=true
    • HBase
      [hbase]

# Comma-separated list of HBase Thrift server 1 for clusters in the format of '(name|host:port)'.
hbase_clusters=(Cluster|<master_node_0_fully_qualified_hostname>:9090)
    • The list of services that must not be loaded at server startup
      [desktop]

# Comma separated list of apps to not load at server startup.
# e.g.: pig,zookeeper
app_blacklist=impala,security
    • Kerberos settings
      [[kerberos]]
  # Path to Hue's Kerberos keytab file
	hue_keytab=/etc/security/keytabs/hue.service.keytab
	# Kerberos principal name for Hue
	hue_principal=hue/<utility_node_hostname>@BDACLOUDSERVICE.ORACLE.COM
  kinit_path=/usr/bin/kinit
  ccache_path=/tmp/krb5cc_54366
  10. Open a new file, name it pseudo-distributed.ini, and then add the following sections:
    sudo vim /usr/local/hue/desktop/conf/pseudo-distributed.ini

[beeswax]

max_number_of_sessions=-1
[[hdfs_clusters]]
    # HA support by using HttpFs

    [[[default]]]
      # Enter the filesystem uri
      fs_defaultfs=hdfs://<master_node_fully_qualified_hostname>:8020

      webhdfs_url=http://<master_node_fully_qualified_hostname>:50070/webhdfs/v1
[[yarn_clusters]]

resourcemanager_api_url=http://<master_node_fully_qualified_hostname>:8088
  11. Update the permissions for Hive.
  12. On the Ambari UI, under HDFS, go to Summary. Ensure the mn0 host is set as Active in the section on the right side of the page. If not, restart the mn1 node from Big Data Service cluster console.
  13. Run the following commands to start Apache Hue:
    sudo chmod -R 777 /etc/hive
sudo su - hue
cd /usr/local/hue/build/env/bin/
./hue syncdb --noinput
./hue migrate
nohup ./hue runcpserver &
crontab -e
0 */12 * * * cd /usr/local/hue/build/env/bin ; ./hue kt_renewer &
  14. Save and exit with the :wq command.