Using Apache Hadoop
Apache Hadoop is an open source distributed processing framework that processes and stores large amounts of data for big data applications in scalable clusters.
Hadoop Configuration Properties
Hadoop configuration properties included in Big Data Service 3.1.1 or later.
| Configuration | Property | Description |
|---|---|---|
hadoop-env |
hadoop_extra_opts |
Extra Java runtime options |
hadoop_log_opts |
Hadoop logging options | |
hadoop_jobtracker_opts |
Hadoop Job tracker Java options | |
hadoop_tasktracker_opts |
Hadoop Task tracker Java options | |
hdfs_namenode_opts_shared |
Shared Java options for primary and secondary HDFS Namenodes | |
hdfs_namenode_pre8_opts_shared |
Shared Java options for primary and secondary HDFS Namenodes if java_version < 8 | |
hdfs_namenode_opts |
Java options for HDFS Namenode | |
hdfs_datanode_opts |
Java options for HDFS Datanode | |
hdfs_datanode_pre8_opts |
Java options for HDFS Datanode if java_version < 8 | |
hdfs_secondary_namenode_opts |
Java options for HDFS secondary Namenode | |
hadoop_client_opts |
Hadoop client Java options | |
hadoop_client_pre8_opts |
Hadoop client options if java_version < 8 | |
hdfs_namenode_secure_opts |
Java options for HDFS Namenode if security is enabled | |
hdfs_secondary_namenode_secure_opts |
Java options for HDFS secondary Namenode if security is enabled | |
hdfs_datanode_secure_opts |
Java options for HDFS Datanode if security is enabled | |
hdfs_journalnode_secure_opts |
Java options for HDFS Journalnode if security is enabled | |
hdfs_nfs3_opts |
Java options for HDFS NFS Gateway | |
hadoop_balancer_opts |
Hadoop Balancer Java options | |
hadoop_ssh_opts |
Extra Hadoop SSH options | |
hadoop_yarn_resourcemanager_opts |
Hadoop Yarn Resource-Manager Java options | |
hadoop_zk_principal_opts |
Hadoop Java options if ZooKeeper principal user is defined | |
hadoop_zk_ssl_params |
ZooKeeper SSL parameters for Hadoop | |
hdfs_namenode_remotejmx_opts |
Java options for HDFS Namenode if remote JMX is enabled | |
hdfs_secondary_namenode_remotejmx_opts |
Java options for HDFS secondary Namenode if remote JMX is enabled | |
hdfs_datanode_remotejmx_opts |
Java options for HDFS Datanode if remote JMX is enabled | |
hdfs_journalnode_remotejmx_opts |
Java options for HDFS Journalnode if remote JMX is enabled | |
hdfs_zkfc_remotejmx_opts |
Java options for HDFS ZooKeeper Failover Controller if remote JMX is enabled | |
hadoop_classpath |
Hadoop classpath containing custom JARs or directories |