1. Integrations Guide
  2. Integration with Apache Hadoop MapReduce
  3. Introduction to Integration with Apache Hadoop MapReduce
  4. Prerequisites

Prerequisites

To minimize the number of non-literal text and tokens that need to be replaced when running the examples that are presented, this document assumes that Apache Hadoop and Oracle NoSQL Database are installed on a Big Data Appliance running Big Data SQL 4.0. Specifically, this document assumes that Apache Hadoop is installed under the directory /opt/cloudera/parcels/CDH, and that Oracle NoSQL Database is installed under /opt/oracle/kv-ee. Thus, if you happen to be using commodity hardware rather than a Big Data Appliance, then you may need to substitute various directory paths presented in this document with values specific to the Apache Hadoop and Oracle NoSQL Database installations on your particular system.

Whether you are using a Big Data Appliance or commodity hardware, in order to work with the examples presented in this document, you will need to install the separate distribution containing the Oracle NoSQL Database Examples. Although you are free to install the example package in any location on your system, for simplicity, this document assumes the example code is installed under the directory /opt/oracle/nosql/apps/kv/examples .

Note:

The host names and ports provided below are for demonstration purpose only. You can provide the value as per the requirement.

Before attempting to execute the example that demonstrates the concepts presented in this document, you should first satisfy the following prerequisites:

  • Become familiar with Apache Hadoop and the MapReduce programming model. Specifically, become familiar with how to write and deploy a MapReduce job.
  • Deploy a Hadoop cluster with 3 DataNodes running on machines with sample host names, dn-host-1, dn-host-2, and dn-host-3.
  • Become familiar with Oracle NoSQL Database and then install, start, and configure an Oracle NoSQL Database that is network reachable from the nodes of the Hadoop cluster. The KVHOME of the store that you start should be configured as the directory /opt/oracle/kv-ee.
  • Deploy a store to 3 machines (real or virtual) with sample host names, kv-host-1, kv-host-2, and kv-host-3. The store's name should be set to the value example-store, and the store's KVROOT should be set to the directories /u01/nosql/sn1/kvroot on kv-host-1, /u02/nosql/sn2/kvroot on kv-host-2, and /u03/nosql/sn3/kvroot on kv-host-3. Finally, an Oracle NoSQL Database admin service, listening on port 5000, should be deployed to each host making up the store.
  • Become familiar with the Oracle NoSQL Database Security model and be able to configure the deployed store for secure access (optional). See Introducing Oracle NoSQL Database Security in the Security Guide.
  • If the deployed store is configured for secure access, start the Oracle NoSQL Database Administrative CLI and securely connect to the store's admin service. See Start the Administration CLI in the Administrator's Guide. Using the CLI, create a user in the store named example-user along with the appropriate security artifacts (login file, trust file, and either password file or Oracle Wallet [Enterprise Edition only]).
  • Obtain and install the separate distribution containing the Oracle NoSQL Database Examples. Although you are free to install that package in any location on your system, for simplicity this document assumes the example code is installed under the directory /opt/oracle/nosql/apps/kv/examples.
  • Be able to compile and execute a Java program and package it and any associated resources in a JAR file.
  • Install the Hadoop JAR files required to compile the example program so that they are available for inclusion in the example program's classpath (see below).

Using specific values for items such as the KVHOME and KVROOT environment variables, as well as the store name, host names, admin port, and example code location described above should allow you to more easily understand and use the example commands presented in this document. Combined with the information contained in the Concepts Guide, along with the Administrator's Guide and Security Guide, you should then be able to generalize and extend these examples to your particular development scenario; substituting the values specific to the given environment where necessary.

Detailed instructions for deploying a non-secure store are provided in the Deploying a Non-Secure Store appendix of this document. Similarly, the Deploying a Secure Store appendix provides instructions for deploying a store configured for security.