1. Administration and Configuration Guide
  2. Data Management Tools (DMT) Module Configurations
  3. Sqoop Configuration
  4. Steps for Configuring Sqoop
  5. Sqoop 1 Cluster Mode

Sqoop 1 Cluster Mode

For configuring Sqoop 1 Cluster Mode, follow these steps:
  1. From the DMT Configurations window, set Sqoop Mode as Cluster.
  2. Specify the path of the HDFS working directory for Sqoop related operations in the Sqoop Working Directory field.
  3. Register a cluster with Target Information domain name as the Cluster Name in case of T2H from the Register Cluster tab in the DMT Configurations window. You can also register a cluster with Source Information domain name as the Cluster Name in case of H2T.
    To enable the Sqoop Cluster in SSH mode, provide the appropriate SSH details in the SSH Server name, SSH Port, and SSH Auth Alias fields.
    To create SSH Auth Alias, follow these steps:
    1. Log in as SYSADMN and navigate to the Database Details window.
    2. Click Add (+ symbol) to view the Database Details window.
    3. Do not enter Name or Schema name. Select ORACLE for DB Type and DEFAULT for Auth Type.
    4. Click Add for Alias Name.
    5. Enter an auth alias in Auth Alias, enter a valid SSH user name in User/Principal Name, and enter an SSH password in Auth String.
    6. Click Save to create and register a new Auth Alias in OFSAA.
    7. Close the Database Details window without saving the details in any of the other fields. For details, see the DMT Configurations section in the OFS Analytical Applications Infrastructure User Guide.
  4. Open the command prompt and execute a manual kinit on the Sqoop node with SSH user credentials before Sqoop execution. Execute kinit periodically based on the Kerberos settings for the life time of the ticket. Also, ensure that the Sqoop commands are executable from the default shell of the SSH user.
  5. Copy the following Third Party Jars from the CDH installation libraries into the $FIC_HOME/ext/lib folder:
    • avro-tools-1.8.2-cdh6.3.0.jar
    • commons-cli-1.4.jar
    • commons-httpclient-3.1.jar
    • hadoop-hdfs-client-3.0.0-cdh6.3.0.jar
    • htrace-core4-4.2.0-incubating.jar
    • jackson-core-2.9.9.jar
    • jackson-mapper-asl-1.9.13.jar
    • protobuf-java-3.1.0.jar
    • re2j-1.1.jar
    • servlet-api-2.5.jar
    • sqoop-1.4.7-cdh6.3.0.jar
    • sqoop-test-1.4.7-cdh6.3.0.jar