Sqoop 1 Client Mode

Step 1: Configuring the Properties

1. From the DMT Configurations window, set Sqoop Mode as Client.
2. Specify the path of the HDFS working directory for Sqoop related operations in the Sqoop Working Directory field.
3. From the Register Cluster tab in the DMT Configurations window, register a cluster with Target Information domain name as the Cluster Name in case of T2H or register a cluster with Source Information domain name as the Cluster Name in case of H2T.
   For details, see DMT Configurations section in the OFS Analytical Applications Infrastructure User Guide.
4. Step 2: Copy Third Party Jars
   Copy the following Third Party Jars from the CDH installation libraries into the $FIC_HOME/ext/lib folder:

   •  commons-net-3.1.jar
   •  hadoop-mapreduce-client-core-3.0.0-cdh6.3.0.jar
   •  avro-tools-1.8.2-cdh6.3.0.jar
   •  commons-cli-1.4.jar
   •  commons-httpclient-3.1.jar
   •  hadoop-hdfs-client-3.0.0-cdh6.3.0.jar
   •  htrace-core4-4.2.0-incubating.jar
   •  jackson-core-2.9.9.jar
   •  jackson-mapper-asl-1.9.13.jar
   •  protobuf-java-3.1.0.jar
   •  re2j-1.1.jar
   •  servlet-api-2.5.jar
   •  sqoop-1.4.7-cdh6.3.0.jar
   •  sqoop-test-1.4.7-cdh6.3.0.jar

   Note:

   The version of Jars depends on the CDH version and the Drivers used. For CDH 5.8.4 version, you should copy htrace-core4-4.0.1-incubating.jar instead of htrace-core-3.0.4.jar.
   Following jars are needed, but may be present in the $FIC_HOME/ext/lib folder as part of CDH Enablement:

   •  commons-configuration-1.6.jar
   •  commons-collections-3.2.2.jar
   •  commons-io-2.4.jar
   •  commons-logging-1.0.4.jar
   •  hadoop-auth-2.0.0-cdh4.7.0.jar
   •  hadoop-common-2.0.0-cdh4.7.0.jar
   •  hadoop-core-2.0.0-mr1-cdh4.7.0.jar
   •  libfb303-0.9.0.jar
   •  libthrift-0.9.0-cdh4-1.jar
   •  slf4j-api-1.6.4.jar

   Note:

   The version of jars to be copied differs depending upon the version of CDH configured.
   Step 3: Copy Configuration XMLs from Hadoop Cluster

   Copy core-site.xml, hdfs-site.xml, mapred-site.xml, hive-site.xml, and yarn-site.xml from the Hadoop Cluster to the Configuration File Path given in the Cluster Configurations window of the registered cluster. Note that only Client Configuration Properties are required.

   If Cloudera Manager is used, the same can be downloaded directly which will contain only the client properties.

   Note:

   If proxy user is enabled and the Job is submitted by the same, the user should be created in every node of the Hadoop Cluster.