Hi team,
I am not able to load data since yesterday from mysql to hive using sqoop. Each time its getting stuck after below, please look into this.
[sayandeepde892819@ip-172-31-38-146 ~]$ sqoop import \
–connect jdbc:mysql://ip-172-31-20-247/sqoopex
–username sqoopuser --password NHkkP876rp
–table account_customer_map2_sd89
–fields-terminated-by ‘,’
–hive-import
–hive-table sde_db.account_customer_map2_sd89 -m 4
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.5.0-292/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.5.0-292/accumulo/lib/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/12/12 07:59:50 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.5.0-292
18/12/12 07:59:50 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/12/12 07:59:51 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/12/12 07:59:51 INFO tool.CodeGenTool: Beginning code generation
18/12/12 07:59:51 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROMaccount_customer_map2_sd89AS t LIMIT 1
18/12/12 07:59:51 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROMaccount_customer_map2_sd89AS t LIMIT 1
18/12/12 07:59:51 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.5.0-292/hadoop-mapreduce
Note: /tmp/sqoop-sayandeepde892819/compile/6449e86c36b3066173807dcfb2b40560/account_customer_map2_sd89.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/12/12 07:59:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-sayandeepde892819/compile/6449e86c36b3066173807dcfb2b40560/account_customer_map2_sd89.jar
18/12/12 07:59:52 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/12/12 07:59:52 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/12/12 07:59:52 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/12/12 07:59:52 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/12/12 07:59:52 INFO mapreduce.ImportJobBase: Beginning import of account_customer_map2_sd89
18/12/12 07:59:53 INFO client.RMProxy: Connecting to ResourceManager at ip-172-31-35-141.ec2.internal/172.31.35.141:8050
18/12/12 07:59:53 INFO client.AHSProxy: Connecting to Application History server at ip-172-31-35-141.ec2.internal/172.31.35.141:10200
18/12/12 07:59:56 INFO db.DBInputFormat: Using read commited transaction isolation
18/12/12 07:59:56 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(acc_id), MAX(acc_id) FROMaccount_customer_map2_sd89
18/12/12 07:59:56 INFO db.IntegerSplitter: Split size: 0; Num splits: 4 from: 1000 to: 1000
18/12/12 07:59:56 INFO mapreduce.JobSubmitter: number of splits:1
18/12/12 07:59:56 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1544284877576_0438
18/12/12 07:59:58 INFO impl.YarnClientImpl: Submitted application application_1544284877576_0438
18/12/12 07:59:59 INFO mapreduce.Job: The url to track the job: http://ip-172-31-35-141.ec2.internal:8088/proxy/application_1544284877576_0438/
18/12/12 07:59:59 INFO mapreduce.Job: Running job: job_1544284877576_0438
18/12/12 08:00:07 INFO mapreduce.Job: Job job_1544284877576_0438 running in uber mode : false
18/12/12 08:00:07 INFO mapreduce.Job: map 0% reduce 0%
18/12/12 08:00:14 INFO mapreduce.Job: map 100% reduce 0%
18/12/12 08:00:14 INFO mapreduce.Job: Job job_1544284877576_0438 completed successfully
18/12/12 08:00:14 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=170970
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=113
HDFS: Number of bytes written=42
HDFS: Number of read operations=4
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Other local map tasks=1
Total time spent by all maps in occupied slots (ms)=10185
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=3395
Total vcore-milliseconds taken by all map tasks=3395
Total megabyte-milliseconds taken by all map tasks=5214720
Map-Reduce Framework
Map input records=1
Map output records=1
Input split bytes=113
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=46
CPU time spent (ms)=1280
Physical memory (bytes) snapshot=231890944
Virtual memory (bytes) snapshot=6382784512
Total committed heap usage (bytes)=216006656
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=42
18/12/12 08:00:14 INFO mapreduce.ImportJobBase: Transferred 42 bytes in 20.4291 seconds (2.0559 bytes/sec)
18/12/12 08:00:14 INFO mapreduce.ImportJobBase: Retrieved 1 records.
18/12/12 08:00:14 INFO mapreduce.ImportJobBase: Publishing Hive/Hcat import job data to Listeners
18/12/12 08:00:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROMaccount_customer_map2_sd89AS t LIMIT 1
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.5.0-292/accumulo/lib/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
18/12/12 08:00:14 WARN hive.TableDefWriter: Column eff_strt_dt had to be cast to a less precise type in Hive
18/12/12 08:00:14 WARN hive.TableDefWriter: Column eff_end_dt had to be cast to a less precise type in Hive
18/12/12 08:00:14 INFO hive.HiveImport: Loading uploaded data into Hive
Logging initialized using configuration in jar:file:/usr/hdp/2.6.5.0-292/hive/lib/hive-common-1.2.1000.2.6.5.0-292.jar!/hive-log4j.properties