Sqoop all tables import is failing

[dbseoracle8661@cxln5 ~]$ sqoop import-all-tables --connect “jdbc:mysql://cxln2.c.thelab-240901.internal/retail_db” --username sqoopuser --password NHkkP
876rp --as-sequencefile -m 4 --warehouse-dir /user/dbseoracle8661/chid/
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.2.0-205/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.6.2.0-205/accumulo/lib/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See SLF4J Error Codes for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
22/06/09 05:56:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.2.0-205
22/06/09 05:56:53 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
22/06/09 05:56:53 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
22/06/09 05:56:54 INFO tool.CodeGenTool: Beginning code generation
22/06/09 05:56:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM Emp AS t LIMIT 1
22/06/09 05:56:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM Emp AS t LIMIT 1
22/06/09 05:56:54 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.2.0-205/hadoop-mapreduce
Note: /tmp/sqoop-dbseoracle8661/compile/a5d7c0b67a5dd8742cdf99fb25a6c588/Emp.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
22/06/09 05:56:56 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-dbseoracle8661/compile/a5d7c0b67a5dd8742cdf99fb25a6c588/Emp.jar
22/06/09 05:56:56 WARN manager.MySQLManager: It looks like you are importing from mysql.
22/06/09 05:56:56 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
22/06/09 05:56:56 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
22/06/09 05:56:56 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
22/06/09 05:56:56 ERROR tool.ImportAllTablesTool: Error during import: No primary key could be found for table Emp. Please specify one with --split-by or
perform a sequential import with ‘-m 1’.

–>I want all tables import to be done with 4 mappers here but my import job is getting terminated with an error saying me to use “-m 1”…Since primary keys are not there on tables, I used “-m 4”…(when I use same command with “-m 1”, my import job is being executed successfully without any error)…Though it is a valid command, it is getting falied… not sure what is causing this to be failed (I tried same command in my cloudera VM… it ran successfully)… Can you please advise what am I missing here? why import is not running with “-m 4”?

Issue if fixed with --autoreset-to-one-mapper(we have to add this in our command), job ran successfully.

How it works?
If there is no primary key on table, then it will take ‘–autoreset-to-one-mapper’ into cosnideration and invoke one mapper on table but if job finds any primary key on table, then it will go with 4 mappers as we have used -m 4 in command.

1 Like