import --connect jdbc:mysql://ip-172-31-13-154/retail_db --username sqoopuser --password NHkkP876rp --table orders -m 1 --target-dir hdfs:///user/ncnkumar7061/orders1
This is my Sqoop command in the Oozie. The job fails.
This video demonstrates SQOOP import in Oozie. But I already have a SQOOP job created which I want to execute through oozie sqoop action. This is the command I mentioned in sqoop action:
job --exec ZDC_TBL_IMPORT_JOB1
But it failed with error stating ‘Cannot restore missing sqoop job ZDC_TBL_IMPORT_JOB1’.
Although when I execute sqoop job --show ZDC_TBL_IMPORT_JOB1, I can see the job.
Please help resolve this.
Sqoop is not an server based but it is an client based tool, it means that, your Sqoop job configurations will be saved in below directory of edge node:
$HOME/.sqoop/
You can browse to this directory & check about your ‘ZDC_TBL_IMPORT_JOB1’ Sqoop job. So now it can be easily executed using below command, since it is being executed in same edge node where you have saved Sqoop job configuration.
sqoop job
Let us understand how Oozie works:
Oozie takes your Sqoop scripts (not job configurations), Hive Scripts, Pig Scripts, JAR’s etc & executes on any node randomly in Hadoop cluster (remember not always on edge node, where your Sqoop job configurations are configured).
To solve this problem you can use “sqoop-metastore” tool. sqoop-metastore
Or other option is generate JAR file out of your job & schedule using Oozie: Sqoop Codegen
But this codegen option is not suitable for passing dynamic arguments.
2 Likes