Oozie jobs not running and hanging for resources

Technical Discussions Oozie
#1

Hi @abhinav, @sgiri,

i’m running Oozie workflow jobs, MapReduce & Hive, they’re hanging with PREP status & not moving forward without any errors or logs. I’m suspecting because of resources issue in NodeManager.

Please find Oozie job id:
0000085-200714153401474-oozie-oozi-W
0000084-200714153401474-oozie-oozi-W
0000083-200714153401474-oozie-oozi-W

Please find job.properties:

nameNode=hdfs://cxln1.c.thelab-240901.internal:8020
jobTracker=cxln1.c.thelab-240901.internal:8050
queueName=default
examplesRoot=examples
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/map-reduce/workflow.xml
outputDir=map-reduce

#2

You can try running from console.

https://oozie.apache.org/docs/4.0.1/DG_Examples.html#Running_the_Examples

after having the Oozie share library installed in HDFS.

https://oozie.apache.org/docs/4.0.1/DG_QuickStart.html#OozieShareLib

1 Like
#3

Thanks for reply, i’ve executed examples by running from console using below command:

$oozie job -config apps/map-reduce/job.properties -run

Below are updated job.properties:

nameNode=hdfs://10.142.1.1:8020
jobTracker=10.142.1.2:8050
queueName=default
examplesRoot=examples

oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/map-reduce/workflow.xml
outputDir=map-reduce

Still same issue, still in prep status.

#4

@abhinav, @satyajit_das, @sgiri

It seems issue with Oozie service, not only my jobs, all other jobs are stuck at PREP. Please check issue.
Please note that, Sqoop action, MR Action, Hive action all jobs are stuck.

#5

I’ve understood by researching about this issue, it can be regarding Database backend for Oozie server, by default it is derby, so some experts migrated to MySQL and didn’t faced issues further.
https://orzota.com/tag/workflow-analytics/

Please let us know by looking into this angle.

#6

@satyajit_das, @sgiri, @abhinav,

Please provide an update on this issue.

#7

@Sachin_Giri, @satyajit_das, @sgiri, @Abhinav_Singh, @abhinav,

we were not able to run single Oozie job from last 8 days. Can you please address this ASAP.
Not a single job is executing, please check below screenshot. This issue is for everyone.

image
image1518×735 77.6 KB

#8

Please loot into it.

#9

I have checked out all the ports and configurations and we have everything setup correctly. There are few open tickets regarding the same in Cloudera forum and there is no solution to this problem there.

If you come to know any solution please let us know and we will be happy to incorporate them.

#11

This may help? @abhinav

#12

Hi @raviteja

Oozie is already on MySQL from day one so derby is not a problem.

.

1 Like