command :
python MovieRatingsBreakdown.py -r hadoop --hadoop-streaming-jar /usr/hdp/2.6.2.0-205/hadoop-mapreduce/hadoop-streaming.jar u.data
Error :
Probable cause of failure:
Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 127
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
Step 1 of 2 failed: Command ‘[’/bin/hadoop’, ‘jar’, ‘/usr/hdp/2.6.2.0-205/hadoop-mapreduce/hadoop-streaming.jar’, ‘-files’, ‘hdfs:///user/mgurnani999463/tmp/mrjob/Movi
eRatingsBreakdown.mgurnani999463.20190928.123257.722804/files/wd/MovieRatingsBreakdown.py#MovieRatingsBreakdown.py,hdfs:///user/mgurnani999463/tmp/mrjob/MovieRatingsBr
eakdown.mgurnani999463.20190928.123257.722804/files/wd/mrjob.zip#mrjob.zip,hdfs:///user/mgurnani999463/tmp/mrjob/MovieRatingsBreakdown.mgurnani999463.20190928.123257.7
22804/files/wd/setup-wrapper.sh#setup-wrapper.sh’, ‘-input’, ‘hdfs:///user/mgurnani999463/tmp/mrjob/MovieRatingsBreakdown.mgurnani999463.20190928.123257.722804/files/u
.data’, ‘-output’, ‘hdfs:///user/mgurnani999463/tmp/mrjob/MovieRatingsBreakdown.mgurnani999463.20190928.123257.722804/step-output/0000’, ‘-mapper’, ‘/bin/sh -ex setup-
wrapper.sh python3 MovieRatingsBreakdown.py --step-num=0 --mapper’, ‘-reducer’, ‘/bin/sh -ex setup-wrapper.sh python3 MovieRatingsBreakdown.py --step-num=0 --reducer’]
’ returned non-zero exit status 256.