when running hadoop streaming, an error is reported. The exception content is as follows:
Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 126
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:538)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
The startup shell script for hadoop is as follows:
hdfs_input="/test/testLine.log"
hdfs_ouput="/out"
python_lib="${fs_default_name}/python/python.tar.gz"
WORK_DIR=$(cd `dirname $0`; pwd)
${hadoop_path}/bin/hadoop jar ${hadoop_path}/libexec/share/hadoop/tools/lib/hadoop-streaming-2.8.2.jar \
-archives ${python_lib}\-sharplib \
-D mapreduce.job.name="didp_log" \
-D mapreduce.job.reduces=1 \
-input ${hdfs_input} \
-output ${hdfs_ouput} \
-mapper "lib/python/bin/python mapper.py" \
-reducer "lib/python/bin/python reducer.py" \
-file ${WORK_DIR}/mapper.py \
-file ${WORK_DIR}/reducer.py \
-inputformat org.apache.hadoop.mapred.TextInputFormat \
-outputformat org.apache.hadoop.mapred.TextOutputFormat \
ask God to tell me how to deal with it. Thank you