Pig-0.16.0 auf Hadoop 2.7.2 - FEHLER 1002: Kann Alias ​​nicht speichern

Question

Ich habe gerade mit Pig Learning begonnen, für die ich ein Pseudo verteilt Hadoop 2.7.2 auf Ubuntu 14.04 LTS mit Pig Version 0.16.0 installiert. Im Folgenden sind meine Konfigurationen für PIG und Hadoop -

Datei: Bashrc

#=============================================================== 
# Hadoop Variable List 

export JAVA_HOME=/usr/lib/jvm/java-9-oracle 
export HADOOP_INSTALL=/home/hadoop/hadoop 
export PATH=$PATH:$HADOOP_INSTALL/bin 
export PATH=$PATH:$HADOOP_INSTALL/sbin 
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL 
export HADOOP_COMMON_HOME=$HADOOP_INSTALL 
export HADOOP_HDFS_HOME=$HADOOP_INSTALL 
export HADOOP_HOME=$HADOOP_INSTALL 
export YARN_HOME=$HADOOP_INSTALL 
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native 
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib/native" 

#=============================================================== 
# PIG variable 
export PIG_HOME="/home/hadoop/pig" 
export PIG_INSTALL="$PIG_HOME" 
export PIG_CONF_DIR="$PIG_HOME/conf" 
export PIG_CLASSPATH="$HADOOP_INSTALL/conf" 
export HADOOPDIR="$HADOOP_INSTALL/conf" 
export PATH="$PIG_HOME/bin:$PATH" 

=======================

und finden Sie das Verzeichnis, aus dem ich die Ausführung Schwein

-rw-rw-r-- 1 hadoop hadoop 540117 Jul 15 12:41 myfile.txt 
hadoop@rajeev:~$ pwd 
/home/hadoop 

ich diese Datei auf HDFS

auch kopiert

hadoop@rajeev:~$ hadoop fs -ls -R /user/hadoop 
-rw-r--r-- 1 hadoop supergroup  540117 2016-07-15 12:48 /user/hadoop/myfile.txt 

Jetzt ... wenn ich folgende Befehle in der Grunt-Shell ausführe, gibt es einen Fehler!

grunt> a = load 'myfile.txt' as line; 
grunt> store a into 'c.out'; 

2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a 
2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4],a[-1,-1] C: R: 
2016-07-15 12:56:38,684 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 
2016-07-15 12:56:38,685 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:57:25,722 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1468556821972_0006 has failed! Stop running all dependent jobs 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete 
2016-07-15 12:57:25,726 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,786 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,839 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed! 
2016-07-15 12:57:25,841 [main] INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics: 

HadoopVersion PigVersion UserId StartedAt FinishedAt Features 
2.7.2 0.16.0 hadoop 2016-07-15 12:56:36 2016-07-15 12:57:25 UNKNOWN 

Failed! 

Failed Jobs: 
JobId Alias Feature Message Outputs 
job_1468556821972_0006 a MAP_ONLY Message: Job failed!  hdfs://localhost:9001/user/hadoop/c.out, 

Input(s): 
Failed to read data from "hdfs://localhost:9001/user/hadoop/myfile.txt" 

Output(s): 
Failed to produce result in "hdfs://localhost:9001/user/hadoop/c.out" 

Counters: 
Total records written : 0 
Total bytes written : 0 
Spillable Memory Manager spill count : 0 
Total bags proactively spilled: 0 
Total records proactively spilled: 0 

Job DAG: 
job_1468556821972_0006 


2016-07-15 12:57:25,842 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed! 

Ich habe versucht, es durch andere Mittel zu lösen, indem nur PIG im lokalen Modus ausgeführt wird und nicht in MapReduce-Modus, aber nichts scheint zu funktionieren. Immer wenn diese einfachen zwei Befehle fehlschlagen.

Die Drucke Protokolldatei Fehler folgende Meldung

2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a 
2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4],a[-1,-1] C: R: 
2016-07-15 12:56:38,684 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 
2016-07-15 12:56:38,685 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:57:25,722 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1468556821972_0006 has failed! Stop running all dependent jobs 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete 
2016-07-15 12:57:25,726 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,786 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,839 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed! 
2016-07-15 12:57:25,841 [main] INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics: 

HadoopVersion PigVersion UserId StartedAt FinishedAt Features 
2.7.2 0.16.0 hadoop 2016-07-15 12:56:36 2016-07-15 12:57:25 UNKNOWN 

Failed! 

Failed Jobs: 
JobId Alias Feature Message Outputs 
job_1468556821972_0006 a MAP_ONLY Message: Job failed!  hdfs://localhost:9001/user/hadoop/c.out, 

Input(s): 
Failed to read data from "hdfs://localhost:9001/user/hadoop/myfile.txt" 

Output(s): 
Failed to produce result in "hdfs://localhost:9001/user/hadoop/c.out" 

Counters: 
Total records written : 0 
Total bytes written : 0 
Spillable Memory Manager spill count : 0 
Total bags proactively spilled: 0 
Total records proactively spilled: 0 

Job DAG: 
job_1468556821972_0006 


2016-07-15 12:57:25,842 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed! 

anfordern Ihre Hilfe!

Answer 1

Geben Sie den vollständigen Pfad und den Datentyp für das Feld an, in das Sie laden.

a = load 'hdfs://localhost:9001/user/hadoop/myfile.txt' AS (line:chararray); 
store a into 'hdfs://localhost:9001/user/hadoop/c.out';