Reducer funktioniert nicht oder nie Anruf

Question

Treiberklasse Geting:

import java.io.IOException; 
import org.apache.hadoop.conf.Configuration; 
import org.apache.hadoop.fs.Path; 
import org.apache.hadoop.io.LongWritable; 
import org.apache.hadoop.io.NullWritable; 
import org.apache.hadoop.io.Text; 
import org.apache.hadoop.mapreduce.Job; 
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; 
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; 
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; 
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; 

public class DRIVER { 


public static void main(String arg[]) 
{ 
    try{ 
    Path in = new Path("aamazon.txt"); 
    Path out = new Path("/output"); 

    Configuration conf = new Configuration(); 

    Job job = Job.getInstance(conf); 
    job.setJarByClass(DRIVER.class); 
    job.setMapperClass(MAPPER.class); 
    job.setReducerClass(REDUCER.class); 
    job.setNumReduceTasks(0); 

    FileInputFormat.addInputPath(job, in); 
    FileOutputFormat.setOutputPath(job, out); 

    job.waitForCompletion(true); 

    System.out.println("Successful");} 

    catch(Exception e){ 
     System.out.println(e.getMessage()); 
    } 


} 
} 

Mapper Klasse:

import java.io.IOException; 
import org.apache.hadoop.io.LongWritable; 
import org.apache.hadoop.io.NullWritable; 
import org.apache.hadoop.io.Text; 
import org.apache.hadoop.mapreduce.Mapper; 

public class MAPPER extends Mapper<LongWritable,Text,LongWritable,Text>{ 

@Override 
public void map(LongWritable key,Text value,Context con) 
{ 
    try 
    { 

     System.out.println(key +"\n"+ value); 
     con.write(key, value); 

    } 

    catch(Exception e) 
    { 
     System.out.println(e.getMessage()); 
    } 
} 
} 

Reducer Klasse:

import java.io.IOException; 
import org.apache.hadoop.io.LongWritable; 
import org.apache.hadoop.io.NullWritable; 
import org.apache.hadoop.io.Text; 
import org.apache.hadoop.mapreduce.Reducer; 

public class REDUCER extends Reducer<LongWritable,Text,LongWritable,Text>{ 

@Override 
public void reduce(LongWritable key,Iterable<Text> value , Context con) 
{ 
    System.out.println("reducer"); 
    try{ 
    for(Text t:value) 
    { 

     con.write(key, t); 
    } 
    } 
    catch (Exception e) 
    { 
     System.out.println(e.getMessage()); 
    } 
} 
} 

---

Probleme:

1. Ausführung bis Mapper Aufarbeiten
2. Reducer nie
3. aufgerufen wird, wenn ich gesetzt setNumReduceTasks(0) dann Mapper nicht

Irgendwelche Ideen, was falsch ist genannt zu werden?

Answer 1

Sie haben die Anzahl der reduzierenden Aufgaben auf Null gesetzt.

Job job = Job.getInstance(conf); 
    job.setJarByClass(DRIVER.class); 
    job.setMapperClass(MAPPER.class); 
    job.setReducerClass(REDUCER.class); 
    job.setNumReduceTasks(0); // this should be greater than 0 

danach Auch wenn es nicht überprüfen, arbeiten Sie Berechtigungen für den „/ Ausgabe“ Pfad auf folgende Zeilen haben -

Path out = new Path("/output"); // it is in the root folder. change it to "./output"