org.apache.nutch.crawl
Class Generator.Selector

java.lang.Object
  extended by org.apache.nutch.crawl.Generator.Selector
All Implemented Interfaces:
Closeable, JobConfigurable, Mapper, Partitioner, Reducer
Enclosing class:
Generator

public static class Generator.Selector
extends Object
implements Mapper, Partitioner, Reducer

Selects entries due for fetch.


Constructor Summary
Generator.Selector()
           
 
Method Summary
 void close()
           
 void configure(JobConf job)
           
 int getPartition(WritableComparable key, Writable value, int numReduceTasks)
          Partition by host.
 void map(WritableComparable key, Writable value, OutputCollector output, Reporter reporter)
          Select & invert subset due for fetch.
 void reduce(WritableComparable key, Iterator values, OutputCollector output, Reporter reporter)
          Collect until limit is reached.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

Generator.Selector

public Generator.Selector()
Method Detail

configure

public void configure(JobConf job)
Specified by:
configure in interface JobConfigurable

close

public void close()
Specified by:
close in interface Closeable

map

public void map(WritableComparable key,
                Writable value,
                OutputCollector output,
                Reporter reporter)
         throws IOException
Select & invert subset due for fetch.

Specified by:
map in interface Mapper
Throws:
IOException

getPartition

public int getPartition(WritableComparable key,
                        Writable value,
                        int numReduceTasks)
Partition by host.

Specified by:
getPartition in interface Partitioner

reduce

public void reduce(WritableComparable key,
                   Iterator values,
                   OutputCollector output,
                   Reporter reporter)
            throws IOException
Collect until limit is reached.

Specified by:
reduce in interface Reducer
Throws:
IOException


Copyright © 2006 The Apache Software Foundation