org.apache.nutch.crawl
Class Generator.CrawlDbUpdater
java.lang.Object
org.apache.hadoop.mapred.MapReduceBase
org.apache.nutch.crawl.Generator.CrawlDbUpdater
- All Implemented Interfaces:
- Closeable, JobConfigurable, Mapper, Reducer
- Enclosing class:
- Generator
public static class Generator.CrawlDbUpdater
- extends MapReduceBase
- implements Mapper, Reducer
Update the CrawlDB so that the next generate won't include the same URLs.
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Generator.CrawlDbUpdater
public Generator.CrawlDbUpdater()
configure
public void configure(JobConf job)
- Specified by:
configure
in interface JobConfigurable
- Overrides:
configure
in class MapReduceBase
map
public void map(WritableComparable key,
Writable value,
OutputCollector output,
Reporter reporter)
throws IOException
- Specified by:
map
in interface Mapper
- Throws:
IOException
reduce
public void reduce(WritableComparable key,
Iterator values,
OutputCollector output,
Reporter reporter)
throws IOException
- Specified by:
reduce
in interface Reducer
- Throws:
IOException
Copyright © 2006 The Apache Software Foundation