org.apache.nutch.fetcher
Class Fetcher
java.lang.Object
org.apache.hadoop.util.ToolBase
org.apache.nutch.fetcher.Fetcher
- All Implemented Interfaces:
- Configurable, JobConfigurable, MapRunnable, Tool
public class Fetcher
- extends ToolBase
- implements MapRunnable
The fetcher. Most of the work is done by plugins.
Field Summary |
static org.apache.commons.logging.Log |
LOG
|
Fields inherited from class org.apache.hadoop.util.ToolBase |
conf |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LOG
public static final org.apache.commons.logging.Log LOG
Fetcher
public Fetcher()
Fetcher
public Fetcher(Configuration conf)
configure
public void configure(JobConf job)
- Specified by:
configure
in interface JobConfigurable
close
public void close()
isParsing
public static boolean isParsing(Configuration conf)
isStoringContent
public static boolean isStoringContent(Configuration conf)
run
public void run(RecordReader input,
OutputCollector output,
Reporter reporter)
throws IOException
- Specified by:
run
in interface MapRunnable
- Throws:
IOException
fetch
public void fetch(Path segment,
int threads)
throws IOException
- Throws:
IOException
main
public static void main(String[] args)
throws Exception
- Run the fetcher.
- Throws:
Exception
run
public int run(String[] args)
throws Exception
- Specified by:
run
in interface Tool
- Throws:
Exception
Copyright © 2006 The Apache Software Foundation