A C E G I L M R S W

A

addURL(ResourceURL) - Method in class com.fatwire.crawler.WebResource

Adds a URL as found on this resource.

addURLs(List<ResourceURL>) - Method in class com.fatwire.crawler.WebResource

Adds a list of URLs as found in this WebResource.

C

com.fatwire.crawler - package com.fatwire.crawler

com.fatwire.crawler.url - package com.fatwire.crawler.url

E

extract(WebResource) - Method in interface com.fatwire.crawler.LinkExtractor

Method is used to parse the WebResource and find a list of links based on the algorithm specified.

G

getBinaryData() - Method in class com.fatwire.crawler.WebResource

The method is used to get the binary data for a resource that is downloaded as part of the crawl session.

getContentEncoding() - Method in class com.fatwire.crawler.WebResource

Get content encoding for the downloaded WebResource.

getContentType() - Method in class com.fatwire.crawler.WebResource

Returns content type of the downloaded WebResource.

getDepth() - Method in class com.fatwire.crawler.url.ResourceURL

The method is used to return the depth of the ResourceURL.

getDownloadTime() - Method in class com.fatwire.crawler.WebResource

Returns the time it took to download this resource in milliseconds.

getHeaders() - Method in class com.fatwire.crawler.WebResource

Used to get all the headers inside the HTTP response for a WebResource.

getResourceSize() - Method in class com.fatwire.crawler.WebResource

Used to get the content size of the downloaded resource.

getStatusCode() - Method in class com.fatwire.crawler.WebResource

Get the status code for the response while accessing the WebResource.

getText() - Method in class com.fatwire.crawler.WebResource

Used to return the HTTP body as a string if text conversion is possible.

getTries() - Method in class com.fatwire.crawler.url.ResourceURL

Return the number of tries made to reach this ResourceURL.

getURI() - Method in class com.fatwire.crawler.url.ResourceURL

Method is used to return the URI for the ResourceURL.

getURI() - Method in class com.fatwire.crawler.WebResource

Returns the URI for the downloaded resource.

getURL() - Method in class com.fatwire.crawler.WebResource

Returns the URL which was used to download this resource.

getURLs() - Method in class com.fatwire.crawler.WebResource

The method is used to return the list of the URLs that are found inside the web resource.

I

incrementTries() - Method in class com.fatwire.crawler.url.ResourceURL

Increment and return the number of retries made to reach this ResourceURL.

L

LinkExtractor - Interface in com.fatwire.crawler

This interface is used to extract the links from the downloaded markup as part of the crawl session.

load(HttpResponse) - Method in class com.fatwire.crawler.WebResource

Reads the HttpResponse and sets various HTTP parameters on this resource.

M

Mailer - Interface in com.fatwire.crawler

This interface provides an extensible point for sending an email after the crawl session has been finished.

R

ResourceRewriter - Interface in com.fatwire.crawler

This interface is used to provide the implementation for rewriting links inside the downloaded markup.

ResourceURL - Class in com.fatwire.crawler.url

This class represents a resource URL which is extracted from a WebResource downloaded as part of the crawl session.

ResourceURL(URI, int) - Constructor for class com.fatwire.crawler.url.ResourceURL

Constructor to create a ResourceURL object.

rewrite(WebResource) - Method in interface com.fatwire.crawler.ResourceRewriter

This method is automatically injected by the crawler framework and is used to provide an implementation for the resource rewriter algorithm.

S

sendMail(String, String) - Method in interface com.fatwire.crawler.Mailer

Send an email after the crawl session is completed.

setContentEncoding(String) - Method in class com.fatwire.crawler.WebResource

Setter for content encoding.

setContentType(String) - Method in class com.fatwire.crawler.WebResource

Setter for the contentType.

setDownloadTime(long) - Method in class com.fatwire.crawler.WebResource

Sets the time taken to download the WebResource in milliseconds.

setStatusCode(int) - Method in class com.fatwire.crawler.WebResource

Setter for the HTTP response status code.

W

WebResource - Class in com.fatwire.crawler

This class is used to represent a resource which is downloaded as part of a crawl session.

WebResource(ResourceURL) - Constructor for class com.fatwire.crawler.WebResource

Constructor for creating an object of WebResource.

A C E G I L M R S W