oracle.ultrasearch.crawler
Interface CrawlerAgent

public interface CrawlerAgent

CrawlerAgent is an interface used by a crawler agent to submit URL-related data to the crawler.

Method Summary

void	close() Shut down the agent
UrlData	fetch() Retrieve the next document URL data
boolean	getAttributeLOVs(LovInfo lovNode) Get the list of values (LOV) for the attributes of this data source
java.lang.String	getCookies(java.lang.String host, java.lang.String realm, java.lang.String url) Get the cookie string of the specified URL; not supported in 9.0.2
java.lang.String	getCredential(java.lang.String host, java.lang.String realm, java.lang.String url) Get the user name and password of the specified URL
boolean	isDeltaCrawlingCapable() Ask if this agent can return only URLs updated since the last crawl
void	open(DataSourceParams params) Initialize the agent
void	received(UrlData urlNode) Acknowledge that the urlNode sent from the fetch method has been received
void	startCrawling(boolean forceRecrawl, java.util.Date lastCrawlTIme) Ask the agent to prepare for fetching of document URL and its attribute from the data source.
void	stopCrawling() Tell the agent to stop crawling; no more fetching request.

Method Detail

open

public void open(DataSourceParams params)
          throws AgentException

Initialize the agent

Parameters:

params - the data source parameters

Throws:

AgentException - if unable to initialize the agent

isDeltaCrawlingCapable

public boolean isDeltaCrawlingCapable()

Ask if this agent can return only URLs updated since the last crawl

Returns:

true if agent can do delta crawling, false otherwise.

startCrawling

public void startCrawling(boolean forceRecrawl,
                          java.util.Date lastCrawlTIme)
                   throws AgentException

Ask the agent to prepare for fetching of document URL and its attribute from the data source.

Parameters:

forceRecrawl - a flag asking agent to return all URLs if true

lastCrawlTime - the time last crawling was performed

Throws:

AgentException - if unable to get ready for fetching

stopCrawling

public void stopCrawling()
                  throws AgentException

Tell the agent to stop crawling; no more fetching request.

Throws:

AgentException - if agent is unable to stop crawling

fetch

public UrlData fetch()
              throws AgentException

Retrieve the next document URL data

Returns:

UrlData if fetch success; null if there are no more URLs to fetch

Throws:

AgentException - if unable to fetch

received

public void received(UrlData urlNode)

Acknowledge that the urlNode sent from the fetch method has been received

getCredential

public java.lang.String getCredential(java.lang.String host,
                                      java.lang.String realm,
                                      java.lang.String url)

Get the user name and password of the specified URL

Parameters:

host - the host name of the URL

realm - the protection realm

url - the URL string where user name and password is needed

Returns:

<user name>:<password> string

getCookies

public java.lang.String getCookies(java.lang.String host,
                                   java.lang.String realm,
                                   java.lang.String url)

Get the cookie string of the specified URL; not supported in 9.0.2

Parameters:

host - the host name of the URL

realm - the protection realm

url - the URL that needs the authentication information

Returns:

cookie string

getAttributeLOVs

public boolean getAttributeLOVs(LovInfo lovNode)
                         throws AgentException

Get the list of values (LOV) for the attributes of this data source

Parameters:

lovNode - the container object to store the attribute LOVs

Returns:

true if lovNode contains attribute LOVs. Return false if there are no attribute LOVs.

Throws:

AgentException - if getting any unknown error

close

public void close()
           throws AgentException

Shut down the agent

Throws:

AgentException - if unable to close the agent