|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
public interface RobotRules
This class holds the rules which were parsed from a robots.txt file, and can test paths against those rules.
Method Summary | |
---|---|
long |
getCrawlDelay()
Get Crawl-Delay, in milliseconds. |
long |
getExpireTime()
Get expire time |
boolean |
isAllowed(URL url)
Returns false if the robots.txt file
prohibits us from accessing the given url , or
true otherwise. |
Method Detail |
---|
long getExpireTime()
long getCrawlDelay()
boolean isAllowed(URL url)
false
if the robots.txt
file
prohibits us from accessing the given url
, or
true
otherwise.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |