The crawl depth (as set by the -d flag) specifies how many levels of page links will be followed. Each URL in the seed has a level of 0 and each link from a seed URL has a level of 1. The links from a level 1 URL have a level of 2 and so on.

For example, if the seed is www.endeca.com, the levels are as follows:

Level 0: www.endeca.com is level 0 and has a link to about.html.
   Level 1: about.html is level 1 and its links are level 2.
      Level 2: contacts.html is level 2 and its links are level 3.

Therefore, if you want to crawl all the level 2 pages, specify -d 2 as the flag argument.


Copyright © Legal Notices