How the Robot Works

How the Robot Works illustrates how the search engine robot works. In How the Robot Works, the robot examines URLs and their associated network resources. Each resource is tested by both the enumerator and the generator. If the resource passes the enumeration test, the robot checks it for additional URLs. If the resource passes the generator test, the robot generates a resource description that is stored in the search engine database.

Figure 16–1 How the Robot Works