At the Data stage, the robot has downloaded the content of the resource at the URL and can access data about the content, such as the description and the author.
If the resource is an HTML file, the Robot parses the <META> tags in the HTML headers. Consequently, any data contained in <META> tags is available at the Data stage.
During the Data phase, the following sources are available to RAFs, in addition to those available during the MetaData phase.
Table 12–3 Sources Available to the RAFs at the Data Phase
Source |
Description |
Example |
---|---|---|
content-charset |
Character set used by the resource | |
content-encoding |
Any form of encoding | |
content-length |
Size of the resource in bytes | |
content-type |
MIME type of the resource |
text/html, image/jpeg |
expires |
Date the resource expires | |
last-modified |
Date the resource was last modified | |
data in <META> tags |
Any data that is provided in <META> tags in the header of HTML resources |
Author, Description, Keywords |
All of these sources except for the data in <META> tags are derived from the HTTP response header returned when retrieving the resource.