Sources Available at the Data Stage

At the Data stage, the robot has downloaded the content of the resource at the URL and can access data about the content, such as the description and the author.

If the resource is an HTML file, the Robot parses the <META> tags in the HTML headers. Consequently, any data contained in <META> tags is available at the Data stage.

During the Data phase, the following sources are available to RAFs, in addition to those available during the MetaData phase.

Table 19–3 Sources Available to the RAFs at the Data Phase

Source	Description	Example
content-charset	Character set used by the resource
content-encoding	Any form of encoding
content-length	Size of the resource in bytes
content-type	MIME type of the resource	text/html, image/jpeg
expires	Date the resource expires
last-modified	Date the resource was last modified
data in <META> tags	Any data that is provided in <META> tags in the header of HTML resources	Author, Description, Keywords

All of these sources except for the data in <META> tags are derived from the HTTP response header returned when retrieving the resource.