Create a Web Crawler Connector

Next, John Allens creates a Web Crawler connector as follows:

1. Go to Tools > AI Agent Studio.
2. Click the Credentials tab.
3. Click the Connectors tab.
4. Click Add Connector.
5. Search for WebCrawler and click +.
6. For Name, enter a name of your choice. For example, Vision Web Connector.
7. For the Code, retain the autogenerated connector code. For example, VISION_WEB_CONNECTOR.
   Note: Make a note of the connector code. You will need to provide this code when you define a Data Security Policy for a role.
8. Select a Family. For example, CX.
9. Select a Product. For example, Service.
10. For Description, enter a description. For example, Use these web pages to answer questions.
11. For the Article ID Prefix, provide a prefix. For example, VWEB.
12. In the Configure section, provide the Starting URL and the Crawl Depth.

    Crawl depth is the page depth from the starting URL up to which you would like the web crawler to process the pages. For example, any child page of the starting web page has a depth value of 2. If you set the crawl depth to 2, it will crawl the starting point URL and all its direct child pages/links.

13. Configure other fields as required and click Create.
14. For User Groups, select Content Intelligence Users.

    User groups enable you to control who can see which content in the Content Intelligence knowledge base. Users assigned to a user group can only access the content targeted for that user group. The predefined user groups help in segregating content that should only be viewed by users within the organization. You can create custom user groups to segment content for additional audiences. For more information, see How do I assign users to a user group in Knowledge?

15. Click Create.

The "Vision Web Connector" content type is created, and it appears under the Connectors tab.