1. Commerce
  2. Store Front
  3. SEO & Analytics
  4. Search Engine Optimization (SEO)
  5. Sitemap Generator

Sitemap Generator

Important:

Customizations such as Catalog Segmentation, Gift Wrap, and Add-on Items can affect the Sitemap Generator output. Therefore, you should not use the Sitemap Generator for sites that use these customizations. Note that the legacy Sitemap Generator that crawls the links on your Commerce web stores has been renamed to Sitemap Crawler. The Sitemap Crawler can work for some Commerce web stores, but your customizations can affect its output. For enhanced performance and experience, you should use the new Sitemap Generator for your Commerce web stores.

Sitemap Generator creates an XML sitemap for your Commerce web store for Search Engine Optimization. By default, the Sitemap Generator includes your Homepage URL with URLs of all Commerce Categories, Subcategories, and Items published to your Commerce web store. In addition to this, by editing the Sitemap Generator settings, you can specify up to an additional 10,000 URLs to be included in the sitemap.

Note:

Site Builder Category URLs are not included by default. However, you can manually add the Site Builder Category URLs with other user-defined URLs.

Sitemap Generator creates your sitemap files and then automatically places them in the file cabinet. In your Domain Settings, the value specified for the HTML Hosting Root field determines the exact folder in which the sitemap files are placed. In most cases, it is the Live Hosting Files folder. For more information, see Set Up Domains for Web Stores.

In accordance with the Sitemap Protocol, the sitemap generation process generates one index file referencing one or more sitemap files for each set of 50,000 URLs.

The XML index file is named sitemap_DomainName_Index.xml and the XML sitemap files use the following naming convention: sitemap_DomainName_Timestamp_SequenceNumber. When there are multiple sitemap files in the Live Hosting Files folder, the timestamp in the file name helps you differentiate between sitemaps generated at different times.

Every time a sitemap is generated, the Sitemap Generator automatically notifies search engines such as Google and Bing that your sitemap has been updated. You can also schedule automatic sitemap generation so that your sitemap is always up-to-date. For more information, see Sitemap Generator Scheduler.

Note:

If you regenerate a sitemap, the new sitemap files overwrite the existing sitemap files in the same file cabinet folder. If the sitemap regeneration is unsuccessful, an error file is created in the same folder with the same name as the index file, but with a .txt extension. Consequently, the existing sitemap files remain intact and are not replaced when the sitemap regeneration is unsuccessful.

This topic includes the following subsections:

Prerequisites

An Item URL is added to the generated sitemap only if the following conditions are fulfilled:

A Commerce Category URL is added to the generated sitemap only if the following conditions are fulfilled:

A Commerce Subcategory URL is added to the generated sitemap only if the following conditions are fulfilled:

Important:

When an item goes out of stock, the item URL is not removed from the sitemap. If your selected Out Of Stock Behavior is to remove an item when out-of-stock, web crawlers might still be able to discover the product details page through the sitemap.

Generate a Sitemap

When you generate a sitemap, your Homepage URL, Item URLs, Commerce Category URLs, and Commerce Subcategory URLs are added by default. To add any additional URLs such as Site Builder Category URLs, Blog posts, and so on, follow the steps outlined in Add User-Defined URLs to the Sitemap. To add the change frequency for each URL in the sitemap, follow the steps outlined in Advanced Sitemap Configuration.

To add URL metadata such as last modified date, change frequency, and priority, follow the steps outlined in Advanced Sitemap Configuration.

Note:

For Matrix Items, URLs of only the matrix parent items are added to the sitemap file.

To generate a sitemap:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

    The Sitemap Generator – Websites page appears.

  2. From the list of domains, select one or more domains for which you would like to generate a sitemap.

  3. Click Generate Sitemap.

    You can view the progress in the Status column.

  4. When you generate the sitemap first time, you must add the sitemap index file name to the robots.txt file as described in Update Sitemap Details in the robots.txt File.

Note:

If the sitemap is generated without any errors, the Status column displays the status as Generated, the timestamp appears in the Last Generated column, and the number of URLs generated appears in the URL Count column. If the sitemap is generated with some errors, the Status column displays the status as Generated with errors. If the sitemap generation fails, the Status column displays the status as Failed. You can then click Generated with errors or Failed to view the error report and troubleshoot the issue.

Add Site Management Tools Landing Pages to the Sitemap

You can add SMT landing page URLs to your sitemap in addition to the default Homepage URL, Item URLs, Commerce Category and Subcategy URLs. For more information about SMT Landing Pages, see Landing Pages.

To add SMT landing page URLs to the sitemap:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Check the Add SMT Landing Pages box.

  4. Click Save.

The next time you generate a sitemap for this domain, URLs of the SMT landing pages in the published state are included in the sitemap. For more information, see Generate a Sitemap.

Note:

If you have configured a landing page to be visible as of a certain date and time, the landing page URL is not added automatically to the sitemap after the landing page is published. The changes are effective in the sitemap when you manually regenerate the sitemap or during the next scheduled time interval (if you have scheduled automatic sitemap generation), subject to the landing page being visible on that date.

Also, if you have configured a landing page to expire as of a certain date and time, the landing page URL is not removed automatically from the sitemap after the landing page expires. The changes are effective in the sitemap when the sitemap is regenerated after expiry of the landing page.

Add User-Defined URLs to the Sitemap

By editing the Sitemap Generator settings, you can add up to 10,000 user-defined URLs to the sitemap in addition to the most important page URLs that are added by default.

To add user-defined page URLs to the sitemap:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Select the Include these Pages in the Sitemap Generation option.

  4. Manually add all the desired page URL paths.

  5. Click Save.

The next time you generate a sitemap for this domain, the user-defined page URLs you added are included in the sitemap. For more information, see Generate a Sitemap.

Sitemap Generator Scheduler

You can configure the Sitemap Generator to automatically update your XML sitemap either daily, weekly, or monthly. Scheduling ensures you always have an up-to-date sitemap available for search engines to index.

For example, if you frequently add new items or update the product details page, you can use the Sitemap Generator Scheduler to configure sitemap generation to occur weekly.

To schedule automatic sitemap generation:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Click the Scheduling subtab.

  4. Check the Enable Scheduling of Sitemap Generation box.

  5. To set the frequency of automatic sitemap generation, select one of the following:

    • Daily – Triggers sitemap generation every day.

    • Weekly – Triggers sitemap generation on the specified day of every week.

    • Monthly – Triggers sitemap generation on the specified day of every month.

  6. Click Save.

Advanced Sitemap Configuration

You can use advanced sitemap configuration to add additional metadata about each URL in the generated XML sitemap. Search engines scan this information when they index your site. Adding metadata such as the last modified date, change frequency, and priority, enables search engine crawlers to index your site more effectively.

You can add the following optional metadata in your XML sitemap through advanced sitemap configuration:

To add or edit the advanced sitemap configuration:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Click the Advanced subtab.

  4. (Optional) To add the last modified date for each URL in the sitemap, select Yes from the Show Last Modified dropdown.

  5. (Optional) To add the relative priority for each URL in the sitemap, select Yes from the Priority dropdown.

  6. (Optional) To include the change frequency for each URL, select a value from the Change Frequency dropdown.

  7. Click Save.

The next time you generate a sitemap for this domain, the metadata you selected is included in the sitemap. For more information, see Generate a Sitemap.

View Generated Sitemap Files

The generated sitemap files are stored in the file cabinet.

To view the generated sitemap files:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Sitemap Files in the View column for the desired site.

    You are redirected to the folder in the file cabinet where the XML sitemap files are stored.

Update Sitemap Details in the robots.txt File

To help search engines locate your sitemap files, you must update your robots.txt file with the name and location of the sitemap index file.

Note:

This process is currently not automated because robots.txt files can be heavily customized.

To update the robots.txt file with the sitemap index file name:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Sitemap Files in the View column for the desired site.

    You are redirected to the Web Site Hosting Files folder in the file cabinet where the robots.txt file is stored.

  3. Note the sitemap index file name. For example, sitemap_mywebstore.com_Index.xml.

  4. Edit the robots.txt file as described in Editing Files in the File Cabinet.

  5. Add the sitemap index file name in the following format: sitemap: http://domain/indexfilename.

    For example, if your sitemap index file name is sitemap_mywebstore.com_Index.xml, add the following entry in the robots.txt file: sitemap: http://mywebstore.com/sitemap_mywebstore.com_Index.xml

  6. Save the robots.txt file.

Because sitemaps are domain-specific, if you have multiple domains sharing the file cabinet root directory, you need to have two robots.txt files pointing to two different sitemaps. You can achieve this by setting up URL redirects as shown in the following example:

For information about URL redirects, see SEO and Redirects.

Best Practices

When you use the Sitemap Generator to generate a sitemap, it is important you consider the following best practices:

Important:

If you use a third-party sitemap generator, you should verify that all URLs are valid in the generated sitemap. If your sitemap contains invalid URLs such as those containing undefined strings (url:undefined), the invalid URLs may be returned in the Google search results.

Related Topics

General Notices