1. Commerce
  2. Store Front
  3. SEO & Analytics
  4. Search Engine Optimization (SEO)
  5. Sitemap Generator

Sitemap Generator

Important:

Customizations such as Catalog Segmentation, Gift Wrap, and Add-on Items can affect Sitemap Generator output. Therefore, you shouldn't use Sitemap Generator for sites with these customizations. Note that the legacy sitemap generator that crawls the links on your Commerce web stores has been renamed Sitemap Crawler. Sitemap Crawler works with some Commerce web stores, but customizations can affect its output. For better performance and experience, you should use the new Sitemap Generator for your Commerce web stores.

Sitemap Generator creates an XML sitemap for your Commerce web store to help with Search Engine Optimization. By default, Sitemap Generator includes your Homepage URL with URLs of all Commerce Categories, Subcategories, and Items published to your Commerce web store. You can also edit Sitemap Generator settings to include up to 10,000 extra URLs in the sitemap.

Note:

Site Builder Category URLs are not included by default. However, you can manually add Site Builder Category URLs with other user-defined URLs.

Sitemap Generator creates your sitemap files and automatically places them in the file cabinet. In your Domain Settings, the value you set for the HTML Hosting Root field determines where your sitemap files are placed. In most cases, it's the Live Hosting Files folder. For more information, see Set Up Domains for Web Stores.

In accordance with the Sitemap Protocol, the sitemap generation process generates one index file referencing one or more sitemap files for each set of 50,000 URLs.

The XML index file is named sitemap_DomainName_Index.xml and the XML sitemap files use the following naming convention: sitemap_DomainName_Timestamp_SequenceNumber. If there are multiple sitemap files in the Live Hosting Files folder, the timestamp in the file name helps you differentiate between sitemaps generated at different times.

Every time a sitemap is generated, Sitemap Generator automatically notifies search engines such as Google and Bing that your sitemap has been updated. You can also schedule automatic sitemap generation so that your sitemap is always up-to-date. For more information, see Sitemap Generator Scheduler.

Note:

If you regenerate a sitemap, the new sitemap files overwrite the existing ones in the same file cabinet folder. If sitemap regeneration fails, an error file with the same name as the index file (but with a .txt extension) is created in the same folder. That way, the existing sitemap files remain intact and aren't replaced if sitemap regeneration fails.

This topic includes the following subsections:

Prerequisites

An Item URL is added to the generated sitemap only if the following conditions are fulfilled:

A Commerce Category URL is added to the generated sitemap only if the following conditions are fulfilled:

A Commerce Subcategory URL is added to the generated sitemap only if the following conditions are fulfilled:

Important:

When an item goes out of stock, the item URL isn't removed from the sitemap. If your selected Out Of Stock Behavior is to remove an item when it's out-of-stock, web crawlers might still be able to discover the product details page through the sitemap.

Generate a Sitemap

When you generate a sitemap, it automatically includes your Homepage URL, Item URLs, Commerce Category URLs, and Commerce Subcategory URLs. To add extra URLs such as Site Builder Category URLs or blog posts, follow the steps outlined in Add User-Defined URLs to the Sitemap. To set how often each URL changes, follow the steps outlined in Advanced Sitemap Configuration.

To add metadata such as last modified date, change frequency, and priority for each URL, follow the steps outlined in Advanced Sitemap Configuration.

Note:

For Matrix Items, only URLs for the matrix parent items are included in the sitemap.

To generate a sitemap:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

    The Sitemap Generator – Websites page appears.

  2. From the list of domains, select one or more domains for which you want to generate a sitemap.

  3. Click Generate Sitemap.

    You can view the progress in the Status column.

  4. The first time you generate the sitemap, you must add the sitemap index file name to the robots.txt file as described in Update Sitemap Details in the robots.txt File.

Note:

If the sitemap generates without errors, the Status column displays the status as Generated, the timestamp appears in the Last Generated column, and the number of URLs generated appears in the URL Count column. If there are errors, the Status column displays the status as Generated with errors. If the sitemap generation fails, the Status column displays the status as Failed. You can click Generated with errors or Failed to view the error report and troubleshoot the issue.

Add Site Management Tools Landing Pages to the Sitemap

You can add SMT landing page URLs to your sitemap along with the default Homepage URL, Item URLs, Commerce Category and Subcategy URLs. For more information about SMT Landing Pages, see Landing Pages.

To add SMT landing page URLs to the sitemap:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Check the Add SMT Landing Pages box.

  4. Click Save.

The next time you generate a sitemap for this domain, URLs for the published SMT landing pages are included. For more information, see Generate a Sitemap.

Note:

If you set a landing page to go live on a certain date and time, its URL isn't added to the sitemap automatically after it's published. The sitemap updates when you manually regenerate it or at the next scheduled interval (if you've set up automatic generation), ifthe landing page is visible on that date.

If you set a landing page to expire on a certain date and time, its URL isn't removed from the sitemap automatically after it expires. The URL is removed after you regenerate the sitemap after the page expires.

Add User-Defined URLs to the Sitemap

You can add up to 10,000 user-defined URLs to the sitemap by editing Sitemap Generator's settings, in addition to the most important page URLs that are added by default.

To add user-defined page URLs to the sitemap:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Select the Include these Pages in the Sitemap Generation option.

  4. Manually add all the desired page URL paths.

  5. Click Save.

The next time you generate a sitemap for this domain, the user-defined page URLs you added are included. For more information, see Generate a Sitemap.

Sitemap Generator Scheduler

You can set Sitemap Generator to update your XML sitemap automatically every day, week, or month. Scheduling ensures you always have an up-to-date sitemap for search engines to index.

For example, if you frequently add new items or update product details, you can set Sitemap Generator Scheduler to run weekly.

To schedule automatic sitemap generation:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Click the Scheduling subtab.

  4. Check the Enable Scheduling of Sitemap Generation box.

  5. To set the frequency of automatic sitemap generation, select one of the following:

    • Daily – Triggers sitemap generation every day.

    • Weekly – Triggers sitemap generation on a specified day every week.

    • Monthly – Triggers sitemap generation on a specified day every month.

  6. Click Save.

Advanced Sitemap Configuration

Advanced sitemap configuration lets you add extra metadata for each URL in your XML sitemap. Search engines use this information when indexing your site. Adding metadata such as the last modified date, change frequency, and priority, helps search engine crawlers to index your site more effectively.

You can add the following optional metadata in your XML sitemap through advanced sitemap configuration:

To add or edit the advanced sitemap configuration:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Edit in the Settings column for the desired domain.

  3. Click the Advanced subtab.

  4. (Optional) To add the last modified date for each URL in the sitemap, select Yes from the Show Last Modified dropdown.

  5. (Optional) To add the relative priority for each URL in the sitemap, select Yes from the Priority dropdown.

  6. (Optional) To include the change frequency for each URL, select a value from the Change Frequency dropdown.

  7. Click Save.

The next time you generate a sitemap for this domain, the metadata you selected is included. For more information, see Generate a Sitemap.

View Generated Sitemap Files

The generated sitemap files are stored in the file cabinet.

To view the generated sitemap files:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Sitemap Files in the View column for the desired site.

    You are redirected to the folder in the file cabinet where the XML sitemap files are stored.

Update Sitemap Details in the robots.txt File

To help search engines locate your sitemap files, you must update your robots.txt file with the name and location of the sitemap index file.

Note:

This process currently isn't automated because robots.txt files can be heavily customized.

To update the robots.txt file with the sitemap index file name:

  1. Go to Commerce > Marketing > SEO > Sitemap Generator.

  2. Click Sitemap Files in the View column for the desired site.

    You are redirected to the Web Site Hosting Files folder in the file cabinet where the robots.txt file is stored.

  3. Note the sitemap index file name. For example, sitemap_mywebstore.com_Index.xml.

  4. Edit the robots.txt file as described in Editing Files in the File Cabinet.

  5. Add the sitemap index file name in the following format: sitemap: http://domain/indexfilename.

    For example, if your sitemap index file name is sitemap_mywebstore.com_Index.xml, add the following entry in the robots.txt file: sitemap: http://mywebstore.com/sitemap_mywebstore.com_Index.xml

  6. Save the robots.txt file.

Because sitemaps are domain-specific, if multiple domains share the file cabinet root directory, you need to have two robots.txt files that point to two different sitemaps. You can do this by setting up URL redirects, as shown in the following example:

For information about URL redirects, see SEO and Redirects.

Best Practices

When you use Sitemap Generator to generate a sitemap, consider the following best practices:

Important:

If you use a third-party sitemap generator, verify that all URLs are valid in the generated sitemap. If your sitemap contains invalid URLs, such as those containing undefined strings (url:undefined), the invalid URLs may be returned in Google search results.

Related Topics

General Notices