20 Optimize Your App for Search Engines
Search Engine Optimization (SEO) covers a variety of techniques that make your application's pages more accessible to web crawlers, the scripts used by search engines to crawl the web and gather pages for indexing. SEO looks to increase the ranking of indexed pages in search results to give your content more visibility.
You can often improve the ranking of your application's pages using sitemaps. Sitemaps list the URLs of your pages, enabling web crawlers to identify your content without relying solely on their ability to crawl and parse the pages. But while sitemaps can help web crawlers find pages for indexing, the indexing process itself might be less than ideal for single-page applications. This is especially true when content is dynamically added to pages based on the results of REST calls. In such scenarios, web crawlers may not wait long enough for the REST calls to complete and the page to be fully rendered before indexing the page. Pages that are indexed before their content can be fully loaded don't feature well in search results.
To address this issue, Visual Builder "prerenders" pages that web crawlers access. When a request for a page is received from Googlebot (or any other search engine), the request is routed through a prerender server, which loads the page, runs any JavaScript required to fully render that page, then strips it out before returning the page to Googlebot. This way, Googlebot always receives a fully rendered page for indexing—just one without any JavaScript in the source.
A page, once prerendered, is always cached, so page markup can be returned immediately when subsequent requests are made for the same URL. This helps reduce page load times that may adversely impact page ranking. If markup is yet to be cached, the process of rendering the page can take some time. To optimize these response times, it's recommended that you warm the prerender server's cache for URLs listed in your application's sitemap, so web crawlers always get the fastest possible response when requesting those URLs.
- Create a sitemap for your web application.
- Add the sitemap to your web application's resources.
- Once your web application is deployed (or redeployed), warm the cache for URLs listed in your app's sitemap (using the
vb-prerender-cache-warm
Grunt task).
Create a Sitemap for a Web App
A sitemap is a document that contains URLs to pages representative of your application. It ensures that important locations in your application are possible for web crawlers to locate and are properly ranked.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://eta.myexample.org/?page=shell</loc>
<lastmod>2023-08-21T16:12:20+03:00</lastmod>
</url>
<url>
<loc>https://eta.myexample.org/?page=shell&shell=bugs&bugs=main-bug</loc>
<lastmod>2023-08-21T16:12:20+03:00</lastmod>
</url>
<url>
<loc>https://eta.myexample.org/?page=shell&shell=tags</loc>
<lastmod>2023-08-21T16:12:20+03:00</lastmod>
</url>
</urlset>
<url>
tag is used to specify the URL of a page. This tag has several child tags, but only the <loc>
and <lastmod>
tags are used to populate the prerender server's cache:
<loc>
is a required tag that specifies the actual URL of a page. The value of a<loc>
tag must begin with the protocol (such as https) and end with a trailing slash, if your web server requires it. This value must be less than 2,048 characters long.<lastmod>
is a required tag that specifies the date the contents of the URL changed between application deployments.
Make sure the sitemap contains each individual URL you want indexed. It should also include up-to-date lastmod
entries if the contents of the URL can change between deployments (for example, when pages contain data from other sources).
For general information on sitemaps, see https://www.sitemaps.org/protocol.html.
Add a Sitemap to a Web App's Resources
You can add a sitemap to your web application's resources to provide web crawlers information about the content in the application.
To add a sitemap to your web application:
Warm the Cache for URLs in a Sitemap
When your web app includes a sitemap that lists URLs to pages representative of your application, you can use the vb-prerender-cache-warm
Grunt task to warm the prerender server's cache for these URLs, so web crawlers get the fastest possible response when requesting those URLs.
To use the vb-prerender-cache-warm
Grunt task, you must set up your file system to build the visual application by installing Node.js
and its package manager (npm
). Once you have installed the necessary tools, you need to save the visual application's sources to your local system. See Build Your Application Locally.
vb-prerender-cache-warm
Grunt task:
- In the command-line interface, navigate to the folder on your local system containing the
package.json
andGruntfile.js
files. - Enter
npm install
to retrieve the node dependencies required to build the application. - Enter
vb-prerender-cache-warm
in the command-line interface to warm the cache. Here are some examples:- To initially warm the cache after a redeployment:
grunt --id=MyApplication --ver=1.0 --sitemap="./webApps/myWebApp/sitemap.xml" --url=http://my.vbinstance.com/ic/builder --clearCache=true vb-prerender-cache-warm
- To update the cache after minor modifications to the sitemap:
grunt --id=MyApplication --ver=1.0 --sitemap="./webApps/myWebApp/sitemap.xml" --url=http://my.vbinstance.com/ic/builder vb-prerender-cache-warm
The
vb-prerender-cache-warm
task reads themanifest.json
file on startup. It will compare the timestamps of the pages in the manifest with the application's sitemap, and warm the cache with any files that have changed since the last run (or which failed on the last run) and write details of the run back to the same file on completion. If theclearCache
flag is set, any existing manifest is ignored and overwritten when the task completes.In the preceding examples, metadata about the last operation is saved tomanifest.json
in the current directory. To save or load from a different manifest file, you can specify the--manifest
option:grunt --id=MyApplication --ver=1.0 --sitemap="./webApps/myWebApp/sitemap.xml" --url=http://my.vbinstance.com/ic/builder --manifest="/tmp/manifest.json" vb-prerender-cache-warm
For more information on the supported command-line options, see vb-prerender-cache-warm.
- To initially warm the cache after a redeployment:
Move Your Sitemap to a Visual Application's Root Directory
When you manually add a sitemap to your web application's resources, you can set up the sitemap file to be copied to the visual application's root directory after the app is staged or published, so the sitemap path does not change.
/webApps/webAppName/version_id
directory. As a result, you'll need to include the newest version_id
in the path to access the application's sitemap.
To automatically copy the sitemap file from the /webApps/webAppName/version_id
directory to the application's root directory each time the app is staged or published, you can edit your application's Gruntfile.js
file and add the vb-post-package
task: