Use the Insights page to review the performance of recommendations and information about the consumers who responded. Review the number of offers made for each product, brand, category, and promotion in a chosen time period, and analyze the accept rates.
Use the Lift Analysis page to see how the adaptive intelligence models are affecting your purchase rates. You can analyze and compare the lift rate of purchases between a control group of consumers who don't receive adaptive intelligence offers and the rest of your consumers.
This screenshot shows the overall lift percentage and compares the purchase rate between the two groups.
Details include the number of consumers for each stage of the offer. The sequence of stages an offer can have are received, clicked, added to cart, and purchased.
Recommendations and Accepts
When consumers respond to recommendations, the application interprets, records, and feeds back their responses in real time to improve the continuous machine learning. Depending on the type of object and the actual response, the Insights pages display the numbers of accepts, as shown here:
This table describes a little more about how accepts are depicted on the Insights page.
|Object||Accept Action||Accept Meaning|
|Product||Purchase||A purchase of the product within two days of it being recommended|
|Promotion||Use toward purchase||An application of a promotion toward a purchase within two days of it being recommended|
Other metrics you can view are the number of consumers who received offers, clicked offers, and added recommended products to their carts. But the only actions that are recorded as accepts are product purchases and the use of promotions in purchases.
Use lift analysis to see how recommendations affect the purchase rate by consumers. You can set the percentage of site visitors to be in the offer group, which is the group that uses adaptive intelligence. The control group is a randomly selected subset of consumers that won't receive offers while the lift analysis is active. You can compare the relative number of clicks, cart-additions, and purchases between the two groups as data is collected.
The top-level measure of lift is the difference between the purchase rate for the offer group and the control group. Purchase rate is the percentage of site visitors who made one or more purchases.
To start lift analysis:
On the Commerce tab of the Insights page, click Start Lift Analysis.
Select the percentage of site visitors to be considered for the randomly-selected offer group.
The remaining percentage is calculated for the control group that you use for comparison.
Enter start and end dates for the duration of the lift analysis.Tip: Leave the end date blank for an open-ended duration. If you set an end date, you can edit it later to either extend the duration or to end it as of the current date. You can't change the start or end date to a date in the past.
To view or edit a recent lift analysis or one that's already in progress, click View Lift Analysis. You can start a new lift analysis, but it will remove all data from the previous lift analysis.
It may take some time before you see responses in the lift analysis. For example, if your data is updated hourly, you won't see any changes until an hour after the next hourly update. Responses might be minimal if your offer group is small, or if you don't have many site visitors.
Percentages and Groups
During the early stages of a lift analysis, the number of consumers in the offer and control groups may look wrong. For example, if your offer group is 50%, and your site had four visitors, you might expect two of them in each group. However, the calculation for randomly selecting who is in each group doesn't simply count and divide the number of consumers. Instead, the percentage controls the likelihood of being put into one group or the other. So in this example, you might see four consumers in one group and none in the other. As more consumers visit your site, the consumer-to-group allocation will normalize as you would expect.
How Boosts and Constraints Affect Lift Analysis
If your lift analysis is in progress for an extended duration, any adjustments you make to individual products, brands, categories, or even promotions will affect lift analysis results. You may want to first monitor the behavior of the offer group and control group without making any individual adjustments to boosts and constraints so that you can see true responses while using only the algorithms for adaptive intelligent recommendations.
It's best practice to avoid using boosts and constraints during lift analysis. If that's unavoidable, keep track of when you made individual adjustments to explain any unexpected changes in responses.