Campaign insights
Use campaign insights to find out exactly who prefers which campaign experience. You can view a campaign's key metrics at a glance and then drill down to focus on segments that are relevant for your business to find out which experience gets maximum conversion or revenue per generation for the segment.
A micro-segment is a combination of attributes. For example, a visitor from New York using a Tablet Micro-segment is uncovered when you drill-down further into campaign insights.
How does campaign insights work?
- Customer profile data is brought in from a variety of sources to expand out-of-the-box attributes. For example, customer data for loyal customers, custom attributes captured on-page for return customers, and DMP type data for new visitors.
- Using all these profile attributes customers are split up into segments, which are then analyzed.
- Segments are indexed against a baseline experience (test winner for Campaign) and ranked based on their potential targeting impact.
How long should I wait before using campaign insights to examine segment metrics?
It depends on factors such as whether there is enough data, how much traffic there is on your site, the confidence that you have in your data, and the confidence that you require in the segments reported. We recommend that you wait at least one week if you have enough traffic for campaign or longer if you have less traffic to your site. Ideally, wait until you have concluded the test campaign.
Working with the Campaign Summary - Viewing all segments
When you first open campaign insights, the summary shows details for the campaign's total traffic share.
- All Visitors: The 100% shows that you are at the top level and viewing data for all segments.
- Generations: The total number of generations for the whole campaign.
- Top Ranked Experience: The best performing experience in the campaign (for the selected action). A combination of variants make up an experience. For example, a welcome page without an image, a welcome page with a medium size banner, and a welcome page with a call-to-action button saying View Our Best Offers.
Tip: Hover over the Top Ranked Experience box to show the elements and the corresponding variant names that make up the experience.
- Conversion Rate: The percentage of generations who have converted out of all the generations who saw the experience. This relates to the top ranking experience for the selected action.
Working with the campaign insights summary
A different summary is displayed when you drill down and view micro-segments.
- Traffic Share: Percentage of visitors who belong in this segment.
- Preferred Experience: Shows the preferred experience for the selected segment and selected action. To show the statistics for the all the preferred experiences for the selected segment, select the Preferred Experience tab.
- Generations: The total number of generations for the selected segment and selected action.
- Conversion Rate: The percentage of generations who have converted out of all the generations who saw the preferred experience.
- Uplift: The uplift of preferred experience of segment against the top ranking experience for the selected action.
As you drill down to micro-segments a breadcrumb trail is displayed to help you navigate back to parent segments.
Working with the segments graph
The graph shows the top 10 highest performing segments for the campaign for all campaign experiences including the control.
- Hover over any segment to view the statistics for that segment including the experience that the particular segment prefers.
- All Experiences: Use to filter graph by experiences, so you can view segments that prefer a specific experience.
- SHOW 10 SEGMENTS: Use to view more experiences on page.
You can drill down and view micro-segments by clicking on any segment in the graph. You can then mine data further by drilling down on the micro-segments (up to a maximum of 3 attributes (for example All Visitors - London, Phone, and Married). When you drill-down to the micro-segment you can select All Actions, Preferred Experience, or Parent Segment to display more details about each micro-segment.
All Actions: Allows you to compare performance of segment for the preferred experience across all available actions.
- Actions: Number of actions for segment for preferred experience for given action.
- Conversion Rate +/- Margin of Error: Conversion metric performance with Margin of Error for segment.
- Conversion Rate Error: Relative error for conversion metric for segment.
- Uplift against Control Experience: Uplift against metric performance of control experience (default experience) in the campaign for segment.
- Confidence against Control Experience: Confidence index for the reported uplift against control experience in the campaign.
- Uplift against Top Ranked Experience: Uplift against metric performance of Top Ranked Experience in campaign for segment.
- Confidence against Top Ranked Experience: Confidence index for the reported uplift against Top Ranked Experience.
Preferred Experiences: Shows statistics for all the preferred experiences.
- Rank: Ranking of experiences based on Conversion Rate for selected action.
- Generations: Number of generations for segment in selected experience.
- Actions: Number of actions by segment in selected experience for selected action.
- Conversion Rate and Margin of Error: Conversion metric performance of segment in selected experience for selected action plus/minus the margin of error.
- Conversion Rate Error: Relative error for the metric performance of selected action of segment in selected experience.
- Parent Segment: View the performance of the micro-segment against its parent.
- Uplift: Uplift of the micro-segment against parent segment. For example, uplift of London and Married against parent segment of London.
- Traffic Share: Traffic share for this micro-segment. For example, traffic share of London and Married.
Campaign insight FAQs
Q. How do I access the campaign insights report?
A. Access campaign insights from the dashboard.
Or access campaign insights from the Reports menu at the top of the page.
Q. How do I view results by date range?
A. You can filter results by a specific date range.
- Click Filters > Date, then select Filter by Action Date or Filter by Generation Date.
Note: Filtering by date may be useful when you want to ignore data from beginning or end of a campaign. For example, say the date range for your campaign is 1st-31st December, but you want to ignore the first 7 days in your report as there was a conflicting media campaign running. For more information on filtering by dates, see Generation Date vs Action Date Filtering.
- Click on the calendar under Date Range and select the dates you require on the right.
- Click Apply.
Q. How do I view results by a particular segment?
A. You can filter results by out-of-the-box visitor attributes or custom attributes. For example, you may want to hide attributes that are not relevant to your business so that you can focus on the relevant attributes only.
Note: You must click on the attribute name to exclude from the report all the attribute values under that name.
- Select the attributes that you want to exclude.
- You can exclude all attributes in a group, or pick and choose specific attributes to exclude from the report:
| Status | Description |
|---|---|
|
Attributes excluded. At the top level this excludes all sub-attributes, for example, if you check Location: City, Country, Metro Area, Region and Zip code are excluded from campaign insights. |
|
Include all attributes. |
|
Specific attributes excluded. |
For a definition of the attributes, see the table in Attributes for targeting.
Q. How do I view the performance of segments for the secondary actions?
A. Campaign insights displays segment preferences for the campaign's primary action by default. You can display metrics for a secondary action by selecting an action from the drop-down at the top of the page.
The top level summary for all segments is displayed along with metrics that relate to the selected secondary action. The segment graph is updated to show segments preferences for action. For example, you may see a different preferred experience, conversion rate, and uplift.
Q. How do I view campaign insights based on revenue per generation?
A. Click the drop-down for Conversion Rate, then select Revenue Per Generation.
You can discover segments based on revenue metrics to understand segments in terms of additional metrics beyond the conversion rate.
- The Revenue per Generation is displayed instead of conversion (revenue per generation = revenue divided by generations for each experience).
When you drill down to a micro-segment the traffic share represents the percentage share of the total campaign. Revenue per Generation is displayed instead of conversion as you drill down in all segment statistics.
Q. How do I change the confidence for campaign insights?
A. Campaign insights uses a confidence of 80% by default. This level is used as a threshold for confidence index. You can change the confidence based on your campaign specification or organization's needs.
- Click Filters > Confidence, then select the level you require. Click Apply. The confidence level is displayed in the list of applied filters. If you select Remove Confidence Threshold, all possible segments will be displayed with a confidence as low as 0%.
Q. How do I filter campaign insights by time frame?
A. You can use the time frame filter to exclude a specific hour, day, or part of day from the report. Use UTC or Local time to filter generation data by UTC timezone or visitor's local time zone respectively.
- Click Filters > Time Frame.
- Select one of the time frame filters.
- Click Apply.
Q. What is the difference between selecting UTC and the Local filter options?
A. If a campaign is targeted across different times zones, using local time attributes can help you analyze statistics during a particular time of day. Say you run a campaign across different US states or EU countries, using local time attributes you can analyze the behavior of the visitors at the same time of day relative to each visitor's time zone.
- Hour, Day of Week, and Part of Day are all UTC times. If you select Hour and 9:00 AM for example, you will exclude visitors who generated into the campaign between UTC time 9:00 and 10:00 AM only - based on the UTC. If you run a campaign across Europe and 20 visitors generated into the campaign in the UK time between 9:00 AM and 10:00 AM and another 30 visitors generated in Germany at a their local time between 9:00 AM and 10:00 PM, you will only exclude 50 total generations from your report.
- Local Hour, Local Day of Week, and Local Part of Day. If you select Local Hour and 9:00 AM for example, you will exclude visitors who generated into the campaign between 9:00 and 10:00 AM in their local time zone. If you run a campaign across Europe and 20 visitors generated into the campaign in the UK time between 9:00 AM and 10:00 AM and another 30 visitors generated in Germany at a their local time between 9:00 AM and 10:00 PM, you will exclude 50 total generations from your report.
Q. What type of campaign can I run campaign insights against?
A. You can run campaign insights on both A/B and multivariate test (MVT).
- The campaign must use MaxTest full factorial optimization method. Taguchi is not supported.
- The campaign must have at least two experiences. In order to identify a valid segment, the report looks for:
- Minimum of 2000 generators per segment.
- Minimum of 100 actions per segment.
- Segment uplift must be more than zero.
Q. How do I view the list of filters that have been applied to campaign insights?
A. The applied filters list shows the filters used in your report. Click the cross to remove a filter.
Q. Can I view campaign insights for a targeted campaign?
A. Yes, you can view insights on a targeted campaign and use campaign insights to uncover micro-segments that may be worth targeting. When you view a targeted campaign, all visitors is replaced with segment visitors (the percentage shown is for the selected segment). You can use the drop-down menu at the top of the page to select one of the targeted segments or unsegmented traffic (the menu is ranked by traffic share in the segment).
When you drill down to a micro-segment, the traffic share represents the percentage share of the total campaign.
Q. Why do I see the message "The current selection for this campaign insights report does not produce any significant segments. Please change your selection or, if the campaign is still collecting data, try again at a later date."?
A. Segment and micro-segments are only shown if your campaign has collected enough data for the segment to be statistically significant. This message is usually displayed if specific algorithm thresholds are not met.
- Minimum of 2000 generators per segment.
- Minimum of 100 actions per segment.
- Segment uplift must be more than zero.
- Minimum confidence of 80% (or any other option provided) has not been reached.
Q. Can I run segment discovery on campaigns that are Live?
A. Yes, you can run segment discovery on campaigns with a status of Live, Paused or Finished. Ideally wait until you have concluded the test.
Q. How many levels of micro-segments can I drill-down?
A. You can drill down to a maximum level of three attributes (for example All Visitors - London, Phone, and Married).
Q. Why would I change the default confidence?
A. Choose a confidence that is suited to your organization's needs or the campaign specification. If you want higher confidence in your results, use a higher confidence level. If you are more risk adverse, you can use a lower confidence level.