Running Multivariate Tests (MVT) for Push campaigns
Important: This feature is available for push campaigns created with the Campaign Designer.
After you create an Multivariate Test (MVT) campaign, you can run the test from Program.
Tip: In the Program Designer, 
indicates an MVT campaign.
You can test any number of versions, and select the sample size for testing. Note that you can run an MVT campaign without testing. In this case, the default version of the campaign is sent.
About winner selection
You can select the winning version either automatically or manually. After the winning version is selected, it will be used for future launches. If the winning version is deleted for any reason, the default version will be used.
About automatic and manual winner selection
If you choose automatic winner selection, Oracle Responsys automatically selects the winning version based on the campaign metric you select:
- Open rate
The number of messages opened. - Conversion rate
The number of conversion events that are attributed to this campaign.
After the winner is selected, Program will send a notification to the specified email address.
Manual winner selection
If you choose manual winner selection, you select the winning version yourself.
With manual winner selection, you wait for the test to run, examine the results, then designate the winning version. When the test completes and you need to pick a winner, Program will send a notification to the specified email address. If you do not pick a winner within 7 days, the default version will be used.
Running an MVT test
You specify the MVT test options in the properties dialog.
To run an MVT test:
- Add a Send push campaign activity to the program and double click it to open the properties dialog.
- In the Campaign section:
- Click Select and select the MVT campaign you want to use, then click OK.
Tip: MVT campaigns are indicated by an
icon in the list. - Select the Enable multivariate testing checkbox.
If you do not select this checkbox, the campaign will run in regular mode using the default version.
- Click Select and select the MVT campaign you want to use, then click OK.
- Click the Multivariate testing section to specify test options.
- To terminate any previously running tests when this test starts, make sure the Start a new test checkbox is selected.
If this checkbox is not selected, the winning version from the previous test will be used if one is already selected. If a test is currently running, it will continue to run.
- Select when the test starts:
- Start on publish—Starts the test when the program is published.
- Start on date/time—Starts the test on a specific date and time.
- Select either Automatic or Manual winner selection mode.
For more information, see About winner selection.
- Select the metric for choosing the winner:
- Open rate–The rate of opened messages to delivered messages.
- Conversion rate–The rate of conversions to delivered messages.
- In Total sample %, type the percentage of your target audience to test.
Oracle Responsys distributes the sample equally between all versions you are testing.
- Select the checkboxes of the versions you want to test.
Note: If you delete a version while the test is running, the default version is used instead and the test results will exclude the deleted version.
Tip: To preview each version, click
. - In the Duration section, specify when the MVT test should end:
- Time: With this option, Program ends the MVT test after the amount of time you have indicated. This means that although all tests will run, results will be accurate only for the first test. However, you can test several campaigns within a program, and you can test the same campaign in multiple programs.
- In the Email notification field, enter the email addresses to receive notifications about the test.
Separate multiple addresses by commas or semicolons.
- Click Done.
Analyzing test results and selecting a winner
To see the test results, double-click on the Analyze tab. The Test Results page shows details of the test and the winning version if one was automatically selected. To select the winner manually or to change the selection, select Designate as winner of the version you want.