A/B testing campaigns

A/B testing is one of the easiest ways to increase conversion rates and learn more about your audience. In an A/B test, you are testing a control version of an email (A) against a different version (B) to measure which one is most successful. Depending on your goals, an A/B test can help you determine such things as the effectiveness of subject lines, content, and calls to action.

Find out more about A/B testing:

Tip: To use a multi-step campaign, you can use the Audience Split step to split contacts down different paths by percentages.

Before you start A/B testing

  • Identify the objective of the A/B test. Typically, you want to identify one element of an email that you want to test. For example, you could focus on content elements like subject lines, headlines, or calls to action, or design elements like color choice, button design, and so on. This blog post can help identify the features you might want to test.
  • Identify the metrics you'll use to measure the effectiveness of your A/B test. When setting up the test, Oracle Eloqua lets you identify one metric that determines the winning email. The winning metric could be one of the following: total opens, unique opens, total clicks, unique clicks, total conversion, or click-through rate. Choose the metric that aligns with your goals for the test. If you're looking for high engagement, you might want to focus on opens or clicks. If you're focused on goal completion, the total conversion metric might be more appropriate. For more information on how these metrics are calculated, see Campaign reporting and metrics.
  • Send the A/B test emails to enough contacts that you're confident the results of the test are trustworthy. The size of your segment, the content of the email, and any time constraints should help you determine an appropriate test group size.
  • Plan to capture the results of the A/B test and share them. You'll want others to benefit from any new discoveries.

Creating an A/B testing campaign

To create an A/B testing campaign:

  1. Navigate to Orchestration An image of the Orchestration icon., then click Campaigns.
  2. Create a new campaign.
    • In the classic UI, click Create a Simple Email Campaign and then click A/B Test in the wizard.
    • In the Redwood experience, click Create > Simple Email Campaign > Create.
  3. First, give your campaign a name. In the classic UI, be sure that you select A/B Test as the campaign type.

    Classic UI

    An image of Step 1 of the simple email campaign design wizard

    Redwood experience

  4. Specify the segment to use for this campaign. You can optionally create a new one or upload a contact list instead. Later you will indicate what percentage of the segment to include in the testing.

    Classic UI

    An image showing step 2 of the simple email campaign design wizard where you select the recipients

    Redwood experience

    Tip: As you configure the campaign, Oracle Eloqua validates your configuration.
    • In the classic UI, click the caution icon An image showing the caution symbol that appears if there are errors associated with the configuration to review the error.
    • In the Redwood experience, click the step in the table of contents that appears with an error indicator .
  5. Next, specify the emails to use in the test.

    Classic UI

    An image of step 3 of the simple email campaign design wizard. In this step you choose the emails to include in the A/B test

    Redwood experience

  6. Specify the size of your test audience and the metric (such as total opens or total clicks) that determines the winning email.

    Classic UI

    Redwood experience

    • To configure the percentage of the segment that you want to participate in the A/B test, drag the slider. For example, when you set it to 50%, this means that 50% of the segment will participate. 25% of the segment will receive the control email (A) and 25% will receive the test email (B).
    • To select the metric to determine which email performed the best, select the Winning Metric.
    • Configure how long you want the A/B testing campaign to run. You can set the test to run for hours, days, weeks, or months or set an end date.
    • To automatically send the winning email to the remaining contacts when the A/B test ends, select Automatically send the winning version. We'll only send out the winner if a winner is declared based on the winning metric. Otherwise, you'll get a notification if the test was inconclusive. In the event of a tie, Oracle Eloqua sends an email and you must manually select a winner.
  7. Next, specify when to launch the campaign.

    Classic UI

    An image of step 5 of the simple email campaign design wizard. In this step you choose when you want to send the email.

    Redwood experience

    • To start the campaign immediately after you active it, click Send now.
    • To start the campaign on a specific date and time, click Schedule for a later time in the classic UI or Send later in the Redwood experience. You can then specify the date and time to start the campaign.

  8. Click Advanced sending options to configure other email options.

    Classic UI

    An image of the advanced sending options available from the simple email campaign design wizard

    Redwood experience

    • Use a Signature: Allows you to configure the email signature. You can choose to Send using a dynamic signature rule or Send on behalf of a single user. If the campaign's email includes a layout, the signature will be dynamically populated based on the sender. Learn more about signature layouts. Learn more about signature rules.

      The sender's user account does not need to be active nor does the person associated to the user account need to have access to Oracle Eloqua. The user account and associated signature fields will be used to populate the email sender and sender information like a signature where applicable. Learn more.

    • Include List Unsubscribe header: Add an unsubscribe header to your email.
    • Break send into smaller batches: Break up sending emails into smaller batches. The period you use depends on the size of the segment. If you are not sure what period to set, start with about 1000 emails per hour. So, for a segment of 3000, sending the email over 3 hours will send 1000 per hour.
    • Only send during certain days and hours: Allows you to configure when to send the email. For example, use this option to send only at off-peak hours or at an optimal time that you've determined.
    • Enable send time optimization: Allows you to configure the email to send based on the contact's individual optimal send time. Available with the Eloqua Advanced Intelligence Cloud Service add-on only. Learn more.
    • Allow emails to be re-sent to past recipients: Allow this email to be re-sent to past recipients. Typically, you do not want to select this option so that you don't send duplicate emails to your contacts.
    • Send email to master exclude members: Allow this email to be sent to contacts on the master exclude list. This ensures that members on this list do not miss out on critical non-marketing emails, such as product information.
    • Send email to unsubscribed members: Allow this email to be sent to unsubscribed contacts. This ensures that unsubscribed members do not miss out on critical non-marketing emails, such as product information.
  9. To configure campaign fields:
    • In the classic UI, click ActionsAn image of the Actions button, then click Settings.
    • In the Redwood experience, on the Review and submit step, click Properties.

    Here you can set campaign fields like product, country, and so on required by your company, and change the CRM sync or closed-loop reporting settings.

  10. Click Save to save the campaign.
  11. Click Activate Campaign to activate the campaign according to its schedule.

    Note: You must resolve all errors before you can active the campaign.

    Oracle Eloqua sends the email or sends it according to the schedule you set. The campaign's status changes to Active or Scheduled. You cannot change the campaign once it is Active or Scheduled. Find out more about changing an active or scheduled campaign.

Monitoring campaign performance

After the campaign is Active, you can see a real-time view of the email's performance. To view the latest information, click Refresh. To view which contacts were included or excluded from the emails, use the View Participants report.

After the A/B test campaign ends, you can review the overall performance of the winning email. If you automatically sent the winning email to the remaining segment, the overall performance will be updated with the details from the winning email. About the results of an A/B test.

Changing an active A/B test

An Active testing campaign cannot be deactivated. If you want to stop sending the test emails and end the test, you must cancel the campaign. In the Redwood experience, click End Campaign on the Campaign Details tab.

Learn more

Choosing the winning email of an A/B testing campaign

About the results of an A/B test

Knowledge Base: Can A/B Testing Can Be Done on Multi-Step Campaigns?

Email Analysis Overview report

Campaign reporting and metrics