Documentation Center

Viewing Experiment statistics

The objective of an Experiment is to allow users to test whether one piece of content (Content Variant) is more successful than others in converting visitors. In the Experiment Statistics view, the performance of individual Content Variants is indicated using Conversion Rates.

About this task

The Conversion Rate is the percentage of Views that resulted in a Conversion (registered when a link is clicked). When one Content Variant is significantly more successful than the others in an Experiment, a Best Performing Variant is declared. The criteria for deciding whether one Content Variant reliably outperforms the others are as follows:
  • to ensure that sufficient data has been collected, each Content Variant must have at least 100 conversions;
  • there must be a significant difference in Conversion Rate between the Content Variants;
  • the difference in Conversion Rate between the Content Variant with the highest Conversion Rate and the next best Content Variant must exceed their combined error margins.

In the following example, Content Variant 1 has a conversion rate of 60% and an error margin of 3%, meaning there is a 95% likelihood that the true conversion rate for the variant lies between 57% and 63%. Content Variant 2 has a conversion rate of 56% and an error margin of 2% so the true conversion rate for the variant likely lies between 54% and 58%. Since the error margin ranges overlap, a winner cannot be reliably declared and so no Best Performing Variant will be displayed:

Conversion Rate (Error Margin)Conversion Rate Error Margin Range
Content Variant 160%(±3)57% - 63%
Content Variant 256%(±2)54% - 58%

Procedure

  1. In the slide out navigation pane, select Targeting.

    The Targeting Dashboard appears.

  2. In the Source drop-down menu:
    1. Select an environment for which you want to manage Experiments. For example, you will typically have a "Staging" environment and a "Live" environment:
      • Staging is an environment that is available only to internal users and is where you develop and test targeting functionality and new promotional content. On this environment, you can use Experience Manager for inline editing and testing.
      • Live is your external, public-facing environment where visitors can access your production content.
    2. Click the Experiments tab to view the list of Experiments available for the environment.
    A list of Experiments for which you have Manage Experiments rights is displayed.
  3. Open an Experiment.
  4. Click .
  5. If the Experiment is available in several Publications, a list of Publications is displayed with the following information so you can compare statistics in all the Publications where the Experiment is running:
    • Publication—the website where the Experiment ran/is running.
    • Total Views—the number of times the Experiment has been displayed on a Page requested by a visitor.
    • Best Performing Variant—the Content Variant with the best conversion rate.
    • Best Conversion Rate—the conversion rate for the Best Performing Variant.
  6. Click a Publication to view the Experiment statistics for that Publication.
  7. At the top of the statistics dialog you can view:
    • Experiment State—Running, Finished or Disabled, and start and end dates.
    • Best performing variant— the Content Variant with the best conversion rate.
    • Total Views—the number of times the Experiment has been displayed on a Page requested by a visitor (all Content Variants).
  8. The Content Variants section displays a list of Components available in the Experiment and or each Content Variant:
    • Views—the number of times the Content Variant has been displayed on a Page requested by a visitor.
    • Conversions—a conversion is registered when a visitor clicks a link in the Content Variant.
    • Conversion Rate—the conversion rate is the number of Views divided by the number of Conversions: 10 Conversions in 100 Views equals a Conversion Rate of 10%.
  9. In the statistics graph, select one of the following views in the drop-down menu:
    • Cumulative view—shows the accrued conversion rates over time (the trend)
    • Daily view—shows conversion rates on a day by day basis (peaks and troughs)

    The Statistics Graph shows Experiment conversion rates over time for each Content Variant in the Experiment. The performance of each Content Variant is plotted on the graph: the vertical axis plots the Conversion Rate, and the horizontal axis the Timeline (the duration of the Experiment, that is the Start Date to End Date). Each Content Variant is assigned a different random color (color has no meaning).

  10. Where the vertical axis meets the horizontal axis, open the tooltip to view variant details for that day.
  11. Zoom in between selected dates:
    • Click and drag on the graph to select the start and end dates you want to zoom in on. When you release the mouse button, the graph shows analytics of the selected dates.
    • Zoom in to within two days (two data points).
    • Click the magnifying glass to zoom out (go back one step).
  12. Click Close to close the Experiment.