In this article, we will provide step-by-step instructions for how to view our most commonly requested metric - Revenue Uplift - as part of Flywheel Campaign Evaluation. For questions regarding how Campaign Evaluation works, please reference the Flywheel Campaign Evaluation Methodology Explained article.

‍

Step 1: Ensure you have an Audience exporting to a destination with a treatment/control split

In order for Campaign Evaluation to begin, you must have an exported audience with a treatment/control split that has some percentage of customers in each group. You can quickly check which audiences fit this criteria when you open up Audience Builder and search through your saved audiences on the home page. The 'Status' column must show 'Exported' and the 'Treatment / Control' column must show any split that isn't 100%/0% (indicating that you exported the audience with zero customers in the control group).

Once you have identified a qualifying audience and would like to analyze its associated Campaign Evaluation, simply click the audience name to open up the audience.

‍

Step 2: Scroll to the 'Performance' graph that populates for all qualifying audiences

This 'Performance' graph is your window into Flywheel Campaign Evaluation. This graph will quickly identify the total number of customers that have entered your audience over time, along with the performance over time of the selected Metric and Value Type.

The above screenshot reveals that 2,494 customers has qualified for the example audience over time, based off the Flex Filters applied in Audience Builder. The Metric selected is 'Customer Count' and the Value Type is 'Total', revealing a time graph of the total number of customers entering or exiting this audience over time. We can click the dropdown boxes in the upper right hand corner to change the visible Metric and Value Type.

Please note: these Metric options are entirely configurable to fit your needs. Since we hook directly into your data warehouse, we can provide customer performance evaluation on any number of metrics available. This can include revenue, app logins, email opens, opportunities won, etc. Please reach out if you have any questions or need different Metrics enabled for your Audiences!

‍

Step 3: Select 'Metric: Revenue' in the Metric dropdown box

In order to analyze Revenue performance of an audience, we must first select the Revenue metric from the Metric dropdown box. This will reveal Revenue totals for the Treatment, Control, and Adjusted Control groups over time.

Since the treatment/control split for the example audience above heavily favored treatment sizing over control, it is expected to witness overall greater revenue totals out of the Treatment group (blue) vs. the Control group (red). But in order to perform an 'apples-to-apples' comparison of these groups, we appropriately scale the Control group's revenue to equate to the number of customers in the Treatment group, resulting in the calculated Adjusted Control (green) performance. The Uplift generated is effectively the difference between the Treatment group and the Adjusted Control Group.

‍

Step 4: Select 'Type: Uplift' in the Value Type dropdown box

To graphically see the difference in these two curves as a calculated Uplift metric, all you need to do is select 'Uplift' in the Value Type dropdown box.

Assuming your campaign is 'successful' like the example audience shown, you will have a calculated Revenue Uplift total presented to you upon selecting 'Metric: Revenue' and 'Type: Uplift' from the dropdown boxes! Check back frequently to observe how this performance changes over time. If the campaign is successful, you should expect to see positive uplift that hopefully grows over time, indicating that this could be an effective audience to target for future campaigns as well.

How to determine if a campaign is 'unsuccessful'?

It should go without saying that, in reality, there are MANY variables and metrics to consider in determining if a campaign is truly successful or not. Revenue, while often an important metric, is just one of these variables to ultimately consider.

With Flywheel Campaign Evaluation, you have the power to quickly observe if Revenue Uplift is being generated via your experimental audience by comparing the treatment vs. control groups. This, combined with the ability to experiment and learn quickly, are just a few of the many reasons our clients have chosen Flywheel services.

Let's say you launch an experimental audience and begin seeing revenue performance that looks like the following:

From the above graph, we can observe that the Treatment group was outperforming the Adjusted Control group for some time, but then suddenly that performance reversed over the course of about 2 weeks. Now the Adjusted Control is generating greater total revenues than the treatment. The reason for 'why' this happens is not always obvious and may require further investigation. But ultimately we must be realistic that this specific campaign may not be generating revenue lift any further. This doesn't mean you are 'losing money', but rather that the Control group is now spending relatively more than the Treatment group for whatever reason. The resulting Uplift graph would then resemble:

While this result may be disappointing to observe at first, it is important for the growth of your sales and marketing teams to observe and learn quickly from. This may indicate that the selling efforts and/or creative is not as effective with the target audience as initially expected, and empowers your teams to test and pivot faster than they likely could before.

‍

Have further questions regarding how the above graphs are generated or how Flywheel Campaign Evaluation works? Check out our Help Center or reach out to solutions@flywheelsoftware.com to learn more!