Uber x Facebook

Growth Experimentation
Growth

Optimizing for better conversion lift in rider acquisition

I analyzed and tested ad campaigns for better cost efficiencies

Quantitative results:
2x cost per sign up for one side of A/B test
Qualitative results:
Insights from experiment drove clear next steps for campaign to increase sign up conversions at lower cost
VISIT PROJECT
work brief

Competition in Facebook ads is high in certain markets for transportation apps, and there is a key decision to be made on whether new creatives would improve cost efficiencies of campaigns. My team and I built an experiment to understand how to proceed considering the increasing costs of Facebook ad spend.

Context

I worked with Paid Social experts in the Uber growth marketing team in APAC in 2018 to driver rider and driver acquisition through the Facebook ads channel. Out of all its channels, Facebook is one of the top spenders. However, Grab was increasing its market presence in the Southeast Asian markets due to increasing local trust since it taps on its strong local brand identity. This created high cost inefficiencies with Facebook spend where spend is high and signup numbers are low leading to relatively higher costs per sign up.

As a result, a management decision had to be made on whether budget needs to be cut on those markets. The team and I had to design an experiment to prove that the markets were still profitable to spend in.

Initial Analysis

To understand how to design this experiment, I first looked at the current state of the campaigns.

The brand and design teams have been pushing many new creatives to the Growth team to use and this has been pushed live without much testing on whether the creatives work well or not. At the same time, there are no comparable campaigns with a clean setup (in terms of optimization settings and creatives) to analyze metrics for. Thus, I proposed to design a simple 28-day experiment that splits the new promotional creatives and best-performing evergreen creatives by 80/20. Spend is controlled at the adset level.

The setup is simple: no A/B test but I created a campaign with the "Dynamic Creative" option switched on. Spend is controlled at the adset level and all the creatives are then uploaded as options for Facebook's algorithms to work its magic and automatically generate the creatives that work best.

The results were pretty clear: 

  • Despite volume of creatives for promo versus evergreen to be split 80/20, spend for promo versus creative was split at 65% to 35%.
  • Cost per signup for promo was 4x higher than the evergreen creatives.

At the same time, cost per signup for evergreen creatives was comparable to historical benchmarks (over the course of one year during the launch of the evergreen creatives). This is a good insight to show that the evergreen creatives are still doing well in those markets from a cost efficiency standpoint but the creatives are important for the campaign performance.

Experiment Design

The initial question of whether the markets are still worth to spend in has been proven right.

However, performance has been stagnant for over 1.5 years for the evergreen creatives. The next question is how do we improve cost efficiencies in those markets?

Since both creatives in this quick experiment were "conversion" creatives which means that the call-to-action is to sign up and download the app, there is a question on whether there needs to be a mix between "conversion" and "consideration" creatives to improve performance.

Therefore the experiment was designed with two campaigns setup: (1) best-performing evergreen creatives focused on conversion funnel and (2) new creatives focused on awareness and conversion funnel stages.

The details of the setup are below:

  • A/B Split test setup (50/50) on Smartly ad optimization platform (reference)
  • Relaunch evergreen creatives in a separate campaign to get rid of historical data vs new creatives
  • Enable Smartly creative optimisation on both campaigns 
  • Disable dynamic creative optimization since only specific copies can be used with specific creatives

Results and Next Steps

The cost per sign up for evergreen conversion creatives is 2x lower than the new awareness and conversion creatives.

Therefore, it means that the market is pretty aware of the brand but needs more push to consider purchasing. The next steps is to develop new "consideration" funnel creatives to mix with the evergreen ones to determine if that helps improve costs per sign up.

LET'S TALK

Connect with me here