To boost weekly active users of the BBC Weather app, the team focused on improving user retention through personalised features like timed notifications using Airship and Optimizely. This case study focuses on the experimentation process along with user research, co-design workshops, content design and lean UX methods.
As one of the oldest weather broadcasting platform based in the UK with 12.5 million weekly active users, BBC Weather is one of BBC's biggest apps. The objective was to increase the weekly active users of the Weather app by enhancing the user experience through personalised and relevant features. Currently, most weather users check the app 1-3 times per week. One of the biggest challenges of BBC is standing out against competitors like Google and Apple weather apps. To address this, we aimed to improve user engagement by introducing features that catered to user's needs and preferences.
“I like to check the weather in the morning as it helps me feel prepared” - 35, BBC Weather user from Surrey
Based on our previous research, we knew that most users typically accessed the weather app in the mornings and only once a day. To encourage habitual use, we developed a feature allowing users to receive weather updates at their preferred time. We used the model of AARRR (Acquisition, Activation,Retention, Referral, Revenue) framework and lean UX model to identify where our opportunity for growth was. Since the BBC is a public service media, revenue as a metric is not applicable, however we do focus on impact instead.
Since we have a wide user audience, our focus was on retention (i.e - Get users to visit the app at least thrice in a week) I led the co-design workshop to brainstorm improvements and ideas where we utilised the research methods such as Crazy 8 and Impact-Effort Analysis to funnel down our ideas.
We shortlisted three ideas and through collaborative decision-making, we decided to experiment with notification banners to increase user engagement and came up with the following hypothesis. The main reason why we picked this idea was because it was easiest to implement and required the least amount of development effort.
.png)
Based on the new hypothesis, I created wireframes for the notification banner that we would show to the new and existing users.
We already had a notification feature but we wanted to highlight that users can set notifications for a certain time during the day. In order to highlight this feature, we decided we would show the following banner to the users twice when they open the app.
The challenge with the notification design was that we could not build this as a native component in a short time. Therefore, we decided to use Airship - an experimentation platform to upsell this banner and use deeplinking to send the users to notification.
.gif)
The constraints with working on Airship was that we could not customise the font style or resize the size of the modal banner. Based on these limitations, I created a new banner in Airship that was in line with the BBC branding. Since the process of setting up notifications was time consuming, we wanted to use the right term for the CTA button to encourage users to sign up for daily notifications.
Since we had multiple options for the call to action (CTA) button, we decided to use the method of MVT testing and conduct an experiment to test out which CTA button performed the best. The control group had the CTA of 'Set up' and Variant A had 'Get Started' and Variant B had 'Continue'.
Experiment Criteria
Length of experiment - Two weeks
Interaction - A dismissible banner would be shown to the user when they open the app
Success metrics - Click through rate
.jpg)
We initially launched this experiment with our Android users. Early data showed a significant increase in morning traffic, indicating positive user response. After running the experiment for two weeks, we concluded our hypothesis.
We think this CTA was successful because it gave users the impression that they are going through a quick guide as to how to set up the daily notification. We considered this experiment to be a success and thus, we released this feature to all our users.
After successfully conducting the experiment on Android, we released the same feature on iOS. Following were my highlights and biggest takeaways from this project -
To know more about this and other BBC projects, get in touch here
