A Peek Behind The Curtain: How OneLaunch A/B Tests Our App
When we at the OneLaunch team look for ways to improve our software, we know only so much. We know how we use OneLaunch and what we like and don’t like. We don’t know how you, our 1 million and growing active daily users, move around the app. What confuses you? What do you use the most? Are you using features the way we intended?
And, as much as we’d love to pull up a chair and watch each of you as you use OneLaunch — we do love to travel — it’s not possible to visit all 1 million+ of you. So we rely on technology to gather data and help us interpret what we’re doing right and what we need to change.
In this article, we are giving you a peek behind the curtain to explain how we use a technology called Mixpanel to test and learn about you and your preferences for using Windows and the OneLaunch app.
Why We Started This Experiment
We faced a challenge. We experienced difficulty in attracting and maintaining new OneLaunch users, which, as you can imagine, was frustrating to our team. We needed a way to properly understand user patterns so we could develop and introduce new features.
That’s when we decided to use Mixpanel. This tool allowed us to test different product features and see the reactions of various user groups. We share updates about these tests company-wide via Slack, a web-based instant messaging app. Thanks to the data provided by Mixpanel, we’ve been able to fix issues, enhance our product’s user-friendliness, and even discovered a way to increase our revenue.
As Neil, our CEO, said, “We are extremely interested in our users’ behaviors, so we love using Mixpanel to delve into why users behave in certain ways.”
Why did we choose Mixpanel?
We decided on Mixpanel after receiving recommendations from trusted industry contacts. After looking into other options, we felt Mixpanel was not only powerful but also user-friendly. As Neil, our CEO, was quoted as saying in a MixPanel case study: “Mixpanel is literally everywhere. Links to boards are in our software, and we share them … during leadership meetings. Every single person supporting OneLaunch consumes insights from Mixpanel.”
Before Mixpanel, we built our own tool and called it “Query Builder.” Although it was somewhat functional, it had numerous shortcomings. We couldn’t accurately track and measure what we wanted, and interpreting the data was challenging. Long story short: We lacked comprehensive knowledge about how people were using OneLaunch. Most of the time, we only learned about problems from our customers or partners.
How Mixpanel works
Mixpanel enables us to run many A/B tests to see how users react to new features. It helps us understand who should be encouraged to take action, how often, and which prompts are beneficial. We also use Mixpanel to identify the things our users like or dislike about our product.
With A/B testing, we can make slight changes to user experiences, such as the color of a button, the placement of an icon, the size of a tab and send one experience to one group of users and another experience to another group. Mixpanel works in the background, without interrupting the user experience and without collecting personal information. The story that Mixpanel tells us is how our customers are using the app and if they’re following the intended user experiences.
That’s the simple view of Mixpanel. It does a lot more and is very sophisticated! We use it to test, learn and iterate on OneLaunch features, but we are still always interested in hearing your feedback, too.
How Mixplanel improved OneLaunch
With Mixpanel, we realized that our new tab screen was generating a lot of views. So, we added weather information and news to it, which doubled our audience. Then, we started displaying ads on this page, earning additional revenue. We’d been hesitant to add display ads to our users’ desktops, but by using Mixpanel, we were able to test this. We learned that our users didn’t mind the ads, which allows us to keep OneLaunch free for you. Without Mixpanel, this opportunity would have gone unnoticed.
We learned that having too many features could be detrimental. Therefore, we removed the features that our users didn’t value.
We were also surprised to find out how significantly holidays impacted our product usage!
So, what’s the result?
One prominent example is that we increased our conversion rate (new active users) by 60% by identifying issues during our installation process. This improvement allowed us to invest more in marketing and acquire more users. Mixpanel has also improved our understanding of technical metrics.
Without Mixpanel, our journey would have been much harder. Our explorations and victories would have been fewer. However, with Mixpanel’s aid, we recently hit a milestone of 1 million daily users. We are grateful for Mixpanel’s assistance and look forward to more success.
Like us, thousands of companies use Mixpanel to enhance their products. Create a free account today to see how it can help your team.