Master This Data Science Skill and You Will Land a Job In Big Tech- Part I
Are you a data scientist dreaming of landing a job in Big Tech but you're not sure what skills you need to get there?
Well, I've got a secret weapon that could be just what you need to land your dream job in top tech companies.
A few months ago, I wrote this article about all the essential skills you need to get hired by the best tech firms, and today, we're going to focus on one of those crucial skills: Experimentation.
Experimentation is a statistical approach that helps us isolate and evaluate the impact of product changes – launching features, UX updates, and all!
But why is experimentation so important for standing out among other data scientists?
It's simple. The biggest tech companies are all about creating great products, and experimentation is a vital tool in achieving that.
If you can become an expert in experimentation, you'll have a significant advantage over other candidates because most job seekers overlook this skill and don't know how to develop it.
You might not be familiar with the term, but you've certainly heard of A/B testing, right? Well, A/B testing is one of the two main methods used in product experimentation, the other being Feature Rollouts.
I'm Khouloud El Alami, a Data Scientist at Spotify, and I'm here to share the experimentation knowledge I've gained from working with the best in the industry. In this article, we'll cover:
- What is experimentation
- Why experimentation is so crucial
- The two main experimentation methods
- How to decide between A/B Tests & Feature Rollouts
By the end of this article, you'll know enough about experimentation to get you started on your journey to landing a job in big tech. Also, this article is the first part of a series on Experimentation.
Make sure to subscribe to my Medium page so that you don't miss future articles on the topic. I also write a free weekly newsletter where I share more tips to get into tech.
What is Experimentation anyway?
In January, I started working on the TV experience of the Spotify app. It's a great place to be in because we're launching a lot of features.
And who says lots of feature releases, says lots of experimentation!
To put it simply, experimentation is a data-driven approach that allows companies to test and evaluate the impact of product changes, such as new features or UX updates, on user behavior and key metrics.
As a data scientist, I can't stress enough the importance of having a strong grasp of statistics and probability theory.
These foundational skills are crucial because they form the basis of many tools and techniques we use daily, and experimentation is a prime example.
At its core, experimentation is deeply rooted in the concept of hypothesis testing, which we all studied in Statistics. In hypothesis testing:
- We start with a question or a hypothesis that we want to test "Will adding playlist shortcuts in the home screen increase user engagement?".
- To test this hypothesis, we make a change (add the shortcuts).
- Then we compare the outcome (user engagement) between two groups: one that experienced the change (treatment group) and one that didn't (control group).
The Scientific Method of Experimentation (+ legit example)
Experimentation follows a similar scientific method to hypothesis testing:
- Observation: Identify an area of interest, a user pain point, or a problem to solve.
- Question: Formulate a specific question or hypothesis based on the observation.
- Hypothesis: Develop a hypothesis that predicts the outcome of the change.
- Experiment: Design & conduct an experiment to test the hypothesis. I will dig more into experiment design in the future!
- Analysis: Collect & analyze the data to see if the results support the hypothesis.
- Conclusion: Draw conclusions & make decisions based on the findings.
Once it's done, we move on to the next problem and repeat this process.
Hypothetical Example: Is it worth adding lyrics to the Spotify app?
When I started working on the TV experience of the Spotify app, we tested many new features. To test each feature, we followed this scientific method. For example:
- Observation: Users enjoy singing along with songs and often look up lyrics on external websites.
- Question: Will adding lyrics to songs on Spotify increase the time users spend on the app, increase user satisfaction, and reduce churn rates?
- Hypothesis: Displaying lyrics will increase user engagement and retention.
-
Experiment: → **** Control group: Users have access to audio content without lyrics. → Treatment Group: Users have access to audio content with lyrics displayed. → Metrics: Average session duration, daily active users (DAUs), & churn rates.
- Analysis: Compare the metrics between the control and treatment groups to check if adding lyrics increased key metrics.
- Conclusion: If the treatment group shows higher engagement and retention, this supports the hypothesis that adding lyrics improves user experience. Spotify can then decide to implement the lyrics feature for all users.
Disclaimer: I don't share any confidential information from my workplace here, this is a hypothetical scenario (I'd like to keep my job
