Getting Started

How to run an A/B test for rate metrics without p-values and confidence intervals.

Have you ever struggled trying to explain your Frequentist test results (namely null/alternative hypothesis, p-values and confidence intervals) to a non-technical audience or… even yourself? Do you wish you could just say “there’s a 95% probability group A has a better click-through rate than group B” after running an A/B test and be done with it? If yes, then you have landed on the right post, if not we have pictures of cute dogs so you might want to stay anyway!

You are working as a Data Scientist for WoofWoof, a dog toy company. The Marketing Department wants to run…


Hands-on Tutorials

Learn about the different Upper Confidence Bound bandit algorithms (UCB1, UCB1-Tuned, UCB1-Normal). Python code provided for all experiments.

Introduction

In this series of posts, we experiment with different bandit algorithms in order to optimise our movie nights — more specifically how we select movies and restaurants for food delivery!

For newcomers, the name bandit comes from slot-machines (known as one-armed bandits). You can think of it as something that can reward you (or not) every time you interact with it (pull its arm). The objective is, given a bunch of bandits that give different rewards, to identify the one that gives the highest ones, as fast as possible. …


A black screen with green vertical letters.
A black screen with green vertical letters.

No idea what Monte-Carlo simulations are? This post is for you. But first, let’s talk about pizza. More specifically, a pizza inside a box.

You are really bored and have decided to calculate the area of your pizza to pass the time (of all the things to do). You know the formula for calculating the area of a circle, Area=𝜋R², but you have no idea what the radius of the pizza is, i.e. the R, and have no rulers around. Suddenly, you remember you have a huge jar full of identical round olive pieces. You also know the area of…


Everything’s great until proven otherwise. Learn about the Optimistic Initial Values algorithm. Python code provided for all experiments.

Introduction

In this series of posts, we experiment with different bandit algorithms in order to optimise our movie nights — more specifically how we select movies and restaurants for food delivery!

For newcomers, the name bandit comes from slot-machines (known as one-armed bandits). You can think of it as something that can reward you (or not) every time you interact with it (pull its arm). The objective is, given a bunch of bandits that give different rewards, to identify the one that gives the highest ones, as fast as possible. …


Learn how Epsilon-Greedy works. Full python code provided for all experiments.

It’s Saturday night and you are getting ready for a movie night. But first, you want to order food! There’s a selection of restaurants in your area and you’ve only tried a few. The question therefore arises, should you order your favourite dish from the ones you’ve tried so far (exploitation) or should you try something new (exploration)?

This dilemma between exploration and exploitation is what multi-armed bandits algorithms try to solve. The aim is to maximise the reward from a sequence of actions (in the restaurant case we want to maximise satisfaction from ordering food) over the long run…

Artemis Nika

Data Scientist.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store