The way A/B testing is done in many—if not most—companies today, p-values and significance are pretty much meaningless. Still, decision models often only allow treatments to be implemented if they achieved a "significant uplift". One better alternative is to employ a Bayesian approach which allows to report probabilities for minimum uplifts and to calculate ROI on a case-by-case basis.
Vamos a Barcelona 🏖️ (ToDUX Newsletter #2)
Dear Subscribers, this is the first newsletter sent from my new home, which is indeed Barcelona 🏖️, and I wanna use it to update you on a couple of things. We still need to get rid of significance in A/B testing First of all, my piece on significance and A/B testing has finally been published… Continue reading Vamos a Barcelona 🏖️ (ToDUX Newsletter #2)
We Need to Get Rid of Significance in A/B Testing (ToDUX Newsletter #1)
Dear Subscribers, Welcome to the first issue of ToDUX (= Tales of Design & User Experience; I’ve skipped the “& Other Stuff” for the acronym). But first things first—thank you all so much for joining my mailing list. I feel very honored. 🙂 As promised, I want to use this newsletter to inform you about… Continue reading We Need to Get Rid of Significance in A/B Testing (ToDUX Newsletter #1)