Tales of Design & User Experience (ToDUX) #16 ⁂ Dear readers & friends, Studies have shown over and over that even professional researchers (including statisticians!) can’t correctly interpret a confidence interval or a p-value most of the time. Then, how could you expect the average Joe to be better at that? Frequentist statistics (the kind that gives you those… Continue reading No, you most probably don’t know what p-values and confidence intervals mean (but don’t feel bad about it, you can just do Bayesian instead)
Tag: online experimentation
“If It’s Not Tested It Might Be Perfectly Fine”: The Counterintuitive Nature of Not Testing Everything
Tales of Design & User Experience (ToDUX) #15 ⁂ Dear readers & friends, So, you might have noticed by now that “research done wrong in industry” is kind of my pet peeve, and I’m sorry you have to endure that. However, I’m not quite done with the topic yet. My new article in The UX Collective,… Continue reading “If It’s Not Tested It Might Be Perfectly Fine”: The Counterintuitive Nature of Not Testing Everything
There is such a thing as too much data
Tales of Design & User Experience (ToDUX) #4 ⁂ Dear readers and friends, I hope you’re all doing good. 😊 I have a two new pieces, both of which have been published in the UX Collective. The first one—“Listen to users, but only 85% of the time”—investigates why it’s not in the best interest of… Continue reading There is such a thing as too much data
“We Need to Get Rid of Significance in A/B Testing, Seriously!” Published in ACM interactions
The way A/B testing is done in many—if not most—companies today, p-values and significance are pretty much meaningless. Still, decision models often only allow treatments to be implemented if they achieved a "significant uplift". One better alternative is to employ a Bayesian approach which allows to report probabilities for minimum uplifts and to calculate ROI on a case-by-case basis.
Vamos a Barcelona 🏖️
Tales of Design & User Experience (ToDUX) #2 ⁂ Dear Subscribers, this is the first newsletter sent from my new home, which is indeed Barcelona 🏖️, and I wanna use it to update you on a couple of things. We still need to get rid of significance in A/B testing First of all, my piece… Continue reading Vamos a Barcelona 🏖️
We Need to Get Rid of Significance in A/B Testing
Tales of Design & User Experience (ToDUX) #1 ⁂ Dear Subscribers, Welcome to the first issue of ToDUX (= Tales of Design & User Experience; I’ve skipped the “& Other Stuff” for the acronym). But first things first—thank you all so much for joining my mailing list. I feel very honored. 🙂 As promised, I… Continue reading We Need to Get Rid of Significance in A/B Testing
2008 – Tales of Design & User Experience 