The way A/B testing is done in many—if not most—companies today, p-values and significance are pretty much meaningless. Still, decision models often only allow treatments to be implemented if they achieved a "significant uplift". One better alternative is to employ a Bayesian approach which allows to report probabilities for minimum uplifts and to calculate ROI on a case-by-case basis.
Tag: split testing
Vamos a Barcelona 🏖️
Tales of Design & User Experience (ToDUX) #2 ⁂ Dear Subscribers, this is the first newsletter sent from my new home, which is indeed Barcelona 🏖️, and I wanna use it to update you on a couple of things. We still need to get rid of significance in A/B testing First of all, my piece… Continue reading Vamos a Barcelona 🏖️
We Need to Get Rid of Significance in A/B Testing
Tales of Design & User Experience (ToDUX) #1 ⁂ Dear Subscribers, Welcome to the first issue of ToDUX (= Tales of Design & User Experience; I’ve skipped the “& Other Stuff” for the acronym). But first things first—thank you all so much for joining my mailing list. I feel very honored. 🙂 As promised, I… Continue reading We Need to Get Rid of Significance in A/B Testing
“Growth Marketing Considered Harmful” Published in i-com
I'm proud that my article "Growth Marketing Considered Harmful" has been published in the latest issue of i-com ‒ Journal of Interactive Media. Abstract: In today’s e-commerce industry, conversion rate optimization is often considered essentially the same as user experience optimization. In addition, there is a strong focus on quantitative experimentation, which some deem a… Continue reading “Growth Marketing Considered Harmful” Published in i-com
KPI-Centered Design
Despite evidence that employing a human-centered design (HCD) approach to product design yields a strong competitive edge, many companies neglect HCD or don't even consider it in the first place. One reason for that is probably that many who think they're working user-centered are practicing something entirely different: KPI-Centered Design. This article inquires into different forms of design processes, into the differences between proper HCD and KPI-Centered Design, and how a resonable compromise can look like: Formulating hypotheses based on evidence rather than just KPIs or gut feeling, and complementing quantitative experimentation with more qualitative insights.
Product Discovery ‒ A Systematic Approach for Customer Research Methods
There is a plethora of customer research methods out there and it can be difficult to stay on top of things when it comes to choosing the right method for a given research question. Specifically in the realm of product discovery, when it comes down to being able to decide whether an idea generates business value or not, certain methods lack validity or are not properly applied. In this article, André Morys provides an overview over when to use which (combination of) methods(s) for product discovery.
How to Infer Usability from User Interactions. My Poster Presented at #ICWE2014
The corresponding publications are: Maximilian Speicher, Andreas Both and Martin Gaedke (2014). “Ensuring Web Interface Quality through Usability-based Split Testing”. In Proc. ICWE. Maximilian Speicher, Andreas Both and Martin Gaedke (2014). “WaPPU: Usability-based A/B Testing”. In Proc. ICWE (Demos). For more information about WaPPU, please see this previous post. Special thanks go to Fred Funke,… Continue reading How to Infer Usability from User Interactions. My Poster Presented at #ICWE2014
2008 – Tales of Design & User Experience 