The way A/B testing is done in many—if not most—companies today, p-values and significance are pretty much meaningless. Still, decision models often only allow treatments to be implemented if they achieved a "significant uplift". One better alternative is to employ a Bayesian approach which allows to report probabilities for minimum uplifts and to calculate ROI on a case-by-case basis.
I'm proud that my article "Growth Marketing Considered Harmful" has been published in the latest issue of i-com ‒ Journal of Interactive Media. Abstract: In today’s e-commerce industry, conversion rate optimization is often considered essentially the same as user experience optimization. In addition, there is a strong focus on quantitative experimentation, which some deem a… Continue reading “Growth Marketing Considered Harmful” Published in i-com
Despite evidence that employing a human-centered design (HCD) approach to product design yields a strong competitive edge, many companies neglect HCD or don't even consider it in the first place. One reason for that is probably that many who think they're working user-centered are practicing something entirely different: KPI-Centered Design. This article inquires into different forms of design processes, into the differences between proper HCD and KPI-Centered Design, and how a resonable compromise can look like: Formulating hypotheses based on evidence rather than just KPIs or gut feeling, and complementing quantitative experimentation with more qualitative insights.
There is a plethora of customer research methods out there and it can be difficult to stay on top of things when it comes to choosing the right method for a given research question. Specifically in the realm of product discovery, when it comes down to being able to decide whether an idea generates business value or not, certain methods lack validity or are not properly applied. In this article, André Morys provides an overview over when to use which (combination of) methods(s) for product discovery.
The corresponding publications are: Maximilian Speicher, Andreas Both and Martin Gaedke (2014). “Ensuring Web Interface Quality through Usability-based Split Testing”. In Proc. ICWE. Maximilian Speicher, Andreas Both and Martin Gaedke (2014). “WaPPU: Usability-based A/B Testing”. In Proc. ICWE (Demos). For more information about WaPPU, please see this previous post. Special thanks go to Fred Funke,… Continue reading How to Infer Usability from User Interactions. My Poster Presented at #ICWE2014