There is such a thing as too much data (ToDUX Newsletter #4)

Dear readers and friends,

I hope you’re all doing good. 😊 I have a two new pieces, both of which have been published in the UX Collective.

The first one—“Listen to users, but only 85% of the time”—investigates why it’s not in the best interest of businesses and designers alike to always listen to users and data.

The second one—“AI will be a thing in Design and Research, but there are some issues to resolve first”—deals with the current challenges and opportunities for artificial intelligence in supporting design processes.

There is such a thing as too much data

Data-driven design is a proven success factor that more and more digital businesses embrace. At the same time, academics and practitioners alike warn that when virtually everything must be tested and proven with numbers, that can stifle creativity and innovation.

But why is that the case?

  1. Users are generally prone to status quo bias and so-called innovation resistance;
  2. “Lean” or “quick ‘n’ dirty” user testing is not always suitable and can lead to faulty results and conclusions; and
  3. business KPIs that are commonly used to verify designs are often short-term in nature.

My article argues that Taleb’s Black Swan theory can solve this dilemma. It shows that online experimentation, and therefore digital design, are fat-tailed phenomena and, hence, prone to Black Swans. It introduces the notion of Black Swan designs—“crazy” designs that make sense only in hindsight—along with four specific criteria.

To ensure incremental improvements and their potential for innovation, businesses should apply Taleb’s barbell strategy: Invest 85‒90% of resources into data-driven approaches and 10‒15% into potential Black Swans.

☞ Listen to users, but only 85% of the time: How Black Swans Can Save Innovation in a Data-Driven World in The UX Collective

Design and research are a thing. Artificial intelligence (AI) is a thing. But they’re not quite a thing together yet.

To what extent do digital designers and user researchers in industry make use of AI-powered systems to support their work? The answer seems to be “not much,” as we found in a survey with 34 practitioners. In general, there seems to be little awareness of the specific advantages AI can bring to a design process and which tools are already out there, despite significant issues with existing processes that could be well mitigated with the help of AI. However, designers and researchers are very open to the topic and would appreciate AI-powered systems to support them in both ideation and evaluation. Providing practitioners with such systems that are easily accessible and demonstrate added value holds great opportunities.

This article describes research that has been conducted in collaboration with Maxim Bakaev, Johanna Jagow, and Sebastian Heil. The research paper was published at the 2022 International Conference on Web Engineering (ICWE).

☞ AI will be a thing in Design and Research, but there are some issues to resolve first in The UX Collective

Enjoy reading and until next time! 👋🏻 Also, I’m always glad to get feedback on my articles and this newsletter. So, please, feel free to just reply to this email.


I love coffee, and if you enjoy what I write, you can spend me one if you like.

This is an issue of my newsletter Tales of Design & User Experience (& Other Stuff)—abbreviated ToDUX—that has been cross-posted here for archival reasons. If you want to get my latest takes and writing delivered straight to your inbox, you can subscribe at 🗞


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.