The U Score: Redesigning Usability Testing

logo-bigUsability testing is often perceived as cumbersome and time-consuming and therefore not thoroughly applied. This was one of the key observations leading to the topic of my PhD thesis. Particularly conducting tests with actual users is often omitted, which results in the release of suboptimal products and websites. In my thesis, I tackle this problem through more automatic evaluation and optimization, however, in the specific context of search engines. Yet, every type of website—no matter if private or professional—should undergo at least one usability test before its release. Therefore, we need to redesign usability testing itself:

  • It must be quicker.
  • It must be cheaper.
  • It must be easier to understand.
  • Still, the result must be as precise as possible.


The U Score is a more general derivative of the findings of my PhD project that provides quick and precise usability evaluation for everyone based on actual research. Any designer or developer who isn’t able to conduct a regular usability test can answer a minimal but exhaustive set of yes/no questions and receives a single usability score for their website or web app. The questions have been designed to be as objective as possible and are based on established research findings. Also, for time reasons I try to minimize the need to involve other people, which, however, cannot be completely eliminated (still, you can receive a complete U Score with the help of only three friends who have a look at your site).

In this way, the U Score provides an approach to usability testing that is as precise as possible given the minimal effort it requires. It’s intended for situations in which designers/developers don’t have the chance to conduct a traditional usability test. Also, it addresses everyone who needs a quick assessment, has never tested the usability of a website before or is new to usability testing. However, please note that the U Score can only be an approximation and is not a complete substitute for established usability testing methods. Still, it gives you a very good baseline according to the motto: Any usability test is better than no usability test!

The current version of the U Score is still in beta development status. Therefore, I highly appreciate your feedback, which you can add to this public Trello board.

For implementation, I’ve relied on a number of well-known technologies and frameworks in combination with some that were new to me (the ones marked with an asterisk):

I hope the U Score can help to reduce the number of websites who’ve never been tested at all and particularly help you to conduct more and quicker usability tests. Enjoy!  🙂


INUIT: The Interface Usability Instrument

INUIT LogoAs one of the building blocks of my PhD thesis, I have developed a novel instrument for measuring the usability of web interfaces, which is simply called Inuit—the Interface Usability Instrument1. This was necessary because a usability instrument that is suited for the automatic methods for Search Interaction Optimization I have developed in my PhD project must fulfill three particular requirements, which are not met by any existing instruments:

(R1) A minimal number of items.
(R2) Items with the right level of abstraction for meaningful correlations with user interactions recorded on the client.
(R3) Items that can be applied to a web interface in terms of a stand-alone webpage.

The Instrument

Inuit has been designed and developed in a two-step process: First, over 250 rules for good usability from established guidelines and checklists were reviewed to identify a set of common underlying factors (or items) according to R2. From these underlying factors, a “structure” of usability based on ISO 9241-11 was created, which was then shown to 9 dedicated usability experts in the second step. The experts—all of which were working in the e-commerce industry—reviewed the given “structure” and proposed changes according to their perception of web interface usability. Finally, seven items have been identified:

  1. Informativeness
  2. Understandability
  3. Confusion
  4. Distraction
  5. Readability
  6. Information Density
  7. Reachability

These items can be translated to, e.g., the following yes/no questions for use in a questionnaire for determining the usability of a webpage:

  1. Did you find the content you were looking for?
  2. Could you easily understand the provided content?
  3. Were you confused while using the webpage?
  4. Were you distracted by elements of the webpage?
  5. Did typography & layout add to readability?
  6. Was there too much information presented on too little space?
  7. Was your desired content easily and quickly reachable (concerning time & distance)?


A confirmatory factor analysis based on a user study with 81 participants has proven that our instrument reasonably well reflects real-world perceptions of web interface usability. Inuit was first introduced at the workshop “Methodological Approaches to Human–Machine Interaction”, which was held as part of the 2013 Mensch & Computer conference. The corresponding paper is named Towards Metric-based Usability Evaluation of Online Web Interfaces (full-text here). The final version of the instrument has been presented at this year’s International Conference on Design, User Experience and Usability (DUXU), which has been held in Los Angeles. The full research paper is titled Inuit: The Interface Usability Instrument and available via Springer (full-text here).

Future Work

In the future, I intend to transfer Inuit into the context of my current work. That is, I intend to use it for evaluating the web interface of HoloBuilder, which enables users to create 3D content for the web, in contrast to the usual 2D content that is consumed nowadays. It will be particularly interesting to see whether both, 2D and 3D web interfaces can be meaningfully evaluated using the same minimal instrument. Furthermore, Inuit will be applied in the context of the research on evidence-based computing that is happening at the VSR research group at Technische Universität Chemnitz.

P.S.: Thanks a lot to Viet Nguyen for the awesome Inuit logo! 🙂

1 Please note the small caps!

[offene Masterarbeit] Was that Page Pleasant to Use? Usability-Metriken in einer echten Suchmaschine

Es gibt viel zu viele schlechte Webseiten! Schon mal versucht, auf die Tagessätze für Auslandsreisekosten zu finden? Falls nicht, einfach mal ausprobieren und viel Spaß dabei! Oder schon mal auf der Seite der Uni Würzburg versucht, herauszufinden, wie genau eine Bewerbung für den Bachelor in Wirtschaftswissenschaften abläuft? Nein? Ist auch eigentlich besser so, weil der Versuch einen leicht in den Wahnsinn treiben kann.

Motivation: Usability? Nein, danke!

Viele Webseiten (auch großer Unternehmen) beweisen keinerlei Gespür für grundlegende Usability-Prinzipen, welche weder sonderlich neu noch sonderlich kompliziert sind. Häufig sind z. B. Informationen, die eine Großzahl an Nutzern betreffen, nicht direkt über die primäre Navigation erreichbar, sondern nur über verschlungene Pfade und zahllose Klicks. Und das trotz einer Fülle an Frameworks und Content-Management-Systemen, die modernste Webdesign- und Usability-Prinzipien unterstützen. Der wohl häufigste Grund für mangelnde Usability einer Webseite ist die Tatsache, dass entsprechende Tests nur unzureichend oder gar nicht durchgeführt werden, häufig aus Kosten- oder Zeitgründen.

The WaPPU dashboardUm dem entgegenzuwirken, habe ich als Teil meiner Doktorarbeit ein prototypisches Tool namens WaPPU entwickelt, welches es ermöglicht, wesentlich günstigere A/B-Tests auf Basis einer neuartigen Metrik für Usability durchzuführen. Das heißt, die Usability zweier leicht unterschiedlicher Versionen derselben Webseite wird während der Benutzung durch echte Nutzer in Form von Metriken in Echtzeit erfasst und in einem Dashboard visualisiert (siehe Abbildung).

Ziel der Arbeit

Mein Dissertationsprojekt ist eingebettet in die Forschungs- und Entwicklungsabteilung der Unister GmbH in Leipzig, welche aktuell eine neuartige Reisesuchmaschine entwickelt. Der entwickelte Prototyp soll im Rahmen einer Masterarbeit in diese reale Suchmaschine integriert werden, um verschiedene Interface-Variationen im produktiven Betrieb anhand ihrer Usability bewerten zu können. Weitere Informationen können der offiziellen Ausschreibung entnommen werden. Interessenten melden sich bitte unter der in der PDF angegebenen E-Mail-Adresse oder über mein Kontakformular.


Ein Demo-Video zum WaPPU-Tool gibt’s hier.