INUIT: The Interface Usability Instrument

INUIT LogoAs one of the building blocks of my PhD thesis, I have developed a novel instrument for measuring the usability of web interfaces, which is simply called Inuit—the Interface Usability Instrument1. This was necessary because a usability instrument that is suited for the automatic methods for Search Interaction Optimization I have developed in my PhD project must fulfill three particular requirements, which are not met by any existing instruments:

(R1) A minimal number of items.
(R2) Items with the right level of abstraction for meaningful correlations with user interactions recorded on the client.
(R3) Items that can be applied to a web interface in terms of a stand-alone webpage.

The Instrument

Inuit has been designed and developed in a two-step process: First, over 250 rules for good usability from established guidelines and checklists were reviewed to identify a set of common underlying factors (or items) according to R2. From these underlying factors, a “structure” of usability based on ISO 9241-11 was created, which was then shown to 9 dedicated usability experts in the second step. The experts—all of which were working in the e-commerce industry—reviewed the given “structure” and proposed changes according to their perception of web interface usability. Finally, seven items have been identified:

  1. Informativeness
  2. Understandability
  3. Confusion
  4. Distraction
  5. Readability
  6. Information Density
  7. Reachability

These items can be translated to, e.g., the following yes/no questions for use in a questionnaire for determining the usability of a webpage:

  1. Did you find the content you were looking for?
  2. Could you easily understand the provided content?
  3. Were you confused while using the webpage?
  4. Were you distracted by elements of the webpage?
  5. Did typography & layout add to readability?
  6. Was there too much information presented on too little space?
  7. Was your desired content easily and quickly reachable (concerning time & distance)?

Conclusions

A confirmatory factor analysis based on a user study with 81 participants has proven that our instrument reasonably well reflects real-world perceptions of web interface usability. Inuit was first introduced at the workshop “Methodological Approaches to Human–Machine Interaction”, which was held as part of the 2013 Mensch & Computer conference. The corresponding paper is named Towards Metric-based Usability Evaluation of Online Web Interfaces (full-text here). The final version of the instrument has been presented at this year’s International Conference on Design, User Experience and Usability (DUXU), which has been held in Los Angeles. The full research paper is titled Inuit: The Interface Usability Instrument and available via Springer (full-text here).

Future Work

In the future, I intend to transfer Inuit into the context of my current work. That is, I intend to use it for evaluating the web interface of HoloBuilder, which enables users to create 3D content for the web, in contrast to the usual 2D content that is consumed nowadays. It will be particularly interesting to see whether both, 2D and 3D web interfaces can be meaningfully evaluated using the same minimal instrument. Furthermore, Inuit will be applied in the context of the research on evidence-based computing that is happening at the VSR research group at Technische Universität Chemnitz.

P.S.: Thanks a lot to Viet Nguyen for the awesome Inuit logo! 🙂

1 Please note the small caps!

Advertisements

#papershizzle

Phew! It was a rather busy (that’s why I haven’t been posting in a while), but also very successful start into the new year. A total of three full papers have been accepted at various conferences and journals. So basically, I’ve been revising and resubmitting papers since Christmas.

First, our paper about Inuit has been accepted at the 4th International Conference on Design, User Experience and Usability (DUXU), which will be held as a part of HCI International 2015 in Los Angeles, California. Inuit is a new usability instrument for interfaces that has been specifically designed for our concept of Usability-based Split Testing. An instrument of this kind contains a set of observable items that are used to predict a latent (i.e., non-observable) variable—in our case, usability. For instance, a person’s intelligence is a latent variable that can only be assessed with a number of specific questions (or items). Therefore, IQ tests are instruments.

Second, an article that is an extended version of our ICWE 2014 paper about SMR has been conditionally accepted by the Journal of Web Engineering (JWE). SMR is a streaming-based system that allows for the prediction of search result relevance from user interactions. In the extended version, we further elaborate on specifics of SMR’s data processing algorithm and complexity. Also, we describe the integration of our system into a real-world industry setting.

Image taken from http://chi2015.acm.org/.

Finally—and probably most importantly—our paper titled “S.O.S.: Does Your Search Engine Results Page (SERP) Need Help?” has been accepted at CHI 2015, which is the premier conference on human–computer interaction and will take place in Seoul! What a great success 🙂 . S.O.S. is the abbreviation for SERP Optimization Suite, which comprises two components. (1) WaPPU, which is a tool for inferring usability scores from users’ interactions with an interface. WaPPU was already presented at ICWE 2014 and has been augmented with (2) a catalog of best practices to form S.O.S. That catalog contains potential causes and corresponding countermeasures for suboptimal usability scores. An extension to WaPPU now automatically detects such suboptimal scores and proposes optimizations based on the catalog.

I am very excited about these accepted papers and definitely looking forward to presenting them to an audience of world-renowned experts. As a side note, a revised and extended version of my post titled What is ›Usability‹? has been published as a technical report in the series “Chemnitzer Informatik-Berichte” (roughly translated: “Computer Science Reports of Chemnitz University of Technology”).

So after this very successful start of the year, let’s see what else 2015 will bring. Stay tuned! 🙂