Lecture Ergonomical Design @ UTwente

Exploration, evaluation and applying theories in HCI

Posted by mschmettow on 22. February 2010

The first home work in ED was to find two examples of each:

  1. A study exploring an innovative technology

  2. Applying a psychological theory

  3. Evaluating an artefact or a method

I will now summarize the solutions to this homework:

Exploring innovative technology

Originally it was the idea to find studies that explore how people are using innovative technology, like it was the case with the Thatcher study on search behaviour. Such studies typically employ relatively open experimental settings and use qualitative data analysis in order to describe the variety of behaviour.

Apparently, most of the suggestions focussed on the term „innovative“. However, a study that proves a new technology to work as expected I would classify as exploratory with respect to feasibility („Can we make this work?“). Comparing two artefacts in terms of performance measures instead is clearly evaluative. It is also something that is seen critical by at least some researchers in HCI. I remember well the provoking talk of Greenberg and Buxton at CH2008. They claimed that usability evaluation can “be ineffective and even harmful if naively done ‘by rule’ rather than ‘by thought”, in particular if “done during early stage design” (Greenberg and Buxton, 2008).

Below I list my three favorite examples for exploration studies from your results:

  • Bailenson et.al. (2007), Virtual interpersonal touch expressing and recognizing emotions through haptic interfaces, Human-Computer Interaction, 22, 325-353

  • Gaver et.al. (2007), Electronic Furniture for the Curious Home: Assessing Ludic Designs in the Field, IJHCI, 22(1), 119-152.

  • Kuber & Yu (2010), Feasibility study of tactile-based authentication. IJHCS 68(3), 158-181

Applying psychological theories

Compelling examples of applying genuine psychological theories are comparably rare in the HCI literature. On the other hands these articles often have some kind of profoundness and timelessness compared to the many evaluation studies. Thus, authors taking the challenge of applying a theory most likely will name the theory in the title or at least in the abstract. Some of you only found some psychology-sounding phrases in the abstracts and merely assumed that there must be some kind of theory behind it. Without at least naming this theory, these solutions didn’t meet the requirements.

My top three examples from your results were:

  • Shin (2009), The Evaluation of User Experience in the Virtual World in Relation to Intrinsic and Extrinsic Emotion. IJHCI, 25(6), 530-553.

  • Anderson, J. R., Matessa, M., and Lebiere, C. 1997. ACT-R: a theory of higher level cognition and its relation to visual attention. Hum.-Comput. Interact. 12, 4 (Dec. 1997), 439-462. DOI= http://dx.doi.org/10.1207/s15327051hci1204_5

  • Xu, S. 2005 Development of Dual-Modal Presentations of Textual Information. Doctoral Thesis. UMI Order Number: AAI3202828., DePaul University, School of Computer Science, Telecommunications, and Information Systems.

Evaluation studies

Evaluation studies are probably the most common type of research. Typically, these studies measure outcomes in a lab situation or even a controlled experiment and argue about the benefit of doing a design this way or the other.

My personal top three were:

  • Wilson, S., Galliers, J. & Fone, J. (2007). Cognitive Artifacts in Support of Medical Shift Handover: An In Use, In Situ Evaluation. International Journal of Human-Computer Interaction, 22(1), 59-80. doi:10.1080/10447310709336955

  • Blandford, A. E.; Hyde, J. K.; Green, T. R. G. & Connell, I. Scoping Analytical Usability Evaluation Methods: A Case Study. Human-Computer Interaction, 2008, 23, 278 – 327

  • Lee, S. and Zhai, S. 2009. The performance of touch screen soft buttons. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 – 09, 2009). CHI ’09. ACM, New York, NY, 309-318. DOI= http://doi.acm.org/10.1145/1518701.1518750

A good joke

There is still sth. to add to my personal list of “honorable mention”: One of you found the silliest evaluation study I ever heard of. Btw., here we see such a vague relation to sth. being “psychological”. I wouldn’t judge this to be about applying a psychological theory, because no such theory is mentioned in the abstract. Still, a really good joke! Enjoy!

Suzuki, N., Kakehi, K., Takeuchi, Y., and Okada, M. 2004. Social effects of the speed of hummed sounds on human-computer interaction. Int. J. Hum.-Comput. Stud. 60, 4 (Apr. 2004), 455-468. DOI= http://dx.doi.org/10.1016/j.ijhcs.2003.09.007

Our research focuses on the nature of voice interaction and activation of psychological tendencies in humans by the power of prosody sounds. This study examines whether people’s impressions and behaviours are affected by variations in the speed of hummed sounds. The sounds consist of just prosodic components similar to continuous humming on the open vowel /a/ or /o/ without any language information. In interaction between individuals as well as among animals, temporal structures including voice speed or duration contribute rhythmic interaction and are closely connected to the participants’ dynamics of mental or emotional states. We think that this phenomenon can be applied to human-computer interaction, even through the variation in temporal structures of hummed sounds used to reduce the influence of content or meaning in language. Our interactive system mimics the prosodic features of the human voice by using humming-only sounds under three different voice speed conditions: (a) faster, (b) normal, and (c) slower than the original speaker. We examine whether the variation in the sound’s speed gives rise to both psychological and behavioural influences in the relationship between the computer and the subject through interaction. Subjects tend to prefer a computer with a normal or faster speaking rate to that with a slower rate on both usefulness and familiarity. Moreover, the speech rate of the subjects changed inversely to the variation in a computer’s hummed sounds. This study demonstrates the importance of temporal structure and emphasizes the need for an investigation of the fundamentals at work in interaction.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: