(A gestural interface tested in South Korea last year)
How Bodies Matter: Five Themes for Interaction Design by Scott Klemmer, Björn Hartmann and Leila Takayama (DIS 2006) gives a relevant overview of different themes of interest for interaction designers focused on tangible/gestural interactions. It covers a broad range of topics concerning how our body is fundamental in our experience with the world.
Drawing on theories of embodiment in philosophy, psychology and sociology, they came up with 5 themes:
"The first, thinking through doing, describes how thought (mind) and action (body) are deeply integrated and how they co-produce learning and reasoning. The second, performance, describes the rich actions our bodies are capable of, and how physical action can be both faster and more nuanced than symbolic cognition. The first two themes primarily address individual corporeality; the next two are primarily concerned with the social affordances. Visibility describes the role of artifacts in collaboration and cooperation. Risk explores how the uncertainty and risk of physical co-presence shapes interpersonal and human-computer interactions. The final theme, thickness of practice, suggests that because the pursuit of digital verisimilitude is more difficult than it might seem, embodied interaction is a more prudent path."
What does that mean for tangible computing? see what the authors say:
" we should not just strive to approach the affordances of tangibility in our interfaces and interactions, but to go beyond what mere form offers. As Dourish notes, “Tangible computing is of interest precisely because it is not purely physical. It is a physical realization of a symbolic reality”. For a combination of virtual representations and physical artifacts to be successful and truly go beyond what each individual medium can offer, we need a thorough understanding what each can offer to us"
(picture taken from one my home ethnography study)
A current research project about the user experience of the Nintendo Wiimote lead me to investigate that last theme concerning the "pursuit of digital verisimilitude. Some excerpts from the paper about it:
"It may seem a platitude, but it is worth repeating that, “if technology is to provide an advantage, the correspondence to the real world must break down at some point” (Grudin). Interaction design is simultaneously drawn in two directions. (...) This section argues that interfaces that are the real world can obviate many of the difficulties of attempting to model all of the salient characteristics of a work process as practiced. This argument builds on Weiser’s exhortation to design for “embodied virtuality” rather than virtual reality. Designing interactions that are the real world instead of ones that simulate or replicate it hedges against simulacra that have neglected an important practice."
Although I fully, "interactions that are the real world" are not so easy to design depending on the technology one have: the hand movement captured when playing Wii tennis is only a basic representation of the complex hand movement when playing tennis. Therefore, as I observe in different field studies, if the interaction per se is relevant for Wii players, there are often misunderstandings between the expected events on the screen (based on what gestures the players felt she did) and what really happens in the game. So what I mean here is that "digital verisimilitude" is also hard in tangible computing as capturing movement is definitely tricky. Think about human physiology, the fact that movement is a dynamic (and capture may imply statefulness), the role of context, etc.