[Research] Can You See Me Now? evaluation

Just read this: M.Flintham, R.Anastasi, S.Benford, T.Hummings, A.Crabtree, C.Greenhalgh, T.Rodden, N.Tandavanith, M.Adams, and J.Row-Farr (2003). Where on-line meets on the streets: experiences with mobile mixed reality games. In Proceedings of the conference on Human factors in computing systems, pages 569–576, Florida, USA.

We describe two games in which online participants collaborated with mobile participants on the city streets. In the first, the players were online and professional performers were on the streets. The second reversed this relationship. Analysis of these experiences yields new insights into the nature of context. We show how context is more socially than technically constructed. We show how players exploited (and resolved conflicts between) multiple indications of context including GPS, GPS error, audio talk, ambient audio, timing, local knowledge and trust. We recommend not overly relying on GPS, extensively using audio, and extending interfaces to represent GPS error.

It's interesting because it deals with how they evaluated their location-based games relying on three sources of data:

  1. offline feedback from players via email and face-to-face conversations (for those known to us) and also debriefing meetings with the project team.
  2. ethnographic observation (utilising video and field notes) of the activities of the runners, players and behind-the-scenes production crew.
  3. statistical and manual analysis of system logs of GPS positions, GPS errors and text messages to reveal broad patterns of activity.