A Survey of Design Issues in Spatial Input by Ken Hickley, Randy Pausch, John C. Goble, and Neal Kassell (1994), Proc. ACM UIST'94 Symposium on User Interface Software & Technology, pp. 213-222. Even though the paper is a bit old, it gives a comprehensive summary of the design issues regarding spatial input, especially regarding the perception of 3D. The authors describes how users have difficulty understanding three-dimensional space, based on user studies.

" Anyone who has tried to build a stone wall knows how difficult it is to look at a pile of available stones and decide which stone will best fit into a gap in the wall. There are some individuals, such as experienced stone masons, who have become proficient with this task, but most people simply have to try different stones until one is found that fits reasonably well. In general, people are good at experiencing 3D and experimenting with spatial relationships between real-world objects, but we possess little innate comprehension of 3D space in the abstract. People do not innately understand three dimensional reality, but rather they experience it. (...) Previous interfaces have demonstrated a number of issues which may facilitate 3D space perception, including the following: Spatial references, Relative gesture vs. absolute gesture Two-handed interaction, Multisensory feedback, Physical constraints"

Why do I blog this? looking for references about how 3D is experienced. Why? because this topic is back again (with the SL frenziness) and that I often find myself in situations where I have to explain why 3D s problematic. It's quite interesting because it engages me in reading old paper from 10-15 years ago, as if everyone forgot the research that has been done.