Overview
This course focuses on issues and problems related to a newly emerging research activity of potentially big industrial impact, "Augmented Reality". AR constitutes a new user-interface paradigm for many "hands-on" application scenarios in which users cannot sit at a conventional desktop computer. While users manipulate real objects in their usual work environment, AR provides them via semi-transparent head-mounted displays or portable monitors with additional information about the objects and the tasks, graphically embedded seamlessly into the real world.In this course, students and instructors will discuss the general paradigm, applications in several industry sectors, and technical issues regarding mobile computing, user tracking, 3D computer graphics and concepts of intelligent environments.
The course is structured as a series of presentations given by students followed by discussions with the participants and the instructors. The objective of the discussion is to clarify the topic and to generate new ideas. Each presentation should last about a hour and a half. The discussion after each presentation is expected to last up to an hour.
Infrastructure
Homepage:
Lotus Notes databases:
If you have any questions or problems regarding the use of these bboards, please send mail to Oliver Creighton.
1. Overview and Concepts of Augmented Reality (Martin Bauer)
This topic provides an overview about the concepts of Augmented Reality and other, related terms that have been proposed during this decade. The session is intended to set the stage for the indepth discussions of the following days.
"A Survey of Augmented Reality"
R. Azuma, Presence, Vol. 6, No. 4, Aug. 1997, pp. 355-385."A Taxonomy of Real and Virtual World Display Integration"
P. Milgram and H. Colquhoun Jr., ISMR 99, Yokohama, March 1999."Computer Science Issues in Ubiquitous Computing"
M. Weiser, Communications of the ACM, July 1993, Vol. 36, No. 7, pp. 75-84."Steps Towards Seamless Mixed Reality"
H. Tamura, H. Yamamoto, A. Katayama, ISMR 99, Yokohama, March 1999.
2. Applications of Augmented Reality (Andreas Löhr)
This session presents prototypical applications across several industry sectors: manufacturing, architecture and exterior construction, and medicine. By comparing the different approaches, the recurring technical problems and potential solutions of AR can be understood in a more general context.
"Several Devils in the Details: Making an AR App Work in the Airplane Factory"
D. Curtis, D. Mizell, P. Gruenbaum, and A. Janin. 1rst International Workshop on Augmented Reality (IWAR'98), San Francisco, 1998."Augmented Reality for Construction Tasks: Doorlock Assembly"
D. Reiners, D. Stricker, G. Klinker, and S. Mueller. 1rst International Workshop on Augmented Reality (IWAR'98), San Franscisco, 1998."Wearing It Out: First Steps Toward Mobile Augmented Realtiy Systems"
S. Feiner, B. MacIntyre, T. Höllerer, ISMR 99, Yokohama, March 1999."Computer-vision-enabled ophthalmic augmented reality: A PC-based prototype"
J.W. Berger and D.S. Shin. 1rst International Workshop on Augmented Reality (IWAR'98), San Franscisco, 1998.
3. Calibration and Tracking (Martin Wagner)
One of the most important aspects of Augmented Reality revolves around tracking the user's eyes in real-time, since virtual objects have to be rendered from the same viewpoint. This session presents the technical problems related to real-time sensing, as well as the currently most commonly used calibration and tracking techniques in Augmented Reality.
"Registration Error Analysis for Augmented Reality",R. Holloway, Presence, Vol. 6, No. 4, Aug. 1997, pp. 413-432."A Multi-ring Color Fiducial System and A Rule-Based Detection Method for Scalable Fiducial-tracking Augmented Reality", Y. Cho, J. Lee, and U. Neumann. 1rst International Workshop on Augmented Reality (IWAR'98), San Franscisco, 1998.
"Confluence of Computer Vision and Interactive Graphics for Augmented Reality", G. Klinker, K. Ahlers, D. Breen, P.-Y. Chevalier, C. Crampton, D. Greer, D. Koller, A. Kramer, E. Rose, M. Tuceryan and R. Whitaker. Presence, Vol. 6, No. 4, AUg. 1997, pp. 433-451.
"Fusion of Data from Head-Mounted and Fixed Sensors", W. Hoff. 1rst International Workshop on Augmented Reality (IWAR'98), San Franscisco, 1998.
4. Mixing Real and Virtual Objects (Stefan Riß)
The seamless, physically correct inclusion of virtual objects into the real world is essential to establishing a truely immersive illusion in Augmented Reality applications. To this end, virtual objects must obey basic physical laws, such as optical occlusion: depending on their position in the scene and on the current vantage point, virtual objects can hide real objects or they can be occluded by them. Optical effects like shadow casting and inter-reflections may also need to be simulated, as well as mechanical relationships like object collisions, gravity and friction between real and virtual objects. In order to correctly analyze and simulate such relationships, the computer needs a good geometrical model of all virtual and real objects involved. This session presents the issues, as well as current solutions.
"Interactive Occlusion and Collision of Real and Virtual Objects in Augmented Reality", D. Breen, E. Rose and R. Whitaker. Technical Report ECRC-95-02, 1995."The Use of Reality Models in Augmented Reality Applications", G. Klinker, D. Stricker and D. Reiners, European Workshop on 3D Structure from Multiple Images of Large-scale Environments (SMILE), in conjunction with ECCV'98, Freiburg, Germany, June 1998.
"Computer Augmented Reality and Illumination", A. Fournier. International Workshop MVD'95: Modeling- Virtual Worlds - Distributed Graphics, St. Augustin, Germany, Nov. 1995.
"A Stereo Machine for Video-Rate Dense Depth Mapping and its New Applications", T. Kanade, A. Yoshida, K. Oda, H. Kano and M. Tanaka, 15th Computer Vision and Pattern Recognition Conference (CVPR), San Francisco, June 1996. (Priority 4)
5. Intelligent Environments (Guido Kraus)
Real industrial applications of Augmented Reality are unlikely to be stand-alone little systems running in some corner of the plant. Rather AR can be expected to be the user interface connecting users with a wealth of information acquired, generated or retrieved dynamically on-line while users are pursuing their jobs. Information will be presented in many different modalities, both on body-worn devices and on wall-mounted installations. This session embeds Augmented Reality into the wider concepts of intelligent environments.
"The KidsRoom: A Perceptually-Based Interactive and Immersive Story Environment", A. Bobick, S. Intille, J. Davis, F. Baird, C. Pinhanez, L. Campbell, U. Ivanov, A. Schuette and A. Wilson. MIT Media Lab Technical Report No. 398, 1997."Design Principles for Intelligent Environments", M. Coen, AAAI-98.
"Context-Aware, Adaptive Wearable Computers as Remote Interfaces to 'Intelligent' Environments", G. Kortuem, Z. Segall, and M. Bauer. 2nd International Symposium on Wearable Computers, Pittsburgh, PA, Oct. 1998, pp. 58-65.
"Building Brains for Rooms: Designing Distributed Software Agents", M. Coen, IAAI'97.
6. Wearable Augmented Reality and Wearable Computing
This session discusses the software engineering issues related to designing distributed system architectures that are capable of supporting AR applications on mobile devices, communicating wirelessly with fixed installations in intelligent environments.
"Software Organization for Dynamic and Adaptable Wearable Systems", S. Fickas, G. Kortuem and Z. Segall, 1rst International Symposium on Wearable Computers, Cambridge, MA, Oct. 1997, pp. 56-63."Adtranz: A Mobile Computing System for Maintenance and Collaboration", D. Siewiorek, A. Smailagic, L. Bass, J. Siegel, R. Martin, and B. Bennington, 2nd International Symposium on Wearable Computers, Pittsburgh, PA, Oct. 1998, pp. 25-32.
"Augmented Reality through Wearable Computing", T. Starner, S. Mann, B. Rhodes, J. Levine, J. Healey, D. Kirsch, R. Picard and A. Pentland. Presence, Vol. 6, No. 4, Aug. 1997, pp. 386-398.
"Towards Situated Computing", R. Hull, P. Neaves and J. Bedford-Roberts. 1rst International Symposium on Wearable Computers, Cambridge, MA, Oct. 1997, pp. 146-153.
7. Multi-Sensory Augmentation (Tobias Weishäupl)
One of the overall goals of virtual reality systems is to create every aspect of the virtual environment such that an observer feels present within, through all the human senses. In spite of this, most VR systems have been designed to appeal only to observer's sense of vision. This same tendency is even more apparent in augmented reality and mixed reality reasearch where, for example, the notion of visual see-through displays easily comes to mind, but auditory or tactile mixtures are less common.
"Auditory Distance Perception in Real, Virtual, and Mixed Environments"
J. Loomis, R. Klatzky, R. Golledge, ISMR 99, Yokohama, March 1999."Feel-through: Augmented Reality with Force Feedback"
H. Iwata, ISMR 99, Yokohama, March 1999."Tangible Bits: Coupling Physicality and Virtuality Through Tangible User Interfaces"
H. Ishii, ISMR 99, Yokohama, March 1999.
8. Communication and Collaboration
An augmented reality system relies on raw data from the real world, the data transmission load becomes a crucial problem if one wants to operate it over a network. Attaining the goal of networked AR clearly requires technological and/or conceptual breakthroughs to address this problem.
"Augmented Telexistence"
S. Tachi, ISMR 99, Yokohama, March 1999."Collaborative Mixed Reality"
M. Billinghurst, H. Kato, ISMR 99, Yokohama, March 1999."Virtual Reality Technologies for Multimedia Communications"
J. Ohya, T. Miyasato, R. Nakatsu, ISMR 99, Yokohama, March 1999.
9. Systems and Design Considerations (Asa MacWilliams)
This session discusses some reports about the development of prototype AR systems.
"Augmented Reality: A Balancing Act Between High Quality and RealTime Constraints"
G. Klinker, D. Stricker, D. Reiners, ISMR 99, Yokohama, March 1999."MR Aided Enineering: Inspection Support Systems Integrating Virtual Instruments and Process Control"
K. Sato, Y. Ban, K Chihara, ISMR 99, Yokohama, March 1999."A Motion-Stabilized Outdoor Augmented Reality System"
R. Azuma, B. Hoff, H. Neely, R. Sarfaty, Proceeding of IEEE Virtual Reality `99, Houston, March 1999.