Context Sensitive Magic Lenses

"Magic Lenses(TM) are filters, that modify the presentation of application objects to reveal hidden information, to enhance data of interest, or to suppress distracting information."

Bier et al.

ABSTRACT

			We present a tool capable of interactively rendering 3D volumetric magic lenses for augmented reality. This interaction tool can apply different rendering styles to portions of a scene depending on contextual information.

TEAM

Dieter Schmalstieg	\| Research Group Leader
Erick Mendez	\| Visualization of Geographical Data
Denis Kalkofen	\| Visualization of Medical Data

CONCEPT

			This is an interaction tool, based on the Magic Lenses technique, that allows a 3D scene to be affected dynamically given contextual information, for example, to support information filtering. With this technique elements of a scene graph are grouped by context in addition to hierarchically, this enables us to locally modify their rendering styles. This research has two major contributions, the use of context sensitivity with 3D Magic Lenses in a scene graph and the implementation of multiple volumetric 3D Magic Lenses for Augmented Reality setups. We have developed our tool for the Studierstube framework which allows us doing rapid prototyping of Virtual and Augmented Reality applications.

IMAGES

We have envisioned a number of applications that can be addressed by the use of CSML. Following, some practical examples are given. The images shown where generated on a Windows PC with 3GHz CPU and NVidia GeForce 7800GT. As tracking systems, we used ARToolkitPlus and ARTTrack. The geographical data of downtown Graz was extracted from a GE Smallworld database, provided by our partner company Grintec GesmbH. Liver and vessel tree models where taken from a surgery planning system.

		As described by Bier et al., two possibilities of the use of Magic Lenses are to enhance data of interest and to reveal hidden information. These two behaviors can simultaneously be achieved with CSML. For example, in this image we show a real model overlaid with 3D representation of buildings, between them, power lines (green) and gas pipes (red) are presented (in this case, for illustration purposes, we use mock-ups). A lens has been placed in the scene intersecting the pipes and the virtual buildings. Those pixels of the buildings that fall inside the lens are cutaway, allowing revealing the information behind them. In turn, the pipes are enhanced with more detailed information on their structure. This is only possible with a context-rich scene.


		An obvious use for Magic Lenses is that of X-Ray vision. In this image, for example, the outside walls of a building that fall inside the lens will be made semi transparent, whereas interior objects such as furniture are rendered in a standard color (blue), while objects of higher importance such as characters are highlighted (red).


		In a dense AR scene, augmentations can often be ambiguous. This is the case of this image where multiple families of objects with different semantics are represented in a similar style. This is the case of this figure where two types of vessel trees are overlaid on top of a liver model. These two vessel types are visually similar, but can be locally disambiguated by intersecting them with a Magic Lens and changing their rendering style accordingly.


		In this image an example of ghosting and cutaway is shown. We define four families of objects, non-occludable, ghosts, cutaways and occluding. Those objects belonging to the family of ghosts will be made semi-transparent when falling in the line of sight between a non-occludable object and the camera. Similarly objects belonging to the family of cutaways will not be displayed and occluding objects will be shown without any rendering style modification.





A single context sensitive 3D Magic Lens can be used to aid the understanding of a scene, by visualizing different parts in easily distinguishable styles. Sometimes, however, the structure of an object assembly is so dense and spatially complex, that, to understand the three dimensional scenario in detail, an interaction with more than one lens will be needed. For example, in this image, the user wants to explore the location and the geometric relationship of the tumors and of both of the two main vessel trees of a liver. To find the tumors, first the user searches for them by interactively intersecting the entire scene with a lens. This lens renders the tumors in green, while at the same time it renders the vessels in transparent (A). Up to here, this shows how a single lens can be used to search for objects in a scene. To furthermore find out about the relationship of the tumors and its surrounding vessels, two more lenses are added that each affect only one of these vessel trees (B)(C). By moving the lenses through the scene the user can interactively visualize parts of the vessel trees (D). Such an interaction helps in developing a mental map of the geometrical relationship between the vessel trees and the tumors, which in the end gives a better understanding of the 3D scene.

VIDEOS

Submition Video to ISMAR 2006 for the article "Interactive Context-Driven Visualisation Tools for Augmented Reality" [ AVI - 48M ] - Codecs: XVID & MP3

PUBLICATIONS

Mendez, E., Kalkofen D., Schmalstieg, D., "Interactive Context-Driven Visualisation Tools for Augmented Reality", In Proceedings, ISMAR 2006, October 22-25, 2006, Santa Barbara, California, USA, 2006, pp. 209-216 [ PDF - 2.4M ]

LINKS

Studierstube: Leading framework for the development of mobile, collaborative and ubiquitous AR applications [ more ... ] Vidente: A project for the Visualization of Subsurface Features that exploits the CSML [ more ... ]

Sections

Context Sensitive Magic Lenses

Document Actions