Technology
Deskotheque Framework
A flexible distributed system coordinates the calibration step and manages multi-user interaction between machines and displays, respectively. Deskotheque is implemented on top of the X Window System for Linux. It is tightly integrated into the windowing system and thus allows for application- and network-transparent cross-display mouse pointer navigation, application sharing, and geometric display compensation. Deskotheque employs a distributed system involving multiple computer nodes in a network. Individual modules are coordinated by a central master module, while the actual X client-server communication uses the standard peer-to-peer approach of the X protocol. The framework functionality can roughly be sub-divided into three components:
- Display management: Geometric compensation and edge blending, drawing primitives on top of the desktop image, and display-adaptive window management are accomplished as plugins for the window managers beryl and compiz.
- Input redirection: Mouse pointer redirection is based on an extended version of Synergy. It was extended by adding multi-input handling and more sophisticated mouse-pointer navigation.
- Multi-pointer navigation: Synergy+MPX supports input redirection for multiple mouse and keyboard pairs. It combines Synergy-plus with Multi-Pointer X.
- Application relocation: Deskotheque uses Xmove to relocate X applications at runtime.
Using multiple calibrated cameras, surfaces in the environment which are suitable for projection are automatically determined.
Geometric Display Compensation
Projected displays allow for a variety of display form factors, as
they can be combined from multiple projectors or spanning non-planar
display surfaces. Software-based compensation of projected imagery is
required in order to provide undistorted display images.
In a camera-assisted offline calibration step a 3D reconstruction of
display surfaces is acquired by showing display-wise structured light
patterns. The three-dimensional model describes the geometric properties
of individual displays and the spatial display topology in the
environment. Warping and edge blending is calculated for each projected
display and can be employed to compensate for:
- oblique projection angles and thereby resulting projective distortions of imagery
- tiled projected displays composed from multiple casually aligned projectors, and
- projections onto multi-planar surfaces.
A uniform display area is created from multiple overlapping projections.
The desktop content is warped and blended using GPU shaders. The image shows a typical raw output.
Cross-Display Mouse Pointer Navigation
Based on a spatial model of the display environment, we automatically derive spatially consistent mouse pointer navigation techniques. We have integrated three navigation techniques into the Deskotheque framework:
- Path navigation connects the closest edge portions of adjacent displays to static paths.
- Free navigation provides a navigation map from the estimated user position.
- The world-in-miniature control allows for input redirection by selecting the desired target location in a miniature model of the environment.
In free navigation mode, the user can traverse irregularly arranged displays directly with the mouse.
Display Space Management
When building environment-adaptive displays from casually aligned projectors, configurations diverging from conventional single- or dual-monitor geometries may evolve. We therefore are designing intelligent (display-adaptive) window management aware of physical display properties and interaction techniques to help users locating and accessing distant content.
Intelligent window management supports a user in locating content.
Collaborative Information Workspaces
We aim to create a collaborative information workspace by combining the Deskotheque multi-display framework with Caleydo, a multiple-view visualization system from the biomedical domain. Research on collaborative information workspaces will cover the following domains:
- adaptations of visualization techniques to display form factors and user preferences and
- interaction techniques to facilitate information sharing and to guide the users' attention to relevant items in the environment
Multiple users can share a joint workspace composed of multiple monitors and projections.
