For the first 3-year phase of the proposed project, we have chosen a number of hot current research topics with concrete goals to jumpstart the joint work of graphics and vision researchers. The goal of these projects is to define topics that require more than one PhD student, fostering the cooperation among students:

• Segmentation of 3D Tubular Tree Structures in Medical Images
• Pose estimation for mobile devices
• Façade Segmentation in Street-side Images
• 3D Models for Object Categorization
• Computing Convex Quadrangulations
• High Performance Ray Tracing

Segmentation of 3D Tubular Tree Structures in Medical Images

Modern volumetric imaging techniques such as CT and MR provide detailed information about branched tubular structures such as blood vessels or the bronchial tree. Analysis of these structures is of vital interest for many clinical applications aiding procedures such as diagnosis, quantification, and monitoring of disease, preoperative planning, or intra-operative navigation. As due to their complexity manual analysis and quantification of these structures is impractical in clinical routine, automatic and robust segmentation methods for tubular tree structures are needed. For clinical applicability of such methods, several issues must be addressed, like the ability to: identify thin low contrast structures, separate multiple interwoven tree structures, handle adjacent structures with similar gray-value such as tumorous regions, and tolerate local disturbances such as motion artifacts. To address these issues we are utilizing a three-step approach consisting of (a) a bottom up tube detection, (b) a grouping of the identified single tubular objects into tree structures, and (c) utilizing this information as a prior for the intrinsic segmentation step. During the grouping step smaller gaps in the tree structures are closed, thus also producing a suitable shape prior in locally disturbed regions. As a consequence of this overall approach, major leakage or under-segmentation can be avoided, thus resulting in an increased robustness compared to other methods. The above outlined approach is of a generic nature and may be applied in a variety of different application domains. However, depending on the objectives of a specific application the requirements and the desired behavior of the individual parts may vary. Therefore, for each of the three steps various novel methods have been developed with different properties that have been evaluated and improved individually. This resulted in an adaptable framework of reusable parts with different properties that are suitable for most tasks. Moreover, this gives us the
possibility to easily combine existing parts to fit the needs of new applications.
Based on the overall approach and the developed methods for the different processing steps various applications have been developed where the overall approach has been evaluated. Therefore, the developed applications were applied to public databases giving a comparison to a large number of competing techniques on clinically relevant data. As the results of this comparison shows, compared to other techniques our utilized three-step approach allows producing high quality results and coping with a variety of problems as they occur in clinical datasets.

Contact: Christian Bauer

Pose estimation for mobile devices

Fast and memory efficient methods for localizing a mobile user’s 6DOF pose from a single camera images. The basic idea is to register views with respect to sparse 3D reconstructions. The tracking dataset is partitioned into pieces based on visibility constraints and occlusion culling, making it scalable and efficient to handle. Starting with a coarse guess, we only consider features that can be seen from the user’s position.

Contact: Manfred Klopschitz

Façade segmentation in street-side images

An idea of a “Virtual Habitat” is to create a 3D model of a real urban environment. The complexity of a real urban scene provides challenge in this research topic. Our primary input data consist of the street-side photographs acquired either via the use of an industrial sensor approach or via crowd sourcing imagery. The primary goal is the classification into major object classes (façade, ground, sky, cloud, vegetation, other). This classification must be robust enough to perform fully automated on a large variety of street-side images with various intra-class objects. For this purpose, the semantic segmentation based on multi-class superpixels was implemented. The novel application of semantic and geometric context via the discriminative random field representation on the level of superpixels was used to improve the results of classification up to approximately 89%. The subsequent research on exploiting the image data redundancy adds additional 5% accuracy into the classification of the building façades.

Reconstruction of the buildings is considered a key part in the modeling of urban environment. Therefore, the next research challenge is the detailed façade description. A new gradient projection approach was developed for this purpose. This method can be considered a step between a traditional gradient projection method and a superpixel based segmentation. As such, it can benefit from previous research on semantic context to provide superior results. Also, the research on new area color descriptors (particularly the k-mean clustering in CIE-Lab color space) shows improved results when compared with traditional descriptors (color histogram, color SIFT). The goal in this step is to identify fine details on the building façade (ornaments, arches, pillars, rims, …) as the specific signatures in the gradient projections in multi-view scenario.

Contact: Michal Recky

3D models for object categorization

The idea behind this topic is to build one single, pose invariant 3D model for a category. This should help to overcome known problems in categorization using 2D models, especially the robustness against view/pose changes.
Our concept extends well known 2D shape models (e.g. Boundary Fragment Model [Opelt et al. 2008]) to 3D shape models by using 3D contour fragments. To learn such models, we capture calibrated stereo videos of hand-held objects of various categories (toy horses, toy cars, etc.) in two stages. First, 3D contour fragments are reconstructed from stereo pairs and second, motion estimation between subsequent pairs is used to generate a so called 3D contour cloud. Pairs of these 3D contour fragments should then be used to learn the shape of a category. This offers the advantage that the 3D nature of a category is exploited such that it is not required to learn each view (e.g. car rear, car front, car side etc. ) independently.
While this approach learns 3D contour models for categories, in recognition these 3D models will be used to generate hypotheses from a single, 2D query image. An active object categorization system will use such category and pose hypotheses from one or more images taken from different viewpoints to generate a reliable category hypothesis for an object.

Contact: Kerstin Pötsch

Computing Convex Quadrangulations

We use projected Delaunay tetrahedra and a maximum independent set approach to compute large subsets of convex quadrangulations on a given set of points in the plane. The new method improves over the popular pairing method based on triangulating the point set.

Contact: Markus Demuth

High Performance Ray Tracing

Raytracing is a widely used image synthesis technique with high computational costs. Modern graphics hardware not only offers enormous parallel computation power but also a fexible programming model. In this thesis CUDA, NVidia's new API for general purpose computations on graphics hardware, and its applications to ray tracing are discussed. A parallel ray tracing system based on CUDA is developed, which has a modular design and provides an extensible set of objects for scene modeling. The ray tracing system is based on a parallel geometry core, that exploits computation power of graphics hardware to perform geometric calculations. Test results show that the CUDA raytracer delivers almost interactive frame rates for scenes containing millions of geometric primitives.

Contact: Thomas Schiffer