Research Projects

The goal of the project is to develop methods for modeling and surveying large construction sites. The project will make use of unmanned aerial vehicles and existing stationary or pan-tilt zoom cameras at the construction site. The goal is to provide accurate 3D models on a regular basis of the whole site. This will generate a 4D data set (3D+time). This data can then be used for documentation, visualization (we will use a mobile augmented reality system to overlay e.g. the plan or a model of the building) as well as measurement (e.g., how much material has been transported). From a scientific point of view we will have to solve following tasks:

• Dense 3D reconstruction from highly overlapping data, we will use variational methods implemented on the GPU.
• Accurate registration of subsequent models over time. Since the 3D reconstruction is changing (per definition) the method needs to handle this. This is an instance of the highly relevant 3D model updating problem.
• Integration of multiple camera sources. Using the 3D model and additional cameras poses the problem of localization of the additional cameras with respect to the 3D model which is again an instance the registration problem.
• Development of a handheld AR platform for visualization. In order to use AR technology the pose of the platform with respect to the model and the reconstruction needs to be determined.

The focus of the project is outdoor mobile computer vision with all of its challenges. Mobile systems need to be compact and energy efficient and are frequently changing locations. Therefore they must be autonomous and perform processing locally. A number of challenges arise from these requirements for which the project aims to provide solutions: Being compact, there is not much space for a large number of sensors such as laser scanners, radar antennas and the like. The work in this project will focus on stereo vision but with two different types of cameras. Often a second camera is already available and stereo information increases detection accuracies. Each time the system moves it needs to adapt to the changing situation. This requires adaptive calibration and online learning. Mobile systems often work from batteries. In addition, there is not much space to include intricate cooling systems. Thus, the system must be designed to be very energy efficient. New approaches for dynamic power management will be explored in the project. To put the work into context, several applications from the area of traffic surveillance/toll enforcement will be implemented and tested in an application oriented setting.

Current traffic enforcement solutions are either very large and costly (section control, toll enforcement) or do not offer much in terms of image processing (radar speed control). The technological output of Mobi Trick makes it possible to design mobile solutions for traffic monitoring, vehicle identification and classification, intelligent incident detection and observation of driver behavior. Mobile devices are also more efficient in enforcement. Their transient nature makes them less predictable. Mobile systems can also react more flexibly to changing road situations such as construction sites.

The ever increasing number of cameras in surveillance system requires automatic video analysis in order to spot critical situations and to alert the monitoring personnel in a timely manner. While most current approaches in this area aim for detecting a large number of specific events on a large set of complex application scenarios, the goal of this project is to go far beyond state of the art by developing novel online learning methods to detect unusual situations in a camera specific scenario. We will exploit the huge amount of data available for a specific camera to reliably learn usual and unusual situations.

In particular the OUTLIER project will carry out basic research in the following areas:

• Improved unsupervised learning methods for huge amounts of data
• Novel methods for semi-supervised learning in huge amounts of unlabeled data

These generic learning algorithms will be applied for the detection of unusual situations in public places and traffic scenarios. Examples are the detection of unusual crowd behavior (upcoming panic, barred emergency exits, or toppled persons), suspicious behavior of pedestrians (e.g. going from one car to another, loitering), vehicles or persons moving on unusual locations, the detection of unusual types of moving objects and detection of unusual situations like accidents, clashes and collisions. Unlike other approaches we do not want to model these situations explicitly and individually, but we will resort to learning to discriminate the usual situation from the unusual one.

Research partners in the project are JRS, TUG for basic and applied research and Siemens for industrial exploitation of project results.

Die klinische Rechtsmedizin gewann in den letzten Jahren aufgrund einer Sensibilisierung der Öffentlichkeit gegenüber häuslicher und sexueller Gewalt, Gewalt gegenüber Kindern und Verdachtsfällen von medizinischen Behandlungsfehlern stark an Bedeutung. Die forensische Untersuchung von Lebenden ist bis heute jedoch auf eine äussere Besichtigung des Körpers beschränkt.

Das neue Ludwig-Boltzmann-Institut (LBI) für klinisch-forensische Bildgebung hat zum Ziel, Verfahren zur Erfassung von inneren Verletzungsbefunden als Grundlage für forensische Gutachten zu entwickeln. Mittels Computertomographie (CT) und Magnetresonanztomographie (MRT), welche in der Klinik etabliert sind, können zusätzliche, objektiv nachweisbare innere Verletzungsbefunde erhoben werden, die eine verbesserte Einschätzung der ausgeübten Gewalt gegen die untersuchte Person ermöglichen. Die Methoden sind jedoch auf klinische Diagnostik ausgerichtet, während forensisch wichtige Befunde nicht oder nicht optimal dargestellt werden.

Das Institut fuer Maschinelles Sehen und Darstellen kooperiert mit dem LBI zur Entwicklung neuer Methoden der Bildverarbeitung und Computergrafik zum Zwecke der Bildgebung.

Automatic and efficient quality analysis of audiovisual content has become a crucial step before storing the material for later use. While most approaches in this area are only dealing with low level signal analysis, the goal of this project is to go far beyond state-of-the-art procedures. On the basis of novel as well as proven computer vision methods, we will attempt to incorporate high level knowledge in the analysis step, thus achieving significant better and faster results than current methods, comparable in their reliability with a human operator.

In particular the vdQA project will carry out research in the following areas:

• Improvement of optical flow field methodologies to deal with multi-frame information

• Application of novel segmentation methods in order to enable semantic quality analysis.

• Knowledge assisted artefact assessment and classification.

• Novel methods for fast and robust detection of difficult impairments like unsteadiness, flicker, freeze frames, test patterns and lost frames.

• Research into methodologies that are particularly well suited for implementations taking advantage of GPU hardware.

The grand challenge in the end is the combination of robustness, speed and integration of human knowledge. The research and industrial partners have dedicated roles in the work programme to achieve those goals. The industrial partners have excellent knowledge of the market and will provide user requirements as well as extensive test material. The academic partners will do research in their respective fields, namely development of basic algorithms for optical flow, tracking, segmentation, classification and usage of GPUs as well as algorithms for content based quality analysis and semantic technologies to represent knowledge. Towards the project end the industrial partners will evaluate and test the developments together with pilot end users.

Project Partners:

Joanneum Research, Institute of Information Systems & Information Management

MediaServices GmbH AudioInspector

HS-ART Digital Service GmbH

Inst. f. Computer Graphics and Vision

The world will witness a tremendous increase in the number of vehicles in the near future. Future traffic monitoring systems will therefore play an important role to improve the throughput and safety of roads. Current monitoring systems capture (usually vision-based) traffic data from a large sensory network; however, they require continuous human supervision which is extremely expensive.

In the proposed EVis research project we investigate the scientific and technological foundations for future autonomous traffic monitoring systems. Autonomy is achieved by a novel combination of three approaches: First, vision-based detection and classification methods are augmented by self-learning and scene adaptation mechanisms which will significantly reduce the effort of manual configuration. Second, visual data is fused with data from other sensors such as radar, infrared or inductive loop sensors. Sensor fusion helps to improve the robustness and confidence, to extend the spatial and temporal coverage as well as to reduce the ambiguity and uncertainty of the processed sensor data. Finally, the developed vision and fusion methods are implemented on a distributed embedded platform which makes them wider applicable and supports real-time operation.

Our autonomous traffic monitoring system will be evaluated using real world traffic data. The evaluation will be conducted in three different case studies: offline testing using recorded data, online testing on a traffic test site, and on a test installation on a public road.

Soft tissue like tendons, arteries, veins or skins are important biological materials. A greater understanding of the foundations and interactions of structure and function of soft tissue, and, in particular, the associated mechanobiology is of great interest in the field. A thorough understanding of the complex interrelations between mechanical factors and the associated biological responses may help to improve diagnostics which allow disease and injury to be treated earlier. The research proposed here will develop a fully automatic system for analyzing macroscopic structures obtained from histological images of arteries by means of modern computer vision techniques. Besides being interesting from the mechanobiological point of view the structural analysis of images of collagen fibers poses also several challenging questions from a computer vision point of view. In particular, due to the wide variety of different appearances of collagen fibers in images this task is non trivial. The main task of this research is the development of novel segmentation techniques for robustly segmenting individual fibril bundles and estimating their parameters, like location and shape, fibril density, mean fibril orientation, wriggling of fibrils etc. This will be achieved by developing novel perceptual grouping methods operating on the extracted orientation data of fibrils. Another major challenge of this research is to extend the structural analysis from 2D to 3D.

Robust and Adaptive Approaches to Scene and Object Recognition: The goal of this joint project is to investigate new robust and adaptive approaches in the area of object and scene recognition. Object and scene recognition is a necessary requirement for developing truly cognitive systems as well as for the development of advanced and novel multimodal interfaces leading to ambient intelligence. Having a robust object and scene recognition system the following applications will greatly benefit: novel user interfaces which understand human activities, intelligent surveillance, indexing multi-media databases and content analysis of images, autonomous mobile systems and robotics, industrial inspection and robotics, etc. The goal is to develop computer vision based systems that can recognize objects, and in the context of environment perform localization and navigation. The major challenge is to develop systems and methods that can work under realistic unconstrained conditions (i.e., outside the lab). The three partners proposing this project (Center for Machine Perception, Czech Technical University Prague, CMP, Computer Vision Lab, Faculty of Computer and Information Science, University of Ljubljana, CVL, and Institute for Computer Graphics and Vision, Graz University of Technology ICG) have considerable expertise in this area and developed complementary methods and techniques. The goal of the project is to join the efforts and combine the expertise. In particular, we do the following activities:

• Organization of joint workshops and colloquia
• Exchange of PhD. students
• Joint research/joint papers

The scanning electron microscope (SEM) is an important tool to examine very small structures. Its large magnification combined with good contrast and large depth of view make it possible to view and characterize microscopic structures in the sub-micron scale. In the recent years, the problem of dense surface reconstruction from multiple SEM images was a research topic on this institute. Reconstruction approaches like shape from stereo and shape from photometric stereo have been evaluated. This work presents a framework for automatic scene reconstruction from three images acquired by a scanning electron microscope. The basic assumption is that the specimen is tilted eucentrically in front of the camera, camera geometry is assumed to be unknown but constant over all views. It is shown that methods for estimating the trifocal tensor as well as modern auto-calibration approaches can be adapted to the imaging conditions in the SEM, and Euclidean scene structure can be retrieved from three uncalibrated views. The performance of the proposed framework is evaluated on synthetic data as well as real images. It is shown that Euclidean scene structure can be retrieved robustly under varying image noise and inaccurate initialization.