Research Projects

This project aims to develop portable measurement tools with in-situ visualization for the construction industry. A future measurement tool will provide a direct augmented reality view of measured properties over the real environment together with instructions as to where and how a certain task can be completed. For example, a metal detection tool should be able to provide direct visual feedback on the location of hidden metallic structures over a live video view of the inspected wall area. Furthermore it can guide a construction engineer to the optimal position for drilling a hole, avoiding any damage to existing structures.

Thus the tools should combine information from several sources to provide interactive and contextaware

guidance: Measurements from built-in sensors; location-aware through online tracking and registration; spatial, semantic information retrieved from a building information system (BIM). At the same time, future tools need to be simple to be used by non-expert users; therefore the system needs to be intuitive and guide users in the correct operation to fulfil their tasks. To accomplish this goal, The project addresses the following challenges:

• Tracking for mobile devices in changing and unknown environments for correct visual overlays. We will investigate the combination of visual online reconstruction methods with range finders and coarse models for absolute registration.
• X-Ray visualization of hidden and abstract information in unknown environments. Here we will investigate automatic approaches that take the environment’s appearance and the virtual information into account to select the best visualization method.
• User guidance based on measured and plan information requires automatic analysis of the spatial arrangements and automatic visualization.

The aim of this research project is holistic scene understanding in large aerial datasets, consisting of thousands of massively redundant high-resolution images. Holistic scene understanding is one of the major problems in computer vision and photogrammetry and has recently got a lot of attention. The problem of holistic image understanding includes two fundamental tasks: 3D scene reconstruction and semantic interpretation of the imaged content at the level of pixels. The tight interaction between semantic classification and 3D reconstruction is often ignored by state of the art aerial image processing workflows, due to the lack of computational power, the absence of efficient algorithms or the enormous effort of manual intervention. However, these tasks are mutually informative and should be solved jointly as a correct class labelling is a valuable source of information for reconstruction, and 3D information can help to improve the semantic interpretation. For instance, a correct classification is a valuable source of information for reconstruction in regions where dense matching methods fail (e.g. sheets of water and reflecting windows / facades), and 3D information can be used as a prior to improve classification (e.g. building and road detection). The high resolution and redundancy due to large overlaps of aerial images requires massive processing power which will be handled by taking advantage of graphic processing units that have proved to give a significant speedup compared to single core machines. In particular, we will focus on algorithms based on variational methods, which provide a high degree of parallelization capability. In order to reduce cost-intensive manual interaction, we further will exploit publicly available user-data from the Internet to improve both interpretation and 3D reconstruction.

In the HOLISTIC project we will provide a flexible framework for scene classification and 3D reconstruction from aerial images that outperforms current state-of-the art and delivers interpretable models at highest possible accuracy. To achieve this goal, we will focus our attention on the following two research subjects: (i) the joint optimization of geometry and semantic classification from aerial images in a unified framework, and (ii) the exploitation of existing geographic information systems and web data to support these two sub-tasks. In addition, we will use web-based standard to efficiently represent the obtained results for fast modeling and data parsing.

In this project we study variational methods for computing highly accurate range data in driver assistance systems.

The generation of realistic 3D models of whole cities has become a vibrant and highly competitive market through the recent activities of, most notably, Goggle Earth and Microsoft Virtual Earth. While the first generation of these systems only delivered high-quality zoomable images of the ground, the current trend is heavily geared towards 3D – that is, users can access three-dimensional height- fields of the terrain, and even 3D models of individual buildings. Simple building models, basically extruded polygons with different types of roofs, can be generated today from aerial images completely automatically. This is a solved problem. Far from solved, however, is the problem of generating automatically detailed buildings with façades. Input data for this problem are registered range maps obtained by stereo matching and sequences of highly overlapping thus redundant images (taken from a car driving in the road) where each pixel has not only a color but also a depth, a z-value. Although range maps can be directly rendered in principle, the data size is huge and, more importantly, the pixels have no semantics: A priori there is no difference between a pixel on the floor, on the wall, or on a door. But these shape semantics are required by all downstream applications using the city model. Shape grammars, on the other hand, have recently become (again) a popular method in research for representing 3D buildings. Their great advantage is that they allow to parameterize buildings, which can be used for populating virtual cities with believable architectural buildings, e.g., for 3D games. The goal of the CITYFIT project is, given highly redundant input imagery and range maps from an arbitrary building in Graz, to synthesize a shape grammar that, when evaluated, creates a clean, CAD- quality reconstruction of that building that fits the original data very closely and makes the semantics of all major architectural features explicit. These shape semantics can even be transferred back to inform the original data, so each of these “semantically enriched” data points can tell whether it belongs to ground, wall, or door.

The systematic creation of models of the real world to support the locational awareness on the Internet can be achieved if previously required massive manual labor gets replaced by automated procedures. A particular challenge exists in the automation of the extraction of the 4 classical map features buildings, circulation spaces (e.g. road networks), vegetation and water bodies, as well as their interaction. Decennia of research have been unable to automate the extraction of these features from classical aerial photography towards an economically viable result. However, we believe that we can succeed in the proposed project to develop automated procedures to create feature data for three reasons. First is the recent advent of digital aerial sensors producing highly redundant digital large format aerial photography. Redundancy will be obtained by using high forward and side overlaps, say at 80% and 60%, so that every point in the terrain is imaged at least 10 times, and any algorithm can rely on multiple analysis results that then can either reinforce or cancel one another. Second, the geometric redundancy gets augmented by a radiometric redundancy using 4 spectral bands, adding an infrared band to the classical red, green and blue color channels. Third, we will combine the classical "object reconstruction" approach available from stereo procedures, by new recognition methods. While classically a "car" on a street may have been seen via a "point cloud" and would have to get recognized simply by a representation of local height anomaly on an otherwise flat reference surface, recognition includes the use of stored images of cars in a data base to actually recognize a car as a human would do when inspecting an aerial image. The project is split up into five work packages which will focus on how reconstruction and recognition techniques can help each other and how additional information either from a previous mission or GIS can be integrated in the 3D modeling framework. One work package will address the assessment of the obtained quality, another will address project management and dissemination activities. Within the project we will develop an extensive library of combined recognition/reconstruction methods, and apply them to a range of test data sets. Test data will vary in geometric resolution (pixel size), overlaps, and types of terrain scenarios.

Electrical impulse discharges in nature are visible as lightning. Their impact point can be electro magnetically located up to a precision of several hundred meters. In some restricted areas such as industrial plants, airports etc. it desirable to know the the impact region and path of the lightning up to a precision of a few meters. If visibility is not too restricted by weather conditions, a multi-camera setup would be a viable option to locate path and impact area of the discharge.

In this project impulse discharges of a few meters are synthetically generated under laboratory conditions and reconstructed using a multi camera setup.

Deformation analysis of material surfaces is a crucial part of material testing and quality control. In this project a stereoscopic surface measurement system has been developed which allows to measure surface deformation during stress/strain tests over a wide range of fields of view. Stereo image pairs are acquired at specified time instances which makes it possible to synchronize the acquisition with the amount of stress applied.

Vehicles driving the wrong way down a motorway represent a serious source of danger. In the year 2005 in Austria, 521 vehicles were counted driving in the wrong direction on motorways. 8 people died due to wrong-way driver accidents.

Immediate detection of a vehicle driving in the wrong direction could help preventing serious accidents by warning the oncoming vehicles (via traffic telematic systems or radio announcements) and by alarming the police. To guarantee immediate detection, one would have to observe every access ramp and every place where a car could turn. Thus a low-cost monitoring system capable of detecting wrong-way drivers and forwarding an alarm to a central traffic control station is needed. Image based detection suffers from one main problem: In case a large vehicle (truck, bus etc.) throws a shadow to the opposite lane of a motorway, that shadow moves against the direction of traffic and may cause a false detection. With a monocular camera system (one single camera), this problem can only be eliminated by large computational costs.

The scanning electron microscope (SEM) is an important tool to examine very small structures. Its large magnification combined with good contrast and large depth of view make it possible to view and characterize microscopic structures in the sub-micron scale. In the recent years, the problem of dense surface reconstruction from multiple SEM images was a research topic on this institute. Reconstruction approaches like shape from stereo and shape from photometric stereo have been evaluated. This work presents a framework for automatic scene reconstruction from three images acquired by a scanning electron microscope. The basic assumption is that the specimen is tilted eucentrically in front of the camera, camera geometry is assumed to be unknown but constant over all views. It is shown that methods for estimating the trifocal tensor as well as modern auto-calibration approaches can be adapted to the imaging conditions in the SEM, and Euclidean scene structure can be retrieved from three uncalibrated views. The performance of the proposed framework is evaluated on synthetic data as well as real images. It is shown that Euclidean scene structure can be retrieved robustly under varying image noise and inaccurate initialization.

Research topics on mobile robotics:

-Visual localisation and map building

-Usage of local visual landmarks

-Localisation using omnidirectional camera