Research Projects

The aim of this research project is holistic scene understanding in large aerial datasets, consisting of thousands of massively redundant high-resolution images. Holistic scene understanding is one of the major problems in computer vision and photogrammetry and has recently got a lot of attention. The problem of holistic image understanding includes two fundamental tasks: 3D scene reconstruction and semantic interpretation of the imaged content at the level of pixels. The tight interaction between semantic classification and 3D reconstruction is often ignored by state of the art aerial image processing workflows, due to the lack of computational power, the absence of efficient algorithms or the enormous effort of manual intervention. However, these tasks are mutually informative and should be solved jointly as a correct class labelling is a valuable source of information for reconstruction, and 3D information can help to improve the semantic interpretation. For instance, a correct classification is a valuable source of information for reconstruction in regions where dense matching methods fail (e.g. sheets of water and reflecting windows / facades), and 3D information can be used as a prior to improve classification (e.g. building and road detection). The high resolution and redundancy due to large overlaps of aerial images requires massive processing power which will be handled by taking advantage of graphic processing units that have proved to give a significant speedup compared to single core machines. In particular, we will focus on algorithms based on variational methods, which provide a high degree of parallelization capability. In order to reduce cost-intensive manual interaction, we further will exploit publicly available user-data from the Internet to improve both interpretation and 3D reconstruction.

In the HOLISTIC project we will provide a flexible framework for scene classification and 3D reconstruction from aerial images that outperforms current state-of-the art and delivers interpretable models at highest possible accuracy. To achieve this goal, we will focus our attention on the following two research subjects: (i) the joint optimization of geometry and semantic classification from aerial images in a unified framework, and (ii) the exploitation of existing geographic information systems and web data to support these two sub-tasks. In addition, we will use web-based standard to efficiently represent the obtained results for fast modeling and data parsing.

Smartphones have evolved considerably in processing power over the last years. They now feature multi-core CPUs as well as GPUs and consumer-quality cameras up to HD resolution. This makes them an interesting platform for graphics and vision and opens new opportunities for research.

The aim of AR4DOC is to facilitate the task of document inspection by a human operator. This requires the person to have detailed knowledge about the nature of a document, which may be outdated or even unavailable at the time of inspection.

We seek to provide this information in an interactive way using Mobile Augmented Reality (AR), so that a well-grounded decision on the vailidity of a document is possible. This involves several tasks such as document localization, recognition, tracking, presentation as well as interaction.

The growth of information is nowadays enormous and at a level which had never been reached before. We currently produce almost more data in one year than was produced in the entire history of mankind so far. In particular the trend to a full digitization of audiovisual content is contributing to this explosion of available material. The exponential growth of online video, most notably YouTube among the many prominent video portals is just one example for that. Even if international studies are not arriving at exactly the same results, the figures are impressive: digital production in 2006 was approximately 160 Exabyte, and is predicted to rise to 990 Exabyte in 2010.

Any video processing /editing software has to keep pace with these extraordinary data rates which requires special efforts from the hardware and the software. Fortunately we see also an extraordinary increase in processing power, especially when looking at recent developments of graphics cards (GPUs). These cards offer massive parallelism (ideally suited for video processing) at a rather modest price. All these facts make this hardware an ideal candidate for video processing. But in order to make full use of the hardware the algorithms have to be highly parallel. Typical tasks encountered in video processing (which will also be tackled by the proposed project are):

Superresolution: With the advent of HDTVs in many homes there is an increasing need to produce also HDTV content. In order to make use of existing (low-resolution) material one can use so called superresolution algorithms. These methods generate from a sequence of low resolution frames a high resolution image by exploiting the high interframe redundancy.

Denoising: There are many sources of noise in a video, either the material is historic or during production/compression etc. noise is added to the video. A basic task is to remove the noise but still preserve all fine scale details.

Interactive video editing: For post production purposes one wants to mark objects in a video (of course the object should only be marked in a single frame and then segmented automatically in all subsequent frames) and either remove them (which requires inpainting methods to fill the holes with meaningful content), place them somewhere else in the video or replace them with different objects. Since these tasks are done interactively this requires interactive framerates.

Fortunately all of these tasks can be addressed by so called variational methods. The basic idea is to formulate the task as a minimization problem of a suitable energy functional. Besides other desirable properties these methods can be implemented in a highly parallel fashion which makes them ideal candidates for implementation on modern GPUs.

In this project we study variational methods for computing highly accurate range data in driver assistance systems.

Different authorities like such as the Ministry of the Interior often require to find certain event or behavior patterns in recordings in large video archives. This "forensic" search is computationally extremely expensive and due to restricted storage permissions often even not possible. Thus, security-critical events can often not prevented or being postpursued. To overcome these problems, the aim of the OUTLIER project is the investigation of algorithms, methods, and processes to alleviate the work of security staff in searching and pursuiting of events in video archives. Furthermore these tasks should be performed more efficiently.

Based on the requirements of the Ministry of the Interior as well as the possibilities of an infrastructure operator these issues should be examined and a research prototype should be created. This should occur in cooperation of AIT and ICG (University of Technology Graz) as research partners and ASE as an industrial partner. Essential research subjects are: (i) detection and segmentation of people, (ii) comparisons and finding of events in different video streams, and (iii) analyses and learning of behavior patterns. In addition, a social-scientific acceptance research will be established by the research institute of the Red Cross (FRK). Based on these results recommendations are compiled for the optimization by use and minimization of problem potentials from social-scientific view.

The ever increasing number of cameras in surveillance system requires automatic video analysis in order to spot critical situations and to alert the monitoring personnel in a timely manner. While most current approaches in this area aim for detecting a large number of specific events on a large set of complex application scenarios, the goal of this project is to go far beyond state of the art by developing novel online learning methods to detect unusual situations in a camera specific scenario. We will exploit the huge amount of data available for a specific camera to reliably learn usual and unusual situations.

In particular the OUTLIER project will carry out basic research in the following areas:

• Improved unsupervised learning methods for huge amounts of data
• Novel methods for semi-supervised learning in huge amounts of unlabeled data

These generic learning algorithms will be applied for the detection of unusual situations in public places and traffic scenarios. Examples are the detection of unusual crowd behavior (upcoming panic, barred emergency exits, or toppled persons), suspicious behavior of pedestrians (e.g. going from one car to another, loitering), vehicles or persons moving on unusual locations, the detection of unusual types of moving objects and detection of unusual situations like accidents, clashes and collisions. Unlike other approaches we do not want to model these situations explicitly and individually, but we will resort to learning to discriminate the usual situation from the unusual one.

Research partners in the project are JRS, TUG for basic and applied research and Siemens for industrial exploitation of project results.

The goal of this project is to develop an interactive visual search method that finds a given pedestrian in a large archive of other camera views efficiently. A user-selected pedestrian image or sequence is used to obtain initial discriminative features and an initial ranked list of hypothetical matches. A discriminative pedestrian recognition model is learned in an on-line manner by user interaction assigning positive and negative labels to the initially retrieved results and on-line boosting for feature selection. This enables that the best discriminative features for the current query are selected.

The generation of realistic 3D models of whole cities has become a vibrant and highly competitive market through the recent activities of, most notably, Goggle Earth and Microsoft Virtual Earth. While the first generation of these systems only delivered high-quality zoomable images of the ground, the current trend is heavily geared towards 3D – that is, users can access three-dimensional height- fields of the terrain, and even 3D models of individual buildings. Simple building models, basically extruded polygons with different types of roofs, can be generated today from aerial images completely automatically. This is a solved problem. Far from solved, however, is the problem of generating automatically detailed buildings with façades. Input data for this problem are registered range maps obtained by stereo matching and sequences of highly overlapping thus redundant images (taken from a car driving in the road) where each pixel has not only a color but also a depth, a z-value. Although range maps can be directly rendered in principle, the data size is huge and, more importantly, the pixels have no semantics: A priori there is no difference between a pixel on the floor, on the wall, or on a door. But these shape semantics are required by all downstream applications using the city model. Shape grammars, on the other hand, have recently become (again) a popular method in research for representing 3D buildings. Their great advantage is that they allow to parameterize buildings, which can be used for populating virtual cities with believable architectural buildings, e.g., for 3D games. The goal of the CITYFIT project is, given highly redundant input imagery and range maps from an arbitrary building in Graz, to synthesize a shape grammar that, when evaluated, creates a clean, CAD- quality reconstruction of that building that fits the original data very closely and makes the semantics of all major architectural features explicit. These shape semantics can even be transferred back to inform the original data, so each of these “semantically enriched” data points can tell whether it belongs to ground, wall, or door.

Computer vision and computer graphics constitute two closely related areas of research: Though both fields rely on the same physical and mathematical principles and on a common set of representations, they mainly differ in how these representations are built. Traditionally these two fields have been treated as separate academic discipline. Exploiting the commonalities between vision and graphics turns out to be a scientifically profitable endeavour. There are many examples of fruitfull combination of graphics and vision, but there is no systematic education of students (especially in Austria). Therefore, the goal of this doctoral program Confluence of Vision and Graphics is to educate highly talented PhD students in this interdisciplinary field and to teach them a common view of this challenging topic from the start. All proposed topics require a significant amount of vision and graphics. The students will be co-supervised jointly by one professor with vision and one professor with graphics expertise. The proposed educational program will ensure that the students will be trained to become future leading scientists, which will face the challenges of research excellence in the interdisciplinary area of graphics and vision, academic leadership, and social competence as a member of a particular research group as well as being a part of the global research network.

The world will witness a tremendous increase in the number of vehicles in the near future. Future traffic monitoring systems will therefore play an important role to improve the throughput and safety of roads. Current monitoring systems capture (usually vision-based) traffic data from a large sensory network; however, they require continuous human supervision which is extremely expensive.

In the proposed EVis research project we investigate the scientific and technological foundations for future autonomous traffic monitoring systems. Autonomy is achieved by a novel combination of three approaches: First, vision-based detection and classification methods are augmented by self-learning and scene adaptation mechanisms which will significantly reduce the effort of manual configuration. Second, visual data is fused with data from other sensors such as radar, infrared or inductive loop sensors. Sensor fusion helps to improve the robustness and confidence, to extend the spatial and temporal coverage as well as to reduce the ambiguity and uncertainty of the processed sensor data. Finally, the developed vision and fusion methods are implemented on a distributed embedded platform which makes them wider applicable and supports real-time operation.

Our autonomous traffic monitoring system will be evaluated using real world traffic data. The evaluation will be conducted in three different case studies: offline testing using recorded data, online testing on a traffic test site, and on a test installation on a public road.