We propose a framework for object tracking in image sequences, following the concept of tracking-by-segmentation. The separation of object and background is achieved by a consecutive semantic superpixel segmentation of the images, yielding tight object boundaries. I.e., in the first image a model of the object’s characteristics is learned from an initial, incomplete annotation. This model is used to classify the superpixels of subsequent images to object and background employing graph-cut. We assume the object boundaries to be tight-fitting and the object motion within the image to be affine. To adapt the model to radiometric and geometric changes we utilize an incremental learner in a co-training scheme. We evaluate our tracking framework qualitatively and quantitatively on several image sequences.

We evaluate the performance of Import Vector Machines (IVM),a sparse Kernel Logistic Regression approach, for the classification of hyperspectral data. The IVM classifier is applied on two different data sets, using different number of training samples. The performance of IVM to Support Vector Machines (SVM) is compared in terms of accuracy and sparsity. Moreover, the impact of the training sample set on the accuracy and stability of IVM was investigated. The results underline that the IVM perform similar when compared to the popular SVM in terms of accuracy. Moreover, the number of import vectors from the IVM is significantly lower when compared to the number of support vectors from the SVM. Thus, the classification process of the IVM is faster. These findings are independent from the study site, the number of training samples and specific classes. Consequently, the proposed IVM approach is a promising classification method for hyperspectral imagery.

Our objective is the categorization of the most dominant objects in facade images, like windows, entrances and balconies. In order to execute an image interpretation of complex scenes we need an interaction between low level bottom-up feature detection and high-level inference from top-down. A top-down approach would use results of a bottom-up detection step as evidence for some high-level inference of scene interpretation. We present a statistically founded object categorization procedure that is suited for bottom-up object detection. Instead of choosing a bag of features in advance and learning models based on these features, it is more natural to learn which features best describe the target object classes. Therefore we learn increasingly complex aggregates of line junctions in image sections from man-made scenes. We present a method for the classification of image sections by using the histogram of diverse types of line aggregates.

@INPROCEEDINGS{wenzel*12:learning, author = {Wenzel, Susanne and F{\" o}rstner, Wolfgang}, title = {Learning a compositional representation for facade object categorization}, booktitle = {ISPRS Annals of Photogrammetry, Remote Sensing and the Spatial Information Sciences; Proc. of 22nd Congress of the International Society for Photogrammetry and Remote Sensing (ISPRS)}, year = {2012}, note = {to appear} }

Three methods of automatic classification of leaf diseases are described based on high-resolution multispectral stereo images. Leaf diseases are economically important as they can cause a loss of yield. Early and reliable detection of leaf diseases has important practical relevance, especially in the context of precision agriculture for localized treatment with fungicides. We took stereo images of single sugar beet leaves with two cameras (RGB and multispectral) in a laboratory under well controlled illumination conditions. The leaves were either healthy or infected with the leaf spot pathogen Cercospora beticola or the rust fungus Uromyces betae. To fuse information from the two sensors, we generated 3-D models of the leaves. We discuss the potential of two pixelwise methods of classification: k-nearest neighbour and an adaptive Bayes classification with minimum risk assuming a Gaussian mixture model. The medians of pixelwise classification rates achieved in our experiments are 91% for Cercospora beticola and 86% for Uromyces betae. In addition, we investigated the potential of contextual classification with the so called conditional random field method, which seemed to eliminate the typical errors of pixelwise classification.

@article{bauer*11:potential, author={Bauer, Sabine and Kor{\v c}, Filip and F\"{o}rstner, Wolfgang}, title={The potential of automatic methods of classification to identify leaf diseases from multispectral images}, journal={Precision Agriculture}, year={2011}, number={3}, pages={361-377}, }

We study the set of domain deformations induced on images of three-dimensional scenes by changes of the vantage point. We parametrize such deformations and derive empirical statistics on the parameters, that show a kurtotic behavior similar to that of natural image and range statistics. Such a behavior would suggest that most deformations are locally smooth, and therefore could be captured by simple parametric maps, such as affine ones. However, we show that deformations induced by singularities and occluding boundaries, although rare, are highly salient, thus warranting the development of dedicated descriptors. We therefore illustrate the development of viewpoint invariant descriptors for singularities, as well as for occluding boundaries. We test their performance on scenes where the current state of the art based on affine-invariant region descriptors fail to establish correspondence, highlighting the features and shortcomings of our approach.

@article{dickscheid*11:coding, author = {Dickscheid, T. and Schindler, F. and F\"{o}rstner, W.}, title = {{Coding Images with Local Features}}, journal = {International Journal of Computer Vision}, publisher = {Springer Netherlands}, issn = {0920-5691}, url = {http://dx.doi.org/10.1007/s11263-010-0340-z}, year = {2011}, pages = {154-174}, volume = {94}, issue = {2}, doi = {10.1007/s11263-010-0340-z} }

High-dimensional data structures occur in many fields of computer vision and machine learning. Transformation between two high-dimensional spaces usually involves the determination of a large amount of parameters and requires much labeled data to be given. There is much interest in reducing dimensionality if a lower-dimensional structure is underlying the data points. We present a procedure to enable the determination of a low-dimensional, projective transformation between two data sets, making use of state-of-the-art dimensional reduction algorithms. We evaluate multiple algorithms during several experiments with different objectives. We demonstrate the use of this procedure for applications like classification and assignments between two given data sets. Our procedure is semi-supervised due to the fact that all labeled and unlabeled points are used for the dimensionality reduction, but only few them have to be labeled. Using test data we evaluate the quantitative and qualitative performance of different algorithms with respect to the classification and assignment task. We show that with these algorithms and our transformation approach high-dimensional data sets can be related to each other. Finally we can use this procedure to match real world facial images with cartoon images from Springfield, home town of the famous Simpsons.

We propose a surface segmentation method based on Fast Marching Farthest Point Sampling designed for noisy, visually reconstructed point clouds or laser range data. Adjusting the distance metric between neighboring vertices we obtain robust, edge-preserving segmentations based on local curvature. We formulate a cost function given a segmentation in terms of a description length to be minimized. An incremental-decremental segmentation procedure approximates a global optimum of the cost function and prevents from under- as well as strong over-segmentation. We demonstrate the proposed method on various synthetic and real-world data sets.

@inproceedings{Schindler*11:fast, author = {Schindler, Falko and F\"orstner, Wolfgang}, booktitle = {LNCS, Photogrammetric Image Analysis}, title = {{Fast Marching for Robust Surface Segmentation}}, year = {2011} }

We present a novel surface model and reconstruction method for man-made environments that take prior knowledge about topology and geometry into account. The model favors but is not limited to horizontal and vertical planes that are pairwise orthogonal. The reconstruction method does not require one particular class of sensors, as long as a triangulated point cloud is available. It delivers a complete 3D segmentation, parametrization and classification for both surface regions and inter-plane relations. By working on a pre-segmentation we reduce the computational cost and increase robustness to noise and outliers. All reasoning is statistically motivated, based on a few decision variables with meaningful interpretation in measurement space. We demonstrate our reconstruction method for visual reconstructions and laser range data.

@INPROCEEDINGS{schindler*11:classification, author = {Schindler, Falko and F{\"o}rstner, Wolfgang and Frahm, Jan-Michael}, title = {Classification and Reconstruction of Surfaces from Point Clouds of Man-made Objects}, booktitle = {International Conference on Computer Vision, IEEE/ISPRS Workshop on Computer Vision for Remote Sensing of the Environment}, year = {2011}, pages = {257--263}, }

We present a method to perform bundle adjustment using long sequences of digital images from an omnidirectional camera. We use the Ladybug3 camera from PointGrey, which consists of six individual cameras pointing in different directions. There is large overlap between successive images but only a few loop closures provide connections between distant camera positions. We face two challenges: (1) to perform a bundle adjustment with images of an omnidirectional camera and (2) implement outlier detection and estimation of initial parameters for the geometry described above. Our program combines the Ladybug’s individual cameras to a single virtual camera and uses a spherical imaging model within the bundle adjustment, solving problem (1). Outlier detection (2) is done using bundle adjustments with small subsets of images followed by a robust adjustment of all images. Approximate values in our context are taken from an on-board inertial navigation system.

@inproceedings{schmeing*11, author = {Schmeing, Benno, Läbe, Thomas and Förstner, Wolfgang}, title = {Trajectory Reconstruction Using Long Sequences of Digital Images From an Omnidirectional Camera}, year = {2011}, city = {Mainz}, proceeding = {Proceedings of the 31th. DGPF Conference}, }

Wir stellen einen Ansatz für eine strenge Bündelausgleichung für Multikamerasysteme vor. Hierzu verwenden wir eine minimale Repräsentation von homogenen Koordinatenvektoren für eine Maximum-Likelihood-Schätzung. Statt den Skalierungsfaktor von homogenen Vektoren durch Verwendung von euklidischen Größen zu eliminieren, werden die homogenen Koordinaten sphärisch normiert, so dass Bild- und Objektpunkte im Unendlichen repräsentierbar bleiben. Dies ermöglicht auch Bilder omnidirektionaler Kameras mit Einzelblickpunkt, wie Fisheyekameras, und weit entfernte bzw. unendlich ferne Punkte zu behandeln. Speziell Punkte am Horizont können über lange Zeiträume beobachtet werden und liefern somit eine stabile Richtungsinformation. Wir demonstrieren die praktische Umsetzung des Ansatzes anhand einer Bildfolge mit dem Multikamerasystem „Ladybug3“ von Point Grey, welches mit sechs Kameras 80 % der gesamten Sphäre abbildet.

@inproceedings{ schneider*11:buendelausgleichung, author = {Schneider, Johannes and Schindler, Falko and F\"{o}rstner, Wolfgang}, title = {B\"{u}ndelausgleichung f\"{u}r Multikamerasysteme}, year = {2011}, city = {Mainz}, proceeding = {Proceedings of the 31th DGPF Conference}, }

A temporal filter approach for real-time detection and reconstruction of curbs and road surfaces from 3D point clouds is presented. Instead of local thresholding, as used in many other approaches, a 3D curb model is extracted from the point cloud. The 3D points are classified to different parts of the model (i.e. road and sidewalk) using a temporally integrated Conditional Random Field (CRF). The parameters of curb and road surface are then estimated from the respectively assigned points, providing a temporal connection via a Kalman filter. In this contribution, we employ dense stereo vision for data acquisition. Other sensors capturing point cloud data, e.g. lidar, would also be suitable. The system was tested on real-world scenarios, showing the advantages over a temporally unfiltered version, due to robustness, accuracy and computation time. Further, the lateral accuracy of the system is evaluated. The experiments show the system to yield highly accurate results, for curved and straight-line curbs, up to distances of 20 meters from the camera.

@INPROCEEDINGS{siegemund*11:curb, author = {Jan Siegemund and Uwe Franke and Wolfgang F\"{o}rstner}, title = {A Temporal Filter Approach for Detection and Reconstruction of Curbs and Road Surfaces based on Conditional Random Fields}, booktitle = {IEEE Intelligent Vehicles Symposium (IV)}, year = {2011}, pages = {637-642}, month = {June}, publisher = {IEEE Computer Society} }

@INPROCEEDINGS{yang*11:crf-classification, author = {Yang, Michael Ying and Förstner, Wolfgang}, title = {Regionwise Classification of Building Facade Images}, booktitle = {Photogrammetric Image Analysis (PIA2011)}, year = {2011}, pages = {209 -- 220}, publisher = {Springer}, series = {LNCS 6952}, }

@INPROCEEDINGS{yang*11:hcrf-scene-classification, author = {Yang, Michael Ying and Förstner, Wolfgang}, title = {A Hierarchical Conditional Random Field Model for Labeling and Classifying Images of Man-made Scenes}, booktitle = {International Conference on Computer Vision, IEEE/ISPRS Workshop on Computer Vision for Remote Sensing of the Environment}, year = {2011}, pages = {accepted}, }

@ARTICLE{yang*11:evaluation, author = {Yang, Michael Ying and Förstner, Wolfgang}, title = {Feature Evaluation for Building Facade Images - An Empirical Study}, journal = {Department of Photogrammetry, Institute of Geodesy and Geoinformation, University of Bonn, TR-IGG-P-2011-02}, year = {2011}, }