We present a method to perform bundle adjustment using long sequences of digital images from an omnidirectional camera. We use the Ladybug3 camera from PointGrey, which consists of six individual cameras pointing in different directions. There is large overlap between successive images but only a few loop closures provide connections between distant camera positions. We face two challenges: (1) to perform a bundle adjustment with images of an omnidirectional camera and (2) implement outlier detection and estimation of initial parameters for the geometry described above. Our program combines the Ladybug’s individual cameras to a single virtual camera and uses a spherical imaging model within the bundle adjustment, solving problem (1). Outlier detection (2) is done using bundle adjustments with small subsets of images followed by a robust adjustment of all images. Approximate values in our context are taken from an on-board inertial navigation system.

@inproceedings{schmeing*11, author = {Schmeing, Benno, Läbe, Thomas and Förstner, Wolfgang}, title = {Trajectory Reconstruction Using Long Sequences of Digital Images From an Omnidirectional Camera}, year = {2011}, city = {Mainz}, proceeding = {Proceedings of the 31th. DGPF Conference}, }

@inproceedings{drauschke*09:improving, author = {Drauschke, Martin and Roscher, Ribana and L\"{a}be, Thomas and F\"{o}rstner, Wolfgang}, title = {Improving Image Segmentation using Multiple View Analysis}, year = {2009}, pages = {211-216}, city = {Paris}, booktitle= {Object Extraction for 3D City Models, Road Databases and Traffic Monitoring - Concepts, Algorithms and Evaluatin (CMRT09)} }

In classical photogrammetry, point observations are manually determined by an operator for performing the bundle adjustment of a sequence of images. In such cases, a comparison of different estimates is usually carried out with respect to the estimated 3D object points. Today, a broad range of automatic methods are available for extracting and matching point features across images, even in the case of widely separated views and under strong deformations. This allows for fully automatic solutions to the relative orientation problem, and even to the bundle triangulation in case that manually measured control points are available. However, such systems often contain random subprocedures like RANSAC for eliminating wrong correspondences, yielding different 3D points but hopefully similar orientation parameters. This causes two problems for the evaluation: First, the randomness of the algorithm has an influence on its stability, and second, we are constrained to compare the orientation parameters instead of the 3D points. We propose a method for benchmarking automatic bundle adjustments which takes these constraints into account and uses the orientation parameters directly. Given sets of corresponding orientation parameters, we require our benchmark test to address their consistency of the form deviation and the internal precision and their precision level related to the precision of a reference data set. Besides comparing different bundle adjustment methods, the approach may be used to safely evaluate effects of feature operators, matching strategies, control parameters and other design decisions for a particular method. The goal of this paper is to derive appropriate measures to cover these aspects, describe a coherent benchmarking scheme and show the feasibility of the approach using real data.

@INPROCEEDINGS{dickscheid*08:benchmarking, author = {Dickscheid, Timo and L{\"a}be, Thomas and F{\" o}rstner, Wolfgang}, title = {Benchmarking Automatic Bundle Adjustment Results}, booktitle = {To appear in: 21st Congress of the International Society for Photogrammetry and Remote Sensing (ISPRS)}, address = {Beijing, China}, year = {2008}, }

This paper presents an empirical investigation into the quality of automatic relative orientation procedures. The results of an in-house developed automatic orientation software called aurelo (Laebe and Foerstner, 2006) are evaluated. For this evaluation a recently proposed consistency measure for two sets of orientation parameters (Dickscheid et. al., 2008) and the ratio of two covariances matrices is used. Thus we evaluate the consistency of bundle block adjustments and the precision level achievable. We use different sets of orientation results related to the same set of images but computed under differing conditions. As reference datasets results on a much higher image resolution and ground truth data from artificial images rendered with computer graphics software are used. Six different effects are analysed: varying results due to random procedures in aurelo, computations on different image pyramid levels and with or without points with only two or three observations, the effect of replacing the used SIFT operator with an approximation of SIFT features, called SURF, repetitive patterns in the scene and remaining non-linear distortions. These experiments show under which conditions the bundle adjustment results reflect the true errors and thus give valuable hints for the use of automatic relative orientation procedures and possible improvements of the software.

@INPROCEEDINGS{laebe*08:quality, author = {L{\"a}be, Thomas and Dickscheid, Timo and F{\" o}rstner, Wolfgang}, title = {On the Quality of Automatic Relative Orientation Procedures}, booktitle = {To appear in: 21st Congress of the International Society for Photogrammetry and Remote Sensing (ISPRS)}, address = {Beijing, China}, year = {2008}, }

This paper presents a new full automatic approach for the relative orientation of several digital images taken with a calibrated camera. This approach uses new algorithms for feature extraction and relative orientation developed in the last few years. There is no need for special markers in the scene nor for approximate values of the orientation data. We use the point operator developed by D. G. Lowe (2004), which extracts points with scale- and rotation-invariant descriptors (SIFT-features). These descriptors allow a successful matching of image points even when dealing with highly convergent or rotated images. The approach consists of the following steps: After extracting image points on all images a matching between every image pair is calculated using the SIFT parameters only. No prior information about the pose of the images or the overlapping parts of the images is used. For every image pair a relative orientation is computed with the help of a RANSAC procedure. Here we use the new 5-point algorithm from D. Nister (2004). Out of this set of orientations approximate values for the orientation parameters and the object coordinates are calculated by computing the relative scales and transforming the models into a common coordinate system. Several tests are made in order to get a reliable input for the currently final step: a bundle block adjustment. The paper discusses the practical impacts of the used algorithms. Examples of different indoor- and outdoor-scenes including a data set of oblique images taken from a helicopter are presented and the results of the approach applied to these data sets are evaluated. These results show that the approach can be used for a wide range of scenes with different types of the image geometry and taken with different types of cameras including inexpensive consumer cameras. In particular we investigate in the robustness of the algorithms, e. g. in geometric tests on image triplets. Further developments like the use of image pyramids with a modified matching are discussed in the outlook. Literature: David G. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, 60, 2 (2004), pp. 91-110. D. Nistér, An efficient solution to the five-point relative pose problem, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 26(6):756-770, June 2004.

@inproceedings{läbe*06:automatic, author = "Läbe, Thomas and Förstner, Wolfgang", title = "AUTOMATIC RELATIVE ORIENTATION OF IMAGES", year = "06", city = "Bonn", proceeding = "Proceedings of the 5th Turkish-German Joint Geodetic Days",}

Der Aufsatz präsentiert ein neues vollautomatisches Verfahren zur relativen Orientierung mehrerer digitaler Bilder kalibrierter Kameras. Es nutzt die in den letzten Jahren neu entwickelten Algorithmen im Bereich der Merkmalsextraktion und der Bildgeometrie und erfordert weder das Anbringen von künstlichen Zielmarken noch die Angabe von Näherungswerten. Es basiert auf automatisch extrahierten Punkten, die mit dem von D. Lowe (2004) vorgeschlagenen Verfahren zur Extraktion skaleninvarianter Bildmerkmale berechnet werden. Diese ermöglichen eine Punktzuordnung auch bei stark konvergenten Aufnahmen. Für die Bestimmung von Näherungswerten der abschließenden Bündelausgleichung wird bei der relativen Orientierung der Bildpaare das direkte Lösungsverfahren von D. Nister (2004) verwendet. Der Aufsatz diskutiert die praktischen Erfahrungen mit den verwendeten Algorithmen anhand von Beispieldatensätzen sowohl von Innenraum- als auch von Aussnaufnahmen.

@inproceedings{läbe*05:erfahrungen, author = "Läbe, Thomas and Förstner, Wolfgang", title = "Erfahrungen mit einem neuen vollautomatischen Verfahren zur Orientierung digitaler Bilder", year = "05", city = "Bonn", proceeding = "Proceedings of DGPF Conference",}

During the last years the number of available low-cost digital consumer cameras has significantly increased while their prices decrease. Therefore for many applications with no high-end accuracy requirements it is an important consideration whether to use low-cost cameras. This paper investigates in the use of consumer cameras for photogrammetric measurements and vision systems. An important aspect of the suitability of these cameras is their geometric stability. Two aspects should be considered: The change of calibration parameters when using the camera's features such as zoom or auto focus and the time invariance of the calibration parameters. Therefore laboratory calibrations of different cameras have been carried out at different times. The resulting calibration parameters, especially the principal distance and the principal point, and their accuracies are given. The usefulness of the information given in the image header, especially the focal length, is compared to the results of the calibration.

@inproceedings{läbe*04:geometric, author = "Läbe, Thomas and Förstner, Wolfgang", title = "Geometric Stability of Low-Cost Digital Consumer Cameras", year = "04", pages = "528-535", city = "Bonn", proceeding = "Proc. of XXth ISPRS Congress 2004",}

The paper describes the automatic learning of parameters for self-diagnosis of a system for automatic orientation of single aerial images used by the State Survey Department of Northrhine--Westfalia. The orientation is based on 3D lines as ground control features, and uses a sequence of probabilistic clustering, search and ML-estimation for robustly estimating the 6 parameters of the exterior orientation of an aerial image. The system is interpreted as a classifier, making an internal evaluation of its success. The classification is based on a number of parameters possibly relevant for self-diagnosis. A hand designed classifier reached 11% false negatives and 2% false positives on appr. 17000 images. A first version of a new classifier using support vector machines is evaluated. Based on appr. 650 images the classifier reaches 2 % false negatives and 4% false positives, indicating an increase in performance.

@inproceedings{förstner*03:learning, author = "Förstner, Wolfgang and Läbe, Thomas", title = "Learning Optimal Parameters for Self-diagnosis in a System for Automatic Exterior Orientation", year = "03", pages = "236-246", city = "Bonn", proceeding = "Computer Vision Systems (ICVS) 2003",}

Eine der notwendigen Voraussetzungen zur Erstellung eines Orthophotos ist die Kenntnis der äußeren Orientierung des zu bearbeitenden Bildes. Hierfür wurde am Institut für Photogrammetrie der Universität Bonn innerhalb eines Kooperationsprojektes mit dem Landesvermessungsamt Nordrhein-Westfalen ein vollautomatisches Verfahren entwickelt, das auf der Suche von projizierten 3D-Kanten im Bild basiert. Das Programm trägt den Namen "AMOR" (Automatische Modellgestützte ORientierung) und beinhaltet sowohl Bildverarbeitung (Kantenextraktion) als auch robuste Schätzverfahren für die Bestimmung der Orientierungselemente. Als Datenbasis zur Bestimmung der äußeren Orientierung werden anders als beim konventionellen manuellen Vorgehen keine Passpunkte sondern sogenannte "Passpunktmodelle" verwendet. Dies sind Mengen georeferenzierter 3D-Kanten, wofür sich insbesondere Gebäudekanten eines digitalen Gebäudemodells eignen. Zur fächendeckenden Orthophotoproduktion wurden in Nordrhein-Westfalen landesweit Gebäude als Passpunktmodelle erfasst. AMOR ist in den Produktionsablauf der Orthophotoherstellung beim Landesvermessungsamt integriert worden und kann aufgrund der Passpunktmodelldatenbank auf einem Großteil der Landesfläche angewendet werden. Der Aufsatz gibt einen Überblick über das Verfahren zur automatischen Orientierungsbestimmung und dessen Integration mit besonderem Schwerpunkt auf den praktischen Einsatz beim Landesvermessungsamt NRW.

@inproceedings{läbe*02:automatische, author = "Läbe, Thomas and Henze, Manfred", title = "Automatische äussere Orientierung in der Orthophotoproduktion - ein Erfahrungsbericht", year = "02", pages = "245-252", city = "Bonn", proceeding = "Proceedings of DGPF Conference",}

The modeling of three-dimensional objects is a current topic in digital photogrammetric research. The modeling of buildings in digital imagery or digital surface models involving automation processes has reached a level where it can compete with classical photogrammetric stereo measurements. There are many different ways on how to integrate automation. We describe our system and its automated features that support the operator in the adaption of parametric models to multiple overlapping images. There do exist tools to automate the measurement of heights, to automate the estimation of the form parameters or for the handling of building aggregates. With such tools we can reach about 20 seconds for the modeling of a volumetric primitive which is fully comparable to the currently used photogrammetric methods.

@inproceedings{gülch*99:integration, author = "Gülch, Eberhard and Müller, Hardo and Läbe, Thomas", title = "Integration of Automatic Processes Into Semi-Automatic Building Extraction", year = "99", city = "Bonn", proceeding = "Proceedings of ISPRS Conference "Automatic Extraction Of GIS Objects From Digital Imagery"",}

This paper describes the contribution of the University of Bonn to the OEEPE test on Automatic Orientation of Aerial Images (Task A). A program for the automatic exterior orientation called AMOR was developed by Wolfgang Schickler at the Institute of Photogrammetry, Bonn. The methods and ideas of this approach are summarized. This program was used to compute the exterior orientation parameters of the given two test images successfully. Results and new solved problems are reported.

@inproceedings{läbe99:contribution, author = "Läbe, Thomas", title = "Contribution to the OEEPE-Test on Automatic Orientation of Aerial Images, Task A - Experiences with AMOR", year = "99", city = "Bonn", proceeding = "OEEPE Seminar on Automatic Orientation of Aerial Images on Database Information",}

A Semi-Automatic Building Extraction system using two or more digitized overlapping aerial images has been enhanced by increased automation for the measurement of saddle-back-roof buildings, hip-roof buildings and boxes. All newly developed modules have been incorporated in the object oriented design of the system. The new methods consist of a ground-point and roof-top matching tool and a robust determination of shape parameters, like e.g. gutter length and width. The current performance of building extraction is quantitatively and qualitatively evaluated. We examine the increased efficiency using the automated tools, the success rate of individual modules and the overall success rate using a combination of methods. A methodology for quantitative comparison is tested on footprints of buildings from classical stereo measurements and from semi-automatic measurements. A qualitative comparison in 3D of multiple measurements of complete buildings is performed on three different datasets.

@inproceedings{gülch*98:performance, author = "Gülch, Eberhard and Müller, Hardo and Läbe, Thomas and Ragia, Lemonia", title = "On the performance of semi-automatic building extraction", year = "98", city = "Bonn", proceeding = "Proceedings of ISPRS Commission III Symposium",}

A semi-automatic building extraction system using two or more digitized overlapping aerial images has been enhanced by increased automation for the measurement of saddleback-roof (lopsided and symmetric) buildings, hip-roof buildings and flat-roof building (boxes). The goal is to minimize the interaction an operator has to do for measuring the form and pose parameters of 3D building models of the above mentioned types. The automated tasks are computed on-line and fully integrated in the work flow. Thus accepting or correcting the results or adapting the automated calculation is possible. The used methods are grey value correlation for absolute heights and the robust estimation techniques RANSAC and Clustering for the determination of heights and the other form parameters of the building primitives. These methods work on automatically extracted line segments. The automated modules have been empirically evaluated on more than 250 buildings in two datasets with different image quality and different densities of built-up areas. The results of these tests show a success rate of up to 88% for a form parameter estimation module and the height measurement.

@inproceedings{läbe*98:robust, author = "Läbe, Thomas and Gülch, Eberhard", title = "Robust Techniques for Estimating Parameters of 3D Building Primitives", year = "98", city = "Bonn", proceeding = "Proceedings of ISPRS Commission II Symposium",}

The bottleneck of today's automation of image orientation is the identification of control points for exterior orientation. A solution for this problem is presented. It is based on 3D-wireframe models of buildings as ground control points. The paper describes the setup of a database of such control points.

@inproceedings{läbe*96:3d-wireframe, author = "Läbe, Thomas and Ellenbeck, Karl Heiko ", title = "3D-Wireframe Models As Ground Control Points For The Automatic Exterior Orientation", year = "96", city = "Bonn", proceeding = "Proceedings of 18th ISPRS Congress",}