Facade Image Parsing

A stochastic attributed structural model for facade elements has been designed. Orthographically rectified facade images are assumed. The structural model forces the elements to be approximately aligned in an array, where configurations with regular spacing are more probable than configurations with irregular spacings. A small number of array elements is allowed to be missing.

Several algorithms for finding an approximate solution in the MAP sense has been tested. We use a grammar-based greedy parser and an MCMC sampling algorithm constructed as a mixture of basic Metropolis-Hastings sampling scheme for data exploration, Hybrid Monte Carlo for speeding up convergence and Reversible Jump MCMC for determining model complexity. All algorithms start from a set of seed detections obtained from a weakly trained AdaBoost window classifier run over a range of image scales.

We experimented with on-line data model adaptation using an on-line classifier update scheme during the parsing. We also experimented with an EM-like procedure which iteratively adapts an image color model using the current output of the parser. Both mechanisms had considerable positive effect on the performance. We also developed an independent self-assessment procedure that measures interpretation difficulty of each given input data instance. This mechanism is based on the MCMC sampler.

All the algorithms were tested on a common eTRIMS benchmark dataset of 60 images. The results are reported in the final eTRIMS deliverables.