Some theory on Single View 3D scene Reconstruction

Introduction

The original purpose of this work was to implement and test the concepts and the algorithms presented on [5,7,8] for extracting geometric information and constructing three-dimensional models from a single, un-calibrated perspective image of a scene. The goal was to present a methodology for the reconstruction and to investigate methods for automating the procedure. In the following sections there is a brief presentation of the techniques used.

Multiple and Single view techniques

Multiple-view geometry [16] algorithms cannot be applied when only one view of a scene (photograph, painting, etc.) is available. Techniques for the partial or complete reconstruction of scenes from a single image have been developed [12,2]. The main challenge of these techniques lies in correctly modeling the perspective distortions introduced by the imaging process from a single input image. They relay on geometric patterns predefined in the scene in order to make the appropriate measurements.

Later work of [15,17,5] exploited the perspective geometry features of the image and achieved measurements of lengths, planar segments and distances of points from planes. These measurements are, in most cases, sufficient for a partial or complete three-dimensional reconstruction of an image. Also, the algorithms proposed,can work in an un-calibrated framework thus there is no need for the camera internal parameters to be known as was the case with previous works.

Scene constraints such as orthogonality and parallelism of structures are exploited, making these algorithms especially suitable for scenes containing man made structures such as architectural elements and geometric patterns.

Semi-automatic approach

Even though the work of Criminisi and Zisserman greatly improved the techniques for single view reconstruction, there is still a big part of the reconstruction process that cannot be completed without user input.

The problem is that when we try to acquire spacial measurements of the world from images, we try to regain the information that was lost by the projection of the 3D space onto those images. Particularly, perspective distortions occur during the acquisition stage. For instance, objects which are far away from the camera look smaller than objects which are close, also artifacts may be introduced in the image due to low quality acquisition media (noise, radial distortion etc.). All these factors lead to incorrect measurement. In order to retrieve the correct results, user input is necessary.

In this document we present both the automatic and the interactive steps of the reconstruction process.