Basic concepts

Before going into the mathematics that support the reconstruction process, we will go through the basic concepts of the methodology. That way we can gain a better understanding of the whole process, its advantages and its limitations. The problem we are addressing is that an image of a scene has lost all information regarding depth. This means that we cannot measure the distance of an object from the camera. Also given an uncalibrated image, (or in the extreme case, a painting) we don't know the internal attributes of the camera.

Multiple view limitations

In multiple-view reconstruction one can measure the missing camera attributes, and also regain some of the the three dimensional information by identifying corresponding views of the same object on the scene on different images [16]. Measurements can be obtained by exploiting the relevant distance between the images of the objects. When only one view of the scene is available we cannot use techniques such us this.

Single view as the reverse of perspective drawing

The approach proposed in Criminisi's and Zisserman's work, is basically the reverse process of perspective drawing. When an artist draws in perspective, she uses a set of three assisting points called ``Vanishing points'' or ``VPs''. Each pair of parallel lines in the world, is drawn to intersect on one of these three points (figure 1).

Figure 1: Perspective drawing of a cube using predefined vanishing points.

Two of these points are on the line of the horizon. The horizon is the line where the ground seems to disappear in a photograph, and is an effect of perspective projection. This line is also called ``Horizontal Vanishing line'' or ``Vl'', in Criminisi literature. The third point is called Vertical vanishing point. Later in this document the Vertical VP will be denoted as and the two on the horizon will be denoted as and .

With the two of the and the vertical vanishing point, we obtain two more lines. These are also vanishing lines, but for different planes (not the ground) [5, p. 109]. All pairs of parallel lines on the ground that move away ¹ from the image plane on the scene, intersect on one of the two horizontal vanishing points. The Vertical vanishing point is the intersection of all parallel lines going upwards. The relative position of these points to the center of the painting controls its perspective attributes.

The main goal in Criminisi's technique is to determine the correct three , and then use them to make an image-two-world mapping for each point in the image².

Unfortunately determining these points is not completely straight forward. Photographs can be quite distorted by imperfections of the camera lens (very common is `radial distortion' due to poor lens quality). Also many photographs contain a significant amount of noise and can be in low resolution (images from surveillance cameras, low cost consumer devices etc.). On the other hand many paintings which appear to have been painted in perspective, contain distortions either on purpose , or by mistake of their creator.

All these factors, introduce artifacts in the image that in most cases cannot be easily removed. The result is that when we apply the proposed methods, on most images, the algorithm cannot decide which are the correct vanishing points so user input is necessary.

Techniques that exploit the geometric patterns on a scene to obtain better results in the determination process have been proposed [17,15] and are discussed below. These techniques, do not provide a fully automated solution. Although they increase the complexity of the system, user guidance is still necessary.

Footnotes

... away¹

An object following that line is moving towards the horizon and increases its distance from the camera

... image²

The mathematics for this will be presented later in this document.