Panoramic Digital Photography

PhD Thesis, very brief outline
The full thesis can be downloaded in pdf format here.

The three most relevant topics of the thesis are summarized here

Robust motion estimation and outlier modelling and detection

A technique is developed to be able to estimate the motion of the camera between two picture in the presence of outliers. For example, in the following image, the pedestrians are likely to perturb the estimation, because most of them are only present in one of the image. The technique is based on outlier modelling, and outperforms the traditional estimation techniques that are used in this kind of situations.
 
(a)
(b)

Fig 1.1: Two images of the same scene. Some pedestrians entered of left the field of view and are sources of confusion to a motion estimation algorithm (the algorithm is supposed to measure the motion of the camera)

Furthermore, the algorithm is able to distinguish an outlier from an inlier without using any a-priori parameter on the error of a pixel-wise comparison of the images. Follow, a sub-image of the above scene, and the associated outlier mask: in black is the probability for each pixel to belong to the outliers (black pixels = outliers, white pixels = inliers, grey pixels = half inliers and outliers)
 
(a)
(b)
(c)

Fig: 1.2: (a) and (b) show two images of the same location, properly aligned. In (b) a pedestrian crossed the field of view of the camera. (c) shows in black, the probability for each pixel to belong to an outlier.

The detection method is shown to work with an arbitrary percentage of outlying data, whereas standard method (based on a median estimator) tend to fail whenever the outlier covers more than 50% of the image.

Colour equalization across a set of images

When the images are taken with the camera set on automatic setting, the shutter speed, as well as the white balancing, may change. In other words, we may end up with an image pair that looks like this one:

(a)
(b)
Fig 2.1: images of a house (a) top (b) bottom. Since the top is much lighter than the bottom of the house, the exposure time had to be changed to get a decent picture. The picture were taken with automatic exposure and white balancing. (a) appears bluewish.

In order to avoid having important differences in the colour or the luminance of the picture, it would be sufficient, if using a digital camera - to keep the settings of the camera constant during the whole capturing process. But this would impair the final image quality, because some pictures would be either over-exposed or under-exposed. We developed the techniques to correct for these changes, as illustrated in the following figure:
 
(a)
(b)
(c)

Fig 2.2: illustration of the colour equalization algorithm. (a) original pictures, blended like a checkerboard: the black cells of the checkerboard belong to the image on the top, the white cells to the image on the bottom. (b) shutter speed difference compensation: the images match in luminance, but not in colour (c) white point (and black point) compensation.

We can see on the right image 2.2(c) that the colour mismatch has been corrected, except - as we could expect - in the region where there was some saturation in the original picture. We should also mention that correcting for colours allow a better geometric registration. This is particularly noticeable when estimating ill-posed parameters like the focal length or the lens distortion.

Rendering of high dynamic range scenes

When the scene contains very bright or very dark regions, it cannot be properly displayed on the screen or on paper: either the bright part of the picture will appear white, or the shade region will appear black. We developed a technique to render this type of images that is based on a human model of contrast perception. An example is given in Figure 3.1.
 
 
 
(a)
(b)

Fig: 3.1: Illustration of the rendering algorithm. (a) original image, rendered with standard techniques: details are lost in the shade (b) rendered image: we notice that the details in the shade are more visible, and the image correspond better to what an observer was able to see in the original place. Note: the dynamic range of the camera was not large enough, thus some noise appears clearly in the shade areas