Most digital cameras use a single sensor to capture the red, green, and blue channels of a color image. For this purpose, the sensor is covered with a color filter array (CFA) measuring a single color value at each pixel location. The Bayer CFA is the most popular one, containing twice as many green as red or blue pixels. Demosaicing is the process of reconstructing a color image from the sampled CFA image, and performs an interpolation of each color channel. The main problem in this interpolation is color aliasing: artificial colors appearing due to undersampling of the separate color channels. Super-resolution algorithms combine the information present in a set of images of the same scene to compute a high resolution image. Typically, a set of aliased images is used that differ by small horizontal and vertical shifts.
|
|
| Figure 1: High resolution, aliasing-free images are reconstructed from sets of low resolution, aliased CFA images. Demosaicing and super-resolution reconstruction are performed jointly. | |
When performing demosaicing and super-resolution imaging separately, the color aliasing artifacts introduced in the demosaicing stage are considered as part of the images for the super-resolution algorithm. In this project, we therefore developed a joint approach that performs demosaicing and super-resolution simultaneously.
Our approach is based on the fact that luminance and chrominance information are encoded in separate regions of the Fourier transform of the CFA image. First, we align the images with high precision using the low frequency, aliasing-free luminance information. Next, we separate the Bayer CFA images into luminance and chrominance information and separately interpolate the luminance and chrominance images using the information from all the input images. The interpolation is performed using a normalized convolution approach. Finally, the resulting luminance and chrominance images are combined into a high resolution color image containing less color artifacts.
The resulting algorithm is computationally more efficient than a separate approach, and gives similar results. No significant improvements in image quality are obtained due to fundamental theoretical limitations. If the luminance and chrominance spectra overlap due to undersampling, it is impossible to separate them again without making assumptions on the specific class of images.
Karim Krichane
Sabine Süsstrunk
EPFL.