English only
Matlab code is available upon request.
(Note that sometimes you have to click twice on a link to bring the pictures up. We are trying to fix the bug)
There is an analogy between single-chip color cameras and the human visual system in that these two systems acquire only one limited wavelength sensitivity band per spatial location. We have exploited this analogy, defining a model that characterizes a one-color per spatial position image as a coding into luminance and chrominance of the corresponding three-colors per spatial position image. Luminance is defined with full spatial resolution while chrominance contains sub-sampled opponent colors. Moreover, luminance and chrominance follow a particular arrangement in the Fourier domain, allowing for demosaicing by spatial frequency filtering. This model shows that visual artifacts after demosaicing are due to aliasing between luminance and chrominance and could be solved using a pre-processing filter. This approach also gives new insights for the representation of single-color per spatial location images and enables formal and controllable procedures to design demosaicing algorithms that perform well compared to concurrent approaches, as demonstrated by experiments.
Figure 1: Decomposition of the Bayer CFA into three sub-sampled lattices
Figure 2: Example of a bilinear interpolation computed with convolution filters. (a) Original image: the frequency spectrum of the red (and green and blue) occupies most of the Fourier spectrum. (b) Reconstructed image with bilinear interpolation. (c) Low pass filtering applied to the original image to reduce the size of the Fourier spectrum of the image. (d) The reconstruction of image (c) after mosaicing according to the Bayer CFA and demosaicing by bilinear interpolation shows no artifacts.
Figure 3: Decomposition of (a) a color image into (b) luminance and (c) chrominance as defined in eq. 5. (d) A CFA image with a single chromatic sensitivity per spatial location according to the Bayer CFA arrangement (the image appears greenish because of the double number of green pixels in the CFA). When subtracting luminance (b) from the CFA image (d), the resulting image is (e), which corresponds to a multiplexed version of sub-sampled chrominance. Selecting pixels in front of each color sample of the Bayer CFA (i.e. de-multiplexing opponent colors) results in image (f). Also, sub-sampling image (c) results exactly in image (f). Thus, image (c) can be recovered from (f) using interpolation.
Figure 4: (a) Fourier representation of a CFA image. The energy of the luminance is concentrated in the center of the image. The energy of chrominance is located on the borders. (b) The luminance filter used in the simulations discussed in Section V. (c) Representation of the transfer function of the luminance estimation filter.
Figure 5: The four artifacts visible after demosaicing: (a) Excessive blurring (b) Grid effect (c) Watercolor (d) False color.
Figure 6: Results of the frequency selection demosaicing algorithm on four images. The same luminance filter (Figure 4-b) is used for the reconstruction.
Figure S1: Comparison of demosaicing by frequency selection and two implementations of the alternating projection algorithm [1]. In the first implementation, we use bilinear interpolation as initialization, one level of decomposition, and eight iterations. In the second implementation, we use a template matching method as initialization, one level of decomposition, and eight iteration. For demosaicing by frequency selection, we use for all images the Filter given in Figure 4 (a).
Figure S2: An image mosaiced with a Bayer CFA containing two times more blue than red and green pixels, demosaiced by frequency selection.
Figure S3: An image mosaiced with a Bayer CFA containing two times more red than blue and green pixels, demosaiced by frequency selection.
Table 1: Comparison of CPSNR obtained with the Bayer CFA, a modified Bayer CFA where the red and green pixels are exchanged, and a CFA where the blue and green pixels are exchanged. An optimal filter is computed for each image and each CFA with respect to CPSNR.
Table 2: Comparison of CPSNR and computation time between different demosaicing algorithms. The same frequency selection filter (Figure 4-b) is used for all images.
[1] B.K. Gunturk, Y. Altunbasak, and R.M. Mersereau, Color plane interpolation using alternating projections, IEEE Transactions on Image Processing, vol. 11, no 9, Sept. 2002, pp. 997-1013