Roberto Costantini and Patrick Vandewalle
In recent years, digital photography has established itself successfully in the consumer electronics market. The improvements of image processing tools, the progress in sensor design and optics manufacturing have contributed to create a highly technological product, able to obtain an excellent performance. Photos taken with a digital camera have become almost indistinguishable from those taken with a classical SLR (single lens reflex) camera of good quality.
Despite their technical differences, traditional (analog) and digital photography share the same principle, i.e. that of capturing light information coming from the world through a special sensing support. In traditional photography, this support is photographic film, a fundamental discovery of the nineteenth century, deeply studied and nowadays well understood. In digital photography, this support is the CCD (Charge Coupled Device) or MOS (Metal-Oxide-Semiconductor) sensor of the camera. Since the invention of CCD in 1970 by W. Boyle and G. Smith at Bell Labs US[1], impressive technological progress has been made in the design of digital sensors: the pixel size has been dramatically reduced, the dynamic range and read-out time have been improved, and the noise resulting from the sensing operations, inherent to all electronic devices, has been reduced as well.
The virtual camera project starts precisely from the idea of modeling this noise, and extends it to the modeling the entire imaging process. The aim is to create a "virtual sensor" able to effectively produce images of a scene as they would look like with a real digital camera. Such a virtual device can be very useful in sensor design, evaluation of different image processing algorithms, etc.
Our current model consists of two main parts. First, an image is computed from a 3D model of a given scene, mimicking the real image of the scene that would be shown to the CCD sensor. The 3D model is based on the XRML ray tracing software developed by Prof. P. Bekaert et al.[2] The light rays arriving on the CCD are traced from the light source, through different reflections on the objects and through the optical system onto the sensor. All light rays are represented by their spectral components (hyperspectral color), which are the components of the light conveyed by the rays to the sensor. We currently use a single lens model for the optical system, allowing us to set the focus as desired.
The second part of our model is based on the vCam Matlab software developed by Prof. B. Wandell et al. at Stanford University [3]. This part models the sensor characteristics, analog-to-digital conversion, different noise sources, demosaicing and post-processing of the image.
This project needs a multidisciplinary approach. Improvements of the overall camera model require knowledge in various domains, such as optics, ray tracing, semiconductors, signal processing, etc. We will first make a detailed analysis of the overall performance of our model. This will then allow us to improve the model for the blocks that cause the major errors in the final image.
Sabine Süsstrunk
Luciano Sbaiz
We want to thank Dr. Philippe Bekaert, Associate Professor of the Expertise Center for Digital Media, University of Limburg (Belgium), for his kind help in illustrating the aspects of RenderPark[2] ray tracing environment that could be useful for our project.
Swiss National Science Foundation (SNF) under grant number 21-067012.01.
National Competence Center in Research on Mobile Information and Communication Systems (NCCR-MICS), a center supported by the Swiss National Science Foundation under grant number 5005-67322.
[1] W.S. Boyle, G.E. Smith, "Charge-coupled semiconductor devices.", Bell Systems Technical Journal, Vol.49, 1970, pp. 587.
[2] RenderPark: http://www.cs.kuleuven.ac.be/cwis/research/graphics/RENDERPARK/
[3] J. E. Farrell, F. Xiao, P. B. Catrysse and B. A. Wandell, A simulation tool for evaluating digital camera image quality, Proc. IS&T/SPIE Electronic Imaging 2004: Image Quality and System Performance, Vol. 5294.