Image Algorithmics Home Page

Disregarding Spectral Overlap - a Unified Approach for Demosaicking, Compressive Sensing and Color Filter Array Design

In proceedings of the IEEE ICIP, 2011

Modern Color Filter Array (CFA) designs avoid overlap of luminance and chrominance spectra as it is widely believed to cause irreversible crosstalk. In this paper, we present a simple formulation of the demosaicking problem that disregards spectral overlap - and thus the geometry of the spectra - and instead connects the luminance and chrominance resolutions in a more fundamental way to the photosite density of the sensor. A linear universal demosaicker consisting of space variant filters results from this formulation along with CFA designs that capture more image information than competing designs. This includes the interesting class of random RGB designs that outperform the Bayer CFA with nonlinear demosaickers in terms of PSNR and have a number of additional desirable properties. Furthermore, we present non-linear enhancements to our universal demosaicker and show them to be either related to compressive sensing or an alternate reconstruction algorithm for it.


Joint Chromatic Aberration Correction and Demosaicking

In proceedings of the IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, 2012

Chromatic Aberration of lenses is becoming increasingly visible with the rise of sensor resolution, and methods to algorithmically correct it are becoming increasingly common in commercial systems. A popular class of algorithms undo the geometric distortions after demosaicking. Since most demosaickers require high frequency correlation of primary colors to work effectively, the result is artifact-ridden as Chromatic Aberration destroys this correlation. The other existing approach of undistorting primary color images before demosaicking requires resampling of sub-sampled primary color images and is prone to aliasing. Furthermore, this algorithm cannot be applied to panchromatic CFAs. We propose a joint demosaicking and Chromatic Aberration correction algorithm that is applicable to both panchromatic and primary color CFAs and suffers from none of the above problems. Our algorithm treats the mosaicing process as a linear transform that is invertible if luminance and chrominance are appropriately bandlimited. We develop and incorporate Chromatic Aberration corrections to this model of the mosaicing process without altering its linearity or invertibility. This correction works for both space variant linear filter demosaicking and the more aggressive compressive sensing reconstruction.


Linear universal demosaicking of regular pattern color filter arrays

In proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012

We show that a recently developed universal demosaicker by the present authors greatly outperforms existing demosaickers when tested with a realistic optical pipeline. We present speed and quality optimizations of this demosaicker for the case of regular pattern color filter arrays. We implement and extensively test optimized versions for several common CFAs including Bayer, CMY and several RGBW patterns. These tests show that the proposed algorithms outperform other demosaickers by a substantial margin while being faster than most of them. High sensitivity RGBW CFAs are shown to have better performance than Bayer demosaicked with previous algorithms. The proposed universal demosaicker is a set of Finite Impulse Response Filters, which allows a single, efficient, Image Signal Processor design to support different CFAs by changing its filter weights. Being linear, the demosaicker is free of noise induced artifacts and outputs images with near Poissonian noise which is noise reduction friendly.


Rectangular pixels for efficient color image sampling

In proceedings of the IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, 2011

We present CFA designs that faithfully capture images with specified luminance and chrominance bandwidths. Previous academic research has mostly been concerned with maximizing PSNR of reconstructed images without specific regard to chrominance bandwidth and cross-talk. Commercial systems, on the other hand, pay close attention to both these parameters as well as to the visual quality of reconstructed images. They commonly sacrifice resolution by using a sufficiently aggressive OLPF to achieve low cross-talk and artifact free images. In this paper, we present the so called Chrominance Bandwidth Ratio, r, model in an attempt to capture both the chrominance bandwidth and the cross-talk between the various signals. Next, we examine the effect of tuning photosite aspect ratio, a hitherto neglected design parameter, and show the benefit of setting it at a different value than the pixel aspect ratio of the display. We derive panchromatic CFA patterns and corresponding photosite aspect ratios that provably minimize the photosite count for all values of r. An interesting outcome is a CFA design that captures full chrominance bandwidth, yet uses fewer photosites than the venerable color-stripe design. Another interesting outcome is a low cost practical CFA design that captures chrominance at half the resolution of luminance using only 4 unique filter colors, that lends itself to efficient linear demosaicking, and yet vastly outperforms the Bayer CFA with identical number of photosites demosaicked with state of the art compute-intensive nonlinear algorithms.