Q1: Does RGBW suffer from low chrominance color resolution because it has fewer R, G, B pixels than Bayer?

A: The Nyquist theorem is commonly misunderstood to be a necessary condition when, in fact, it is a sufficient condition. Systems that capture compressible images, such as some leading Bayer systems and our RGBW system, can perform above the Nyquist limit. All natural images are compressible, a fact leveraged by image compression systems and also by the Human Visual System.

Reducing R, G, B pixels in a CFA is akin to increasing the compression ratio of an image encoder. It will result in loss of quality, but not in the obvious way of reduced resolution. If done well, the loss of quality will be subtle and hard to detect. In our RGBW processor, the gain in SNR from W pixels is vastly more important than the subtle loss in quality from the reduced R, G, B pixel density.

Q2: Is the color accuracy of RGBW as good as that of Bayer? Is it robust to mixed lighting and white balance errors?

A: The color accuracy of RGBW is equal to that of Bayer, within the range of experimental error. In RGBW, color pixels are subject to crosstalk from W, but this is easily corrected in processing. Answer to question 6 explains why crosstalk correction does not impact the SNR of RGBW.

Color accuracy and mixed light performance is a concern in RWB and RYYB systems since they omit the G pixel. Since RGBW does not skip pixels of any color, it’s color performance is of reference quality.

Q3: Since W pixels are more sensitive than R, G, B pixels, doesn’t it saturate early in bright light and so reduce the dynamic range?

A: We overcome this problem by ignoring W pixels in clipped highlights and demosaicking only the RGB pixels. This leads to a slight decrease in sensitivity in highlights but retains the W pixels’ advantage in the rest of the image. This is not a problem in practice because highlights already have high SNR and do not need increased sensitivity.

We recommend exposing for RGB, and allowing W to clip if it will. After our processing, the image will have the same upper end of usable dynamic range as Bayer but extended lower end.

Q4: How can RGBW have double the sensitivity of Bayer when only half of its pixels are W, and W has only double the sensitivity of G?

We have observed the following on a gray image patch: the W signal from the image sensor is roughly twice as strong as the G signal which is roughly twice as strong as the R, B signal. Both the Quantum Efficiency of the photodiode and the transmissivity of the color filters account for these sensitivity differences.

In RGBW, 1/2 the pixels are W, 1/4 are G and 1/8 each are R, B. If we assume W to generate 1 unit of signal, G generates 1/2 and R, B each generate 1/4. Total signal for a gray image patch then is 1/2 + 1/4 * 1/2 + (1/8 + 1/8) * 1/4 = 11/16

Bayer, on the other hand, has 1/2 G pixels and 1/4 R, B pixels each and generates 1/2 * 1/2 + (1/4 + 1/4)*1/4 = 6/16 on the same image patch.

This rough analysis indicates that RGBW should be roughly twice as sensitive as Bayer. In practice the relative sensitivities of R, G, B, W vary from sensor to sensor with W often being a little less than twice as sensitive as G but G often being more than twice as sensitive as one or both of R, B.

We have prototype sensors manufactured by 3 vendors and their sensitivity improvement over Bayer range from 6dB to 6.9 dB in low light where the sensor is read noise limited, and 3dB to 3.5dB in bright light where the sensor is photon shot noise limited.

Q5: How does RGBW compare to frame stacking and is it compatible with frame stacking?

RGBW is compatible with frame stacking and other low light technologies such as camera arrays, larger aperture lenses, improved sensors etc, so that the benefits of these technologies is additive.

Stacking two Bayer frames leads to 3dB improvement in sensitivity, regardless of the light level. Since RGBW demonstrates 6+dB improvement in low light, one low light RGBW image is equivalent to stacking 4 low light Bayer images. In practice you will need to take more than 4 Bayer images to account for subject and camera movement.

Q6: Is RGBW robust to crosstalk? Does crosstalk erode RGBW’s SNR advantage over Bayer?

In both RGBW and Bayer systems, crosstalk desaturates colors which requires stronger color correction to compensate. In Bayer systems, strong color correction leads to an increase in luminance noise. In RGBW systems, however, luminance noise is primarily determined by noise in the W color plane with R, G, B playing a minor role. Since color correction does not act on W, it has little influence on the luminance noise.

Crosstalk does stress the choma denoiser, and this effect is greater for RGBW than Bayer since the stronger W signal leaks into R,G, B. However chroma denoising is also easier for RGBW because the clean W color plane better defines the edges and so prevents desaturation and color bleed. The latter effect more than offsets the former so that RGBW has better color performance than Bayer in the presence of crosstalk.

We have experimentally verified RGBW’s SNR lead over Bayer on a range of sensors, some with high crosstalk and others with low crosstalk and the lead remains unchanged.

Q7: Is RGB-IR’s color accuracy good in mixed lighting?

Our color science ensures that the IR color plane is accurately reconstructed and its leakage into RGB completely neutralized - under all lighting conditions. Mixed lighting stresses color reproduction, but no more than it would a traditional Bayer system.

Q8: Can the RGBW demosaicking technology be used to improve Bayer and other RGB demosaicking?

Yes, the basic principles of RGBW that allow us to capture 4 color planes from a single sensor can also be used to capture 3 color planes from RGB sensors, such as Bayer, with greater accuracy than any existing system we know of. When used in this way, the primary benefit of our technology is of alias reduction and improved resolution.