LMS COMPARED WITH OTHER CFAs
Cyan-Magenta-Yellow
CMY and LMS share a similarity in their high transmissivity of Cyan, Magenta, and Yellow color filters, which increases luma SNR, but their conversion to RGB introduces chroma noise. When coupled with chroma denoising, CMY yields a noise profile similar to LMS.
However, CMY suffers from poor color accuracy, which is even worse than conventional RGB sensors due to difficulties in manufacturing its color filters. While the luma sensitivity of CMY is roughly similar to LMS, the quantum efficiency of M falls between that of Magenta and Cyan, and S has lower quantum efficiency than all CMY colors. But, only 1 in 8 pixels in the LMS CFA is S, raising the possibility of a small luma SNR advantage of CMY over LMS. To quantify this potential advantage, one needs to carefully account for the quantum efficiencies of the sensor silicon in question.
Nevertheless, this small luma SNR lead of CMY over LMS is almost certainly not worth its poor color accuracy.
RYYB
The RYYB sensor, proposed by erstwhile Aptina and implemented by Huawei, features a Bayer pattern with G replaced by Yellow. As L is similar to Y and accounts for half of the pixels in both LMS and RYYB, the difference between LMS and RYYB lies in the coloring of the other half of the pixels.
RYYB obtains G by differencing Y-R, while LMS obtains R by differencing L-M, which is similar to Y-G. However, RYYB suffers from poor R and G accuracy due to difficulties in manufacturing Y such that Y-R is high-quality green, and accurate R is impossible to sense directly. On the other hand, LMS captures high-quality G by directly sensing G (M), and its calculated R=L-M is more accurate than any directly sensed R can be, making LMS’ colors more accurate than conventional RGB imagers.
In the LMS CFA, the noise of R=L-M is controlled by the high density of both L and M, while in the RYYB CFA, the noise of G=Y-R is not as well controlled due to the low density of R. Additionally, any loss of G quality – either as increased noise or increased chroma denoising artifacts – is very visible since G is the color our eyes see best.
LMS also has a luma SNR advantage over RYYB since it has a lot of M pixels that are more sensitive than either R or B. The only advantage of RYYB over LMS is higher blue SNR since it has twice as many B pixels as LMS has the equivalent S pixels. However, this is of minor importance since our retinas have very few S and have difficulty sensing the loss of blue quality. The improved SNR of G in LMS alone is more valuable than the improved SNR of B in RYYB, with the improved color accuracy and the higher luma SNR being additional benefits of LMS.
RGBW
RGBW CFA patterns, with 50% white or clear pixels, allow more light to pass through than LMS. However, the luma SNR of LMS is surprisingly close to RGBW, owing to the better numerical stability of LMS demosaicking.
At this stage, it’s premature to conclude which of the two, RGBW or LMS, has a better SNR. However, it’s safe to say that LMS offers better color accuracy but requires the new L color filter to be manufactured, whereas RGBW is available now.
Foveon and other Multi-Layer Image Sensors
The Foveon X3 image sensor has 3 stacked photosites at each pixel, with the top layer sensing blue, the middle layer green, and the bottom layer red. Other image sensors with similar designs have also been proposed, featuring different photo-sensitive layers for red, green, and blue.
However, Foveon and other multi-layer image sensors cannot be used for LMS because of the significant spectral overlap between L and M. Once a layer senses M, it removes most of L’s signal since there is a large overlap between the two. Similarly, placing the L sensing layer over M presents the reverse problem.
Random Color Filter Array
The L, M, S cones in the human retina are hexagonal and arranged in a random pattern, while the LMS image sensor pixels are square and arranged in a regular pattern. Both are optimal for their pixel shape.
Square pixels admit higher resolution along the diagonal directions than the horizontal and vertical directions. Since lenses have the same resolution along all directions, this extra diagonal resolution goes unused. Color filter arrays with regular patterns take advantage of this unused diagonal channel capacity to convey color information.
Hexagonal pixels, on the other hand, have almost the same resolution in every direction and are, thus, less wasteful. You need 20% fewer hexagonal pixels than square pixels for the same resolution monochrome image. The simple mechanism of encoding color using regular CFA patterns does not work with hexagonal pixels; instead more sophisticated compressive sensing techniques with random CFA patterns must be used. This is what the HVS likely does.
While hexagonal pixels are more efficient, yielding higher frame rates, or consuming less power, or both, the imaging industry is standardized on square pixels for now. Random color patterns do not outperform regular color patterns on square pixel sensors.