Color-Blind Image Sensors: Towards Digital Twin of Human Retina

The human retina contains a complex arrangement of photoreceptors that convert light into visual information. Conventional image sensors mimic the trichromacy of the retina using periodic filter mosaics responsive to three primary colors. However, this is, at best, an approximation, as an actual retina exhibits a quasi-random spatial distribution of light-sensitive rod and cone photoreceptors, where the ratio of rods to cones and their concentrations vary across the retina. Hence, the periodic mosaics are limited to accurately simulate the properties of the eye. Here, we present an image sensor with similar distribution, spacing, ratios and spectral characteristics of an actual foveal mosaic for emulating eye-like sampling and mimicking color blindness. To perform image reconstruction, we use a fully convolutional U-Net neural network adopting the concept of receptive fields in the retinal circuitry. Our research will enable the development of digital twin of a retina to further understand color vision deficiencies.