FPM image enhancement with bi-modal deep learning

Digital pathology based on whole slide imaging system is about to permit major breakthrough in automated diagnosis for a rapid and highly sensitive disease detection. High-resolution FPM (Fourier Ptychographic Microscopy) slide scanners delivering rich information on the sample such as intensity and its complementary phase image are becoming available. They allow new effective bi-modal data exploitation strategies that can be useful for more efficient automated diagnosis. However, when the thickness of the sample becomes comparable or greater than the microscope depth of field, we report observation of undesirable contrast change of sub-cellular compartments in phase images around optimal focus plan, reducing their usability. In this article, an U-Net artificial neural network is trained to reinforce specifically targeted sub-cellular compartments contrast (for both intensity and phase images). The procedure used to construct a reference database with virtual z-stacking calculations is detailed. It is obtained exploiting FPM reconstruction algorithm to explore images around optimal focus plane (virtual Z-stacking) and select those with adequate contrast and focus. Furthermore intensity and phase images are advantageously exploited in a bi-modal deep-neural network architecture. By construction and once trained, the U-Net is able to simultaneously reinforce targeted cell compartments visibility and to compensate any focus imprecision. It is efficient over a large field of view at high resolution. The interest of the approach is illustrated considering the use-case of Plasmodium falciparum detection in blood smear where improvement in the detection sensitivity is demonstrated without degradation of the specificity. Post-reconstruction FPM image processing with such U-Net and its training procedure is general and applicable to demanding biological screening applications.