All-Optical Doubly Resonant Cavities for ReLU Function in Nanophotonic Deep Learning

We present a novel approach to implementing all-optical Rectified Linear Unit (ReLU) activation functions using compact doubly-resonant cavities with dimensions of approximately $10\,\mu\mathrm{m}$. Our design leverages $\chi^{(2)}$ nonlinear processes within carefully engineered photonic structures that simultaneously resonate at both fundamental and second-harmonic frequencies. By exploiting the phase-sensitive nature of second-harmonic generation, we demonstrate an optical analog to the ReLU function, achieving femtojoule-level activation energy-comparable to state-of-the-art approaches-while reducing device footprint by two orders of magnitude compared to previous implementations. We develop the theoretical framework using coupled-mode theory and validate it through rigorous finite-difference time-domain simulations. Beyond ReLU, we show that the same physical structure can implement alternative activation functions such as ELU and GELU through simple adjustments to input conditions. Neural network simulations demonstrate that our proposed optical activation functions achieve classification accuracy within $0.4\%$ of ideal electronic implementations while offering significant advantages in energy efficiency and processing speed. This work represents a significant advancement toward realizing energy-efficient, high-density optical neural networks for next-generation artificial intelligence hardware.