Breaking reciprocity enables new regimes of light-matter interaction with broad implications for fundamental physics and emerging quantum technologies. Although various approaches have been explored to achieve optical nonreciprocity, realizing it at the single-photon level has remained a major challenge. Here, we demonstrate magnetic-free optical nonreciprocity -- including both isolation and circulation -- in the quantum regime, enabled by efficient and noiseless all-optical frequency conversion on an integrated photonic chip. Our device preserves the quantum coherence and entanglement of the input photons while delivering exceptional performance parameters, meeting the stringent demands of classical and quantum information systems. This realization of quantum optical nonreciprocity in a scalable photonic platform opens a pathway toward directional quantum communication and noise-resilient quantum networks.