Rigorous theory of thin-vapor-layer linear optical properties: The case of quenching of atomic polarization upon collisions of atoms with dielectric walls

Hot alkali metal vapors enclosed in sub-micron spectroscopic cells provide an ideal system for fundamental studies of the atom-wall and atom-light interactions at nanoscale. Here, we propose a novel approach for calculating the eigenmodes of a thin-vapor-layer beyond the limitations of the first-order perturbation theory in optical density for the case of quenching of atomic polarization upon collisions of atoms with dielectric walls. We show that higher-order optical density corrections lead to a remarkable density-dependent blue shift and deformation of the spectral line shapes of reflection, transmission, and absorption. We also demonstrate that the eigenmodes of the thin-vapor-layer can be calculated independently of the choice of optical boundary conditions. This greatly extends the applicability of the constructed theory for the development of miniature atomic sensors.