Optica Open
arXiv.svg (5.58 kB)

Hand Gestures Recognition in Videos Taken with Lensless Camera

Download (5.58 kB)
posted on 2023-01-10, 03:02 authored by Yinger Zhang, Zhouyi Wu, Peiying Lin, Yang Pan, Yuting Wu, Liufang Zhang, Jiangtao Huangfu
A lensless camera is an imaging system that uses a mask in place of a lens, making it thinner, lighter, and less expensive than a lensed camera. However, additional complex computation and time are required for image reconstruction. This work proposes a deep learning model named Raw3dNet that recognizes hand gestures directly on raw videos captured by a lensless camera without the need for image restoration. In addition to conserving computational resources, the reconstruction-free method provides privacy protection. Raw3dNet is a novel end-to-end deep neural network model for the recognition of hand gestures in lensless imaging systems. It is created specifically for raw video captured by a lensless camera and has the ability to properly extract and combine temporal and spatial features. The network is composed of two stages: 1. spatial feature extractor (SFE), which enhances the spatial features of each frame prior to temporal convolution; 2. 3D-ResNet, which implements spatial and temporal convolution of video streams. The proposed model achieves 98.59% accuracy on the Cambridge Hand Gesture dataset in the lensless optical experiment, which is comparable to the lensed-camera result. Additionally, the feasibility of physical object recognition is assessed. Furtherly, we show that the recognition can be achieved with respectable accuracy using only a tiny portion of the original raw data, indicating the potential for reducing data traffic in cloud computing scenarios.



This arXiv metadata record was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.

Usage metrics



    Ref. manager