Optica Open
Optica-template.pdf (3.49 MB)

MHPCG:Multi-modal Hyperspectral Point Cloud Generation Based on Single RGB Image

Download (3.49 MB)
posted on 2023-04-17, 10:31 authored by weitao cai, Genping Zhao, zhuowei wang, Yeping Peng, he su, Lianglun Cheng
Multimodal hyperspectral point cloud which has both spectral features and three-dimensional spatial features, has presented great potential in various remote sensing applications. However, direct acquisition of this multimodal data is expensive and difficult as for the rareness of this type of imaging equipment. Moreover, acquisition through heterogeneous fusion of independently collected hyperspectral image and point cloud data brings great challenges. In this study, a multimodal hyperspectral point cloud data generation method is proposed using data-driven learning based deep learning approach with only a single RGB image. The whole network unifies spectral super-resolution reconstruction, monocular 3D reconstruction and data fusion to generate hyperspectral point cloud. The quality of the generated data is testified through experiments on practical plants. Both single modality and multi-modality data quality are evaluated by estimation of the growth status of plants. In this paper, hyperspectral point cloud obtained though low-cost RGB imaging not only avoids the independent acquisition of single modality data using expensive professional equipment, and also gets rid of challenging fusion of the multi-source heterogeneous data. Of more importance, it renders simultaneous acquisition of multidimensional data of high resolution with respect to both spectral and spatial information of the target. The synchronous acquisition of rich spectral and physical geometry information throws light on the comprehensive understanding of the physical and biochemical information of the target object.


Funder Name

Guangzhou Fundamental and Applied Research; Collaborative Education Project of Ministry of Education with Beijing Piesat Information Technology Co.Ltd; Guangdong Provincial Key Laboratory of Cyber-Physical System; the National Natural Science Foundation of China; Science and technology research in key areas in Foshan; Provincial Agricultural Science and Technology Innovation and Extension project of Guangdong Province; the Opening Foundation of Key Laboratory of Environment Change and Resources Use in Beibu Gulf, Ministry of Education, Nanning Normal University

Preprint ID