Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

Zhao, Jiangsan and Kumar, Ajay and Rajalakshmi, P and et al, . (2022) Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping. Remote Sensing, 14 (5). pp. 1-19. ISSN 2072-4292

Text
Remote_Sensing.pdf - Published Version
Available under License Creative Commons Attribution.
Download (5MB)

Abstract

Multispectral images (MSIs) are valuable for precision agriculture due to the extra spectral information acquired compared to natural color RGB (ncRGB) images. In this paper, we thus aim to generate high spatial MSIs through a robust, deep-learning-based reconstruction method using ncRGB images. Using the data from the agronomic research trial for maize and breeding research trial for rice, we first reproduced ncRGB images from MSIs through a rendering model, Model-True to natural color image (Model-TN), which was built using a benchmark hyperspectral image dataset. Subsequently, an MSI reconstruction model, Model-Natural color to Multispectral image (Model-NM), was trained based on prepared ncRGB (ncRGB-Con) images and MSI pairs, ensuring the model can use widely available ncRGB images as input. The integrated loss function of mean relative absolute error (MRAEloss) and spectral information divergence (SIDloss) were most effective during the building of both models, while models using the MRAEloss function were more robust towards variability between growing seasons and species. The reliability of the reconstructed MSIs was demonstrated by high coefficients of determination compared to ground truth values, using the Normalized Difference Vegetation Index (NDVI) as an example. The advantages of using “reconstructed” NDVI over Triangular Greenness Index (TGI), as calculated directly from RGB images, were illustrated by their higher capabilities in differentiating three levels of irrigation treatments on maize plants. This study emphasizes that the performance of MSI reconstruction models could benefit from an optimized loss function and the intermediate step of ncRGB image preparation. The ability of the developed models to reconstruct high-quality MSIs from low-cost ncRGB images will, in particular, promote the application for plant phenotyping in precision agriculture. © 2022 by the authors. Licensee MDPI, Basel, Switzerland.

[error in script]

IITH Creators:

IITH Creators	ORCiD
Rajalakshmi, P	https://orcid.org/0000-0002-7252-6728

Item Type:

Article

Additional Information:

Funding: This study is partially funded by the Japan Science and Technology Agency (JST) and India Department of Science and Technology (DST), SICORP Program JPMJSC16H2, and JST AIP Acceleration Research “Studies of CPS platform to raise big-data-driven AI agriculture”. B.R. was funded by the University of Natural Resources and Life Sciences Vienna.

Uncontrolled Keywords:

Deep learning; Function Optimization; Images reconstruction; Loss function optimization; Loss functions; Multispectral image reconstruction; Multispectral images; Natural color RGB image; Natural colour; Precision Agriculture; RGB images

Subjects:

Others > Agricultural engineering
Electrical Engineering

Divisions:

Department of Electrical Engineering

Depositing User:

. LibTrainee 2021

Date Deposited:

22 Jul 2022 06:10

Last Modified:

22 Jul 2022 06:10

URI:

http://raiithold.iith.ac.in/id/eprint/9587