UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+

ZHAO Yugang, LIU Wenping, ZHOU Yan, CHEN Riqiang, ZONG Shixiang, LUO Youqing

JOURNAL OF NANJING FORESTRY UNIVERSITY ›› 2024, Vol. 48 ›› Issue (4) : 93-103.

PDF(3918 KB)
南京林业大学学报（自然科学版）
PDF(3918 KB)
JOURNAL OF NANJING FORESTRY UNIVERSITY ›› 2024, Vol. 48 ›› Issue (4) : 93-103. DOI: 10.12302/j.issn.1000-2006.202209055

UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+

Author information +
History +

Abstract

【Objective】This study proposes the feature segmentation method Tree-DeepLab for unmanned aerial vehicle (UAV) forest images, based on an attention mechanism and the DeepLabV3+ semantic segmentation network, to extract the main feature distribution information in forest areas.【Method】First, the forest images were annotated according to feature types from six categories (Platanus orientalis, Ginkgo biloba, Populus sp., grassland, road, and bare ground) to obtain the semantic segmentation datasets. Second, the following improvements were made to the semantic segmentation network: (1) the Xception network, the backbone of the DeepLabV3+ semantic segmentation network, was replaced by ResNeSt101 with a split attention mechanism; (2) the atrous convolutions of different dilation rates in the atrous spatial pyramid pooling were connected using a combination of serial and parallel forms, while the combination of the atrous convolution dilation rates was simultaneously changed; (3) a shallow feature fusion branch was added to the decoder; (4) spatial attention modules were added to the decoder; and (5) efficient channel attention modules were added to the decoder.【Result】Training and testing were performed based on an in-house dataset. The experimental results revealed that the Tree-DeepLab semantic segmentation model had mean pixel accuracy (mPA) and mean intersection over union (mIoU) values of 97.04% and 85.01%, respectively, exceeding those of the original DeepLabV3+ by 4.03 and 14.07 percentage points, respectively, and outperforming U-Net and PSPNet.【Conclusion】The study demonstrates that the Tree-DeepLab semantic segmentation model can effectively segment UAV aerial photography images of forest areas to obtain the distribution information of the main feature types in forest areas.

Key words

unmanned aerial vehicle(UAV) / land-cover image segmentation / forestry images / DeepLabV3+ / attention mechanism / ResNeSt

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations
ZHAO Yugang , LIU Wenping , ZHOU Yan , et al . UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+[J]. JOURNAL OF NANJING FORESTRY UNIVERSITY. 2024, 48(4): 93-103 https://doi.org/10.12302/j.issn.1000-2006.202209055

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis
[1]
王静, 高建中. 林地地块特征对农户林业生产效率的影响[J]. 林业经济问题, 2021, 41(6):577-582.
WANG J, GAO J Z. The effects of the characteristics of forest land parcels on farmers’ forestry production efficiency[J]. News For Econ, 2021, 41(6):577-582. DOI: 10.16832/j.cnki.1005-9709.20210072.
Cited in this article [1]
[2]
DALPONTE M, ØRKA H O, GOBAKKEN T, et al. Tree species classification in boreal forests with hyperspectral data[J]. IEEE Trans Geosci Remote Sens, 2013, 51(5):2632-2645. DOI: 10.1109/TGRS.2012.2216272.
Cited in this article [1]
[3]
BLANCO S R, HERAS D B, ARGÜELLO F. Texture extraction techniques for the classification of vegetation species in hyperspectral imagery: bag of words approach based on superpixels[J]. Remote Sens, 2020, 12(16):2633. DOI: 10.3390/rs12162633.
Cited in this article [1]
[4]
THANH NOI P, KAPPAS M. Comparison of random forest, k-nearest neighbor, and support vector machine classifiers for land cover classification using sentinel-2 imagery[J]. Sensors, 2017, 18(1):18. DOI: 10.3390/s18010018.
Cited in this article [1]
[5]
YUAN Y, HU X Y. Random forest and objected-based classification for forest pest extraction from UAV aerial imagery[J]. Int Arch Photogramm Remote Sens Spatial Inf Sci, 2016, XLI-B1:1093-1098. DOI: 10.5194/isprs-archives-xli-b1-1093-2016.
Cited in this article [1]
[6]
赵庆展, 江萍, 王学文, 等. 基于无人机高光谱遥感影像的防护林树种分类[J]. 农业机械学报, 2021, 52(11):190-199.
ZHAO Q Z, JIANG P, WANG X W, et al. Classification of protection forest tree species based on UAV hyperspectral data[J]. Trans Chin Soc Agric Mach, 2021, 52(11):190-199. DOI: 10.6041/j.issn.1000-1298.2021.11.020.
Cited in this article [1]
[7]
戴鹏钦, 丁丽霞, 刘丽娟, 等. 基于FCN的无人机可见光影像树种分类[J]. 激光与光电子学进展, 2020, 57(10):36-45.
DAI P Q, DING L X, LIU L J, et al. Tree species identification based on FCN using the visible images obtained from an unmanned aerial vehicle[J]. Laser Optoelectron Prog, 2020, 57(10):36-45. DOI: 10.3788/LOP57.101001.
Cited in this article [1]
[8]
张军国, 冯文钊, 胡春鹤, 等. 无人机航拍林业虫害图像分割复合梯度分水岭算法[J]. 农业工程学报, 2017, 33(14):93-99.
ZHANG J G, FENG W Z, HU C H, et al. Image segmentation method for forestry unmanned aerial vehicle pest monitoring based on composite gradient watershed algorithm[J]. Trans Chin Soc Agric Eng, 2017, 33(14):93-99. DOI: 10.11975/j.issn.1002-6819.2017.14.013.
Cited in this article [1]
[9]
张增, 王兵, 伍小洁, 等. 无人机森林火灾监测中火情检测方法研究[J]. 遥感信息, 2015, 30(1):107-110, 124.
ZHANG Z, WANG B, WU X J, et al. An algorithm of forest fire detection based on UAV remote sensing[J]. Remote Sens Inf, 2015, 30(1):107-110, 124. DOI: 10.3969/j.issn.1000-3177.2015.01.018.
Cited in this article [1]
[10]
刘文萍, 仲亭玉, 宋以宁. 基于无人机图像分析的树木胸径预测[J]. 农业工程学报, 2017, 33(21):99-104.
LIU W P, ZHONG T Y, SONG Y N. Prediction of trees diameter at breast height based on unmanned aerial vehicle image analysis[J]. Trans Chin Soc Agric Eng, 2017, 33(21):99-104. DOI: 10.11975/j.issn.1002-6819.2017.21.012.
Cited in this article [1]
[11]
MARTINS J, JUNIOR J M, MENEZES G, et al. Image segmentation and classification with SLIC superpixel and convolutional neural network in forest context[C]// IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: IEEE, 2019:6543-6546. DOI: 10.1109/IGARSS.2019.8898969.
Cited in this article [1]
[12]
LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015:3431-3440. DOI: 10.1109/CVPR.2015.7298965.
Cited in this article [1]
[13]
RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015:234-241. DOI: 10.1007/978-3-319-24574-4_28.
Cited in this article [1]
[14]
ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:6230-6239. DOI: 10.1109/CVPR.2017.660.
Cited in this article [1]
[15]
LIN G S, MILAN A, SHEN C H, et al. RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:5168-5177. DOI: 10.1109/CVPR.2017.549.
Cited in this article [1]
[16]
CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[DB/OL]. (2014-09-14)[2022-05-25]. https://arXiv.org/abs/1412.7062. DOI: 10.48550/arXiv.1412.7062.
Cited in this article [1]
[17]
CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans Pattern Anal Mach Intell, 2018, 40(4):834-848. DOI: 10.1109/TPAMI.2017.2699184.
Cited in this article [1]
[18]
CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[DB/OL]. (2017-07-19)[2022-05-05]. https://arXiv.org/abs/1706.05587. DOI: 10.48550/arXiv.1706.05587.
Cited in this article [1]
[19]
CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// European Conference on Computer Vision. Cham: Springer, 2018:833-851. DOI: 10.1007/978-3-030-01234-2_49.
Cited in this article [1]
[20]
韩蕊, 慕涛阳, 赵伟, 等. 基于无人机多光谱影像的柑橘树冠分割方法研究[J]. 林业工程学报, 2021, 6(5):147-153.
HAN R, MU T Y, ZHAO W, et al. Research on citrus canopy segmentation method based on UAV multispectral image[J]. Journal of Forestry Engineering, 2021, 6(5):147-153. DOI: 10.13360/j.issn.2096-1359.202011021.
Cited in this article [1]
[21]
刘文定, 田洪宝, 谢将剑, 等. 基于全卷积神经网络的林区航拍图像虫害区域识别方法[J]. 农业机械学报, 2019, 50(3):179-185.
LIU W D, TIAN H B, XIE J J, et al. Identification methods for forest pest areas of UAV aerial photography based on fully convolutional networks[J]. Trans Chin Soc Agric Mach, 2019, 50(3):179-185. DOI: 10.6041/j.issn.1000-1298.2019.03.019.
Cited in this article [1]
[22]
徐辉, 祝玉华, 甄彤, 等. 深度神经网络图像语义分割方法综述[J]. 计算机科学与探索, 2021, 15(1):47-59.
XU H, ZHU Y H, ZHEN T, et al. Survey of image semantic segmentation methods based on deep neural network[J]. J Front Comput Sci Technol, 2021, 15(1):47-59. DOI: 10.3778/j.issn.1673-9418.2004039.
Cited in this article [1]
[23]
HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018:7132-7141. DOI: 10.1109/CVPR.2018.00745.
Cited in this article [1]
[24]
WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018:3-19. DOI: 10.1007/978-3-030-01234-2_1.
Cited in this article [1]
[25]
WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2020:11531-11539. DOI: 10.1109/CVPR42600.2020.01155.
Cited in this article [1]
[26]
ZHANG H, WU C R, ZHANG Z Y, et al. ResNeSt: split-attention networks[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New Orleans, LA, USA: IEEE, 2022:2735-2745. DOI: 10.1109/CVPRW56347.2022.00309.
Cited in this article [1]
[27]
CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:1800-1807. DOI: 10.1109/CVPR.2017.195.
Cited in this article [1]
[28]
HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016:770-778. DOI: 10.1109/CVPR.2016.90.
Cited in this article [1]
[29]
YANG M K, YU K, ZHANG C, et al. DenseASPP for semantic segmentation in street scenes[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018:3684-3692. DOI: 10.1109/CVPR.2018.00388.
Cited in this article [2]
[30]
李丹, 张俊杰, 赵梦溪. 基于FCM和分水岭算法的无人机影像中林分因子提取[J]. 林业科学, 2019, 55(5):180-187.
LI D, ZHANG J J, ZHAO M X. Extraction of stand factors in UAV image based on FCM and watershed algorithm[J]. Sci Silvae Sin, 2019, 55(5):180-187. DOI: 10.11707/j.1001-7488.20190520.
Cited in this article [1]
[31]
刘旭光, 肖啸, 兰玉彬, 等. 应用可见光遥感影像的林区植被分割方法[J]. 东北林业大学学报, 2023, 51(4):62-67.
LIU X G, XIAO X, LAN Y B, et al. Forest vegetation segmentation method with UAV visible light remote sensing images[J]. Journal of Northeast Foresrty University, 2023, 51(4):62-67. DOI: 10.13759/j.cnki.dlxb.2023.04.008.
Cited in this article [1]
PDF(3918 KB)

Accesses

Citation

Detail
Sections
Recommended
The full text is translated into English by AI, aiming to facilitate reading and comprehension. The core content is subject to the explanation in Chinese.

Author

Reader

Reviewer

www.nldxb.njfu.edu.cn

Edited ＆ Published by Editorial Department of Nanjing Forestry University(Natural Sciences Edition)
Address： No.159 Longpan Road,Nanjing 210037 Jiangsu,P.R.China
E-mail ：xuebao@njfu.edu.cn , xuebao@njfu.com.cn
Telephone +86-25-85428247,85427076
Distributed by China International Book Trading Corporation P.O. Box 399, Beijing ,P.R. China

/

〈 〉