基于注意力机制和改进DeepLabV3+的无人机林区图像地物分割方法

赵玉刚; 刘文萍; 周焱; 陈日强; 宗世祥; 骆有庆

doi:10.12302/j.issn.1000-2006.202209055

赵玉刚, 刘文萍, 周焱, 陈日强, 宗世祥, 骆有庆

南京林业大学学报（自然科学版） ›› 2024, Vol. 48 ›› Issue (4) : 93-103.

PDF(3918 KB)

国家林草科技领军期刊
中国精品科技期刊
中国高校百佳科技期刊
江苏省新闻出版政府奖期刊奖
RCCSE林学权威期刊（A+）
CSCD核心期刊
Scopus数据库收录期刊
中文核心期刊
SCD核心期刊

作者加群：102861116

微信公众号：南京林业大学学报

高级检索

PDF(3918 KB)

南京林业大学学报（自然科学版） ›› 2024, Vol. 48 ›› Issue (4) : 93-103. DOI: 10.12302/j.issn.1000-2006.202209055

专题报道Ⅲ：智慧林业之森林可视化研究（执行主编李凤日、张怀清、曹林）

基于注意力机制和改进DeepLabV3+的无人机林区图像地物分割方法

赵玉刚 ¹ ,
刘文萍 ¹^,^* ,
周焱 ¹ ,
陈日强 ¹ ,
宗世祥 ² ,
骆有庆 ²

作者信息 +

UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+

Author information +

文章历史 +

摘要

【目的】为提取林区主要地物分布信息,基于注意力机制和DeepLabV3+语义分割网络提出一种面向无人机林区图像的地物分割方法Tree-DeepLab。【方法】根据不同的林区地物类型对图像进行标注,标注类型分为法国梧桐(Platanus orientalis)、银杏(Ginkgo biloba)、杨树(Populus sp.)、草地、道路和裸地6类,以获取语义分割数据集。对语义分割网络进行改进:①将带有分组注意力机制的ResNeSt101网络作为DeepLabV3+语义分割网络的主干网络;②将空洞空间卷积池化金字塔模块的连接方式设置成串并行相结合形式,同时改变空洞卷积的扩张率组合;③解码器增加浅层特征融合分支;④解码器增加空间注意力模块;⑤解码器增加高效通道注意力模块。【结果】在自制数据集基础上进行训练和测试,试验结果表明:Tree-DeepLab语义分割模型的平均像素精度和平均交并比分别为97.04%和85.01%,较原始DeepLabV3+分别提升4.03和14.07个百分点,且优于U-Net和PSPNet语义分割网络。【结论】Tree-DeepLab语义分割网络能够有效分割无人机航拍林区图像,以获取林区主要地物类型的分布信息。

Abstract

【Objective】This study proposes the feature segmentation method Tree-DeepLab for unmanned aerial vehicle (UAV) forest images, based on an attention mechanism and the DeepLabV3+ semantic segmentation network, to extract the main feature distribution information in forest areas.【Method】First, the forest images were annotated according to feature types from six categories (Platanus orientalis, Ginkgo biloba, Populus sp., grassland, road, and bare ground) to obtain the semantic segmentation datasets. Second, the following improvements were made to the semantic segmentation network: (1) the Xception network, the backbone of the DeepLabV3+ semantic segmentation network, was replaced by ResNeSt101 with a split attention mechanism; (2) the atrous convolutions of different dilation rates in the atrous spatial pyramid pooling were connected using a combination of serial and parallel forms, while the combination of the atrous convolution dilation rates was simultaneously changed; (3) a shallow feature fusion branch was added to the decoder; (4) spatial attention modules were added to the decoder; and (5) efficient channel attention modules were added to the decoder.【Result】Training and testing were performed based on an in-house dataset. The experimental results revealed that the Tree-DeepLab semantic segmentation model had mean pixel accuracy (mPA) and mean intersection over union (mIoU) values of 97.04% and 85.01%, respectively, exceeding those of the original DeepLabV3+ by 4.03 and 14.07 percentage points, respectively, and outperforming U-Net and PSPNet.【Conclusion】The study demonstrates that the Tree-DeepLab semantic segmentation model can effectively segment UAV aerial photography images of forest areas to obtain the distribution information of the main feature types in forest areas.

导出引用

赵玉刚, 刘文萍, 周焱, 等. 基于注意力机制和改进DeepLabV3+的无人机林区图像地物分割方法[J]. 南京林业大学学报（自然科学版）. 2024, 48(4): 93-103 https://doi.org/10.12302/j.issn.1000-2006.202209055

ZHAO Yugang, LIU Wenping, ZHOU Yan, et al. UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+[J]. JOURNAL OF NANJING FORESTRY UNIVERSITY. 2024, 48(4): 93-103 https://doi.org/10.12302/j.issn.1000-2006.202209055

中图分类号： S758;TP391

参考文献

列表( 原文顺序 | 文献年度倒序 | 文中引用次数倒序 ) 可视化分析

[1]	王静, 高建中. 林地地块特征对农户林业生产效率的影响[J]. 林业经济问题, 2021, 41(6):577-582. WANG J, GAO J Z. The effects of the characteristics of forest land parcels on farmers’ forestry production efficiency[J]. News For Econ, 2021, 41(6):577-582. DOI: 10.16832/j.cnki.1005-9709.20210072. 本文引用 [1]

[2]	DALPONTE M, ØRKA H O, GOBAKKEN T, et al. Tree species classification in boreal forests with hyperspectral data[J]. IEEE Trans Geosci Remote Sens, 2013, 51(5):2632-2645. DOI: 10.1109/TGRS.2012.2216272. 本文引用 [1]

[3]	BLANCO S R, HERAS D B, ARGÜELLO F. Texture extraction techniques for the classification of vegetation species in hyperspectral imagery: bag of words approach based on superpixels[J]. Remote Sens, 2020, 12(16):2633. DOI: 10.3390/rs12162633. 本文引用 [1]

[4]	THANH NOI P, KAPPAS M. Comparison of random forest, k-nearest neighbor, and support vector machine classifiers for land cover classification using sentinel-2 imagery[J]. Sensors, 2017, 18(1):18. DOI: 10.3390/s18010018. 本文引用 [1]

[5]	YUAN Y, HU X Y. Random forest and objected-based classification for forest pest extraction from UAV aerial imagery[J]. Int Arch Photogramm Remote Sens Spatial Inf Sci, 2016, XLI-B1:1093-1098. DOI: 10.5194/isprs-archives-xli-b1-1093-2016. 本文引用 [1]

[6]	赵庆展, 江萍, 王学文, 等. 基于无人机高光谱遥感影像的防护林树种分类[J]. 农业机械学报, 2021, 52(11):190-199. ZHAO Q Z, JIANG P, WANG X W, et al. Classification of protection forest tree species based on UAV hyperspectral data[J]. Trans Chin Soc Agric Mach, 2021, 52(11):190-199. DOI: 10.6041/j.issn.1000-1298.2021.11.020. 本文引用 [1]

[7]

戴鹏钦, 丁丽霞, 刘丽娟, 等. 基于FCN的无人机可见光影像树种分类[J]. 激光与光电子学进展, 2020, 57(10):36-45.

DAI

P Q

, DING

L X

, LIU

L J

, et al. Tree species identification based on FCN using the visible images obtained from an unmanned aerial vehicle[J]. Laser Optoelectron Prog, 2020, 57(10):36-45. DOI: 10.3788/LOP57.101001.

本文引用 [1]

[8]

张军国, 冯文钊, 胡春鹤, 等. 无人机航拍林业虫害图像分割复合梯度分水岭算法[J]. 农业工程学报, 2017, 33(14):93-99.

ZHANG

J G

, FENG

W Z

, HU

C H

, et al. Image segmentation method for forestry unmanned aerial vehicle pest monitoring based on composite gradient watershed algorithm[J]. Trans Chin Soc Agric Eng, 2017, 33(14):93-99. DOI: 10.11975/j.issn.1002-6819.2017.14.013.

本文引用 [1]

[9]	张增, 王兵, 伍小洁, 等. 无人机森林火灾监测中火情检测方法研究[J]. 遥感信息, 2015, 30(1):107-110, 124. ZHANG Z, WANG B, WU X J, et al. An algorithm of forest fire detection based on UAV remote sensing[J]. Remote Sens Inf, 2015, 30(1):107-110, 124. DOI: 10.3969/j.issn.1000-3177.2015.01.018. 本文引用 [1]

[10]	刘文萍, 仲亭玉, 宋以宁. 基于无人机图像分析的树木胸径预测[J]. 农业工程学报, 2017, 33(21):99-104. LIU W P, ZHONG T Y, SONG Y N. Prediction of trees diameter at breast height based on unmanned aerial vehicle image analysis[J]. Trans Chin Soc Agric Eng, 2017, 33(21):99-104. DOI: 10.11975/j.issn.1002-6819.2017.21.012. 本文引用 [1]

[11]

MARTINS

, JUNIOR

J M

, MENEZES

, et al. Image segmentation and classification with SLIC superpixel and convolutional neural network in forest context[C]// IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: IEEE, 2019:6543-6546. DOI: 10.1109/IGARSS.2019.8898969.

本文引用 [1]

[12]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015:3431-3440. DOI: 10.1109/CVPR.2015.7298965. 本文引用 [1]

[13]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015:234-241. DOI: 10.1007/978-3-319-24574-4_28. 本文引用 [1]

[14]	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:6230-6239. DOI: 10.1109/CVPR.2017.660. 本文引用 [1]

[15]	LIN G S, MILAN A, SHEN C H, et al. RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:5168-5177. DOI: 10.1109/CVPR.2017.549. 本文引用 [1]

[16]	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[DB/OL]. (2014-09-14)[2022-05-25]. https://arXiv.org/abs/1412.7062. DOI: 10.48550/arXiv.1412.7062. 本文引用 [1]

[17]	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans Pattern Anal Mach Intell, 2018, 40(4):834-848. DOI: 10.1109/TPAMI.2017.2699184. 本文引用 [1]

[18]	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[DB/OL]. (2017-07-19)[2022-05-05]. https://arXiv.org/abs/1706.05587. DOI: 10.48550/arXiv.1706.05587. 本文引用 [1]

[19]	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// European Conference on Computer Vision. Cham: Springer, 2018:833-851. DOI: 10.1007/978-3-030-01234-2_49. 本文引用 [1]

[20]	韩蕊, 慕涛阳, 赵伟, 等. 基于无人机多光谱影像的柑橘树冠分割方法研究[J]. 林业工程学报, 2021, 6(5):147-153. HAN R, MU T Y, ZHAO W, et al. Research on citrus canopy segmentation method based on UAV multispectral image[J]. Journal of Forestry Engineering, 2021, 6(5):147-153. DOI: 10.13360/j.issn.2096-1359.202011021. 本文引用 [1]

[21]

刘文定, 田洪宝, 谢将剑, 等. 基于全卷积神经网络的林区航拍图像虫害区域识别方法[J]. 农业机械学报, 2019, 50(3):179-185.

LIU

W D

, TIAN

H B

, XIE

J J

, et al. Identification methods for forest pest areas of UAV aerial photography based on fully convolutional networks[J]. Trans Chin Soc Agric Mach, 2019, 50(3):179-185. DOI: 10.6041/j.issn.1000-1298.2019.03.019.

本文引用 [1]

[22]	徐辉, 祝玉华, 甄彤, 等. 深度神经网络图像语义分割方法综述[J]. 计算机科学与探索, 2021, 15(1):47-59. XU H, ZHU Y H, ZHEN T, et al. Survey of image semantic segmentation methods based on deep neural network[J]. J Front Comput Sci Technol, 2021, 15(1):47-59. DOI: 10.3778/j.issn.1673-9418.2004039. 本文引用 [1]

[23]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018:7132-7141. DOI: 10.1109/CVPR.2018.00745. 本文引用 [1]

[24]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018:3-19. DOI: 10.1007/978-3-030-01234-2_1. 本文引用 [1]

[25]	WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2020:11531-11539. DOI: 10.1109/CVPR42600.2020.01155. 本文引用 [1]

[26]	ZHANG H, WU C R, ZHANG Z Y, et al. ResNeSt: split-attention networks[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New Orleans, LA, USA: IEEE, 2022:2735-2745. DOI: 10.1109/CVPRW56347.2022.00309. 本文引用 [1]

[27]	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:1800-1807. DOI: 10.1109/CVPR.2017.195. 本文引用 [1]

[28]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016:770-778. DOI: 10.1109/CVPR.2016.90. 本文引用 [1]

[29]	YANG M K, YU K, ZHANG C, et al. DenseASPP for semantic segmentation in street scenes[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018:3684-3692. DOI: 10.1109/CVPR.2018.00388. 本文引用 [2]

[30]	李丹, 张俊杰, 赵梦溪. 基于FCM和分水岭算法的无人机影像中林分因子提取[J]. 林业科学, 2019, 55(5):180-187. LI D, ZHANG J J, ZHAO M X. Extraction of stand factors in UAV image based on FCM and watershed algorithm[J]. Sci Silvae Sin, 2019, 55(5):180-187. DOI: 10.11707/j.1001-7488.20190520. 本文引用 [1]

[31]	刘旭光, 肖啸, 兰玉彬, 等. 应用可见光遥感影像的林区植被分割方法[J]. 东北林业大学学报, 2023, 51(4):62-67. LIU X G, XIAO X, LAN Y B, et al. Forest vegetation segmentation method with UAV visible light remote sensing images[J]. Journal of Northeast Foresrty University, 2023, 51(4):62-67. DOI: 10.13759/j.cnki.dlxb.2023.04.008. 本文引用 [1]