基于注意力机制和改进DeepLabV3+的无人机林区图像地物分割方法

doi:10.12302/j.issn.1000-2006.202209055

国家林草科技领军期刊
中国精品科技期刊
中国高校百佳科技期刊
江苏省新闻出版政府奖期刊奖
RCCSE林学权威期刊（A+）
CSCD核心期刊
Scopus数据库收录期刊
中文核心期刊
SCD核心期刊

作者加群：102861116

微信公众号：南京林业大学学报

高级检索

南京林业大学学报（自然科学版） ›› 2024, Vol. 48 ›› Issue (4): 93-103.doi: 10.12302/j.issn.1000-2006.202209055

所属专题：专题报道Ⅲ：智慧林业之森林可视化研究

• 专题报道Ⅲ：智慧林业之森林可视化研究（执行主编李凤日、张怀清、曹林） • 上一篇下一篇

基于注意力机制和改进DeepLabV3+的无人机林区图像地物分割方法

赵玉刚¹(), 刘文萍¹^,^*(), 周焱¹, 陈日强¹, 宗世祥², 骆有庆²

1.北京林业大学信息学院,国家林业和草原局林业智能信息处理工程技术研究中心, 北京 100083
2.北京林业大学林学院,北京 100083

收稿日期:2022-09-24 修回日期:2022-11-01 出版日期:2024-07-30 发布日期:2024-08-05
通讯作者: *刘文萍(wendyl@vip.163.com),教授。
作者简介:
赵玉刚(15621377528@163.com)。
基金资助:
国家林业和草原局重大应急科技项目(ZD202001);国家重点研发计划(2021YFD1400900)

UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+

ZHAO Yugang¹(), LIU Wenping¹^,^*(), ZHOU Yan¹, CHEN Riqiang¹, ZONG Shixiang², LUO Youqing²

1. School of Information, Beijing Forestry University, Engineering Research Center for Forestry-oriented Intelligent Information Processing of National Forestry and Grassland Administration, Beijing 100083, China
2. School of Forestry, Beijing Forestry University, Beijing 100083, China

Received:2022-09-24 Revised:2022-11-01 Online:2024-07-30 Published:2024-08-05

摘要/Abstract

摘要：

【目的】为提取林区主要地物分布信息,基于注意力机制和DeepLabV3+语义分割网络提出一种面向无人机林区图像的地物分割方法Tree-DeepLab。【方法】根据不同的林区地物类型对图像进行标注,标注类型分为法国梧桐(Platanus orientalis)、银杏(Ginkgo biloba)、杨树(Populus sp.)、草地、道路和裸地6类,以获取语义分割数据集。对语义分割网络进行改进:①将带有分组注意力机制的ResNeSt101网络作为DeepLabV3+语义分割网络的主干网络;②将空洞空间卷积池化金字塔模块的连接方式设置成串并行相结合形式,同时改变空洞卷积的扩张率组合;③解码器增加浅层特征融合分支;④解码器增加空间注意力模块;⑤解码器增加高效通道注意力模块。【结果】在自制数据集基础上进行训练和测试,试验结果表明:Tree-DeepLab语义分割模型的平均像素精度和平均交并比分别为97.04%和85.01%,较原始DeepLabV3+分别提升4.03和14.07个百分点,且优于U-Net和PSPNet语义分割网络。【结论】Tree-DeepLab语义分割网络能够有效分割无人机航拍林区图像,以获取林区主要地物类型的分布信息。

关键词: 无人机, 地物分割, 林区图像, DeepLabV3+, 注意力机制, ResNeSt

Abstract:

【Objective】This study proposes the feature segmentation method Tree-DeepLab for unmanned aerial vehicle (UAV) forest images, based on an attention mechanism and the DeepLabV3+ semantic segmentation network, to extract the main feature distribution information in forest areas.【Method】First, the forest images were annotated according to feature types from six categories (Platanus orientalis, Ginkgo biloba, Populus sp., grassland, road, and bare ground) to obtain the semantic segmentation datasets. Second, the following improvements were made to the semantic segmentation network: (1) the Xception network, the backbone of the DeepLabV3+ semantic segmentation network, was replaced by ResNeSt101 with a split attention mechanism; (2) the atrous convolutions of different dilation rates in the atrous spatial pyramid pooling were connected using a combination of serial and parallel forms, while the combination of the atrous convolution dilation rates was simultaneously changed; (3) a shallow feature fusion branch was added to the decoder; (4) spatial attention modules were added to the decoder; and (5) efficient channel attention modules were added to the decoder.【Result】Training and testing were performed based on an in-house dataset. The experimental results revealed that the Tree-DeepLab semantic segmentation model had mean pixel accuracy (mPA) and mean intersection over union (mIoU) values of 97.04% and 85.01%, respectively, exceeding those of the original DeepLabV3+ by 4.03 and 14.07 percentage points, respectively, and outperforming U-Net and PSPNet.【Conclusion】The study demonstrates that the Tree-DeepLab semantic segmentation model can effectively segment UAV aerial photography images of forest areas to obtain the distribution information of the main feature types in forest areas.

Key words: unmanned aerial vehicle(UAV), land-cover image segmentation, forestry images, DeepLabV3+, attention mechanism, ResNeSt

中图分类号:

S758
TP391

赵玉刚,刘文萍,周焱,等. 基于注意力机制和改进DeepLabV3+的无人机林区图像地物分割方法[J]. 南京林业大学学报（自然科学版）, 2024, 48(4): 93-103.

ZHAO Yugang, LIU Wenping, ZHOU Yan, CHEN Riqiang, ZONG Shixiang, LUO Youqing. UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+[J].Journal of Nanjing Forestry University (Natural Science Edition), 2024, 48(4): 93-103.DOI: 10.12302/j.issn.1000-2006.202209055.

图/表 12

图1

图2

图3

图4

图5

图6

图7

图8

表1

表2

表3

图9

参考文献 31

[1]	王静, 高建中. 林地地块特征对农户林业生产效率的影响[J]. 林业经济问题, 2021, 41(6):577-582.
	WANG J, GAO J Z. The effects of the characteristics of forest land parcels on farmers’ forestry production efficiency[J]. News For Econ, 2021, 41(6):577-582. DOI: 10.16832/j.cnki.1005-9709.20210072.
[2]	DALPONTE M, ØRKA H O, GOBAKKEN T, et al. Tree species classification in boreal forests with hyperspectral data[J]. IEEE Trans Geosci Remote Sens, 2013, 51(5):2632-2645. DOI: 10.1109/TGRS.2012.2216272.
[3]	BLANCO S R, HERAS D B, ARGÜELLO F. Texture extraction techniques for the classification of vegetation species in hyperspectral imagery: bag of words approach based on superpixels[J]. Remote Sens, 2020, 12(16):2633. DOI: 10.3390/rs12162633.
[4]	THANH NOI P, KAPPAS M. Comparison of random forest, k-nearest neighbor, and support vector machine classifiers for land cover classification using sentinel-2 imagery[J]. Sensors, 2017, 18(1):18. DOI: 10.3390/s18010018.
[5]	YUAN Y, HU X Y. Random forest and objected-based classification for forest pest extraction from UAV aerial imagery[J]. Int Arch Photogramm Remote Sens Spatial Inf Sci, 2016, XLI-B1:1093-1098. DOI: 10.5194/isprs-archives-xli-b1-1093-2016.
[6]	赵庆展, 江萍, 王学文, 等. 基于无人机高光谱遥感影像的防护林树种分类[J]. 农业机械学报, 2021, 52(11):190-199.
	ZHAO Q Z, JIANG P, WANG X W, et al. Classification of protection forest tree species based on UAV hyperspectral data[J]. Trans Chin Soc Agric Mach, 2021, 52(11):190-199. DOI: 10.6041/j.issn.1000-1298.2021.11.020.
[7]	戴鹏钦, 丁丽霞, 刘丽娟, 等. 基于FCN的无人机可见光影像树种分类[J]. 激光与光电子学进展, 2020, 57(10):36-45.
	DAI P Q, DING L X, LIU L J, et al. Tree species identification based on FCN using the visible images obtained from an unmanned aerial vehicle[J]. Laser Optoelectron Prog, 2020, 57(10):36-45. DOI: 10.3788/LOP57.101001.
[8]	张军国, 冯文钊, 胡春鹤, 等. 无人机航拍林业虫害图像分割复合梯度分水岭算法[J]. 农业工程学报, 2017, 33(14):93-99.
	ZHANG J G, FENG W Z, HU C H, et al. Image segmentation method for forestry unmanned aerial vehicle pest monitoring based on composite gradient watershed algorithm[J]. Trans Chin Soc Agric Eng, 2017, 33(14):93-99. DOI: 10.11975/j.issn.1002-6819.2017.14.013.
[9]	张增, 王兵, 伍小洁, 等. 无人机森林火灾监测中火情检测方法研究[J]. 遥感信息, 2015, 30(1):107-110, 124.
	ZHANG Z, WANG B, WU X J, et al. An algorithm of forest fire detection based on UAV remote sensing[J]. Remote Sens Inf, 2015, 30(1):107-110, 124. DOI: 10.3969/j.issn.1000-3177.2015.01.018.
[10]	刘文萍, 仲亭玉, 宋以宁. 基于无人机图像分析的树木胸径预测[J]. 农业工程学报, 2017, 33(21):99-104.
	LIU W P, ZHONG T Y, SONG Y N. Prediction of trees diameter at breast height based on unmanned aerial vehicle image analysis[J]. Trans Chin Soc Agric Eng, 2017, 33(21):99-104. DOI: 10.11975/j.issn.1002-6819.2017.21.012.
[11]	MARTINS J, JUNIOR J M, MENEZES G, et al. Image segmentation and classification with SLIC superpixel and convolutional neural network in forest context[C]// IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: IEEE, 2019:6543-6546. DOI: 10.1109/IGARSS.2019.8898969.
[12]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015:3431-3440. DOI: 10.1109/CVPR.2015.7298965.
[13]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015:234-241. DOI: 10.1007/978-3-319-24574-4_28.
[14]	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:6230-6239. DOI: 10.1109/CVPR.2017.660.
[15]	LIN G S, MILAN A, SHEN C H, et al. RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:5168-5177. DOI: 10.1109/CVPR.2017.549.
[16]	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[DB/OL]. (2014-09-14)[2022-05-25]. https://arXiv.org/abs/1412.7062. DOI: 10.48550/arXiv.1412.7062.
[17]	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans Pattern Anal Mach Intell, 2018, 40(4):834-848. DOI: 10.1109/TPAMI.2017.2699184.
[18]	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[DB/OL]. (2017-07-19)[2022-05-05]. https://arXiv.org/abs/1706.05587. DOI: 10.48550/arXiv.1706.05587.
[19]	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// European Conference on Computer Vision. Cham: Springer, 2018:833-851. DOI: 10.1007/978-3-030-01234-2_49.
[20]	韩蕊, 慕涛阳, 赵伟, 等. 基于无人机多光谱影像的柑橘树冠分割方法研究[J]. 林业工程学报, 2021, 6(5):147-153.
	HAN R, MU T Y, ZHAO W, et al. Research on citrus canopy segmentation method based on UAV multispectral image[J]. Journal of Forestry Engineering, 2021, 6(5):147-153. DOI: 10.13360/j.issn.2096-1359.202011021.
[21]	刘文定, 田洪宝, 谢将剑, 等. 基于全卷积神经网络的林区航拍图像虫害区域识别方法[J]. 农业机械学报, 2019, 50(3):179-185.
	LIU W D, TIAN H B, XIE J J, et al. Identification methods for forest pest areas of UAV aerial photography based on fully convolutional networks[J]. Trans Chin Soc Agric Mach, 2019, 50(3):179-185. DOI: 10.6041/j.issn.1000-1298.2019.03.019.
[22]	徐辉, 祝玉华, 甄彤, 等. 深度神经网络图像语义分割方法综述[J]. 计算机科学与探索, 2021, 15(1):47-59.
	XU H, ZHU Y H, ZHEN T, et al. Survey of image semantic segmentation methods based on deep neural network[J]. J Front Comput Sci Technol, 2021, 15(1):47-59. DOI: 10.3778/j.issn.1673-9418.2004039.
[23]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018:7132-7141. DOI: 10.1109/CVPR.2018.00745.
[24]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018:3-19. DOI: 10.1007/978-3-030-01234-2_1.
[25]	WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2020:11531-11539. DOI: 10.1109/CVPR42600.2020.01155.
[26]	ZHANG H, WU C R, ZHANG Z Y, et al. ResNeSt: split-attention networks[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New Orleans, LA, USA: IEEE, 2022:2735-2745. DOI: 10.1109/CVPRW56347.2022.00309.
[27]	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017:1800-1807. DOI: 10.1109/CVPR.2017.195.
[28]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016:770-778. DOI: 10.1109/CVPR.2016.90.
[29]	YANG M K, YU K, ZHANG C, et al. DenseASPP for semantic segmentation in street scenes[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018:3684-3692. DOI: 10.1109/CVPR.2018.00388.
[30]	李丹, 张俊杰, 赵梦溪. 基于FCM和分水岭算法的无人机影像中林分因子提取[J]. 林业科学, 2019, 55(5):180-187.
	LI D, ZHANG J J, ZHAO M X. Extraction of stand factors in UAV image based on FCM and watershed algorithm[J]. Sci Silvae Sin, 2019, 55(5):180-187. DOI: 10.11707/j.1001-7488.20190520.
[31]	刘旭光, 肖啸, 兰玉彬, 等. 应用可见光遥感影像的林区植被分割方法[J]. 东北林业大学学报, 2023, 51(4):62-67.
	LIU X G, XIAO X, LAN Y B, et al. Forest vegetation segmentation method with UAV visible light remote sensing images[J]. Journal of Northeast Foresrty University, 2023, 51(4):62-67. DOI: 10.13759/j.cnki.dlxb.2023.04.008.

模型 model	原始ASPP original ASPP	D-ASPP 模块	SP-ASPP 模块	ASPP 扩张率组合 ASPP dilation rate combinations	模型参数/MB model parameters	训练时间/min training time	测试时间/s testing time	平均像素精度/% mPA	平均交并比/% mIoU
1	√			(6,12,18)	49.41	259	23.83	95.82	78.24
2		√		(6,12,18,24)	50.43	330	23.91	96.74	81.69
3		√		(3,6,9,12)	50.43	333	23.66	96.81	82.01
4			√	(6,12,18,24)	48.94	271	25.51	96.63	82.41
5			√	(3,6,9,12)	48.94	252	22.62	97.02	83.18

模型 model	主干网络 backbone	原始ASPP original ASPP	SP-ASPP 模块	浅层特征融合分支 shallow feature fusion branche	SA 模块	ECA 模块	训练时间/min training time	测试时间/s testing time	平均像素精度/% mPA	平均交并比/% mIoU
1	Xception	√					256	22.43	93.01	70.94
2	ResNet101	√					237	19.41	95.27	75.52
3	ResNeSt101	√					259	23.83	95.82	78.24
4	ResNeSt101	√		√			268	23.72	96.78	82.13
5	ResNeSt101		√				252	22.62	97.02	83.18
6	ResNeSt101		√	√			261	23.23	96.94	83.96
7	ResNeSt101		√	√	√		270	23.97	97.01	84.02
8	ResNeSt101		√	√		√	264	23.44	97.05	84.13
9	ResNeSt101		√	√	√	√	272	24.79	97.04	85.01

语义分割模型 semantic segmentation models	平均像素精度/% mPA	训练时间/min training time	测试时间/s testing time	平均交并比/% mIoU	交并比/% IoU
语义分割模型 semantic segmentation models	平均像素精度/% mPA	训练时间/min training time	测试时间/s testing time	平均交并比/% mIoU	杨树 Populus sp.	银杏 Ginkgo biloba	法国梧桐 Platanus orientalis	道路 road	草地 grassland	裸地 bare groud
U-Net	94.01	241	20.25	67.50	74.67	93.99	90.87	66.53	33.31	45.62
PSPNet	93.28	608	42.93	71.59	80.18	94.18	88.37	51.94	44.17	70.69
DeepLabV3+	93.01	256	22.43	70.94	84.67	93.12	82.05	55.75	48.32	61.74
Tree-DeepLab	97.04	272	24.79	85.01	92.55	97.01	91.95	81.53	68.64	78.35

基于注意力机制和改进DeepLabV3+的无人机林区图像地物分割方法

UAV forestry land-cover image segmentation method based on attention mechanism and improved DeepLabV3+

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 31

相关文章 12

编辑推荐

Metrics

本文评价

作者

读者

审稿

[1]	钟浩, 王楚虹, 林文树. 联合地基激光雷达与无人机影像的树种识别[J]. 南京林业大学学报（自然科学版）, 2024, 48(4): 104-112.
[2]	牛弘健, 刘文萍, 陈日强, 宗世祥, 骆有庆. 基于Resnet的林地无人机图像去雾改进算法[J]. 南京林业大学学报（自然科学版）, 2024, 48(2): 175-18.
[3]	张华聪, 谭新建, 喻龙华, 厉月桥, 陈永富, 刘仁, 张怀清. 基于增强Frost局部滤波及单木距离图重构标记的CHM树冠分割[J]. 南京林业大学学报（自然科学版）, 2023, 47(5): 9-18.
[4]	杨乐, 黄晓君, 包玉海, 包刚, 佟斯琴, 苏都毕力格. 无人机航高对落叶松毛虫虫害遥感监测精度的影响[J]. 南京林业大学学报（自然科学版）, 2023, 47(4): 13-22.
[5]	杨堃, 范习健, 薄维昊, 刘婕, 王俊玲. 基于视觉加强注意力模型的植物病虫害检测[J]. 南京林业大学学报（自然科学版）, 2023, 47(3): 11-18.
[6]	吴炅, 蒋馥根, 彭邵锋, 马开森, 陈松, 孙华. 结合树冠体积的油茶树高与产量估测研究[J]. 南京林业大学学报（自然科学版）, 2022, 46(2): 53-62.
[7]	赵颖慧, 杨海城, 甄贞. 基于ULS、TLS和超声测高仪的天然次生林中不同林冠层树高估测[J]. 南京林业大学学报（自然科学版）, 2021, 45(4): 23-32.
[8]	程建斌,汪继斌,王年金,姚小华. 无人机辅助授薄壳山核桃花粉对山核桃的结实效应[J]. 南京林业大学学报（自然科学版）, 2019, 43(04): 199-202.
[9]	陶欢,李存军,谢春春,周静平,淮贺举,蒋丽雅,李凤涛. 基于HSV阈值法的无人机影像变色松树识别[J]. 南京林业大学学报（自然科学版）, 2019, 43(03): 99-106.
[10]	代婷婷,马骏,徐雁南. 基于Agisoft PhotoScan 的无人机影像自动拼接在风景园林规划中的应用[J]. 南京林业大学学报（自然科学版）, 2018, 42(04): 165-170.
[11]	王玮,王浩,李卫正,乌日汗,田晓冬,郭苏明,陈周翔,张浩峰. 基于小型无人机摄影测量的江南景观水资源综合利用分析[J]. 南京林业大学学报（自然科学版）, 2018, 42(01): 7-14.
[12]	张青萍,梁慧琳,李卫正,杨梦珂,朱灵茜,黄安. 数字化测绘技术在私家园林中的应用研究[J]. 南京林业大学学报（自然科学版）, 2018, 42(01): 1-6.