基于深度学习多目标检测技术的林火图像识别研究

何乃磊, 张金生, 林文树

南京林业大学学报(自然科学版) ›› 2024, Vol. 48 ›› Issue (3) : 207-218.

PDF(15287 KB)
PDF(15287 KB)
南京林业大学学报(自然科学版) ›› 2024, Vol. 48 ›› Issue (3) : 207-218. DOI: 10.12302/j.issn.1000-2006.202205025
研究论文

基于深度学习多目标检测技术的林火图像识别研究

作者信息 +

Forest fire image recognition based on deep learning multi-target detection technology

Author information +
文章历史 +

摘要

【目的】 林火的发生不仅会对森林生态环境造成严重的破坏,影响生态系统功能,还会给人类造成一定危害和损失。基于深度学习对森林火灾图像进行识别,旨在更高效精准地对森林火灾发生初期的图像进行识别并预警,从而降低森林火灾对森林生态系统和人类社会产生的危害。【方法】 借助SSD算法目标检测算法在TensorFlow上的实现,根据林火特征对其网络结构进行适当优化,提出一种可以识别森林火灾图像中火焰特征的模型方法。首先对获取的图像进行归一化处理,然后使用Imgaug图像数据增强库对林火图像进行数据增强以构建林火数据集,搭建深度学习运行环境并设定超参数。通过对测试集中的数据进行测试获取模型对图像中林火特征的识别效果,并利用Loss曲线、P-R曲线的可视化对模型进行评估,最后得到模型对于林火的识别精度。【结果】 随着迭代次数的增加,损失值由训练初期的35.31逐渐下降,训练至20 000步时损失值稳定在7.10左右,此时模型的识别精度达到较高水平,对测试数据中林火特征的识别置信度达到0.9以上。基于FLAME公开数据集中的林火图像搭建测试集,经过测试评估本模型对林火特征的mAP可以达到97.40%,漏检率为0.03,对测试图片的平均检测时间仅为0.07 s,对比Faster R-CNN模型在同等数据集上的实现,SSD可以获得更为理想的检测速度。【结论】 提出的针对林火特征识别的SSD算法能兼顾检测速度和检测精度,对林火早期的图像能够快速识别并拥有较低的漏检率,有助于林区工作人员对火灾及时做出处理,从而为森林火灾早期预防提供技术参考。

Abstract

【Objective】 Forest fires not only cause serious damage to the natural environment of forests and affect their ecosystem functions, but also cause harm and losses to human society. Based on the use of a deep learning process to identify forest fire images, it is possible to efficiently and accurately identify images showing the early stages of forest fires, thereby enabling prompt action to reduce the harm caused by forest fires to forest ecosystems and human society. 【Method】 With the implementation of the single shot multibox detector (SSD) target detection algorithm based on the TensorFlow framework, the network structure was optimized according to the characteristics of forest fires, and a model was developed that could identify flame features in forest fire images. First, the acquired images were normalized and then the “Imgaug” image data enhancement library was used to perform data enhancement on forest fire images and build a forest fire dataset, followed by the building of a deep learning operating environment and the setting of hyper-parameters. The recognition of forest fire features in the image by the model was obtained by testing the data in the test set. By visualizing both the Loss curve and the precision-recall (P-R) curve, the recognition accuracy of the model for forest fires was ultimately obtained.【Result】 With an increase in the number of iterations, the loss value gradually decreased from 35.31 in the early stage of training, and stabilized at about 7.10 when the training reached 20 000 iterations. At this point, the recognition accuracy of the model reached a high level. The recognition confidence of fire features was greater than 0.9. A test set was built based on the forest fire images in the fire luminosity airborne-based machine learning evaluation (FLAME) public data set. After testing and evaluation, the model achieved a mean average precision for forest fire characteristics of 97.40%, the missed detection rate was 0.03, and the average detection time of the test image was only 0.07 s. Compared with the faster- region-based convolutional neural network (Faster R-CNN) model using the same data set, the SSD achieved a better detection speed and recognition accuracy. 【Conclusion】 The SSD algorithm for forest fire feature recognition proposed here can take into account the detection speed and precision, and can quickly identify images in the early stage of a forest fire with a low missed detection rate, which will enable forest staff to deal with the fire in time, providing a technical reference for the early prevention of forest fires.

关键词

深度学习 / 林火 / 图像识别 / 目标检测 / SSD算法

Key words

deep learning / forest fire / image identification / object detection / single shot multibox detector (SSD) algorithm

引用本文

导出引用
何乃磊, 张金生, 林文树. 基于深度学习多目标检测技术的林火图像识别研究[J]. 南京林业大学学报(自然科学版). 2024, 48(3): 207-218 https://doi.org/10.12302/j.issn.1000-2006.202205025
HE Nailei, ZHANG Jinsheng, LIN Wenshu. Forest fire image recognition based on deep learning multi-target detection technology[J]. JOURNAL OF NANJING FORESTRY UNIVERSITY. 2024, 48(3): 207-218 https://doi.org/10.12302/j.issn.1000-2006.202205025
中图分类号: S762   

参考文献

[1]
国家统计局. 中国统计年鉴[M]. 北京: 中国统计出版社, 2020: 259.
National Bureau of Statistics. China Statistical Yearbook[M]. Beijing: China Statistics Press, 2020: 259.
[2]
李宾. 面向森林火灾的烟雾检测系统的设计与开发[D]. 青岛: 山东科技大学, 2018.
LI B. Design and development of smoke detection system for forest fire[D]. Qingdao: Shandong University of Science and Technology, 2018. DOI:10.27275/d.cnki.gsdku.2018.000552.
[3]
GIGLIO L, DESCLOITRES J, JUSTICE C O, et al. An enhanced contextual fire detection algorithm for MODIS[J]. Remote Sensing of Environment, 2003, 87(2/3): 273-282.
[4]
付迎春, 徐颂军, 陈蜜. 基于MODIS影像梯度的林火边界提取方法[J]. 林业科学, 2008, 44(7):56-61.
FU Y C, XU S J, CHEN M. Extraction of forest fire edge line based on MODIS imagery gradient[J]. Scientia Silvae Sinicae, 2008, 44(7):56-61.
[5]
杨光, 宁吉彬, 舒立福, 等. 黑龙江大兴安岭卫星热点预报森林火灾准确性研究[J]. 北京林业大学学报, 2017, 39(12):1-9.
YANG G, NING J B, SHU L F, et al. Study on the accuracy of forest fire prediction by satellite hot spots in Daxing’an Mountains of Heilongjiang,northeastern China[J]. Journal of Beijing Forestry University, 2017, 39(12):1-9. DOI:10.13332/j.1000-1522.20170147.
[6]
单会见. 基于无人机的林火识别和定位算法研究[D]. 杭州: 浙江农林大学, 2019.
SHAN H J. Research on forest fire identification and location algorithm based on UAV[D]. Hangzhou: Zhejiang Agriculture and Forestry University, 2019.
[7]
马越豪, 卢晓, 董佩, 等. 基于图像特征的无人机火灾检测[J]. 消防科学与技术, 2019, 38(5): 658-660.
MA Y H, LU X, DONG P, et al. UAV fire detection based on image features[J]. Fire Science and Technology, 2019, 38(5):658-660.
[8]
孙鹏. 基于改进SSD模型的小目标检测研究[D]. 南京: 南京邮电大学, 2021.
SUN P. Small object detection research via optimized SSD model[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2021.
[9]
王飞. 基于深度学习的森林火灾识别检测系统的研究与实现[D]. 成都: 电子科技大学, 2020.
WANG F. Research and implementation of forest fire detection system based on deep learning[D]. Chengdu: University of Electronic Science and Technology of China, 2020.
[10]
刘青, 刘志国, 刘守全, 等. 基于改进YOLOv3的无人机林火监测系统设计与实现[J]. 消防科学与技术, 2021, 40(4): 557-561.
LIU Q, LIU Z G, LIU S Q, et al. Design and implementation of UAV forest fire monitoring system based on improved YOLOv3[J]. Fire Science and Technology, 2021, 40(4):557-561.
[11]
傅天驹. 基于深度学习的林火图像识别算法及实现[D]. 北京: 北京林业大学, 2016.
FU T J. Forest fire image recognition algorithm and realization based on deep learning[D]. Beijing: Beijing Forestry University, 2016.
[12]
陈燕红. 基于卷积神经网络的无人机森林火灾检测研究[D]. 西安: 西安理工大学, 2019.
CHEN Y H. Research on forest fire detection using unmanned aerial vehicles based on convolutional neural network[D]. Xi'an: Xi'an University of Technology, 2019.
[13]
周浪, 樊坤, 瞿华, 等. 基于Sparse-DenseNet模型的森林火灾识别研究[J]. 北京林业大学学报, 2020, 42(10): 36-44.
ZHOU L, FAN K, QU H, et al. Forest fire identification based on Sparse-DenseNet model[J]. Journal of Beijing Forestry University, 2020, 42(10):36-44.DOI: 10.12171/j.1000-1522.20190371.
[14]
马茂平. 基于特征显著性增强的改进SSD算法研究[D]. 成都: 西南交通大学, 2021.
MA M P. Research on improved SSD algorithm based on feature enhanced significantly[D]. Chengdu: Southwest Jiaotong University, 2021.
[15]
LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector[C]// European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[16]
李青援, 邓赵红, 罗晓清, 等. 注意力与跨尺度融合的SSD目标检测算法[J]. 计算机科学与探索, 2022, 16(11):2575-2586.
LI Q Y, DENG Z H, LUO X Q, et al. SSD object detection algorithm with attention and cross-scale fusion[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11):2575-2586.DOI:10.3778/j.issn.1673-9418.2102001.
[17]
SHAMSOSHOARA A, AFGHAH F, RAZI A, et al. Aerial imagery pile burn detection using deep learning: the FLAME dataset[J]. Computer Networks, 2021, 193: 108001.
[18]
刘嘉政, 王雪峰, 王甜. 基于深度学习的树种图像自动识别[J]. 南京林业大学学报(自然科学版), 2020, 44(1):138-144.
LIU J Z, WANG X F, WANG T. Automatic identification of tree species based on deep learning[J]. Nanjing For Univ(Nat Sci Ed), 2020, 44(1):138-144. DOI:103969/j.issn.1000-2006.201809004.
[19]
齐浩. 基于SSD的目标检测算法及其应用研究[D]. 邯郸: 河北工程大学, 2021.
QI H. Research and application object detection algorithm based on SSD[D]. Handan: Hebei University of Engineering, 2021.
[20]
张倩如, 王云飞, 吕帅朝, 等. 基于改进GhostNet的小麦秸秆表皮结构完整性分类方法[J]. 南京农业大学学报, 2022, 45(4):788-798.
ZHANG Q R, WANG Y F, LYU S C, et al. Integrity classification of wheat straw epidermis based on improved GhostNet[J]. J Nanjing Agric Univ, 2022, 45(4):788-798.DOI: 10.7685/jnau.202108015.
[21]
缪伟志, 陆兆纳, 王俊龙, 等. 基于视觉的火灾检测研究[J]. 森林工程, 2022, 38(1):86-92,100.
MIAO W Z, LU Z N, WANG J L, et al. Fire detection research based on vision[J]. Forest Engineering, 2022, 38(1):86-92,100. DOI:10.16270/j.cnki.slgc.2022.01.007.
[22]
LIU W, ANGUELOV D, ERHAN D, et al. Ssd: Single shot multibox detector[C]// Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Springer International Publishing, 2016: 21-37.
[23]
BAI G, HOU J, ZHANG Y, et al. An intelligent water level monitoring method based on SSD algorithm[J]. Measurement, 2021, 185: 110047.
[24]
SUTANTO A R, KANG D K. A novel diminish smooth L1 loss model with generative adversarial network[C]// Intelligent Human Computer Interaction: 12th International Conference, IHCI 2020, Daegu, South Korea: Springer International Publishing, 2021: 361-368.
[25]
张坚鑫, 郭四稳, 张国兰, 等. 基于多尺度特征融合的火灾检测模型[J]. 郑州大学学报(工学版), 2021, 42(5):13-18.
ZHANG J X, GUO S W, ZHANG G L, et al. Fire detection model based on multi-scale feature fusion[J]. J Zhengzhou Univ(Eng Sci), 2021, 42(5):13-18.DOI:10.13705/j.issn.1671-6833.2021.05.016.
[26]
张勇. 基于改进型SSD的火焰及烟雾检测系统的设计与实现[D]. 长沙: 湖南师范大学, 2021.
ZHANG Y. Design and implementation of fire and smoke detection system based on improved SSD[D]. Changsha: Hunan Normal University, 2021.
[27]
张倩, 周平平, 王公堂, 等. 基于合成图像的Faster R-CNN森林火灾烟雾检测[J]. 山东师范大学学报(自然科学版), 2019, 34(2):180-185.
ZHANG Q, ZHOU P P, WANG G T, et al. Faster R-CNN forest fire smoke detection based on synthetic images[J]. J Shandong Nor Univ(Nat Sci), 2019, 34(2):180-185.
[28]
陈琼, 谢家亮. 基于自适应采样的不平衡分类方法[J]. 华南理工大学学报(自然科学版), 2022, 50(4):26-34,45.
CHEN Q, XIE J L. An Imbalanced classification method based on adaptive sampling[J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(4):26-34,45.
[29]
南玉龙, 张慧春, 郑加强, 等. 深度学习在林业中的应用[J]. 世界林业研究, 2021, 34(5): 87-90.
NAN Y L, ZHANG H C, ZHENG J Q, et al. Application of deep learning to forestry[J]. World Forestry Research, 2021, 34(5):87-90.DOI:10.13348/j.cnki.sjlyyj.2021.0020.y.
[30]
高桂雨, 李丁祎, 赵娟娟, 等. 基于目标检测SSD算法的森林火灾监测模型研究[J]. 电脑知识与技术, 2022, 18(2):13-14,23.
GAO G Y, LI D Y, ZHAO J J, et al. Research on forest fire monitoring model based on object detection SSD algorithm[J]. Computer Knowledge and Technology, 2022, 18(2): 13-14,23. DOI:10.14004/j.cnki.ckt.20220061.

基金

黑龙江省自然科学基金联合引导项目(LH2020C049)
中央高校基本科研业务费专项资金项目(2572019BL03)

编辑: 李燕文
PDF(15287 KB)

Accesses

Citation

Detail

段落导航
相关文章

/