Crown segmentation of CHM based on the enhanced frost local filtering and distance map reconstruction

ZHANG Huacong, TAN Xinjian, YU Longhua, LI Yueqiao, CHEN Yongfu, LIU Ren, ZHANG Huaiqing

JOURNAL OF NANJING FORESTRY UNIVERSITY ›› 2023, Vol. 47 ›› Issue (5) : 9-18.

PDF(4636 KB)
PDF(4636 KB)
JOURNAL OF NANJING FORESTRY UNIVERSITY ›› 2023, Vol. 47 ›› Issue (5) : 9-18. DOI: 10.12302/j.issn.1000-2006.202209011

Crown segmentation of CHM based on the enhanced frost local filtering and distance map reconstruction

Author information +
History +

Abstract

【Objective】We used the enhanced Frost local filtering and single-tree distance map reconstruction marking technology to segment a canopy height model (CHM), improving the accuracy and efficiency of unpiloted aerial system (UAV)-light detection and ranging (LiDAR) segmentation in single-tree crowns.【Method】 We selected three forest types - coniferous mixed, coniferous-broad mixed, and broad-leaved mixed - in the Shanxia Experimental Forest Farm of Fenyi, Jiangxi Province. We then used UAV-LiDAR data to construct the CHM. To combat the increased pores in the crown area of the high-resolution CHM, we used the enhanced Frost local filtering to optimize the CHM and results were compared with different filtering methods. Next we applied the distance map reconstruction marker segmentation technology to segment and analyze the CHM-with resolutions of 0.1, 0.2, 0.5 and 1.0 m after optimization of the enhanced Froest local filter. Finally, we determined the CHM with the optimal resolution, and compared segmentation results with that of a watershed algorithm with the same resolution and mean-shift segmentation algorithm. 【Result】 Applying the enhanced Frost local filter indeed optimized the CHM-preserving image details while suppressing phase crown noise. A resolution of 0.2 m performed best for the CHM segmentation. An overall accuracy of 0.96, 0.84 and 0.75 was observed for coniferous mixed, coniferous-broad mixed, and broad-leaved mixed forests, respectively. The crown width of a single tree was calculated according to crown segmentation results, and the R2 estimated at 0.83, 0.82 and 0.71, respectively. 【Conclusion】Through the enhanced Frost local filtering and distance map reconstruction marking technology, the single-tree segmentation and crown estimation of laser point cloud CHMs can be realized, meeting key requirements of forest surveys and monitoring.

Key words

UAV laser point cloud / cannopy height model(CHM) / enhance Frost / distance map marker and reconstruction / crown segmentation

Cite this article

Download Citations
ZHANG Huacong , TAN Xinjian , YU Longhua , et al . Crown segmentation of CHM based on the enhanced frost local filtering and distance map reconstruction[J]. JOURNAL OF NANJING FORESTRY UNIVERSITY. 2023, 47(5): 9-18 https://doi.org/10.12302/j.issn.1000-2006.202209011

References

[1]
解宇阳, 王彬, 姚扬, 等. 基于无人机激光雷达遥感的亚热带常绿阔叶林群落垂直结构分析[J]. 生态学报, 2020, 40(3):196-207.
XIE Y Y, WANG B, YAO Y, et al. Quantification of vertical community structure of subtropical evergreen broadleaved forest community using UAV-Lidar data[J]. Acta Ecologica Sinica, 2020, 40(3):196-207. DOI:10.5846/stxb201809282112.
[2]
陈宗铸, 杨琦, 雷金睿, 等. 基于激光雷达数据的热带森林冠高模型生成及平均树高估计[J]. 中南林业科技大学学报, 2018, 38(7):1-7.
CHEN Z T, YANG Q, LEI J R, et al. Generation of tropical forest canopy height model and estimation of average tree height based on lidar data[J]. Journal of Central South University of Forestry and Technology, 2018, 38(7):1-7. DOI:10.14067/j.cnki.1673-923x.2018.07.001.
[3]
巨一琳, 姬永杰, 黄继茂, 等. 联合LiDAR和多光谱数据森林地上生物量反演研究[J]. 南京林业大学学报(自然科学版), 2022, 46(1):58-68.
JU Y L, JI Y J, HUANG J M, et al. Inversion of forest aboveground biomass using combination of LiDAR and multispectral data[J]. Journal of Nanjing Forestry University(Natural Sciences Edition), 2022, 46(1):58-68. Doi: 10.12302/j.issn.1000-2006.202109029.
[4]
ALVITES C I, SANTOPUOLI G, MAESANO M, et al. Unsupervised algorithms to detect single trees in a mixed-species and multi-layered Mediterranean forest using LiDAR data[J]. Canadian Journal of Forest Research, 2021, 51(12):1766-1780. DOI:10.1139/cjfr-2020-0510.
Accurate measurement of forest growing stock is a prerequisite for implementing climate-smart forestry strategies. This study deals with the use of airborne laser scanning data to assess carbon stock at the tree level. It aims to demonstrate that the combined use of two unsupervised techniques will improve the accuracy of estimation supporting sustainable forest management. Based on the heterogeneity of tree height and point cloud density, we classified 31 forest stands into four complexity categories. The point cloud of each stand was further divided into three horizontal layers, improving the accuracy of tree detection at tree level for which we calculated volume and carbon stock. The average accuracy of tree detection was 0.48. The accuracy was higher for forest stands with lower tree density and higher frequency of large trees, as well as a dense point cloud (0.65). The prediction of carbon stock was higher with a bias ranging from –0.3% to 1.5% and a root mean square error ranging from 0.14% to 1.48%.
[5]
JIANG X, LI G, LU D, et al. Modelingforest aboveground carbon density in the brazilian amazon with integration of MODIS and airborne LiDAR data[J]. Remote Sensing, 2020, 12(20): 3330. DOI:10.3390/rs12203330.
Timely updates of carbon stock distribution are needed to better understand the impacts of deforestation and degradation on forest carbon stock dynamics. This research aimed to explore an approach for estimating aboveground carbon density (ACD) in the Brazilian Amazon through integration of MODIS (moderate resolution imaging spectroradiometer) and a limited number of light detection and ranging (Lidar) data samples using linear regression (LR) and random forest (RF) algorithms, respectively. Airborne LiDAR data at 23 sites across the Brazilian Amazon were collected and used to calculate ACD. The ACD estimation model, which was developed by Longo et al. in the same study area, was used to map ACD distribution in the 23 sites. The LR and RF methods were used to develop ACD models, in which the samples extracted from LiDAR-estimated ACD were used as dependent variables and MODIS-derived variables were used as independent variables. The evaluation of modeling results indicated that ACD can be successfully estimated with a coefficient of determination of 0.67 and root mean square error of 4.18 kg C/m2 using RF based on spectral indices. The mixed pixel problem in MODIS data is a major factor in ACD overestimation, while cloud contamination and data saturation are major factors in ACD underestimation. These uncertainties in ACD estimation using MODIS data make it difficult to examine annual ACD dynamics of degradation and growth, however this method can be used to examine the deforestation-induced ACD loss.
[6]
张冬, 云挺, 薛联凤, 等. 基于力场的点云树木骨架提取方法[J]. 南京林业大学学报(自然科学版), 2016, 40(2):160-166.
ZHANG D, YUN T, XUE L F, et al. Skeleton extraction for point cloud trees based on force field[J]. Journal of Nanjing Forestry University(Natural Sciences Edition), 2016, 40(2):160-166. DOI:10.3969/j.issn.1000-2006.2016.02.027.
[7]
BLACKBURN R C, BUSCAGLIA R, MEADOR A. Mixtures of airborne lidar-based approaches improve predictions of forest structure[J]. Canadian Journal of Forest Research, 2021, 51(8):1106-1116. DOI:10.1139/cjfr-2020-0506
The most common method for modeling forest attributes with airborne lidar, the area-based approach, involves summarizing the point cloud of individual plots and relating this to attributes of interest. Tree- and voxel-based approaches have been considered as alternatives to the area-based approach but are rarely considered in an area-based context. We estimated three forest attributes (basal area, overstory biomass, and volume) across 1680 field plots in Arizona and New Mexico. Variables from the three lidar approaches (area, tree, and voxel) were created for each plot. Random forests were estimated using subsets of variables based on each individual lidar approach and mixtures of each approach. Boruta feature selection was performed on variable subsets, including the mixture of all lidar-approach predictors (KS-Boruta). A corrected paired t test was utilized to compare six validated models (area-Boruta, tree-Boruta, voxel-Boruta, KS-Boruta, KS-all, and ridge-all) for each forest attribute. Based on significant reductions in error (SMdAPE), basal area and biomass were best modeled with KS-Boruta, while volume was best modeled with KS-all. Analysis of variable importance shows that voxel-based predictors are critical for the prediction of the three forest attributes. This study highlights the importance of multiresolution voxel-based variables for modeling forest attributes in an area-based context.
[8]
李文娟, 赵传燕, 别强, 等. 基于机载激光雷达数据的森林结构参数反演[J]. 遥感技术与应用, 2015, 30(5):917-924.
LI W J, ZHAO C Y, BIE Q, et al. Retrieval of the forest structural parameters using airborne LiDAR data[J]. Remote Sensing Technology and Application, 2015, 30(5):917-924. DOI:10.11873/j.issn.1004-0323.2015.5.0917.
[9]
VALERO S, KNAPP D E, TOCHON G, et al. On the use of binary partition trees for the tree crown segmentation of tropical rainforest hyperspectral images[J]. Remote Sensing of Environment, 2015, 159:318-331.DOI:10.1016/j.rse.2014.12.020.
[10]
王辉, 韩娜娜, 吕程序, 等. 基于Mask R-CNN的单株柑橘树冠识别与分割[J]. 农业机械学报, 2021, 52(5):169-174.
WANG H, HAN N N, LYU C X, et al. Recognition and segmentation of individual citrus tree crown based on Mask R-CNN[J]. Transactions of the Chinese Society of Agricultural Machinery, 2021, 52(5):169-174. DOI:10.6041/j.issn.1000-1298.2021.05.018.
[11]
程晓菲, 武刚. 基于高分辨率卫星影像的CV模型单木定位法[J]. 南京林业大学学报(自然科学版), 2022, 46(5):143-151.
CHEN X F, WU G. Locating individual tree from high resolution satellite images based on CV model[J]. Journal of Nanjing Forestry University(Natural Sciences Edition), 2022, 46(5):143-151. DOI:10.12302/j.issn.1000-2006.202102020.
[12]
孙振峰, 张晓丽, 李霓雯. 机载与星载高分遥感影像单木树冠分割方法和适宜性对比[J]. 北京林业大学学报, 2019, 41(11):66-75.
SUN Z F, ZHANG X L, LI N W. Comparison of individual tree crown extraction method and suitability of airborne and spaceborne high-resolution remote sensing images[J]. Journal of Beijing Forestry University, 2019, 41(11):66-75. DOI:10.13332/j.1000-1522.20180446.
[13]
YANG L, SHENG Y H, WANG B. 3D reconstruction of building facade with fused data of terrestrial LiDAR data and optical image[J]. Optik, 2016, 127(4):2165-2168. DOI:10.1016/j.ijleo.2015.11.147.
[14]
MAN Q, DONG P, GUO H. Pixel and feature-level fusion of hyperspectral and lidar data for urban land-use classification[J]. International Journal of Remote Sensing, 2015, 36(5/6):1618-1644. DOI:10.1080/01431161.2015.1015657.
[15]
LI Y, JAVIER I G. LiDAR for autonomous driving: the principles, challenges, and trends for automotive lidar and perception systems[J]. IEEE Signal Processing Magazine, 2020, 37(4):50-61. DOI:10.1109/MSP.2020.2973615.
[16]
FABAIN D, REIK L, FELIX M, et al. Simulating imaging spectrometer data: 3D forest modeling based on LiDAR and in situ data[J]. Remote Sensing of Environment, 2014, 152:235-250. DOI: 10.1016/j.rse.2014.06.015.
[17]
徐志扬, 刘浩栋, 陈永富, 等. 基于无人机LiDAR的杉木树冠上部外轮廓模拟与可视化研究[J]. 林业科学研究, 2021, 34(4):9.
XU Z Y, LIU H D, CHEN Y F, et al. Outer upper crown profile aimulation and visualization for Cunninghamia lanceolata based on UAV-borne LiDAR data[J]. Forest Research, 2021, 34(4):9. DOI:10.13275/j.cnki.lykxyj.2021.04.005.
[18]
HUO L, LINDBERG E, HOLMGREN J. Towards low vegetation identification:a new method for tree crown segmentation from LiDAR data based on a symmetrical structure detection algorithm (SSD)[J]. Remote Sensing of Environment, 2022, 270: 112857. DOI:10.1016/j.rse.2021.112857.
[19]
AYREY E, FARVER S, KERSHAW, et al. Layer Stacking: a novel algorithm for individual forest tree segmentation from LiDAR point clouds[J]. Canadian Journal of Remote Sensing, 2017, 43(1):16-27. DOI:10.1080/07038992.2017.1252907.
[20]
HUI Z, LI N, XIA Y, et al. Individual tree extraction from UAV LIDAR point clouds based on self-adaptive mean shift segmentation[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021(1):25-30. DOI:10.5194/isprs-annals-V-1-2021-25-2021.
[21]
CHEN X, JIANG K, ZHU Y, et al. Individual tree crown segmentation directly from UAV-borne LiDAR data using thePointNet of deep learning[J]. Forests, 2021, 12(2):131. DOI:10.3390/f12020131.
Accurate individual tree crown (ITC) segmentation from scanned point clouds is a fundamental task in forest biomass monitoring and forest ecology management. Light detection and ranging (LiDAR) as a mainstream tool for forest survey is advancing the pattern of forest data acquisition. In this study, we performed a novel deep learning framework directly processing the forest point clouds belonging to the four forest types (i.e., the nursery base, the monastery garden, the mixed forest, and the defoliated forest) to realize the ITC segmentation. The specific steps of our approach were as follows: first, a voxelization strategy was conducted to subdivide the collected point clouds with various tree species from various forest types into many voxels. These voxels containing point clouds were taken as training samples for the PointNet deep learning framework to identify the tree crowns at the voxel scale. Second, based on the initial segmentation results, we used the height-related gradient information to accurately depict the boundaries of each tree crown. Meanwhile, the retrieved tree crown breadths of individual trees were compared with field measurements to verify the effectiveness of our approach. Among the four forest types, our results revealed the best performance for the nursery base (tree crown detection rate r = 0.90; crown breadth estimation R2 > 0.94 and root mean squared error (RMSE) < 0.2m). A sound performance was also achieved for the monastery garden and mixed forest, which had complex forest structures, complicated intersections of branches and different building types, with r = 0.85, R2 > 0.88 and RMSE < 0.6 m for the monastery garden and r = 0.80, R2 > 0.85 and RMSE < 0.8 m for the mixed forest. For the fourth forest plot type with the distribution of crown defoliation across the woodland, we achieved the performance with r = 0.82, R2 > 0.79 and RMSE < 0.7 m. Our method presents a robust framework inspired by the deep learning technology and computer graphics theory that solves the ITC segmentation problem and retrieves forest parameters under various forest conditions.
[22]
WANG J, CHEN X, CAO L, et al. Individual rubber tree segmentation based on ground-based LiDAR data and faster R-CNN of deep learning[J]. Forests, 2019, 10(9):793-813. DOI:10.3390/f10090793.
Rubber trees in southern China are often impacted by natural disturbances that can result in a tilted tree body. Accurate crown segmentation for individual rubber trees from scanned point clouds is an essential prerequisite for accurate tree parameter retrieval. In this paper, three plots of different rubber tree clones, PR107, CATAS 7-20-59, and CATAS 8-7-9, were taken as the study subjects. Through data collection using ground-based mobile light detection and ranging (LiDAR), a voxelisation method based on the scanned tree trunk data was proposed, and deep images (i.e., images normally used for deep learning) were generated through frontal and lateral projection transform of point clouds in each voxel with a length of 8 m and a width of 3 m. These images provided the training and testing samples for the faster region-based convolutional neural network (Faster R-CNN) of deep learning. Consequently, the Faster R-CNN combined with the generated training samples comprising 802 deep images with pre-marked trunk locations was trained to automatically recognize the trunk locations in the testing samples, which comprised 359 deep images. Finally, the point clouds for the lower parts of each trunk were extracted through back-projection transform from the recognized trunk locations in the testing samples and used as the seed points for the region’s growing algorithm to accomplish individual rubber tree crown segmentation. Compared with the visual inspection results, the recognition rate of our method reached 100% for the deep images of the testing samples when the images contained one or two trunks or the trunk information was slightly occluded by leaves. For the complicated cases, i.e., multiple trunks or overlapping trunks in one deep image or a trunk appearing in two adjacent deep images, the recognition accuracy of our method was greater than 90%. Our work represents a new method that combines a deep learning framework with point cloud processing for individual rubber tree crown segmentation based on ground-based mobile LiDAR scanned data.
[23]
黄昕晰, 夏凯, 冯海林, 等. 基于无人机影像与Mask R-CNN的单木树冠检测与分割[J]. 林业工程学报, 2021, 6(2):133-140.
HUANG X X, XIA K, FENG H L, et al. Research on individual tree crown detection and segmentation using UAV imaging and Mask R-CNN[J]. J Fore Eng, 2021, 6(2):133-140.DOI:10.13360/j.issn.2096-1359.202009004.
[24]
VINCENT L, SOILLE P. Watersheds in digital spaces: an efficient algorithm based on immersion simulations[J]. IEEE Computer Society, 1991, 13(6):583-598. DOI:10.1109/34.87344.
[25]
CHEN Q, DENNIS B, GONG P, et al. Isolating individual trees in a savanna woodland using small footprint LIDAR data[J]. Photogrammetric Engineering and Remote Sensing, 2006, 72(8):923-932. DOI:10.14358/PERS.72.8.923.
[26]
王鑫运, 黄杨, 邢艳秋, 等. 基于无人机高密度LiDAR点云的人工针叶林单木分割算法[J]. 中南林业科技大学学报, 2022, 42(8):66-77.
WANG X Y, HUANG Y, LI D J, et al. The single tree segmentation of UAV high-density LiDAR point cloud data based on coniferous plantations[J]. Journal of Central South University of Forestry and Technology, 2022, 42(8):66-77. DOI:10.14067/j.cnki.1673-923x.2022.08.007.
[27]
李岩, 史泽林, 程坤, 等. 运用激光雷达数据的单木树冠提取算法对帽儿山林场单木参数估测的影响[J]. 东北林业大学学报, 2019, 47(11):61-67.
LI Y, SHI Z L, CHEN K, et al. Effect of three individual tree crown delineation algorithms on estimating individual tree parameters with LiDAR data[J]. Journal of Northeast Forestry University, 2019, 47(11):61-67. DOI:10.13759/j.cnki.dlxb.2019.11.012.
[28]
张海清, 李向新, 王成, 等. 结合DSM的机载LiDAR单木树高提取研究[J]. 地球信息科学学报, 2021, 23(10):1873-1881.
Abstract
机载LiDAR在提取地形坡度较大区域的冠层高度模型(CHM)时易产生畸变,降低单木树高的提取精度,为此提出一种CHM与数字表面模型(DSM)相结合的树高估算方法。首先基于预处理后的点云生成的CHM,利用局部最大值算法和标记控制分水岭分割算法进行分割,得到单木树冠轮廓多边形;然后结合DSM,采用固定窗口的局部最大值算法探测树顶点并提取其高程,继而与使用狄洛尼三角网和高程内插得到的地面点相减获取树高;最后,以广西兴安县富江村附近地形起伏较大的针叶林为试验区,测试3种不同坡度下,在CHM、CHM结合DSM获得的树高与实测树高分别进行精度分析。结果表明,当树木分别位于平均坡度为32&#x000b0;、27&#x000b0;和15&#x000b0;的试验区时,CHM中提取的树高与实测数据拟合的R<sup>2</sup>分别为0.84、0.85和0.87,RMSE为1.48、1.41和1.58 m,结合DSM后R<sup>2</sup>为0.92、0.91和0.93,RMSE为0.93、1.02和1.16 m;在地形坡度较大的区域,本文方法可以有效提高单木树高的估算精度。
ZHANG H Q, LI X X, WANG C, et al. Individual tree height extraction from airborne LiDAR data by combining with DSM[J]. Journal of Geo-information Science, 2021, 23(10):1873-1881. DOI:10.12082/dqxxkx.2021.210030.
[29]
LOPES A, NEZRY E, TOUZI R, et al. Structure detection and statistical adaptive speckle filtering in SAR images[J]. International Journal of Remote Sensing, 1993, 14(9):1735-1758.
[30]
贾惠珍, 王同罕. 基于自适应微调因子的改进 frost 滤波[J]. 计算机工程与设计, 2011, 32(11): 3793-3795.
JIA H Z, WANG T H. Modified frost filtering based on adaptive parameter[J]. Computer Engineering and Design, 2011, 32(11):3793-3795.
[31]
VINCENT L. Morphological grayscale reconstruction in image analysis: applications and efficient algorithms[J]. IEEE transactions on image processing, 1993, 2(2):176-201. DOI:10.1109/83.217222.
Two different formal definitions of gray-scale reconstruction are presented. The use of gray-scale reconstruction in various image processing applications discussed to illustrate the usefulness of this transformation for image filtering and segmentation tasks. The standard parallel and sequential approaches to reconstruction are reviewed. It is shown that their common drawback is their inefficiency on conventional computers. To improve this situation, an algorithm that is based on the notion of regional maxima and makes use of breadth-first image scannings implemented using a queue of pixels is introduced. Its combination with the sequential technique results in a hybrid gray-scale reconstruction algorithm which is an order of magnitude faster than any previously known algorithm.
[32]
李增元, 庞勇, 刘清旺, 等. 激光雷达森林参数反演技术与方法[M]. 北京: 科学出版社, 2015.
LI Z Y, PANG Y, LIU Q W, et al. Lidar forest parameter inversion technology and method[M]. Beijing: Science Press, 2015.
[33]
王娟, 张超, 陈巧, 等. 结合无人机可见光和激光雷达数据的杉木树冠信息提取[J]. 西南林业大学学报, 2022, 42(1):133-141.
WANG J, ZHANG C, CHEN Q, et al. The method of extracting information of Cunninghamia lanceolata crown combined with RGB and LiDAR based on UAV[J]. Journal of Southwest Forestry University. 2022, 42(1):133-141. DOI: 10.11929/j.swfu.202101013.
[34]
汪垚, 张志玉, 倪文俭, 等. 基于机载LiDAR数据的林下地形提取算法比较与组合分析[J]. 北京林业大学学报, 2017, 39(12):25-35.
WANG Y, ZHANG Z Y, NI W J, et al. Comparison of filter algorithms and combination analysis for DEM extracting based on airborne laser scanning point clouds[J]. Journal of Beijing Forestry University, 2017, 39(12):25-35. DOI: 10.13332/j.1000-1522.20170300.
PDF(4636 KB)

Accesses

Citation

Detail

Sections
Recommended
The full text is translated into English by AI, aiming to facilitate reading and comprehension. The core content is subject to the explanation in Chinese.

/