【目的】尝试融合多视图特征来最大化特征信息,提出多视图级联集成卷积神经网络(MVC-CNN)鸟鸣音分类方法,构建泛化性较强的鸟鸣分类模型,以促进鸟类物种多样性保护和生态环境智能监测的深入研究。【方法】以16种鸟鸣音频数据为研究对象,采用短时傅里叶变换(STFT)、小波变换(WT)和希尔伯-特黄变换(HHT)等特征提取方法生成鸟鸣音的3类谱图以构成多视图特征数据,并作为卷积神经网络(CNN)的输入,训练不同视图的基分类器STFT-CNN、WT-CNN和HHT-CNN;分别采用Bagging和Stacking集成方法构建了多视图Bagging集成卷积神经网络(MVB-CNN)模型和多视图Stacking集成卷积神经网络(MVS-CNN)模型。以CNN强大的特征提取能力,提出了多视图级联集成卷积神经网络(MVC-CNN)模型,将不同视图经CNN提取得到的深度特征级联融合,以支持向量机(SVM)为最终分类器获得分类结果。【结果】构建的基分类模型WT-CNN、STFT-CNN、HHT-CNN的准确率分别为89.11%、88.36%和81.00%;多视图集成模型MVB-CNN和MVS-CNN的准确率分别为89.92%和93.54%,多视图级联集成模型MVC-CNN的准确率为95.76%。MVC-CNN模型准确率比单一视图基分类模型提升6.65%~14.76%,比MVB-CNN和MVS-CNN提升5.84%和2.22%。【结论】研究提出的MVC-CNN模型能充分结合鸟鸣多视图特征的优势,有效提升鸟鸣分类效果,具有较高的稳定性和更好的泛化能力,为多视图鸟鸣音分类研究提供技术方案。

【Objective】This study aimed to build a birdsong classification model with strong generalization integrating multi-view features and maximizing feature information to promote profound research on bird species diversity protection and ecological environmentally-intelligent monitoring.【Method】Using 16 types of birdsong audio data as the research objects, the short-time Fourier transform (STFT), wavelet transform (WT) and Hilbert-Huang transform (HHT) feature extraction methods were used to generate three types of birdsong spectrograms to constitute multi-view feature data, and as the input of the convolutional neural network (CNN), the base classifiers STFT-CNN, WT-CNN, and HHT-CNN for different views were trained. The multi-view bagging ensemble convolutional neural network (MVB-CNN) and multi-view stacking ensemble convolutional neural network (MVS-CNN) models were constructed using bagging and stacking integration methods, respectively. With the powerful feature extraction capability of CNN, the multi-view cascaded ensemble convolutional neural network (MVC-CNN) model was proposed to cascade and fuse the deep features extracted from different views through CNN. The classification results were obtained by using a support vector machine (SVM). 【Result】The accuracy rates of the base classification models WT-CNN, STFT-CNN, and HHT-CNN constructed in this study were 89.11%, 88.36%, and 81.00%, respectively; the accuracy rates of the ensemble models MVB-CNN and MVS-CNN were 89.92% and 93.54%, respectively; and the accuracy of the multi-view cascade ensemble model MVC-CNN was 95.76%. The accuracy of the MVC-CNN model improved by 6.65%-14.76% over the single-view-based classification model and by 5.84% and 2.22% over the MVB-CNN and MVS-CNN models, respectively.【Conclusion】The MVC-CNN model proposed in this study fully combined the advantages of multi-view features of birdsong, effectively improving the birdsong classification effects with a greater stability and better generalizational ability, providing a technical solution for multi-view birdsong classification researches.