【Objective】This study aims to address the challenges of small fire point targets and complex environments in aerial forest fire images, we propose FireViT, a self-attention-based image recognition method. 【Method】This method aims to enhance the accuracy and robustness of aerial forest fire image recognition. We used forest fire videos collected by drones in Chongli District, Zhangjiakou City, to construct a dataset through data preprocessing. A 10-layer vision transformer (ViT) was selected as the backbone network. Images were serialized using overlapping sliding windows, with embedded positional information fed into the first layer of ViT. The region selection modules, extracted from the preceding nine layers of ViT, were integrated into the tenth layer through multi-head self-attention and multi-layer perceptron mechanisms. This effectively amplified minor differences between subgraphs to capture features of small targets. Finally, a contrastive feature learning strategy was employed to construct an objective loss function for model prediction. We validated the model’s effectiveness by establishing training and testing sets with sample ratios of 8∶2, 7∶3, 6∶4, and 4∶6, and compared its performance with five classical models.【Result】With the allocation ratio of four training and test sets, the model achieved a recognition rate of 100% and accuracy of 94.82%, 95.05%, 94.90%, and 94.80%, respectively, with an average accuracy of 94.89%. This performance surpassed that of the five comparison models. The model converged rapidly, maintained a high recognition accuracy rate, and demonstrated stability in subsequent iterations. It showed strong generalization ability. The recognition rates were 99.97%, 99.89%, 99.80% and 99.77%, also higher than the five comparison models.【Conclusion】This research employed a model that integrated a self-attention mechanism with weakly supervised learning to reveal distinct local feature variations in aerial forest fire images across various environments. The approach exhibited strong generalization capability and robustness, which was significant for improving the capacity, efficiency, and effectiveness of fire situation management and hazard response. It also played a crucial role in preventing forest wildfires.