Efficient Hybrid Spatiotemporal Attention for Climate Change Detection: A ConvLSTM-Based Encoder with Linear and Local Window Attention
DOI:
https://doi.org/10.15680/IJCTECE.2026.0903010Keywords:
Environmental Monitoring, Earth Observation, Explainable AI, Time Series Forecasting, Attention-based Neural Networks, AI, ML, Cloud, Data Learning, Data ScienceAbstract
We propose a novel hybrid spatiotemporal attention mechanism integrated with ConvLSTM for climate change detection in remote sensing data, addressing the limitations of existing methods in handling multi-source, high-dimensional spatiotemporal dependencies. The proposed encoder combines linear attention and local window attention to efficiently capture both global and local patterns while maintaining computational tractability, where linear attention reduces quadratic complexity through kernel approximation and local window attention preserves fine-grained spatial details. A dynamic gating mechanism adaptively fuses features from both branches, then the fused representations are processed by ConvLSTM to model temporal dynamics. Moreover, the attention maps are aggregated to provide interpretable visual explanations for detected climate trends, bridging the gap between model performance and human-understandable insights. The framework is implemented with hardware-aware optimizations, including FlashAttention and shifted window partitioning, to ensure scalability for large-scale remote sensing datasets. Experiments demonstrate superior accuracy and efficiency compared to conventional ConvLSTM and Transformer-based approaches, particularly in long-sequence climate anomaly detection tasks. The method not only advances the state-of-the-art in spatiotemporal modeling but also offers a practical solution for real-world climate monitoring applications where interpretability and computational efficiency are critical
References
1) L Zhang & B Du (2019) Deep Learning for Remote Sensing: A Comprehensive Review. IEEE Geoscience and Remote Sensing Magazine, 2019.
2) X Shi, Z Chen, H Wang, DY Yeung, et al. (2015) Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in Neural Information Processing Systems.
3) A Vaswani, N Shazeer, N Parmar, et al. (2017) Attention is all you need. In Advances in Neural Information Processing Systems.
4) A Katharopoulos, A Vyas, N Pappas, et al. (2020) Transformers are rnns: Fast autoregressive transformers with linear attention. In Proceedings of the 37th International Conference on Machine Learning.
5) I Beltagy, ME Peters & A Cohan (2020) Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
6) D Chakraborty, H Başağaoğlu & J Winterle (2021) Interpretable vs. noninterpretable machine learning models for data-driven hydro-climatological process modeling. Expert Systems with Applications.
7) A Asokan & J Anitha (2019) Change detection techniques for remote sensing applications: A survey. Earth Science Informatics.
8) W Li, H Liu, Y Wang, Z Li, Y Jia & G Gui (2019) Deep learning-based classification methods for remote sensing images in urban built-up areas. Ieee Access.
9) Y Lu, S Wang, B Wang, X Zhang, X Wang & Y Zhao (2024) Enhanced window-based self-attention with global and multi-scale representations for remote sensing image super-resolution. Remote Sensing.
10) W Boulila, H Ghandorh, S Masood, A Alzahem, et al. (2024) A transformer-based approach empowered by a self-attention technique for semantic segmentation in remote sensing. Heliyon.
11) Mulajkar, R. M., Khatri, A. A., Gunjal, S. D., Galhe, D. S., Bhosale, S. B., & Bangar, A. P. (2025). Blockchain and AI Synergy in Vascular Data Management: Enhancing Trust, Traceability, and Diagnostic Accuracy in Healthcare Systems. Vascular and Endovascular Review, 8(15s), 315-330.
12) Wen, B., Li, Y., & Bresler, Y. (2020). Image recovery via transform learning and low-rank modeling: The power of complementary regularizers. IEEE Transactions on Image Processing, 29, 5310-5323.
13) Lande, R., & Mulajkar, R. M. (2018). Moving object detection using foreground detection for video surveillance system. Int. Res. J. Eng. Technol.(IRJET), 17(6), 517-519.
14) Mulajkar, R. M., & Gohokar, V. V. (2017, February). Development of Semi-Automatic Methodology for Extraction of Depth for 2D-to-3D Conversion. In Proceedings of the 9th International Conference on Machine Learning and Computing (pp. 373-378).
15) Mulajkar, R. M., & Gohokar, V. V. (2017, February). Development of methodology for extraction of depth for 2D-to-3D conversion. In 2017 Second international conference on electrical, computer and communication technologies (ICECCT) (pp. 1-5). IEEE.
16) Elminir, H. K., Sabbeh, S. F., ElSoud, M. A., & Gamal, A. (2012). Multi feature content based video retrieval using high level semantic concept. International Journal of Computer Science Issues (IJCSI), 9(4), 254.
17) Mulajkar, R. M., & Gohokar, V. V. (2016). Effect of Various Edge Detection Techniques for Depth Estimation Based on Defocus Monocular Cue.
18) Padwal, R. A., & Mulajkar, R. M. (2016). A COMPARATIVE STUDY OF IMAGE SEGMENTATION METHOD. International Journal of Advance Research in Engineering, Science & Technology, 3(7), 151-163.
19) Navas, V. M. T., Buljac, A., Hild, F., Morgeneyer, T., Helfen, L., Bernacki, M., & Bouchard, P. O. (2019). A comparative study of image segmentation methods for micromechanical simulations of ductile damage. Computational Materials Science, 159, 43-65.
20) Wable, A. A., Khapre, G. P., & Mulajkar, R. M. (2016). Intelligent farming robot for plant health detection using image processing and sensing device. Int J Eng Sci, 8320.
21) BANKHELE, N. B., MULAJKAR, R. M., & DUMBRE, S. T. (2016). Polyp detection in colon capsule endoscopy by using texure segmentation method. International Journal of Innovations in Engineering Research and Technology, 1-7.
22) Mulajkar Rahul, M., & Gohakar, D. M. V. Design of Efficient Method for Extraction of Scene Depth Information for 2D-TO-3D Conversion.
23) Mulajkar, R., & Gohokar, V. V. (2016). ‘Design of Efficient Method for Extraction of Scene Depth Information for 2D-TO-3D Conversion. IJGIP, 3(August), 269-274.
24) Bankhele, M. N. B., & Mulajkar, R. M. (2016). Detection of Protrusion on Curved Folded Surface In Colon Capsule Endoscopy.
25) Bharti, N. S., & Mulajkar, R. M. (2015). Detection and classification of plant diseases. International Research Journal of Engineering and Technology, 2(2), 2267-2272.
26) Amoda, N., Jadhav, B., & Naikwadi, S. (2014). Detection and classification of plant diseases by image processing. International Journal of Innovative Science, Engineering & Technology, 1(2), 211-217.
27) Suryabhan, S. Y., Dhede, V. M., & Mulajkar, R. M. A Novel Approach to Image Resolution Enhancement Through Histogram Processing.
28) Rathod, D. T., Mulajkar, R. M., & Dhede, V. M. Enhanced Real-Time Wearable Health Monitoring System with Multi-Modal Sensor Fusion and Predictive Analytics for Personalized Healthcare.
29) Lande, M. R. B., & Mulajkar, M. R. Tracking Human Motion Based on Bounding Box Concept.
30) Rafii, Z., & Pardo, B. (2012). Repeating pattern extraction technique (REPET): A simple method for music/voice separation. IEEE transactions on audio, speech, and language processing, 21(1), 73-84.
31) Waman, V. B., & Mulajkar, R. M. Monocular defocus and texture cue based depth map estimation of 2d image.
32) Waman, V. B., & Mulajkar, R. M. Graph Cut Based approach in creating superpixel for Depth estimation.
33) Ashok, M. G. A., Jadhavar, P., & Mulajkar, R. M. Android Application Implemented for Distance Measurement using Image Processing.
34) Madale, G. M., & Mulajkar, R. Detection of Diabetic Retinopathy by Red Sores Detection.
35) Wable, A. A., & Mulajkar, R. M. Intelligent Robot for Agricultural Plants Health Detection Using Image Processing.
36) Mulajkar, R., & Gohokar, V. V. (2016). ‘Design of Efficient Method for Extraction of Scene Depth Information for 2D-TO-3D Conversion. IJGIP, 3(August), 269-274.
37) Mulajkar Rahul, M., & Gohakar, D. M. V. Design of Efficient Method for Extraction of Scene Depth Information for 2D-TO-3D Conversion.
38) A Sharma, A Jain, P Gupta & V Chowdary (2020) Machine learning applications for precision agriculture: A comprehensive review. IEEE Access.
39) MFI Sumon, MA Khan & A Rahman (2023) Machine Learning for Real-Time Disaster Response and Recovery in the US. Unable to determine the complete publication venue.
40) A Bostrom, JL Demuth, CD Wirz, MG Cains, et al. (2024) Trust and trustworthy artificial intelligence: A research agenda for AI in the environmental sciences. Risk Analysis.
41) R Pappagari, P Zelasko, J Villalba, et al. (2019) Hierarchical transformers for long document classification. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop.
42) TJ Ham, SJ Jung, S Kim, YH Oh, Y Park, et al. (2020) A^3: Accelerating attention mechanisms in neural networks with approximation. In IEEE Symposium on High-Performance Interconnects.
