(50-5) 11 * << * >> * Русский * English * Содержание * Все выпуски

  Полный текст (PDF)

DOI: 10.18287/COJ2005

ID статьи: 2005

Аннотация:
В работе предложен метод оценки поля прозрачности облачности с использованием фундаментальной нейросетевой модели TerraMind, использующей в качестве мультимодальных данных территории съемки мультиспектральные изображения и изображения радара с синтезированной апертурой. Исходными данными для обучения модели TerraMind выступал набор SEN12MSCR, содержащий пары изображений спутников Sentinel-1 и Sentinel-2 как с облачностью, так и без нее. Для дообучения фундаментальной модели с целью решения искомой задачи мы использовали аугментацию, синтезируя облачность и соответствующие ей истинные поля прозрачности с использованием Satellite Cloud Generation Toolbox. Численные результаты проведенных экспериментальных исследования показывают высокие качественные показатели предложенного метода оценки, а качественные результаты оценки полей прозрачности на натурных снимках подтверждают работоспособность метода на реальных данных.

Ключевые слова:
Sentinel 2, Sentinel 1, радар с синтезированной апертурой, мультиспектральные изображения, нейронные сети, устранение облачности, устранение дымки, синтетический набор данных, TerraMind.

Благодарности:
Результаты исследования были получены при финансовой поддержке Российской Федерации в лице Минобрнауки России (Соглашение № 075-15-2024-558).

Цитирование:
Коновалов, В.Ф. Оценка поля прозрачности облачности с использованием оптических и радарных изображений дистанционного зондирования Земли / В.Ф. Коновалов, В.В. Мясников, В.В. Сергеев // Компьютерная оптика. – 2026. – Т. 50, № 5. – 2005. – doi: 10.18287/COJ2005.

Citation:
Konovalov VF, Myasnikov VV, Sergeev VV. A method for cloud soft segmentation using multimodal remote sensing data. Computer Optics 2026; 50(5): 2005. doi: 10.18287/COJ2005.

References:

1. King MD, Platnick S, Menzel WP, Ackerman SA, Hubanks PA. Spatial and temporal distribution of clouds observed by MODIS onboard the Terra and Aqua satellites. IEEE Transactions on Geoscience and Remote Sensing 2013; 51(7): 3826-3852. DOI: 10.1109/TGRS.2012.2227333.
2. Konovalov VF, Myasnikov VV, Sergeev VV. Reconstruction of optical remote sensing images using synthetic aperture radar data and diffusion neural networks. Optoelectronics, Instrumentation and Data Processing 2025; 61(4): 449-457.
3. Meraner A, Ebel P, Zhu XX, Schmitt M. Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion. ISPRS Journal of Photogrammetry and Remote Sensing 2020; 166: 333-346. DOI: 10.1016/j.isprsjprs.2020.05.013.
4. Valovcin FR. Snow/Cloud Discrimination. Hanscom AFB, MA: Air Force Geophysics Laboratory, Air Force Systems Command, United States Air Force; 1976. 17 p. Air Force Surveys in Geophysics No. 349. Report AFGL-TR-76-0174.
5. Kyle HL, Curran RJ, Barnes WL, Escoe D. A cloud physics radiometer. In: Proceedings of the Third Conference on Atmospheric Radiation; June 28-30, 1978; Davis, CA, USA. Boston, MA: American Meteorological Society; 1978: 107.
6. Hall DK, Riggs GA, Salomonson VV. Development of methods for mapping global snow cover using Moderate Resolution Imaging Spectroradiometer data. Remote Sensing of Environment 1995; 54(2): 127-140.
7. Wang Z, Zhao L, Meng J, Han Y, Li X, Jiang R, Chen J, Li H. Deep learning-based cloud detection for optical remote sensing images: A survey. Remote Sensing 2024; 16(23): 4583. DOI: 10.3390/rs16234583.
8. Li Z, Shen H, Weng Q, Zhang Y, Dou P, Zhang L. Cloud and cloud shadow detection for optical satellite imagery: Features, algorithms, validation, and prospects. ISPRS Journal of Photogrammetry and Remote Sensing 2022; 188: 89-108. DOI: 10.1016/j.isprsjprs.2022.04.018.
9. Huang L, Jiang B, Lv S, Liu Y, Fu Y. Deep learning-based semantic segmentation of remote sensing images: A survey. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2024; 17: 8370-8396. DOI: 10.1109/JSTARS.2024.3388508.
10. Mahajan S, Fataniya B. Cloud detection methodologies: Variants and development—a review. Complex & Intelligent Systems 2020; 6: 251-261. DOI: 10.1007/s40747-019-00128-0.
11. Zhu X, Helmer EH. An automatic method for screening clouds and cloud shadows in optical satellite image time series in cloudy regions. Remote Sensing of Environment 2018; 214: 135-153. DOI: 10.1016/j.rse.2018.05.024.
12. Frantz D, Haß E, Uhl A, Stoffels J, Hill J. Improvement of the Fmask algorithm for Sentinel-2 images: Separating clouds from bright surfaces based on parallax effects. Remote Sensing of Environment 2018; 215: 471-481. DOI: 10.1016/j.rse.2018.04.046.
13. Qiu S, Zhu Z, He B. Fmask 4.0: Improved cloud and cloud shadow detection in Landsats 4-8 and Sentinel-2 imagery. Remote Sensing of Environment 2019; 231: 111205. DOI: 10.1016/j.rse.2019.05.024.
14. Ning J, Xie L, Yin J, Liu Y. Cloud removal advances: A comprehensive review and analysis for optical remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2025; 18: 18621-18641. DOI: 10.1109/JSTARS.2025.3580718.
15. Joglekar M, Deshpande A. Remote sensing image dehazing techniques: Systematic review, performance evaluation and future directions. Archives of Computational Methods in Engineering 2026. DOI: 10.1007/s11831-026-10500-2.
16. Aybar C, Bautista L, Montero D, Contreras J, Ayala D, Prudencio F, Loja J, Ysuhuaylas L, Herrera F, Gonzales K, et al. CloudSEN12+: The largest dataset of expert-labeled pixels for cloud and cloud shadow detection in Sentinel-2. Data in Brief 2024; 56: 110852. DOI: 10.1016/j.dib.2024.110852.
17. Main-Knorn M, Pflug B, Louis J, Debaecker V, Müller-Wilm U, Gascon F. Sen2Cor for Sentinel-2. In: Proceedings of the Living Planet Symposium 2016; May 9-13, 2016; Prague, Czech Republic. ESA Special Publication SP-740; 2017: 1-8.
18. Hu J, Wei Y, Chen Z, Zhi X, Zhang W. CM-YOLO: Typical object detection method in remote sensing cloud and mist scene images. Remote Sensing 2025; 17(1): 125. DOI: 10.3390/rs17010125.
19. Zhang J, Li Y, Yang X, Jiang R, Zhang L. RSAM-Seg: A SAM-based model with prior knowledge integration for remote sensing image semantic segmentation. Remote Sensing 2025; 17(4): 590. DOI: 10.3390/rs17040590.
20. Yu Z, Idris MYI, Wang P. When cloud removal meets diffusion model in remote sensing. arXiv preprint arXiv:2504.14785; 2025. DOI: 10.48550/arXiv.2504.14785.
21. Liu Y, Li W, Guan J, Zhou S, Zhang Y. Effective cloud removal for remote sensing images by an improved mean-reverting denoising model with elucidated design space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2025: 17851-17861.
22. Grohnfeldt C, Schmitt M, Zhu XX. A conditional generative adversarial network to fuse SAR and multispectral optical data for cloud removal from Sentinel-2 images. In: Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. IEEE; 2018: 1726-1729. DOI: 10.1109/IGARSS.2018.8519215.
23. Chen S, Zhang W, Li Z, Wang Y, Zhang B. Cloud removal with SAR-optical data fusion and graph-based feature aggregation network. Remote Sensing 2022; 14(14): 3374. DOI: 10.3390/rs14143374.
24. Tu K, Yang C, Qing Y, Qi K, Chen N, Gong J. Cloud removal with optical and SAR imagery via multimodal similarity attention. ISPRS Journal of Photogrammetry and Remote Sensing 2025; 226: 116-126. DOI: 10.1016/j.isprsjprs.2025.05.004.
25. Zhu ZH, Lu W, Chen SB, Ding C, Tang J, Luo B. Real-world remote sensing image dehazing: Benchmark and baseline. IEEE Transactions on Geoscience and Remote Sensing 2025; 63: 1-16. DOI: 10.1109/TGRS.2025.3584234.
26. Zhao Z, Yan J, Li C, Wang X, Jiang P, Tang J. DehazeMamba: SAR-guided optical remote sensing image dehazing with adaptive state space model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2026; 19: 778-794. DOI: 10.1109/JSTARS.2025.3634360.
27. Makarau A, Richter R, Müller R, Reinartz P. Haze detection and removal in remotely sensed multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing 2014; 52(9): 5895-5905. DOI: 10.1109/TGRS.2013.2293662.
28. Huang B, Li Z, Yang C, Sun F, Song Y. Single satellite optical imagery dehazing using SAR image prior based on conditional generative adversarial networks. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision. IEEE; 2020: 1795-1802. DOI: 10.1109/WACV45572.2020.9093471.
29. Shen H, Zhong T, Jia Y, Wu C. Remote sensing image dehazing using generative adversarial network with texture and color space enhancement. Scientific Reports 2024; 14: 12382. DOI: 10.1038/s41598-024-63259-6.
30. Zhou H, Wu X, Chen H, Chen X, He X. RSDehamba: Lightweight vision Mamba for remote sensing satellite image dehazing. arXiv preprint arXiv:2405.10030; 2024. DOI: 10.48550/arXiv.2405.10030.
31. Ma X, Wang Q, Tong X. Nighttime light remote sensing image haze removal based on a deep learning model. Remote Sensing of Environment 2025; 318: 114575. DOI: 10.1016/j.rse.2024.114575.
32. Zhu Z, Luo Y, Wei H, Li Y, Qi G, Mazur N, Li Y, Li P. Atmospheric light estimation based remote sensing image dehazing. Remote Sensing 2021; 13(13): 2432. DOI: 10.3390/rs13132432.
33. Jakubik J, Yang F, Blumenstiel B, et al. TerraMind: Large-scale generative multimodality for Earth observation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2025.
34. Ebel P, Xu Y, Schmitt M, Zhu XX. SEN12MS-CR-TS: A remote sensing data set for multimodal multitemporal cloud removal. IEEE Transactions on Geoscience and Remote Sensing 2022; 60: 5222414. DOI: 10.1109/TGRS.2022.3146246.
35. Song Y, He Z, Qian H, Du X. Vision transformers for single image dehazing. arXiv preprint arXiv:2204.03883; 2022.
36. Schmitt M, Hughes LH, Qiu C, Zhu XX. SEN12MS: A curated dataset of georeferenced multi-spectral Sentinel-1/2 imagery for deep learning and data fusion. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2019; IV-2/W7: 153-160. DOI: 10.5194/isprs-annals-IV-2-W7-153-2019.
37. Jiang B, Chen G, Wang J, Ma H, Wang L, Wang Y, Chen X. Deep dehazing network for remote sensing image with non-uniform haze. Remote Sensing 2021; 13(21): 4443. DOI: 10.3390/rs13214443.
38. Czerkawski M, Atkinson R, Michie C, Tachtatzis C. SatelliteCloudGenerator: Controllable cloud and shadow synthesis for multispectral optical satellite images. Remote Sensing 2023; 15(17): 4138. DOI: 10.3390/rs15174138.
39. Kong F, Huang B, Bradbury K, Malof J. The Synthinel-1 dataset: A collection of high-resolution synthetic overhead imagery for building segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE; 2020: 1814-1823. DOI: 10.1109/WACV45572.2020.9093500.
40. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929; 2021. DOI: 10.48550/arXiv.2010.11929.
41. Mentzer F, Minnen D, Agustsson E, Tschannen M. Finite scalar quantization: VQ-VAE made simple. arXiv preprint arXiv:2309.15505; 2023. DOI: 10.48550/arXiv.2309.15505.

---

Россия, 443001, Самара, ул. Молодогвардейская, 151; электронная почта: journal@computeroptics.ru; тел: +7 (846) 242-41-24 (ответственный секретарь), +7 (846) 332-56-22 (технический редактор), факс: +7 (846) 332-56-20