(50-3) 14 * << * >> * Russian * English * Content * All Issues

Сommon domain construction method for radar and optical Earth remote sensing images matching
A.N. Borisov1, V.V. Myasnikov1, V.V. Sergeev1

1 Samara National Research University, Moskovskoye Shosse 34, Samara, 443086, Russia

  Full text (PDF)

DOI: 10.18287/COJ1980

Article ID: 1980

Language: English

Abstract:
This paper presents a new method for constructing a common domain for digital Earth remote sensing images acquired by two types of sensors: a multispectral optical sensor and a synthetic aperture radar. After translation to a common domain, initially heterogeneous images become sufficiently similar to be coregistered by image matching methods, developed for standard images. The problem of constructing a common domain is considered as a task of simultaneous training of two U-Net neural networks. A proposed method is compared with an existing method based on the pix2pix generative neural network, which performs domain translation from the radar to the optical domain. Results of image matching using the well-known SIFT algorithm, the state-of-the-art RoMa and DeDoDe algorithms, and the RIFT2 heterogenous image matching algorithm are presented. Experimental studies demonstrate the clear advantage of a proposed method: a proposed method in combination with the trained RoMa network achieves the matching success rate of 99.9% with the average corner error of 0.84 pixels; pix2pix in combination with DeDoDe achieves the matching success rate of 98% with the average corner error of 4.3 pixels.

Keywords:
digital Earth remote sensing images, multispectral images, radar images, common domain, image coregistration.

Acknowledgements:
This work was supported by the Ministry of Science and Higher Education, Russia (Agreement No. 075-15-2024-558).

Citation:
Borisov AN, Myasnikov VV, Sergeev VV. Сommon domain construction method for radar and optical Earth remote sensing images matching. Computer Optics 2026; 50(3): 1980. DOI: 10.18287/COJ1980.

References:

  1. Verykokou S, Ioannidis C. Image matching: a comprehensive overview of conventional and learning-based methods. Encyclopedia 2025; 5(1): 4. doi:10.3390/encyclopedia5010004.
  2. Zhang S, Wang J, Li S, et al. Deep learning reforms image matching: a survey and outlook. arXiv preprint 2025; arXiv:2506.04619. doi:10.48550/arXiv.2506.04619.
  3. Aharchi M, Ait Kbir M. A review on 3D reconstruction techniques from 2D images. In: Proc 3rd Int Conf Smart City Appl (SCA 2020); 2020. pp. 510-522. doi:10.1007/978-3-030-37629-1_37.
  4. Campos C, Elvira R, Rodríguez JJG, Montiel JMM, Tardós JD. ORB-SLAM3: an accurate open-source library for visual, visual--inertial, and multimap SLAM. IEEE Trans Robot 2021; 37(6): 1874-1890. doi:10.1109/TRO.2021.3075644.
  5. Chen H, Ye T, Li C, et al. Bright: a globally distributed multimodal building damage assessment dataset with very-high-resolution for all-weather disaster response. Earth Syst Sci Data 2025; 17(12): 6217-6253. doi:10.5194/essd-17-6217-2025.
  6. Liu C, Zhang Y, Wang X, et al. A review of optical and SAR image deep feature fusion in semantic segmentation. IEEE J Sel Top Appl Earth Obs Remote Sens 2024; 17: 12910-12930. doi:10.1109/JSTARS.2024.3424831.
  7. Klaus B, Horn P. Robot vision. Cambridge, MA: MIT Press; 1986. ISBN: 978-0-262-08159-7.
  8. Astruc G, Gonthier N, Mallet C, Landrieu L. AnySat: one Earth observation model for many resolutions, scales, and modalities. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR); 2025. pp. 19530-19540.
  9. Jakubik J, Felsberg M, et al. Terramind: large-scale generative multimodality for Earth observation. In: Proc IEEE/CVF Int Conf Comput Vis (ICCV); 2025. pp. 7383-7394.
  10. Kang Q, Zhang J, Huang G, Liu F. Robust optical and SAR image matching via attention-guided structural encoding and confidence-aware filtering. Remote Sens 2025; 17(14): 2501. doi:10.3390/rs17142501.
  11. Liao Y, Zhang X, Wang Z, et al. Feature matching and position matching between optical and SAR with local deep feature descriptor. IEEE J Sel Top Appl Earth Obs Remote Sens 2022; 15: 448-462. doi:10.1109/JSTARS.2021.3134676.
  12. Zhang Y, Qui C, Li S, et al. PLISA: an optical--SAR remote sensing image registration method based on pseudo-label learning and interactive spatial attention. Remote Sens 2025; 17(21): 3571. doi:10.3390/rs17213571.
  13. Liao J, Yao Y, Yuan L, Hua G, Kang SB. Visual attribute transfer through deep image analogy. ACM Trans Graph 2017; 36(4): 120-1--120-15. doi:10.1145/3072959.3073683.
  14. Isola P, Zhu JY, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR); 2017. pp. 1125-1134.
  15. Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proc IEEE Int Conf Comput Vis (ICCV); 2017. pp. 2223-2232. doi:10.1109/ICCV.2017.244.
  16. Quan D, Li X, Yang J, et al. Deep generative matching network for optical and SAR image registration. In: Proc IEEE Int Geosci Remote Sens Symp (IGARSS); 2018. pp. 6215-6218. doi:10.1109/IGARSS.2018.8518653.
  17. Du W, Zhou Y, Zhao J, Tian X. K-means clustering guided generative adversarial networks for SAR--optical image matching. IEEE Access 2020; 8: 217554-217572. doi:10.1109/ACCESS.2020.3042213.
  18. Ma T, Ma J, Yu K, Zhang J, Fu W. Multispectral remote sensing image matching via image transfer by regularized conditional generative adversarial networks and local feature. IEEE Geosci Remote Sens Lett 2021; 18(2): 351-355. doi:10.1109/LGRS.2020.2972361.
  19. Edstedt J, Sun Q, Bökman G, Wadenbäck M, Felsberg M. RoMa: robust dense feature matching. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR); 2024. pp. 19790-19800. doi:10.1109/CVPR52733.2024.01871.
  20. Li L, Han L, Ye Y, Xiang Y, Zhang T. Deep learning in remote sensing image matching: a survey. ISPRS J Photogramm Remote Sens 2025; 225: 88-112. doi:10.1016/j.isprsjprs.2025.04.001.
  21. Sommervold O, Gazzea M, Arghandeh R. A survey on SAR and optical satellite image registration. Remote Sens 2023; 15(3): 850. doi:10.3390/rs15030850.
  22. Chen HM, Arora MK, Varshney PK. Mutual information-based image registration for remote sensing data. Int J Remote Sens 2003; 24(18): 3701-3706. doi:10.1080/0143116031000117047.
  23. Wang Y, Yu Q, Yu W. An improved normalized cross correlation algorithm for SAR image registration. In: Proc IEEE Int Geosci Remote Sens Symp (IGARSS); 2012. pp. 2086-2089. doi:10.1109/IGARSS.2012.6350961.
  24. Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR); 2005; 1: 886-893. doi:10.1109/CVPR.2005.177.
  25. Ye Y, Bruzzone L, Shan J, Bovolo F, Zhu Q. Fast and robust matching for multimodal remote sensing image registration. IEEE Trans Geosci Remote Sens 2019; 57(11): 9059-9070. doi:10.1109/TGRS.2019.2924684.
  26. Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 1981; 24(6): 381-395. doi:10.1145/358669.358692.
  27. Lowe DG. Object recognition from local scale-invariant features. In: Proc 7th IEEE Int Conf Comput Vis (ICCV); 1999; 2: 1150-1157. doi:10.1109/ICCV.1999.790410.
  28. Xiang Y, Wang F, You H. OS-SIFT: a robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas. IEEE Trans Geosci Remote Sens 2018; 56(6): 3078-3090. doi:10.1109/TGRS.2018.2790483.
  29. Ma W, Wen Y, Wu Y, et al. Remote sensing image registration with modified SIFT and enhanced feature matching. IEEE Geosci Remote Sens Lett 2017; 14(1): 3-7. doi:10.1109/LGRS.2016.2600858.
  30. Li J, Hu Q, Ai M. RIFT: multi-modal image matching based on radiation-variation insensitive feature transform. IEEE Trans Image Process 2020; 29: 3296-3310. doi:10.1109/TIP.2019.2959244.
  31. Li J, Shi P, Hu Q, Zhang Y. RIFT2: speeding-up RIFT with a new rotation-invariance technique. arXiv preprint 2023; arXiv:2303.00319. doi:10.48550/arXiv.2303.00319.
  32. Zhao J, Yang D, Li Y, Xiao P, Yang J. Intelligent matching method for heterogeneous remote sensing images based on style transfer. IEEE J Sel Top Appl Earth Obs Remote Sens 2022; 15: 6723-6731. doi:10.1109/JSTARS.2022.3197748.
  33. Chen J, Zhang Y, Li S, et al. SAR and optical image registration based on deep learning with co-attention matching module. Remote Sens 2023; 15(15): 3879. doi:10.3390/rs15153879.
  34. Borisov AN, Myasnikov VV, Sergeev VV. Method of automatic coregistration of digital remote sensing images from different sources. Computer Optics 2024; 48(6): 932-943. doi:10.18287/2412-6179-CO-1604.
  35. Pearson K. On lines and planes of closest fit to systems of points in space. Philos Mag 1901; 2(11): 559-572. doi:10.1080/14786440109462720.
  36. Hu C, Zhu R, Sun X, Li X, Xiang D. Optical and SAR image registration based on pseudo-SAR image generation strategy. Remote Sens 2023; 15(14): 3528. doi:10.3390/rs15143528.
  37. Dellinger F, Delon J, Gousseau Y, Michel J, Tupin F. SAR-SIFT: a SIFT-like algorithm for SAR images. IEEE Trans Geosci Remote Sens 2015; 53(1): 453-466. doi:10.1109/TGRS.2014.2323552.
  38. Liaghat A, Helfroush MS, Norouzi J, Danyali H, Guerrero JM. OS-CycleGAN: modified CycleGAN-based descriptors for optical and SAR image matching. IEEE Trans Geosci Remote Sens 2025; 63: 5214918. doi:10.1109/TGRS.2025.3575193.
  39. Wu B, Wang H, Zhang C, Chen J. Optical-to-SAR translation based on CDA-GAN for high-quality training sample generation for ship detection in SAR amplitude images. Remote Sens 2024; 16(16): 3001. doi:10.3390/rs16163001.
  40. Sghaier MO, Hadzagic M, Yu JY, Shton S. Change detection in heterogeneous satellite remote-sensing imagery using deep features comparison. In: Proc IEEE Int Geosci Remote Sens Symp (IGARSS); 2024. pp. 8599-8602. doi:10.1109/IGARSS53475.2024.10641813.
  41. Du Z, Zhang Y, Li M, et al. Concatenated deep-learning framework for multitask change detection of optical and SAR images. IEEE J Sel Top Appl Earth Obs Remote Sens 2024; 17: 719-731. doi:10.1109/JSTARS.2023.3333959.
  42. Schmitt M, Hughes LH, Zhu XX. The SEN1-2 dataset for deep learning in SAR-optical data fusion. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci 2018; IV(1): 141-146. doi:10.5194/isprs-annals-IV-1-141-2018.
  43. Schmitt M, Hughes LH, Qui C, Zhu XX. SEN12MS---a curated dataset of georeferenced multi-spectral Sentinel-1/2 imagery for deep learning and data fusion. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci 2019; IV(2/W7): 153-160. doi:10.5194/isprs-annals-IV-2-W7-153-2019.
  44. Xiang Y, Tao R, Wang F, You H, Han B. Automatic registration of optical and SAR images via improved phase congruency model. IEEE J Sel Top Appl Earth Obs Remote Sens 2020; 13: 5847-5861. doi:10.1109/JSTARS.2020.3026162.
  45. Wenger R, Puissant A, Weber J, Idoumghar L, Forestier G. MultiSenGe: a multimodal and multitemporal benchmark dataset for land use/land cover remote sensing applications. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci 2022; V-3-2022: 635-640. doi:10.5194/isprs-annals-V-3-2022-635-2022.
  46. Borisov A. Methods of spectral matching for remote sensing data. Opt Mem Neural Netw 2025; 34(Suppl 2): S221-S229. doi:10.3103/S1060992X25602398.
  47. Borisov AN, Myasnikov VV, Sergeev VV. Methods of spectral matching of heterogeneous remote sensing images. Optoelectron Instrum Proc 2025; 61(4): 422-429. doi:10.3103/S8756699025700475.
  48. Zhao H, Gallo O, Frosio I, Kautz J. Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 2017; 3(1): 47-57. doi:10.1109/TCI.2016.2644865.
  49. Zhou W, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004; 13(4): 600-612. doi:10.1109/TIP.2003.819861.
  50. Kramer MA. Autoassociative neural networks. Comput Chem Eng 1992; 16(4): 313-328. doi:10.1016/0098-1354(92)80051-A.
  51. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Proc 18th Int Conf Med Image Comput Comput Assist Interv (MICCAI); 2015. pp. 234-241. doi:10.1007/978-3-319-24574-4_28.
  52. Tan M, Le Q. EfficientNet: rethinking model scaling for convolutional neural networks. arXiv preprint 2019; arXiv:1905.11946. doi:10.48550/arXiv.1905.11946.
  53. Edstedt J, Bökman G, Zhao Z. DeDeDo v2: analyzing and improving the DeDoDe keypoint detector. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit Workshops (CVPRW); 2024. pp. 4245-4253. doi:10.1109/CVPRW63382.2024.00428.
  54. Iakubovskii P. Segmentation Models PyTorch. 2020. Available at: https://github.com/qubvel-org/segmentation_models.pytorch.
  55. Dozat T. Incorporating Nesterov momentum into Adam. 2016. Available at: https://openreview.net/forum?id=OM0jvwB8jIp57ZJjtNEZ.
  56. Smith LN, Topin N. Super-convergence: very fast training of neural networks using large learning rates. In: Proc SPIE 2019; 11006: 369-386. doi:10.1117/12.2520589.
  57. Zhu JY. CycleGAN and pix2pix in PyTorch. Available at: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix.

151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20