(48-4) 16 * << * >> * Russian * English * Content * All Issues

Neural network algorithm for optical-SAR image registration based on a uniform grid of points
V.V. Volkov 1, E.A. Shvets 1

1 Institute for Information Transmission Problems (IITP RAS),
127051, Moscow, Russia, Bolshoy Karetny per. 19, build. 1

 PDF, 2739 kB

DOI: 10.18287/2412-6179-CO-1426

Pages: 610-618.

Full text of article: English language.

Abstract:
The paper considers the problem of satellite multimodal image registration, in particular, optical and SAR (Synthetic Aperture Radar). Such algorithms are used in object detection, change detection, navigation. The paper considers algorithms for optical-to-SAR image registration in conditions of rough image pre-alignment. It is known that optical and SAR images have an inaccuracy in registration with georeference (up to 100 pixels with a spatial resolution of 10 m/pixel).
     This paper presents a neural network algorithm for optical-to-SAR image registration based on descriptors calculated for a uniform grid of points. First, algorithm find uniform grid of points for both images. Next, the neural network calculates descriptors for each point and finds descriptor distances between all possible pairs of points between optical and SAR images. Using obtained descriptor distances, a matching is made between the points on the optical and SAR images. The found matches between points are used to calculate the geometric transformation between images using the RANSAC algorithm with a limited (to combinations of translation, rotation and uniform scaling) affine transformation model.
     The accuracy of the proposed algorithm for optical-to-SAR image registration was investigated with different distortions in rotation and scale.

Keywords:
image registration, optical-to-SAR, resnet18, neural network descriptor.

Citation:
Volkov VV, Shvets EA. Neural network algorithm for optical-SAR image registration based on a uniform grid of points. Computer Optics 2024; 48(4): 610-618. DOI: 10.18287/2412-6179-CO-1426.

Acknowledgements:
This work was supported by the Russian Science Foundation (project No. 20-61-47089).

References:
  1. Errico A, Angelino CV, Cicala L, Persechino G, Ferrara C, Lega M, Vallario A, Parente C, Masi G, Gaetano R, Scarpa G. Detection of environmental hazards through the feature-based fusion of optical and SAR data: A case study in southern Italy. Int J Remote Sens 2015; 36(13): 3345-3367. DOI: 10.1080/01431161.2015.1054960.
  2. Plank S, Twele A, Martinis S. Landslide mapping in vegetated areas using change detection based on optical and polarimetric SAR data. Remote Sens 2016; 8(4): 307. DOI: 10.3390/rs8040307.
  3. Yu Q, Ni D, Jiang Y, Yan Y, An J, Sun T. Universal SAR and optical image registration via a novel SIFT framework based on nonlinear diffusion and a polar spatial-frequency descriptor. ISPRS J Photogramm Remote Sens 2021; 171: 1-17.
  4. Ye SP, Chen CX, Nedzved A, Jiang J. Building detection by local region features in SAR images. Computer Optics 2020; 44(6): 944-950. DOI: 10.18287/2412-6179-CO-703.
  5. Hamdi I, Tounsi Y, Benjelloun M, Nassim A. Evaluation of the change in synthetic aperture radar imaging using transfer learning and residual network. Computer Optics 2021; 45(4): 600-607. DOI: 10.18287/2412-6179-CO-814.
  6. Sidorchuk DS, Volkov VV. Fusion of radar, visible and thermal imagery with account for differences in brightness and chromaticity perception [In Russian]. Sensory Systems 2018; 32(1): 14-18. DOI: 10.7868/S0235009218010031.
  7. Schmitt M, Hughes LH, Zhu XX. The SEN1-2 dataset for deep learning in SAR-optical data fusion. arXiv Preprint. 2018. Source: <https://arxiv.org/abs/1807.01569>.
  8. Ye Y, Yang C, Zhu B, Zhou L, He Y, Jia H. Improving co-registration for Sentinel-1 SAR and Sentinel-2 optical images. Remote Sens 2021; 13(5): 928.
  9. Wang Z, Yu A, Zhang B, Dong Z, Chen X. A fast registration method for optical and SAR images based on SRAWG feature description. Remote Sens 2022; 14(19): 5060.
  10. Hansson N. Investigation of registration methods for high resolution SAR-EO imagery. Master of Science Thesis in Electrical Engineering. Linköping, Sweden: Linköping University; 2022.
  11. Shi W, Su F, Wang R, Fan J. A visual circle based image registration algorithm for optical and SAR imagery. 2012 IEEE Int Geoscience and Remote Sensing Symposium 2012; 2109-2112. DOI: 10.1109/IGARSS.2012.6351089.
  12. Suri S, Reinartz P. Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. IEEE Trans Geosci Remote Sens 2009; 48(2): 939-949. DOI: 10.1109/TGRS.2009.2034842.
  13. Gong M, Zhao S, Jiao L, Tian D, Wang S. A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Trans Geosci Remote Sens 2013; 52(7): 4328-4338. DOI: 10.1109/TGRS.2013.2281391.
  14. Fan B, Huo C, Pan C, Kong Q. Registration of optical and SAR satellite images by exploring the spatial relationship of the improved SIFT. IEEE Geosci Remote Sens Lett 2012; 10(4): 657-661. DOI: 10.1109/LGRS.2012.2216500.
  15. Ma W, Wen Z, Wu Y, Jiao L, Gong M, Zheng Y, Liu L. Remote sensing image registration with modified SIFT and enhanced feature matching. IEEE Geosci Remote Sens Lett 2016; 14(1): 3-7. DOI: 10.1109/LGRS.2016.2600858.
  16. Paul S, Pati UC. Optical-to-SAR image registration using modified distinctive order based self-similarity operator. 2018 IEEE Int Students’ Conf on Electrical, Electronics and Computer Science (SCEECS) 2018: 1-5. DOI: 10.1109/SCEECS.2018.8546950.
  17. Xiang Y, Wang F, You H. OS-SIFT: A robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas. IEEE Trans Geosci Remote Sens 2018; 56(6): 3078-3090. DOI: 10.1109/TGRS.2018.2790483.
  18. Paul S, Pati UC. Automatic optical-to-SAR image registration using a structural descriptor. IET Image Process 2019; 14(1): 62-73. DOI: 10.1049/iet-ipr.2019.0389.
  19. Xiong X, Xu Q, Jin G, Zhang H, Gao X. Rank-based local self-similarity descriptor for optical-to-SAR image matching. IEEE Geosci Remote Sens Lett 2019; 17(10): 1742-1746. DOI: 10.1109/LGRS.2019.2955153.
  20. Wang H, Wang C, Li P, Chen Z, Cheng M, Luo L, Liu Y. Optical-to-SAR image registration based on Gaussian mixture model. Int Arch Photogramm Remote Sens Spat Inf Sci 2012; 39: 179-183.
  21. Kunina I, Panfilova E, Gladkov A. Matching of SAR and optical images by independent referencing to vector map. Proc SPIE 2019; 11041: 1104102. DOI: 10.1117/12.2523132.
  22. Zhang W, Zhao Y. SAR and optical image registration based on uniform feature points extraction and consistency gradient calculation. Appl Sci 2023; 13(3): 1238.
  23. Kouyama T, Kanemura A, Kato S, Imamoglu N, Fukuhara T, Nakamura R. Satellite attitude determination and map projection based on robust image matching. Remote Sens 2017; 9(1): 90.
  24. Li X, Du Z, Huang Y, Tan Z. A deep translation (GAN) based change detection network for optical and SAR remote sensing images. ISPRS J Photogramm Remote Sens 2021; 179: 14-34.
  25. Merkle N, Luo W, Auer S, Müller R, Urtasun R. Exploiting deep matching and SAR data for the geo-localization accuracy improvement of optical satellite images. Remote Sens 2017; 9(6): 586.
  26. Li Z, Zhang H, Huang Y. A rotation-invariant optical and SAR image registration algorithm based on deep and Gaussian features. Remote Sens 2021; 13(13): 2628.
  27. Volkov VV, Shvets EA. Dataset and method for evaluating optical-to-Sar image registration algorithms based on keypoints [In Russian]. Information Technologies and Computing Systems 2021; 2: 44-57. DOI: 10.14357/20718632210205.
  28. Terms of the Copernicus Data Hub portals and Data supply conditions. 2024. Source: <https://scihub.copernicus.eu/twiki/do/view/SciHubWebPortal/TermsConditions>.
  29. Copernicus Open Access Hub website. 2024. Source: <https://scihub.copernicus.eu/>.
  30. Volkov V. Modification of the method of detecting and describing keypoints SIFT for optical-to-SAR image registration [In Russian]. Sensory Systems 2022; 36(4): 349-365. DOI: 10.31857/S0235009222040060.
  31. Tropin DV, Shemiakina JA, Konovalenko IA, Faradjev IA. Localization of planar objects on the images with complex structure of projective distortion [In Russian]. Information Processes 2019; 19(2): 208-229.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20