(48-1) 16 * << * >> * Russian * English * Content * All Issues

Intelligent methods for natural data analysis: application to space weather
O.V. Mandrikova 1

1 Institute of Cosmophysical Research and Radio Wave Propagation,
Far Eastern Branch of the Russian Academy of Sciences,
684034, Kamchatskiy Kray, Paratunka, Russia, Mirnaya st. 7

 PDF, 1548 kB

DOI: 10.18287/2412-6179-CO-1367

Pages: 139-148.

Full text of article: Russian language.

Abstract:
The paper describes methods for detecting anomalies in geophysical monitoring data. This work studies a highly relevant class of problems in this area, aimed at creating methods for space weather forecasting. The negative impact of space weather anomalies on human health and practically all modern infrastructure objects requires the development of methods and the creation of effective means of detecting anomalies. Threshold wavelet filtering methods widely used for data analysis and anomaly detection allow one to obtain fairly accurate estimates using a “greed” strategy, even in the case of incomplete noise data. With this approach, the signal is estimated by isolating coherent structures. But these methods have high computational complexity, failing to provide accurate estimates when the signal-to-noise ratio is low. For such signals, we propose using adaptive probabilistic thresholds. Threshold function parameters are introduced that make it possible to estimate the variability of the process, suppress noise, and detect nonstationary features of different time-frequency structures. The paper also considers ways to combine threshold wavelet filtering with neural networks of the NARX and Autoencoder architectures. Schemes for the implementation of such approaches in the problems of detecting space weather anomalies are proposed. Using the problem of detecting ionospheric anomalies as an example, the efficiency of combining threshold wavelet filtering with the NARX network is shown. The efficiency of network sharing an Autoencoder with adaptive wavelet threshold filtering is shown in the problem of anomaly detection in cosmic ray flux intensity data.

Keywords:
data analysis methods, anomaly detection, space weather, wavelet transform, neural networks.

Citation:
Mandrikova OV. Intelligent methods for natural data analysis: application to space weather. Computer Optics 2024; 48(1): 139-148. DOI: 10.18287/2412-6179-CO-1367.

Acknowledgements:
The work was funded within the government project AAAA-A21-121011290003-0, “Physical processes in the system of near space and geospheres under solar and lithospheric influences” IKIR FEB RAS.

References:
  1. Vorobev AV, Vorobeva GR. Geoinformation system for amplitude-frequency analysis of geomagnetic variations and space weather observation data. Computer Optics 2017; 41(6): 963-972. DOI: 10.18287/2412-6179-2017-41-6-963-972.
  2. Mandrikova OV, Zhizhikina EA. An automatic method for estimating the geomagnetic field. Computer Optics 2015; 39(3): 420-428. DOI: 10.18287/0134-2452-2015-39-3-420-428.
  3. Mandrikova BS. A method for analyzing complex structured data with elements of machine learning. Computer Optics 2022; 46(3): 506-512. DOI: 10.18287/2412-6179-CO-1088.
  4. Mandrikova OV, Fetisova NV, Polozov YA. Hybrid model for time series of complex structure with ARIMA components. Mathematics 2021; 9; 1122.
  5. Geppener VV, Mandrikova BS. An automated method for analyzing cosmic ray data and isolating sporadic effects [In Russian]. Journal of Computational Mathematics and Mathematical Physics 2021; 61(7): 1137-1148. DOI: 10.31857/S0044466921070061.
  6. Kuznetsov VD. Space weather and risks of space activity [In Russian]. Space technology and technology. 2014; 3(6): 3-13.
  7. Murzin BC. Astrophysics of cosmic rays: Textbook for universities [In Russian]. Moscow: "Logos" Publisher; 2007. ISBN: 978-5-98704-171-6.
  8. National Space Weather Program. Strategic Plan. Office of Federal Coordinator for Meteorological Services and Supporting Research FCM-P30-1995. Washington DC; 1995, August.
  9. World Meteorological Organization. Source: <https://public.wmo.int/ru>.
  10. Mallat SG. A wavelet tour of signal processing. San Diego, CA: Academic Press; 1999.
  11. Herley C, Kovacevic J, Ramchandran K, Vetterli M. Tilings of the time-frequency plane: Construction of arbitrary orthogonal bases and feist tiling algorithms. IEEE Trans Signal Proc 1993; 41: 3341-3359.
  12. Chen S, Donoho D. Atomic decomposition by basis pursuit. Technical Report. Stanford, CA: Stanford University; 1995.
  13. Mallat SG, Zhang ZF. Matching pursuits with time-frequency dictionaries. IEEE Trans Signal Process 1993; 41: 3397-3415.
  14. Coifman, RR, Wickerhauser MV. Entropy-based algorithms for best basis selection. IEEE Trans Inf Theory 1992; 38: 713-718.
  15. Mandrikova O, Mandrikova B, Rodomanskay A. Method of constructing a nonlinear approximating scheme of a complex signal: Application pattern recognition. Mathematics 2021; 9: 737.
  16. Danilov DL, Zhiglyavsky AA. Principal components of time series: The Caterpillar method. Saint-Petersburg: “Presskom” Publisher; 1997.
  17. Singh J, Barabanov N. Stability of discrete time recurrent neural networks and nonlinear optimization problems. Neural Netw 2016; 74: 58-72.
  18. Diaconescu E. The use of NARX neural networks to predict chaotic time series. WSEAS Trans Comp Res 2008; 3: 182-191.
  19. Lin T, Horne BG, Tino P, Giles L. Learning long-term dependencies in NARX recurrent neural networks. IEEE Trans Neural Netw 1996; 7: 1329-1338.
  20. Tsungnan L, Giles CL, Horne BG, Kung SY. A delay damage model selection algorithm for NARX neural networks. IEEE Trans Signal Process 1997; 45: 2719-2730.
  21. Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 1994; 5: 157-166.
  22. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997; 9: 1735-1780.
  23. Kühnert C, Gonuguntla NM, Krieg H, Nowak D, Thomas JA. Application of LSTM networks for water demand prediction in optimal pump control. Water 2021; 13: 644.
  24. Li P, Zhang J, Krebs P. Prediction of flow based on a CNN-LSTM combined deep learning approach. Water 2022; 14: 993.
  25. Greff K, Srivastava RK, Koutnik J, Steunebrink BR, Schmidhuber J. LSTM: A search space odyssey. IEEE Trans Neural Netw Learn Syst 2017; 28: 2222-2232.
  26. Pascanu R, Mikolov T, Bengio Y. On the difficulty of training Recurrent Neural Networks. Proc 30th Int Conf on Machine Learning 2013: 1-9.
  27. Gers F. Long short-term memory in recurrent neural networks. Lausanne, Switzerland: EPFL; 2001.
  28. Vizilter YuV, Gorbatsevich VS, Zheltov SY. Structure-functional analysis and synthesis of deep convolutional neural networks. Computer Optics 2019; 43(5): 886-900. DOI: 10.18287/2412-6179-2019-43-5-886-900.
  29. Soldatova OP, Lyozin IA, Lyozina IV, Kupriyanov AV, Kirsh DV. Application of fuzzy neural networks for defining crystal lattice types in nanoscale images. Computer Optics 2015; 39(5): 787-794. DOI: 10.18287/0134-2452-2015-39-5-787-794.
  30. Rodin IA, Khonina SN, Serafimovich PG, Popov SB. Recognition of wavefront aberrations types corresponding to single Zernike functions from the pattern of the point spread function in the focal plane using neural networks. Computer Optics 2020; 44(6): 923-930. DOI: 10.18287/2412-6179-CO-810.
  31. Goodfellow Y, Benjio I, Courville A. Deep learning [In Russian]. Moscow: "DMK Press" Publisher; 2018.
  32. Rudin W. Functional analysis. New York: McGraw-Hill, 1973.
  33. Riesz F. Sur uneespèce de géométrieanalytique des systèmes de fonctionssommables. Comptesrendus de l'Académie des Sciences 1907; 144: 1409-1411.
  34. Chui CK. Introduction to wavelets [In Russian]. Moscow: "Mir" Publisher; 2001. ISBN: 5-03-003397-1.
  35. Jaffard S. Pointwise smoothness, two-microlocalization and wavelet coefficients. Publ Mat 1991; 35: 155-168.
  36. Witte RS, Witte JS. Statistics. 11th ed. New York, NY: Wiley; 2017.
  37. Haykin SS. Neural networks: A comprehensive foundation. 2nd ed. Upper Saddle River, NJ, USA: Prentice Hall; 1999.
  38. Danilov AD. Ionospheric F-region response to geomagnetic disturbances. Adv Space Res 2013; 52(3): 343-366.
  39. Real-time database of high-resolution neutron monitors. Source: <www.nmdb.eu>.
  40. Schlickeiser R. Cosmic ray astrophysics. Berlin, Heidelberg: Springer GmbH & Co KG; 2002.
  41. Abunina MA, Belov AV, Eroshenko EA, Abunin AA, Oleneva VA, Yanke VG, Melkumyan AA. Ring of station method in research of cosmic ray variations: 1. General description. Geomagn Aeron 2020; 60: 38-45.
  42. Geomagnetic Equatorial Dst Index Home Page. Source: <https://wdc.kugi.kyoto-u.ac.jp/dstdir/>.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20