(44-3) 21 * << * >> * Russian * English * Content * All Issues

Motor imagery recognition in electroencephalograms using convolutional neural networks
A.D. Bragin 1, V.G. Spitsyn 1,2

1 National Research Tomsk Polytechnic University, 634050, Russia, Tomsk, Lenin Avenue 30,
2 National Research Tomsk State University, 634050, Russia, Tomsk, Lenin Avenue 36

 PDF, 892 kB

DOI: 10.18287/2412-6179-CO-669

Pages: 482-487.

Full text of article: Russian language.

Abstract:
Electroencephalography is a widespread method to record brain signals with the use of electrodes located on the surface of the head. This method of recording the brain activity has become popular because it is relatively cheap, compact, and does not require implanting the electrodes directly into the brain. The article is devoted to a problem of recognition of motor imagery by electroencephalogram signals. The nature of such signals is complex. Characteristics of electroencephalograms are individual for every person, also depending on their age and mental state, as well as the presence of noise and interference. The multitude of these parameters should be taken into account when analyzing encephalograms. Artificial neural networks are a good tool for solving this class of problems. Their application allows combining the tasks of extracting, selecting and classifying features in one signal processing unit. Electroencephalograms are time signals and we note that Gramian Angular Fields and Markov Transition Field transforms are used to represent time series in the form of images. The article shows the possibility of using the Gramian Angular Fields and Markov Transition Field transformations of the electroencephalogram (EEG) signal for motor imagery recognition using examples of imaginary movements with the right and left hand, also studying the effect of the resolution of Gramian Angular Fields and Markov Transition Field images on the classification accuracy. The best classification accuracy of the EEG signal into the motion and state-of-rest classes is about 99%. In future, the research results can be applied in constructing the brain-computer interface.

Keywords:
image analysis, pattern recognition, neural networks, electroencephalogram, Gramian angular field, Markov transition field, motor imagery recognition, convolutional neural networks.

Citation:
Bragin AD, Spitsyn VG. Motor imagery recognition in electroencephalograms using convolutional neural networks. Computer Optics 2020; 44(3): 482-487. DOI: 10.18287/2412-6179-CO-669.

Acknowledgements:
The reported study was funded by the Russian Foundation for Basics Research under RFBR research project No. 18-08-00977 А and supported by Tomsk Polytechnic University Competitiveness Enhancement Program.

References:
  1. Sivakami A, Devi SSh. Analysis of EEG for motor imagery based classification of hand activities. Int J Biomed Eng Sci (IJBES) 2015; 2(3): 2015.
  2. van Luijtelaar G, Lüttjohann A, Makarov VV, Maksimenko VA, Koronovskii AA, Hramov AE. Methods of automated absence seizure detection, interference bystimulation, and possibilities for prediction in genetic absence models. J Neurosci Methods 2016; 260: 144-158.
  3. Koronovskii AA, Hramov AE, Grubov VV, Moskalenko OI, Sitnikova EY, Pavlov AN. Coexistence of intermittencies in the neuronal network of the epileptic brain. Phys Rev E 2016; 93: 032220. DOI: 10.1103/PhysRevE.93.032220.
  4. Grubov VV, Runnova AE, Kurovskaуa MK, Pavlov AN, Koronovskii AA, Hramov AE. Demonstration of brain noise on human EEG signals in perception of bistable images. Proc SPIE 2016; 9707: 97070Z. DOI: 10.1117/12.2207390.
  5. Hramov AE, Koronovskii AA, Makarov VA, Pavlov AN, Sitnikova EY. Wavelets in neuroscience. Heidelberg, New York, Dordrecht, London: Springer; 2015.
  6. Sotnikov P, Finagin K, Vidunova S. Bands of the electro-encephalogram signal in eye-brain-computer interface. Procedia Computer Science 2017; 103: 168-175.
  7. Vasilyev AN, Liburkina SP, Kaplan AY. Lateralization of EEG patterns in humans during motor imagery of arm movements in the brain-computer interface. Zhurnal Vysshei Nervnoi Deyatelnosti Imeni IP Pavlova 2016; 66(3): 302-312.
  8. Maksimenko VA, Heukelum S, Makarov VV, Kelderhuis J, Lüttjohann A, Koronovskii AA, Hramov AE, Luijtelaar G. Absence seizure control by a Brain computer interface. Sci Rep 2017; 7: 2487.
  9. Hsu W, Chiang I. Application of neural network to brain-computer interface. 2012 IEEE International Conference on Granular Computing 2012: 163-168. DOI: 10.1109/GrC.2012.6468559.
  10. Nakayama K, Inagaki K. A Brain computer interface based on neural network with efficient preprocessing. International Symposium on Intelligent Signal Processing and Communications 2006: 673-676. DOI: 10.1109/ISPACS.2006.364745.
  11. Östberg R. Robustness of a neural network used for image classification: The effect of applying distortions on adversarial examples. Dissertation. 2018.
  12. Wang Q, Guo W, Zhang K, Ororbia AG, Xing X, Liu X, Lee Giles C. Adversary resistant deep neural networks with an application to malware detection. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2017: 1145-1153.
  13. Yim J, Sohn K. Enhancing the performance of convolu-tional neural networks on quality degraded datasets. 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA) 2017: 1-8. DOI: 10.1109/DICTA.2017.8227427.
  14. Hatami N, Gavet Y, Debayle J. Classification of time-series images using deep convolutional neural networks. Proc SPIE 2017; 10696: 106960Y. DOI: 10.1117/12.2309486.
  15. Wang Z, Oates T. Spatially encoding temporal correlations to classify temporal data using convolutional neural networks. Source: <https://arxiv.org/abs/1509.07481>.
  16. Wang Z, Oates T. Imaging time-series to improve classification and imputation. Proceedings of the 24th International Joint Conference on Artificial Intelligence 2015: 3939-3945.
  17. Wang Z, Oates T. Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. Association for the Advancement of Artificial Intelligence (AAAI) Conference 2015: 40-46.
  18. Lin J, Keogh E, Wei L, Lonardi S. Experiencing SAX: a novel symbolic representation of time series. Data Min Knowl Discov2007; 15(2): 107-144.
  19. Cho H, Ahn M, Ahn S, Kwon M, Jun SC. EEG datasets for motor imagery brain–computer interface. GigaScience 2007; 6(7): gix034. DOI: 10.1093/gigascience/gix034.
  20. Blankertz B, Müller KR, Curio G, Vaughan TM, Schalk G, Wolpaw JR, Schlögl A, Neuper C, Pfurtscheller G, Hinter-berger T, Schröder M, Birbaumer N. The BCI competition 2003: Progress and perspectives in detection and discrimination of EEG single trials. IEEE Transactions on Biomedical Engineering 2004; 6(51): 1044-1051.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: ko@smr.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20