(47-1) 17 * << * >> * Russian * English * Content * All Issues

Joint analysis of radiological reports and CT images for automatic validation of pathological brain conditions
Y.D. Agafonova 1, A.V. Gaidel 1,2, P.M. Zelter 3, A.V. Kapishnikov 3, A.V. Kuznetsov 1,4,5, E.N. Surovtsev 3, A.V. Nikonorov 1,2

1 Samara National Research University, 443086, Samara, Russia, Moskovskoye Shosse 34;
2 IPSI RAS – Branch of the FSRC "Crystallography and Photonics" RAS,
443001, Samara, Russia, Molodogvardeyskaya 151;
3 FSBEI HE SamSMU MOH Russia, 443099, Samara, Russia, Chapayevskaya 89;
4 Sber AI, 121170, Moscow, Russia, Kutuzovsky prospekt, 32 building 2;
5 Artificial Intelligence Research Institute (AIRI), 105064, Moscow, Russia, Nizhniy Susalnyy pereulok, 5

 PDF, 1372 kB

DOI: 10.18287/2412-6179-CO-1201

Pages: 152-159.

Full text of article: Russian language.

Abstract:
We consider a problem of validation of radiological medical reports and computed tomography images for an automated analysis of brain structures. Two methods for solving the problem are proposed: a method based on the ruCLIP multimodal model, and a method based on the joint use of two separate classifiers – for a text report and for a brain CT image. We discuss methods evaluation and the obtained results. The proposed approaches make it possible to correctly classify 99.6% of radiological reports from a test sampling into 15 possible diagnoses.

Keywords:
deep learning, computed tomography, computer-aided diagnosis, pattern recognition, natural language processing.

Citation:
Agafonova YD, Gaidel AV, Zelter PM, Kapishnikov AV, Kuznetsov AV, Surovtsev EN, Nikonorov AV. Joint analysis of radiological reports and CT images for automatic validation of pathological brain conditions. Computer Optics 2023; 47(1): 152-159. DOI: 10.18287/2412-6179-CO-1201.

Acknowledgements:
This work was supported by the Russian Science Foundation (Project No. 19-29-01235).

References:
  1. Yanase J, Triantaphyllou E. A systematic survey of computer-aided diagnosis in medicine: Past and present developments. Expert Syst Appl 2019; 138: 112821. DOI: 10.1016/j.eswa.2019.112821.
  2. Choplin RH, Boehme JM, Maynard CD, Picture archiving and communication systems: an overview. Radiographics 1992; 12(1): 127-129. DOI: 10.1148/radiographics.12.1.1734458.
  3. Zingmond D, Lenert L. Monitoring free-text data using medical language processing. Comput Biomed Res 1993; 26(5): 467-481. DOI: 10.1006/cbmr.1993.1033.
  4. Wahlang I, Maji AK, Saha G, Chakrabarti P, Jasinski M, Leonowicz Z, Jasinska E. Brain magnetic resonance imaging classification using deep learning architectures with gender and age. Sensors 2022; 22: 1766.
  5. Yagis E, Atnafu SW, de Herrera AG, et al. Effect of data leakage in brain MRI classification using 2D convolutional neural networks. Sci Rep 2021; 11: 22544. DOI: 10.1038/s41598-021-01681-w.
  6. Bala W, Steinkamp J, Feeney T, Gupta A, Sharma A, Kantrowitz J, Cordella N, Moses J, Draken FT. A web application for adrenal incidentaloma identification, tracking, and management using machine learning. Appl Clin Inform 2020; 11(4): 606-616. DOI: 10.1055/s-0040-1715892.
  7. Dantas R, Bertoldi M, Wangenheim F. An approach for retrieval and knowledge communication using medical documents. Proc 23rd Int Conf on Software Engineering and Knowledge Engineering (SEKE) 2011: 169-174.
  8. Sludnova A, Shutko V, Gaidel A, Zelter P, Kapishnikov A, Nikonorov A. Identification of pathological changes in the lungs using an analysis of radiological reports and tomographic images. Computer Optics 2021; 45(2): 261-266. DOI: 10.18287/2412-6179-CO-793.
  9. Wu Y, Mukunoki M, Funatomi T, Minoh M, Lao S. Optimizing mean reciprocal rank for person re-identification. 8th IEEE Int Conf on Advanced Video and Signal Based Surveillance (AVSS) 2011: 408-413. DOI: 10.1109/AVSS.2011.6027363.
  10. Chen CH, Lin PH, Hsieh JG, Cheng SL, Jeng JH. Robust multi-class classification using linearly scored categorical cross-entropy. 3rd IEEE Int Conf on Knowledge Innovation and Invention (ICKII) 2020: 200-203. DOI: 10.1109/ICKII50300.2020.9318835.
  11. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017; 60(6): 84-90. DOI: 10.1145/3065386.
  12. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997; 9(8): 1735-1780. DOI: 10.1162/neco.1997.9.8.1735.
  13. Shonenkov A, Kuznetsov A, Dimitrov D, Shavrina T, Chesakov D, Maltseva A, Fenogenova A, Pavlov I, Emelyanov A, Markov S, Bakshandaeva D, Shybaeva V, Chertok A. RuCLIP – new models and experiments: a technical report. arXiv Preprint. 2022. Source: <https://arxiv.org/abs/2202.10784>.
  14. Radford A, Kim JW, Hallacy C, Ramesh A, Goh G, Agarwal S, Sastry G, Askell A, Mishkin P, Clark J, Krueger G. Learning transferable visual models from natural language supervision. Int Conf on Machine Learning 2021; 139: 8748-8763.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20