(46-6) 16 * << * >> * Russian * English * Content * All Issues

Fine-tuning the hyperparameters of pre-trained models for solving multiclass classification problems
D. Kaibassova 1, М. Nurtay 1, А. Tau 1, М. Kissina 1

1 Abylkas Saginov Karaganda technical university,
100000, Kazakhstan, Karaganda city, 56 N. Nazarbayev avenue

 PDF, 1967 kB

DOI: 10.18287/2412-6179-CO-1078

Pages: 971-979.

Full text of article: English language.

Abstract:
This study is devoted to the application of fine-tuning methods for Transfer Learning models to solve the multiclass image classification problem using the medical X-ray images. To achieve this goal, the structural features of such pre-trained models as VGG-19, ResNet-50, InceptionV3 were studied. For these models, the following fine-tuning methods were used: unfreezing the last convolutional layer and updating its weights, selecting the learning rate and optimizer. As a dataset chest X-Ray images of the Society for Imaging Informatics in Medicine (SIIM), as the leading healthcare organization in its field, in partnership with the Foundation for the Promotion of Health and Biomedical Research of Valencia Region (FISABIO), the Valencian Region Medical ImageBank (BIMCV) ) and the Radiological Society of North America (RSNA) were used. Thus, the results of the experiments carried out illustrated that the pre-trained models with their subsequent tuning are excellent for solving the problem of multiclass classification in the field of medical image processing. It should be noted that ResNet-50 based model showed the best result with 82.74 % accuracy. Results obtained for all models are reflected in the corresponding tables.

Keywords:
multiclass classification, transfer learning, fine-tuning, CNN, image augmentation, X-ray.

Citation:
Kaibassova D, Nurtay М, Таu А, Кissina М. Fine-tuning the hyperparameters of pre-trained models for solving multiclass classification problems. Computer Optics 2022; 46(6): 971-979. DOI: 10.18287/2412-6179-CO-1078.

Acknowledgements:
Text.

References:
  1. Narin A, Kaya C, Pamuk Z. Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks. Pattern Anal Appl 2021; 24: 1207-1220.
  2. Narayanan BN, Hardie RC, Krishnaraja V, Karam C, Davuluru VSP. Transfer-to-transfer learning approach for computer aided detection of COVID-19 in Chest Radiographs. AI 2020; 1(4): 539-557.
  3. Tahir H, Khan MS, Tariq MO. Performance analysis and comparison of Faster R-CNN, Mask R-CNN and ResNet50 for the detection and counting of vehicles. Proc 2021 Int Conf on Computing, Communication, and Intelligent Systems (ICCCIS), Greater Noida, India, 19-20 February 2021: 587-594.
  4. Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng 2010; 22: 1345-1359.
  5. Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A survey on deep transfer learning – ICANN 2018. In Book: Kůrková V, Manolopoulos Y, Hammer B, Iliadis L, Maglogiannis I, eds. Artificial neural networks and machine learning – ICANN 2018. Springer; 2018: 270-279.
  6. Rifkin R, Klautau A. In defense of one-vs-all classification. J Mach Learn Res 2004; 5: 101-141.
  7. Ikechukwu AV, Murali S, Deepu R, Shivamurthy RC. ResNet-50 vs VGG-19 vs training from scratch: A comparative analysis of the segmentation and classification of Pneumonia from chest X-ray images. Global Transitions Proceedings 2021; 2(2): 375-381. DOI: 10.1016/j.gltp.2021.08.027.
  8. Setiawan W, Damayanti F. Layers modification of convolutional neural network for pneumonia detection. J Phys Conf Ser 2020; 1477: 052055. DOI: 10.1088/1742-6596/1477/5/052055.
  9. Gouabou ACF, et al. Ensemble method of convolutional neural networks with directed acyclic graph using dermoscopic images: Melanoma detection application. Sensors 2021; 21(12): 3999.
  10. Anilkumar KK, Manoj VJ, Sagi TM. Automated detection of leukemia by pretrained deep neural networks and transfer learning: A comparison. Med Eng Phys 2021; 98: 8-19. DOI: 10.1016/j.medengphy.2021.10.006.
  11. Liu Y, Yang P, Pi Y, et al. Automatic identification of suspicious bone metastatic lesions in bone scintigraphy using convolutional neural network. BMC Med Imaging 2021; 21: 131. DOI: 10.1186/s12880-021-00662-9.
  12. Zhou J, Yang X, Zhang L, Shao S, Bian G. Multisignal VGG19 network with transposed convolution for rotating machinery fault diagnosis based on deep transfer learning. Shock Vib 2020; 2020: 8863388. DOI: 10.1155/2020/8863388.
  13. Fengzi L, Kant S, Araki S, Bangera S, Shukla SS. Neural networks for fashion image classification and visual search. SSRN Electronic Journal 2020. DOI: 10.2139/ssrn.3602664.
  14. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016 IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
  15. Szegedy C, et al. Rethinking the inception architecture for computer vision. Proc IEEE conf on computer vision and pattern recognition 2016: 2818-2826. DOI: 10.1109/CVPR.2016.308.
  16. Pashina TA, Gaidel AV, Zelter PM, Kapishnikov AV, Nikonorov AV. Automatic highlighting of the region of interest in computed tomography images of the lungs. Computer Optics 2020; 44(1): 74-81. DOI: 10.18287/2412-6179-CO-659.
  17. Kaibassova D, La L, Smagulova A, et al. Methods and algorithms of analyzing syllabuses for educational programs forming intellectual system. J Theor Appl Inf Technol 2020; 98(5): 876-888.
  18. Nurtay M, Kaibassova D. Methods of filtering medical images for solving the segmentation problem. Youth and Modern Information Technologies. Proc XVIII Int Scientific and Practical Conf of Students, Postgraduates and Young Scientists. Tomsk: Tomsk Polytechnic University Publisher; 2021: 34-35.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20