(50-5) 13 * << * >> * Russian * English * Content * All Issues

Preventing Overfitting of Partial Least Squares – Discriminant Analysis and Principal Component Analysis – Linear Discriminant Analysis Models for Classification of Raman Spectra of Skin Cancer, Trained on a Small Data Sample
D.V. Vlasova1, I.A. Bratchenko2

1 Samara National Research University, Moskovskoye Shosse 34, Samara, 443086, Russia;
2 Immanuel Kant Baltic Federal University, 14 A. Nevskogo Street, Kaliningrad, 236041, Russia

  Full text (PDF)

DOI: 10.18287/COJ1721

Article ID: 1721

Language: Russian

Abstract:
Modern medical diagnostics actively implements optical spectroscopy methods, among which Raman spectroscopy occupies a special place. This technology demonstrates exceptional sensitivity to the smallest molecular changes in biological tissues, which makes it a promising tool for non-invasive detection of malignant skin tumors. In research practice, machine learning algorithms are widely used to process spectral data, but their implementation often does not take into account the specifics of spectroscopic information, which leads to the creation of overfitted "black-box" models.

This study presents a method for training a classifier on a small data sample, which guarantees its stability and high prediction accuracy. The presented work considers a method for constructing a classifier based on two methods: PLS-DA (Least Squares Discriminant Analysis) and PCA-LDA (Principal Component Analysis - Linear Discriminant Analysis), trained on a limited sample of Raman spectra. The study focuses on differentiating two most common types of malignant skin lesions - basal cell carcinoma and melanoma. A comprehensive analysis, including mean square error estimation, cross-validation procedures, visualization of scores and loadings, and calculation of classification accuracy, allowed us to determine the optimal complexity of models and the optimal number of their principal components for robust classifier prediction.

Keywords:
Partial Least Squares Discriminant Analysis, Principal Component Analysis Linear Discriminant Analysis, principal components, overfitting of machine learning models, score, loading, cross-validation.

Citation:
Vlasova DV, Bratchenko IA. Preventing Overfitting of Partial Least Squares Discriminant Analysis and Principal Component Analysis – Linear Discriminant Analysis Models for Classification of Raman Spectra of Skin Cancer, Trained on a Small Data Sample. Computer Optics 2026; 50(5): 1721. doi: 10.18287/COJ1721.

References:

  1. Abdalla BMZ, Abdalla CMZ. Epidemiology of skin cancer. In: Abdalla CMZ, ed. Oncodermatology. Cham: Springer; 2023: 29-37. DOI: 10.1007/978-3-031-29277-4_3.
  2. Lim L, Nichols BS, Migden MR, Rajaram N, Reichenberg J, Markey MK, Ross MI, Tunnell JW. Clinical study of noninvasive in vivo melanoma and nonmelanoma skin cancers using multimodal spectral diagnosis. Journal of Biomedical Optics 2014; 19(11): 117003. DOI: 10.1117/1.JBO.19.11.117003.
  3. Bratchenko IA, Alonova MV, Myakinin OO, Moryatov AA, Kozlov SV, Zakharov VP. Hyperspectral visualization of skin pathologies in the visible region. Computer Optics 2016; 40(2): 240-248. DOI: 10.18287/2412-6179-2016-40-2-240-248.
  4. Khristoforova YA, Bratchenko IA, Myakinin OO, Artemyev DN, Moryatov AA, Orlov AE, Kozlov SV, Zakharov VP. Portable spectroscopic system for in vivo skin neoplasms diagnostics by Raman and autofluorescence analysis. Journal of Biophotonics 2019; 12(4): e201800400. DOI: 10.1002/jbio.201800400.
  5. Vestergaard ME, Macaskill P, Holt PE, Menzies SW. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: A meta-analysis of studies performed in a clinical setting. British Journal of Dermatology 2008; 159(3): 669-676. DOI: 10.1111/j.1365-2133.2008.08713.x.
  6. Vinokurov VO, Matveeva IA, Khristoforova YA, Myakinin OO, Bratchenko IA, Bratchenko LA, Moryatov AA, Kozlov SV, Machikhin AS, Abdulhalim I, Zakharov VP. Neural network classifier of hyperspectral images of skin pathologies. Computer Optics 2021; 45(6): 879-886. DOI: 10.18287/2412-6179-CO-832.
  7. Shipp DW, Sinjab F, Notingher I. Raman spectroscopy: Techniques and applications in the life sciences. Advances in Optics and Photonics 2017; 9(2): 315-428. DOI: 10.1364/AOP.9.000315.
  8. Sahu A, Deshmukh A, Hole AR, Chaturvedi P, Krishna CM. In vivo subsite classification and diagnosis of oral cancers using Raman spectroscopy. Journal of Innovative Optical Health Sciences 2016; 9(5): 1650017. DOI: 10.1142/S1793545816500176.
  9. Wu M, Gao B, Wei X. Recent advances in Raman spectroscopy for skin diagnosis. Journal of Innovative Optical Health Sciences 2023; 16(3): 2330003. DOI: 10.1142/S1793545823300033.
  10. Al-Sammarraie SZ, Bratchenko LA, Tupikova EN, Skuratova MA, Wang S, Lebedev PA, Bratchenko IA. Human blood plasma SERS analysis using silver nanoparticles for cardiovascular diseases detection. Journal of Biomedical Photonics & Engineering 2024; 10(1): 010301. DOI: 10.18287/JBPE24.10.010301.
  11. Tu Q, Chang C. Diagnostic applications of Raman spectroscopy. Nanomedicine: Nanotechnology, Biology and Medicine 2012; 8(5): 545-558. DOI: 10.1016/j.nano.2011.09.013.
  12. Das RS, Agrawal YK. Raman spectroscopy: Recent advancements, techniques and applications. Vibrational Spectroscopy 2011; 57(2): 163-176. DOI: 10.1016/j.vibspec.2011.08.003.
  13. Qi Y, Chen EX, Hu D, Yang Y, Wu Z, Zheng M, Sadi MA, Jiang Y, Zhang K, Chen Z, Chen YP. Applications of Raman spectroscopy in clinical medicine. Food Frontiers 2024; 5(2): 392-419. DOI: 10.1002/fft2.335.
  14. Orlando A, Franceschini F, Muscas C, Pidkova S, Bartoli M, Rovere M, Tagliaferro A. A comprehensive review on Raman spectroscopy applications. Chemosensors 2021; 9(9): 262. DOI: 10.3390/chemosensors9090262.
  15. Le Mezo-Burtin C. Industrial Application of the Raman-Laser-Fiber Optics Method of Analysis of Sugars in Fruit. Doctoral dissertation. France; 1989.
  16. Yang D, Ying Y. Applications of Raman spectroscopy in agricultural products and food analysis: A review. Applied Spectroscopy Reviews 2011; 46(7): 539-560. DOI: 10.1080/05704928.2011.593216.
  17. Jin H, Lu Q, Chen X, Ding H, Gao H, Jin S. The use of Raman spectroscopy in food processes: A review. Applied Spectroscopy Reviews 2016; 51(1): 12-22. DOI: 10.1080/05704928.2015.1087404.
  18. Petersen M, Yu Z, Lu X. Application of Raman spectroscopic methods in food safety: A review. Biosensors 2021; 11(6): 187. DOI: 10.3390/bios11060187.
  19. Zheng J, He L. Surface-enhanced Raman spectroscopy for the chemical analysis of food. Comprehensive Reviews in Food Science and Food Safety 2014; 13(3): 317-328. DOI: 10.1111/1541-4337.12062.
  20. Xiao L, Feng S, Lu X. Raman spectroscopy: Principles and recent applications in food safety. Advances in Food and Nutrition Research 2023; 106: 1-29. DOI: 10.1016/bs.afnr.2023.03.007.
  21. Yaseen T, Sun DW, Cheng JH. Raman imaging for food quality and safety evaluation: Fundamentals and applications. Trends in Food Science & Technology 2017; 62: 177-189. DOI: 10.1016/j.tifs.2017.01.012.
  22. Scotter CNG. Non-destructive spectroscopic techniques for the measurement of food quality. Trends in Food Science & Technology 1997; 8(9): 285-292. DOI: 10.1016/S0924-2244(97)01053-4.
  23. Fontecha J, Bellanato J, Juarez M. Infrared and Raman spectroscopic study of casein in cheese: Effect of freezing and frozen storage. Journal of Dairy Science 1993; 76(11): 3303-3309. DOI: 10.3168/jds.S0022-0302(93)77667-5.
  24. Vankeirsbilck T, Vercauteren A, Baeyens W, Van der Weken G, Verpoort F, Vergote G, Remon JP. Applications of Raman spectroscopy in pharmaceutical analysis. Trends in Analytical Chemistry 2002; 21(12): 869-877. DOI: 10.1016/S0165-9936(02)01208-6.
  25. Kalantri PP, Somani RR, Makhija DT. Raman spectroscopy: A potential technique in analysis of pharmaceuticals. Der Chemica Sinica 2010; 1(1): 1-12.
  26. Silge A, Weber K, Cialla-May D, Müller-Bötticher L, Fischer D. Trends in pharmaceutical analysis and quality control by modern Raman spectroscopic techniques. Trends in Analytical Chemistry 2022; 153: 116623. DOI: 10.1016/j.trac.2022.116623.
  27. Wartewig S, Neubert RHH. Pharmaceutical applications of mid-IR and Raman spectroscopy. Advanced Drug Delivery Reviews 2005; 57(8): 1144-1170. DOI: 10.1016/j.addr.2005.01.022.
  28. Esmonde-White KA, Cuellar M, Uerpmann C, Lenain B, Lewis IR. Raman spectroscopy as a process analytical technology for pharmaceutical manufacturing and bioprocessing. Analytical and Bioanalytical Chemistry 2017; 409(3): 637-649. DOI: 10.1007/s00216-016-9824-1.
  29. Pinzaru SC, Pavel IE. SERS and pharmaceuticals. In: Pinzaru SC, ed. Surface-Enhanced Raman Spectroscopy: Analytical, Biophysical and Life Science Applications. Weinheim: Wiley-VCH; 2010: 129-154. DOI: 10.1002/9783527632756.
  30. Bratchenko IA, Bratchenko LA. Comment on “Combining derivative Raman with autofluorescence to improve the diagnosis performance of echinococcosis”. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy 2021; 252: 119514. DOI: 10.1016/j.saa.2021.119514.
  31. Khristoforova YA, Bratchenko LA, Skuratova MA, Lebedeva EA, Lebedev PA, Bratchenko IA. Raman spectroscopy in chronic heart failure diagnosis based on human skin analysis. Journal of Biophotonics 2023; 16(7): e202300016. DOI: 10.1002/jbio.202300016.
  32. Liland KH, Kohler A, Afseth NK. Model-based pre-processing in Raman spectroscopy of biological samples. Journal of Raman Spectroscopy 2016; 47(6): 643-650. DOI: 10.1002/jrs.4886.
  33. Monici M. Cell and tissue autofluorescence research and diagnostic applications. Biotechnology Annual Review 2005; 11: 227-256. DOI: 10.1016/S1387-2656(05)11007-2.
  34. Ferraro JR, Nakamoto K. Special techniques. In: Ferraro JR, ed. Introductory Raman Spectroscopy. 2nd ed. Elsevier; 2003: 147-202.
  35. Hoskuldsson A. PLS regression methods. Journal of Chemometrics 1988; 2(3): 211-228. DOI: 10.1002/cem.1180020306.
  36. Lee LC, Liong CY, Jemain AA. Partial least squares-discriminant analysis (PLS-DA) for classification of high-dimensional data: A review of contemporary practice strategies and knowledge gaps. Analyst 2018; 143(15): 3526-3539. DOI: 10.1039/C8AN00599K.
  37. Ruiz-Perez D, Guan H, Madhivanan P, Mathee K, Narasimhan G. So you think you can PLS-DA? BMC Bioinformatics 2020; 21: 1-10. DOI: 10.1186/s12859-019-3310-7.
  38. Chevallier S, Bertrand D, Kohler A, Courcoux P. Application of PLS-DA in multivariate image analysis. Journal of Chemometrics 2006; 20(5): 221-229. DOI: 10.1002/cem.994.
  39. Bratchenko LA, Bratchenko IA, Khristoforova YA, Artemyev DN, Konovalova DY, Lebedev PA, Zakharov VP. Raman spectroscopy of human skin for kidney failure detection. Journal of Biophotonics 2021; 14(2): e202000360. DOI: 10.1002/jbio.202000360.
  40. Madden MG, Ryder AG. Machine learning methods for quantitative analysis of Raman spectroscopy data. In: Proceedings of SPIE 4876, Opto-Ireland 2002: Optics and Photonics Technologies and Applications. SPIE; 2003: 1130-1139. DOI: 10.1117/12.464039.
  41. Kongklad G, Chitaree R, Taechalertpaisarn T, Panvisavas N, Nuntawong N. Discriminant analysis PCA-LDA assisted surface-enhanced Raman spectroscopy for direct identification of malaria-infected red blood cells. Methods and Protocols 2022; 5(3): 49. DOI: 10.3390/mps5030049.
  42. Anowar F, Sadaoui S, Selim B. Conceptual and empirical comparison of dimensionality reduction algorithms: PCA, KPCA, LDA, MDS, SVD, LLE, ISOMAP, LE, ICA, and t-SNE. Computer Science Review 2021; 40: 100378. DOI: 10.1016/j.cosrev.2021.100378.
  43. Sharma A, Paliwal KK. Linear discriminant analysis for the small sample size problem: An overview. International Journal of Machine Learning and Cybernetics 2015; 6(3): 443-454. DOI: 10.1007/s13042-013-0226-9.
  44. Pichardo-Molina JL, Frausto-Reyes C, Barbosa-Garcia O, Huerta-Franco R, Gonzalez-Trujillo JL, Ramirez-Alvarado CA, Gutierrez-Juarez G, Medina-Gutierrez C. Raman spectroscopy and multivariate analysis of serum samples from breast cancer patients. Lasers in Medical Science 2007; 22(4): 229-236. DOI: 10.1007/s10103-006-0432-8.
  45. Sahu A, Sawant S, Mamgain H, Krishna CM. Raman spectroscopy of serum: An exploratory study for detection of oral cancers. Analyst 2013; 138(14): 4161-4174. DOI: 10.1039/C3AN00308F.
  46. Liu W, Sun Z, Chen J, Jing C. Raman spectroscopy in colorectal cancer diagnostics: Comparison of PCA-LDA and PLS-DA models. Journal of Spectroscopy 2016; 2016: 1603609. DOI: 10.1155/2016/1603609.
  47. Berrar D. Cross-validation. 2019. Available at: https://www.researchgate.net/profile/Daniel-Berrar/publication/381674773_Cross-Validation/links/6791b19e82501639f5063948/Cross-Validation.pdf (accessed 24.08.2026).
  48. Guo S, Bocklitz T, Neugebauer U, Popp J. Common mistakes in cross-validating classification models. Analytical Methods 2017; 9(30): 4410-4417. DOI: 10.1039/C7AY01363A.
  49. Fushiki T. Estimation of prediction error by using K-fold cross-validation. Statistics and Computing 2011; 21(2): 137-146. DOI: 10.1007/s11222-009-9153-8.
  50. Junjuri R, Calvarese M, Meyer-Zedler T, et al. Estimation of biological variance in coherent Raman microscopy data of two cell lines using chemometrics. Analyst 2024; 149(17): 4395-4406. DOI: 10.1039/D4AN00648H.

151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20