Methods of two-dimensional projection of digital images into eigen-subspaces: peculiarities of implementation and application
Kukharev G.A., Shchegoleva N.L.

 

Electrotechnical University "LETI" (ETU), Saint-Petersburg, Russia

 PDF

Abstract:
The history of development of algorithms for projection of digital images into their eigensubspaces using linear methods based on PCA (principal component analysis), LDA (linear discriminant analysis), PLS (partial least squares), and CCA (canonical correlation analysis) is considered. We show that the emergence of new application areas has changed  the requirements for the methods, with major changes involving (i) the use of PCA, LDA, PLS and CCA methods for both small and extremely large face image (FI) samples in the initial sets; (ii) a criterion for determining the eigen-basis, which also should provide the solution of a particular problem (the minimum error of face image approximation, etc.); (iii) the applicability of the methods under consideration to the processing of two or more image sets from different sensors or several sets of any number of matrices; and (iv) the possibility of realizing two-dimensional projections of face images (or other numerical matrices) directly into the layers of convolutional neural networks (NN) and/or integrating their functions into the NN as separate blocks. Estimates of the computational complexity and examples of solving image processing problems are also given.

Keywords:
face image (FI) sets and numeric matrices, an eigen-basis and eigensubspaces, principal components analysis (PCA), linear discriminant analysis (LDA), partial least squares (PLS), canonical correlation analysis (CCA), Karhunen-Loeve transformation (KLT), 2DPCA/2DKLT, 2DPLS/2DKLT, 2DCCA/2DKLT, CNN, Deep NN.

Citation:
Kukharev GA, Shchegoleva NL. Methods of two-dimensional projection of digital images into eigen-subspaces: peculiarities of implementation and application. Computer Optics 2018; 42(4): 637-656. DOI: 10.18287/2412-6159-2018-42-4-637-656.

References:

  1. Pearson K. On lines and planes of closest fit to systems of points in space. The London, Edinburgh and Dublin Philosophical Magazine and Journal of Sciences 1901; 6(2): 559-572. DOI: 10.1080/14786440109462720.
  2. Hoteling H. Analysis of complex variables into principal components. Journal of Educational Psychology 1933; 24(6): 415-441. DOI: 10.1037/h0071325.
  3. Fisher RA. The use of multiple measurements in taxonomic problems. Annals of Eugenics 1936; 7(2): 159-188. DOI: 10.1111/j.1469-1809.1936.tb02137.x.
  4. Hoteling H. Relations between two sets of variates. Biometryka 1936; 28(3/4): 321-377. DOI: 10.2307/2333955.
  5. Sirovich L, Kirby M. Low-dimensional procedure for the characterization of human faces. J Opt Soc Am A 1987; 4(3): 519-524. DOI: 10.1364/JOSAA.4.000519.
  6. Turk М, Pentland A. Eigenfaces for recognition. Journal of Cognitive Neuroscience 1991; 3(1): 71-86. DOI: 10.1162/jocn.1991.3.1.71.
  7. Belhumeur PN, Hespanha JP, Kriegman DJ. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997; 19(7): 711-720. DOI: 10.1109/34.598228.
  8. Tsapatsoulis N, Alexopoulos V, Kollias S. A vector based approximation of KLT and its application to face recognition. Proc EUSIPCO-98 1998; III: 1581-1584. DOI: 10.5281/zenodo.36612.
  9. Kukharev G. Biometric systems: Methods and means for people recognition [In Russian]. Sankt-Petersburg: “Politehnika” Publisher; 2001. ISBN: 5-7325-0623-3.
  10. Kukharev G, Kuzminski A. Techniki biometryczne: Czesc 1. Metody rozpoznawania twarzy. Szczecin: „Pracownia Poligraficzna WI PS” Publisher, 2003.
  11. Kukharev G, Forczmanski P. Data dimensionality reduction for face recognition. Machine GRAPHICS & VISION 2004; 13(1/2): 99-121.
  12. Kukharev G, Forczmanski P. Face recognition by means of two-dimensional direct linear discriminant analysis. Proc PRIP’2005: 280-283.
  13. Kukharev GA, Shchegoleva NL. Human face recognition systems [In Russian]. Saint-Petersburg: Saint-Petersburg Electrotechnical University “LETI” Publisher; 2006. ISBN: 5-7629-0665-5.
  14. Kukharev G, Tujaka A, Binh N. System of face recognition using LDA with one training image per person. Metody Informatyki Stosowanej 2008; 3(16): 167-185.
  15. Yang J, Zhang D, Frangi AF, Yang J-Y. Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 2004; 26(1): 131-137. DOI: 10.1109/TPAMI.2004.1261097.
  16. Zhang D, Zhou ZH. (2D)2PCA: Two-directional two-dimensional PCA for efficient face representation and recognition. Neurocomputing 2005; 69(1-3): 224-231. DOI: 10.1016/j.neucom.2005.06.004.
  17. Ye J. Generalized low rank approximations of matrices. Machine Learning 2005; 61(1-3): 167-191. DOI: 10.1007/s10994-005-3561-6.
  18. Kong H, Wang L, Teoh EK, Li X, Wang J-G, Venkateswarlu R. Generalized 2D principal component analysis for face image representation and recognition. Neural Networks 2005; 18(5-6): 585-594. DOI: 10.1016/j.neunet.2005.06.041.
  19. Ding ChHQ, Ye J. Two-dimensional singular value decomposition (2DSVD) for 2D maps and images. Proc 2005 SIAM International Conference on Data Mining 2005.
  20. Gu Z, Lin W, Lee B-S, Lau CT, Paul M. Two-dimensional singular value decomposition (2D-SVD) based video coding. IEEE International Conference on Image Processing 2010: 181-184. DOI: 10.1109/ICIP.2010.5650998.
  21. Gurumoorthy KS, Rajwade A, Banerjee A, Rangarajan A. A method for compact image representation using sparse matrix and tensor projections onto exemplar orthonormal bases // IEEE Transactions on Image Processing 2010; 19(2): 322-334. DOI: 10.1109/TIP.2009.2034991.
  22. Inoue K, Urahama K. Equivalence of non-iterative algorithms for simultaneous low rank approximations of matrices. CVPR'06 2006. DOI: 10.1109/CVPR.2006.112.
  23. Tang X, Wang X. Face sketch recognition. IEEE Transactions on Circuits and Systems for Video Technology 2004; 14(1): 50-57. DOI: 10.1109/TCSVT.2003.818353.
  24. CUHK Face Sketch Database (CUFS). Source: áhttp://mmlab.ie.cuhk.edu.hk/archive/facesketch.htmlñ.
  25. Kukharev G, Oleinik A. Face photo-sketch transformation and population generation. In Book: Chmielewski L, Datta A, Kozera R, Wojciechowski K, eds. Computer vision and graphics. ICCVG 2016: Computer vision and graphics 2016: 329-340. DOI: 10.1007/978-3-319-46418-3_29.
  26. Borga M. Learning multidimensional signal processing. Linköping, Sweden: Linköpings Universitet; 1998. ISBN: 91-7219-202-X.
  27. Reiter M, Donner R, Langs G, Bischof H. 3D and infrared face reconstruction from rgb data using canonical correlation analysis. Proc 18th Int Conf Patt Recogn (ICPR 2006) 2006; 1: 425-428. DOI: 10.1109/ICPR.2006.24.
  28. Reiter M, Donner R. Estimation of face depth maps from color textures using canonical correlation analysis. In Book: Chum O, Franc V, eds. Proceedings of the Computer Vision Winter Workshop 2006 (CWW' 06). Telc: Czech Society for Cybernetics and Informatics; 2006. ISBN: 80-239-6530-1.
  29. Donner R, Reiter M, Langs G, Peloschek P, Bischof H. Fast active appearance model search using canonical correlation analysis. IEEE Trans Pattern Anal Mach Intell 2006; 28(10): 1960-1964. DOI: 10.1109/TPAMI.2006.206.
  30. Alonso J, Zepeda Y, Davoine F, Charbit M. Face tracking using canonical correlation analysis. Proc VISAPP 2007 2007; 2: 396-402.
  31. Sharma A, Jacobs DW. Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch. CVPR 2011: 593-600. DOI: 10.1109/CVPR.2011.5995350.
  32. Lee SH, Choi S. Two-dimensional canonical correlation analysis. IEEE Signal Processing Letters 2007; 14(10): 735-738. DOI: 10.1109/LSP.2007.896438.
  33. Zou C-R, Sun N, Ji Z-H, Li Z. 2DCCA: a novel method for small sample size face recognition. WACV’07 2007. DOI: 10.1109/WACV.2007.1.
  34. Shao M, Wang Y. Joint features for face recognition under variable illuminations. Fifth International Conference on Image and Graphics 2009: 922-927. DOI: 10.1109/ICIG.2009.128.
  35. Gong X. Application to three-dimensional canonical correlation analysis for feature fusion in image recognition. Journal of Computers 2011; 6(11): 2427-2433.
  36. Kamencay P, Hudec R, Benco M, Zachariasova M. 2D-3D face recognition method based on a modified CCA-PCA algorithm. International Journal of Advanced Robotic Systems 2014; 11(3). DOI: 10.5772/58251.
  37. Kukharev G, Kamenskaya E. Two-dimensional canonical correlation analysis for face image processing and recognition. Metody Informatyki Stosowanej 2009; 3(21): 103-112.
  38. Kukharev G, Tujaka A, Forczmanski P. Face recognition using two-dimensional CCA and PLS. International Journal of Biometrics 2011; 3(4): 300-321. DOI: 10.1504/IJBM.2011.042814.
  39. Kukharev G, Kamenskaya E, Matveev Y, Shchegoleva N. Methods of facial images processing and recognition in biometrics [In Russian]. Saint-Peterburg: “Politechnika” Publisher; 2013. ISBN: 978-5-73251-028-7.
  40. Why dogs look like hosts [In Russian]. Source: áhttp://www.house-dog.ru/about_391.htmlñ.
  41. Gupta S, Markey MK, Castleman KR, Bovik AC. Texas 3D face recognition database. Source: áhttp://live.ece.utexas.edu/research/texas3dfr /index.htmñ.
  42. Tu C-T, Ho M-C, Lin M-Y. A new approach for face hallucination based on a two-dimensional direct combined model. Patt Recogn 2017; 62: 1-20. DOI: 10.1016/j.patcog.2016.07.020.
  43. An L, Bhanu B. Face image super-resolution using 2D CCA. Signal Processing 2014; 103: 184-194. DOI: 10.1016/j.sigpro.2013.10.004.
  44. Hou S, Sun Q, Xia D. A two-dimensional partial least squares with application to biological image recognition. 2010 Sixth International Conference on Natural Computation 2010: 57-61. DOI: 10.1109/ICNC.2010.5583135.
  45. Velkov VV. Multidimensional biology and multidimensional medicine [In Russian]. Chemistry and Life 2007; 3: 10-15.
  46. Meng C, Zeleznik OA, Thallinger GG, Kuster B, Gholami AM, Culhane AC. Dimension reduction techniques for the integrative analysis of multi-omics data. Briefings in Bioinformatics 2016; 17(4): 628-641. DOI: 10.1093/bib/bbv108.
  47. Qiu J, Wang H, Lu J, Zhang B, Du K-L. Neural network implementations for PCA and its extensions. ISRN Artificial Intelligence 2012; 2012: 847305. DOI: 10.5402/2012/847305.
  48. Chan T-H, Jia K, Gao S, Lu J, Zeng Z, Ma Y. PCANet: A Simple Deep Learning Baseline for Image Classification? IEEE Transactions on Image Processing 2015; 24(12): 5017-5032. DOI: 10.1109/TIP.2015.2475625.
  49. Tian L, Fan C, Ming Y. Multiple scales combined principle component analysis deep learning network for face recognition. J Electron Imaging 2016; 25(2): 023025. DOI: 10.1115/1.JEI.25.2.023025.
  50. Hasegawa R, Hotta K. PLSNet: A simple network using Partial Least Squares regression for image classification. Proc 23rd ICPR 2016: 1601-1606. DOI: 10.1109/ICPR.2016.7899865.
  51. Andrew G, Arora R, Bilmes J, Livescu K. Deep canonical correlation analysis. Proc 30th ICMP: PMLR 2013; 28(3): 1247-1255.
  52. Benton A, Khayrallah H, Gujral B, Reisinger D, Zhang Sh, Arora R. Deep generalized canonical correlation analysis. Source: áarXiv:1502.02519v2ñ.
  53. Kukharev GA, Shchegoleva NL. Algorithms of two-dimensional projection of digital images in eigensubspace: History of development, implementation and application // Pattern Recognition and Image Analysis 2018; 28(2): 185-206. DOI: 10.1134/S1054661818020116.
© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20