Face photo retrieval based on sketches
G.A. Kukharev, N.L. Shchegoleva

 

West Pomeranian University of Technology, Szczecin, Poland,
Saint Petersburg Electrotechnical University "LETI", St. Petersburg, Russia

Full text of article: English language.

Abstract:

The paper deals with the problem of the automatic retrieval of face photos using sketch drawings based on the witness description. We propose new methods for the generation of a sketch population from the initial one to improve the performance of sketch-based photo image retrieval systems. The method based on the computation of an average sketch from the generated population has been applied to increase the index of similarity in sketch-photo pairs. It is shown that such sketches are more similar to the original photographic images and their use leads to good results. Results of the experiments on CUHK Face Sketch and CUHK Face Sketch FERET databases and open access databases of photo-sketches pairs are discussed.

Keywords:
photo-sketch retrieval, population of sketches.

Citation:
Kukharev GA, Shchegoleva NL. Face photo retrieval based on sketches.Computer Optics 2016; 40(5): 729-739. DOI: 10.18287/2412-6179-2016-40-5-729-739.

References:

  1. Uhl R, Lobo N da V. A framework for recognizing a facial image from a police sketch, Proc IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, Jun 18-20, 1996: 586-593. DOI: 10.1109/CVPR.1996.517132.
  2. Konen W. Comparing facial line drawings with gray-level images: A case study on PHANTOMAS. Proc International Conference on Artificial Neural Networks, Bochum, Germany, Jul 16-19, 1996: 727-734. DOI: 10.1007/3-540-61510-5_123.
  3. Identi-Kit, Identi-Kit Solutions. Source: <http://www.iden­tikit.net>.
  4. FACES 4.0. Source: <http://www.iqbiometrix.com>.
  5. Yuen PCA, Man CH. Human face image searching system using sketch. Proc Workshop on Machine Vision Applications (MVA2002), Nara-ken New Public Hall, Nara, Japan, Dec 11-13, 2002: 500-503.
  6. Tang X, Wang X. Face photo-sketch synthesis and recognition. Proc 9th IEEE International Conference on Computer Vision, Nice, France, Oct 13-16, 2003; 1: 687-694. DOI: 10.1109/ICCV.2003.1238414.
  7. Tang X, Wang X. Face photo-sketch synthesis and recognition. IEEE Transactions on PAMI 2009; 31(11): 1955-1967. DOI: 10.1109/TPAMI.2008.222.
  8. Klare BF, Li Z, Jain AK. Matching forensic sketches to mug shot photos. IEEE Transactions on PAMI 2011; 33(3): 639-646. DOI: 10.1109/TPAMI.2010.180.
  9. Han H, Klare BF, Bonnen K, Jain AK. Matching composite sketches to face photos: a component-based approach. IEEE Transactions on Information Forensics and Security 2013: 8(1): 191-204. DOI: 10.1109/TIFS.2012.2228856.
  10. Davies GM, Valentine T. Facial composites: forensic utility and psychological research. In: Rod CL, Ross DF, Read JD, Toglia MP, eds. Handbook of eyewitness psychology. Vol II: Memory for people. Mahwah, NJ: Lawrence Erlbaum Associates Publishers; 2007; xii: 59-83.
  11. Gibson SJ, Solomon CJ, Pallares-Bejarano A. Synthesis of photographic quality facial composites using evolutionary algorithms. In: Proceedings of the British Machine Vision Conference 2003, Sep 9-11. Norwich, UK: BMVA Press, University of East Anglia; 2003: 221-230.
  12. Frowd CВ, Hancock PJB, Carson D. EvoFIT: A holistic, evolutionary facial imaging technique for creating composites. ACM Transactions on Applied Perceptions 2004; 1(1): 19-39. DOI: 10.1145/1008722.1008725.
  13. George B, Gibson SJ, Maylin MIS, Solomon CJ. EFIT-V – Interactive evolutionary strategy for the construction of photo-realistic facial composites. Proceedings of the 10th annual conference on Genetic and evolutionary computation (GECCO), Atlanta, GA, USA, July 12-16, 2008: 1485-1490. DOI: 10.1145/1389095.1389384.
  14. Frowd CD, Pitchford M, Skelton F, Petkovic A. Catching even more offenders with EvoFIT facial composites. Proc Third International Conference on Emerging Security Technologies (EST), Lisbon, Portugal, Sep 5-7, 2012: 20-26. DOI: 10.1109/EST.2012.26.
  15. Student sketch database. Source: <http://mmlab.ie.cuhk.edu.hk/facesketch. html>.
  16. Face sketch FERET database. Source: <http://mmlab.ie.cuhk.edu.hk/cufsf>.
  17. Zhang W, Wang X, Tang X. Coupled information-theo­retic encoding for face photo-sketch recognition. Proc IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, June 20-25, 2011: 513-520. DOI: 10.1109/CVPR.2011.5995324.
  18. Li X, Cao X. A simple framework for face photo-sketch synthesis. Mathematical Problems in Engineering 2012; 2012: 910719. DOI: 10.1155/2012/910719.
  19. Galoogahi HK, Sim T. Face photo retrieval by sketch example. Proc 20th ACM international conference on Multimedia, Nara, Japan, 29 Oct – 02 Nov, 2012: 949-952. DOI: 10.1145/2393347.2396354.
  20. Sharma A, Jacobs DW. Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch. Proc 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA, June 20-25, 2011: 593-600. DOI: 10.1109/CVPR.2011.5995350.
  21. Liang C, Mingquan Z, Yanjun H, Xiaoming D. Face sketch synthesis via sparse representation. Proc 20th International Conference on Pattern Recognition (ICPR), Istanbul, Aug 23-26, 2010: 2146-2149. DOI: 10.1109/ICPR.2010.526.
  22. Kukharev GA, Buda K, Shchegoleva NL. Methods of face photo-sketch comparison. Pattern Recognition and Image Analysis 2014; 24(1): 102-113. DOI: 10.1134/S1054661814010076.
  23. Kukharev GA, Buda K, Shchegoleva NL. Sketch generation from photo to create test databases. Przeglad Elektrotechniczny (Electrical Review) 2014; 90(2): 97-100. DOI: 10.12915/pe.2014.02.26.
  24. Yu H, Zhang JJ. Mean value coordinates–based caricature and expression synthesis. Signal, Image and Video Processing 2013; 7(5): 899-910. DOI: 10.1007/s11760-011-0279-8.
  25. Kukharev GA, Matveev YuN, Shchegoleva NL. Matching of a sketches with an original photos. Proc XVIII International Conference on Soft Computing and Measurements (SCM) 2015: 157-159. DOI: 10.1109/SCM.2015.7190441.
  26. Wang Z, Bovik AC. A universal image quality index. IEEE Signal Processing Letters 2002; 9(3): 81-84. DOI: 10.1109/97.995823.
  27. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 2004; 13(4): 600-612. DOI: 10.1109/TIP.2003.819861.
  28. Saeed M, Resa E. Mixture of experts: a literature survey. Artificial Intelligence Review 2014; 42(2): 275-293. DOI: 10.1007/s10462-012-9338-y.
  29. Hitrov MV, ed, Kukharev GA, Kamenskaya EI, Matveev YuN, Shchegoleva NL. Methods of facial images processing and recognition in biometrics [In Russian]. Saint-Peterburg: “Politechnika” Publisher, 2013. ISBN: 978-5-7325-1028-7.
© 2009, IPSI RAS
Institution of Russian Academy of Sciences, Image Processing Systems Institute of RAS, Russia, 443001, Samara, Molodogvardeyskaya Street 151; e-mail: ko@smr.ru; Phones: +7 (846) 332-56-22, Fax: +7 (846) 332-56-20