PDF

Abstract:
In this work, we research the applicability of generative adversarial neural networks for generating training samples for a traffic sign classification task. We consider generative neural networks trained using the Wasserstein metric. As a baseline method for comparison, we take image generation based on traffic sign icons. Experimental evaluation of the classifiers based on convolutional neural networks is conducted on real data, two types of synthetic data, and a combination of real and synthetic data. The experiments show that modern generative neural networks are capable of generating realistic training samples for traffic sign classification that outperform methods for generating images with icons, but are still slightly worse than real images for classifier training.

Keywords:
traffic sign classification, synthetic training sample, generative neural network.

Citation:
Shakhuro VI, Konushin AS. Image synthesis with neural networks for traffic sign classification. Computer Optics 2018; 42(1): 105-112. DOI: 10.18287/2412-6179-2018-42-1-105-112.

References:

1. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A, Fei-Fei L. ImageNet large scale visual recognition challenge. International Journal of Computer Vision 2015; 115(3): 211-252. DOI: 10.1007/s11263-015-0816-y.
2. Lin T, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Doll'ar P, Zitnick L. Microsoft COCO: Common objects in context. In Book: Fleet D, Pajdla T, Schiele B, Tuytelaars T, eds. Computer Vision – ECCV 2014. Switzerland: Springer International Publishing; 2014: 740-755. ISBN: 978-3-319-10592-5.
3. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B. The cityscapes dataset for semantic urban scene understanding. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition 2016: 3213-3223. DOI: 10.1109/CVPR.2016.350.
4. Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R. Real-time human pose recognition in parts from single depth images. Communications of the ACM 2013; 56(1): 116-124. DOI: 10.1145/2398356.2398381.
5. Richter SR, Vineet V, Roth S, Koltun V. Playing for data: Ground truth from computer games. In book: Leibe B, Matas J, Sebe N, Welling M, eds. Computer Vision – ECCV 2016. Cham, Switzerland: Springer; 2016: 102-118. DOI: 10.1007/978-3-319-46475-6_7.
6. Moiseyev B, Konev A, Chigorin A, Konushin A. Evaluation of traffic sign recognition methods trained on synthetically generated data. ACIVS 2013: 576-583. DOI: 10.1007/978-3-319-02895-8_52.
7. Chigorin A, Konushin A. A system for large-scale automatic traffic sign recognition and mapping. CMRT13 – City Models, Roads and Traffic 2013: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2013; II-3/W3: 13-17. DOI: 10.5194/isprsannals-II-3-W3-13-2013.
8. Fischer P, Dosovitskiy A, Ilg E, Hausser P, Hazirbas C, Golkov V, van der Smagt P, Cremers D, Brox T. Flownet: Learning optical flow with convolutional networks. arXiv preprint arXiv:1504.06852 2015.
9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. Proc NIPS 2014; 2: 2672-2680.
10. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 2015. Source: <https://arxiv.org/abs/1511.06434>.
11. Denton EL, Chintala S, Fergus R. Deep generative image models using a Laplacian pyramid of adversarial networks. Proc NIPS 2015; 1: 1486-1494.
12. Mirza M, Osindero S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 2014. Source: <https://arxiv.org/abs/1411.1784>.
13. Zheng Z, Zheng L, Yang Y. Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. arXiv preprint arXiv:1701.07717 2017. Source: <https://arxiv.org/abs/1701.07717>.
14. Arjovsky M, Chintala S, Bottou L. Wasserstein gan. arXiv preprint arXiv:1701.07875 2017. Source: <https://arxiv.org/abs/1701.07875>.
15. Stallkamp J, Schlipsing M, Salmen J, Igel C. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks 2012; 32: 323-332. DOI: 10.1016/j.neunet.2012.02.016.
16. Shakhuro VI, Konushin AS. Russian traffic signs images dataset [In Russian]. Computer Optics 2016; 40(2): 294-300. DOI: 10.18287/2412-6179-2016-40-2-294-300.
17. Lisitsyn SO, Bayda OA. Road sign recognition using support vector machines and histogram of oriented gradients [In Russian]. Computer Optics 2012; 36(2): 289-295.
18. Ciresan D, Meier U, Masci J, Schmidhuber J. Multi-column deep neural network for traffic sign classification. Neural Networks 2012; 32: 333-338. DOI: 10.1016/j.neunet.2012.02.023.

---

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20