Full text of article: Russian language.

 PDF

Abstract:
Object-oriented logical means for a real-time analysis of video were developed. The software includes a translator of the Actor Prolog object-oriented logic language to Java, an open source Java library that contains built-in classes of Actor Prolog, and a programming environment of the Actor Prolog language. An example of using the software for intelligent video monitoring of abnormal behavior in people is considered. Object-oriented means of the Actor Prolog language make it possible to divide a logic program into several parallel processes corresponding to various stages of the video processing. The translation of logic programs to Java provides reliability, portability, and openness of the intelligent video surveillance programs.

Keywords:
real-time video analysis, intelligent visual surveillance, parallel object-oriented logic programming, abnormal activity detection, Actor Prolog, complex event recognition, computer vision, technical vision, translation of Prolog to Java.

Citation:
Morozov AA, Sushkova OS. Real-time analysis of video by means of the Actor Prolog language. Computer Optics 2016; 40(6): 947-957. DOI: 10.18287/2412-6179-2016-40-6-947-957.

References:

1. Vishwakarma S, Agrawal A. A survey on activity recognition and behavior understanding in video surveillance. Visual Computer 2013; 29(10): 983-1009. DOI: 10.1007/s00371-012-0752-6.
2. Borges PVK, Conci N, Cavallaro A. Video-based human behavior understanding: A survey. IEEE Transactions on Circuits and Systems for Video Technology 2013; 23(11): 1993-2008. DOI: 10.1109/TCSVT.2013.2270402.
3. Haritaoglu I, Harwood D, Davis LS. W4: Who? When? Where? What? A real time system for detecting and tracking people. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, 1998, Nara, Japan, 1998: 222-227. DOI: 10.1109/AFGR.1998.670952.
4. Shet V., Harwood D., Davis L. VidMAP: Video monitoring of activity with Prolog. AVSS 2005: 224-229. DOI: 10.1109/AVSS.2005.1577271.
5. O’Hara S. VERSA – video event recognition for surveillance applications. M.S. thesis. University of Nebraska at Omaha; 2008.
6. Machot F, Kyamakya K, Dieber B, Rinner B. Real time complex event detection for resource-limited multimedia sensor networks. AVSS 2011: 468-473. DOI: 10.1109/AVSS.2011.6027378.
7. Shet V, Singh M, Bahlmann C, Ramesh V, Neumann J, Davis L. Predicate logic based image grammars for complex pattern recognition. International Journal of Computer Vision 2011; 93(2): 141-161. DOI: 10.1007/s11263-010-0343-9.
8. Skarlatidis A., Artikis A., Filippou J., Paliouras G. A probabilistic logic programming event calculus. Theory and Practice of Logic Programming 2015; 15(2): 213-245. DOI: 10.1017/S1471068413000690.
9. Artikis A, Sergot M, Paliouras G. A logic programming approach to activity recognition. EiMM '10 Proceedings of the 2nd ACM International Workshop on Events in Multimedia 2010: 3-8. DOI: 10.1145/1877937.1877941.
10. Ma J, Liu W, Miller P, Yan W. Event composition with imperfect information for bus surveillance. AVSS 2009: 382-387.  DOI: 10.1109/AVSS.2009.25.
11. Ferryman J, Hogg D, Sochman J, Behera A, Rodriguez-Serrano JA, Worgan S, Li L, Leung V, Evans M, Cornic P, Herbin S, Schlenger S, Dose M. Robust abandoned object detection integrating wide area visual surveillance and social context. Pattern Recognition Letters 2013; 34(7): 789-798. DOI: 10.1016/j.patrec.2013.01.018.
12. Wielemaker J, Schrijvers T, Triska M, Lager T. SWI-Prolog. Theory and Practice of Logic Programming 2012; 12(1-2): 67-96.
13. Baral C., Gelfond G., Son T.C., Pontelli E. Using answer set programming to model multi-agent scenarios involving agents’ knowledge about other’s knowledge. Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, Toronto, 2010; 259-266.
14. Ginsberg ML. Multivalued logics: A uniform approach to reasoning in artificial intelligence. Computational intelligence 1988; 4(3): 265-316. DOI: 10.1111/j.1467-8640.1988.tb00280.x.
15. Kowalski R, Sergot M. A logic-based calculus of events. New Generation Computing 1986; 4(1): 67-96. DOI: 10.1007/BF03037383.
16. Kimmig A, Demoen B, Raedt LD, Costa VS, Rocha R. On the implementation of the probabilistic logic programming language ProbLog. Theory and Practice of Logic Programming 2011; 11(2-3): 235-262.  DOI: 10.1017/S1471068410000566.
17. Ballan L, Bertini M, Del Bimbo A, Seidenari L, Serra G. Event detection and recognition for semantic annotation of video. Multimedia Tools and Applications 2011; 51(1): 279-302. DOI: 10.1007/s11042-010-0643-7.
18. Neumann B. Natural language description of time-varying scenes. Semantic Structures: Advances in Natural Language Processing 1984: 167-207.
19. Dance S, Caelli T, Liu Z-Q. A concurrent, hierarchical approach to symbolic dynamic scene interpretation. Pattern Recognition 1996; 29(11): 1891-1903. DOI: 10.1016/0031-3203(96)00029-5.
20. Dönderler ME, Ulusoy Ö, Güdükbay U. Rule-based spatiotemporal query processing for video databases. The VLDB Journal 2004; 13(1): 86-103. DOI: 10.1007/s00778-003-0114-0.
21. QasemiZadeh B, Shen J, O'Neill I, Miller P, Hanna P, Stewart D, Wang H. A speech based approach to surveillance video retrieval. AVSS '09 2009: 336-339. DOI: 10.1109/AVSS.2009.54.
22. Tran SD, Davis LS. Event modeling and recognition using Markov logic networks. ECCV '08 2008; Pt II: 610-623. DOI: 10.1007/978-3-540-88688-4_45.
23. Morariu VI. Analyzing structured scenarios by tracking people and their limbs. Ph.D. Thesis. University of Maryland; 2010.
24. Chakraborty I., Elgammal A. Video based activity recognition in a trauma center. 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) 2013: 1-8.
25. Han S., Koo B., Hutter A., Shet V.D., Stechele W. Subjective logic based hybrid approach to conditional evidence fusion for forensic visual surveillance. AVSS 2010: 337-344. DOI: 10.1109/AVSS.2010.19.
26. Selman J, Amer M, Fern A, Todorovic S. PEL-CNF: Probabilistic event logic conjunctive normal form for video interpretation. ICCV Workshops 2011: 680-687. DOI: 10.1109/ICCVW.2011.6130308.
27. Minsky M. A framework for representing knowledge. In book: Winston PH, Hom B, eds. The psychology of computer vision. New York: McGraw-Hill, 1975: 211-277.
28. Davison A. From parlog to polka in two easy steps. In Book: International symposium on programming language implementation and logic programming. Berlin Heidelberg: Springer; 1991; 171-182. ISBN: 978-3-540-54444-9.
29. Richardson M, Domingos P. Markov logic networks. Machine Learning 2006; 62(1): 107-136. DOI: 10.1007/s10994-006-5833-1.
30. Josang A. A logic for uncertain probabilities. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 2001; 9(3): 279-311. DOI: 10.1142/S0218488501000831.
31. Morozov AA, Vaish A, Polupanov AF, Antciperov VE, Lychkov II, Alfimtsev AN, Deviatkov VV. Development of concurrent object-oriented logic programming platform for the intelligent monitoring of anomalous human activities. In Book: Plantier G, Schultz T, Fred A, Gamboa H, eds. Biomedical Engineering Systems and Technologies. Springer International Publishing; 2015; 82-97. DOI: 10.1007/978-3-319-26129-4_6.
32. Morozov AA, Polupanov AF. Development of the logic programming approach to the intelligent monitoring of anomalous human behaviour. In Book: Paulus D, Fuchs C, Droege D, eds. Proceedings of 9th Open German-Russian Workshop on Pattern Recognition and Image Understanding (OGRW 2014). Koblenz: University of Koblenz-Landau; 2015: 82-85. Source: <https://kola.opus.hbz-nrw.de/files/915/OGRW_2014_Proceedings.pdf>.
33. Morozov AA, Polupanov AF. Intelligent visual surveillance logic programming: Implementation issues. In Book: Ströder T, Swift T, eds. Proceedings of the International Joint Workshop on Implementation of Constraint and Logic Programming Systems and Logic-based Methods in Programming Environments 2014 (July 17-18, 2014, Vienna, Austria). Aachen: RWTH Aachen University; 2014: 31-45. Source: <http://aib.informatik.rwth-aachen.de/2014/2014-09.pdf>.
34. Morozov AA, Polupanov AF, Sushkova OS. A Translator of Actor Prolog to Java. In Book: Bassiliades N, Fodor P, Giurca A, Gottlob G, Kliegr T, Nalepa GJ, Palmirani M, Paschke A, Proctor M, Roman D, Sadri F, Stojanovic N, eds. RuleML 2015 DC and Challenge. Proceedings of the 9th International Rule Challenge and the 5th RuleML Doctoral Consortium (Berlin, Germany, August 2-5). Berlin: CEUR; 2015. Source: <https://www.csw.inf.fu-berlin.de/ru­leml2015-ceur/>.
35. Davison A. A survey of logic programming-based object-oriented languages. In Book: Wegner P, Yonezawa A, Agha G, eds. Research Directions in Concurrent Object Oriented Programming. Cambridge, MA: MIT Press; 1993: 42-106. ISBN: 0-262-01139-5.
36. Morozov AA. Actor Prolog: An object-oriented language with the classical declarative semantics. In: Sagonas K, Tarau P, eds. Proc IDL 1999. Paris, France: 1999: 39-53. Source: <http://www.cplire.ru/Lab144/paris.pdf>.
37. Morozov A.A. Logic object-oriented model of asynchronous concurrent computations. Pattern Recognition and Image Analysis 2003; 13(4): 640-649. Source: <http://www.cplire.ru/Lab144/pria640.pdf>.
38. Chang C-L, Lee R. Symbolic logic and mechanical theorem proving. New York: Academic Press; 1973. ISBN: 978-0-08-091728-3.
39. Actor-prolog-java-library: A GitHub repository containing source codes of Actor Prolog built-in classes. Source: <https://github.com/Morozov2012/actor-prolog-java-library/>.
40. Morozov A.A. Development of a method for intelligent video monitoring of abnormal behavior of people based on parallel object-oriented logic programming. Pattern Recognition and Image Analysis 2015; 25(3): 481-492. DOI: 10.1134/S1054661815030153.
41. Blunsden S.J., Fisher R.B. The BEHAVE video dataset: Ground truthed video for multi-person behavior classification. Annals of the BMVA 2010; 4: 1-12. DOI: 10.3390/s131114398.
42. Gascue>a JM, Fern<ndez-Caballero A. On the use of agent technology in intelligent, multisensory and distributed surveillance. The Knowledge Engineering Review 2011; 26(2): 191-208. DOI: 10.1017/S0269888911000026.
43. Morozov AA, Sushkova OS, Polupanov AF. Incorporation of distributed multiagent programming means in a strongly typed logic language [In Russian]. Zhurnal Radioelektroniki [Journal of Radioelectronics] 2016; 7. Source: <http://jre.cplire.ru/ jre/jul16/9/text.pdf>.
44. Development of logic programming approach to intelligent monitoring of anomalous human activities. Source: <http://www.fullvision.ru/actor_prolog/show_eng.php>.

---

© 2009, IPSI RAS
Institution of Russian Academy of Sciences, Image Processing Systems Institute of RAS, Russia, 443001, Samara, Molodogvardeyskaya Street 151; E-mail: journal@computeroptics.ru; Phones: +7 (846) 332-56-22, Fax: +7 (846) 332-56-20