A Survey of Deep Learning Solutions for Anomaly Detection in Surveillance Videos
DOI:
https://doi.org/10.24203/ijcit.v10i5.166Keywords:
Deep Learning, Anomaly Detection, Anomaly Detection in Videos, Intelligence Video Surveillance, Deep Anomaly DetectionAbstract
Deep learning has proven to be a landmark computing approach to the computer vision domain. Hence, it has been widely applied to solve complex cognitive tasks like the detection of anomalies in surveillance videos. Anomaly detection in this case is the identification of abnormal events in the surveillance videos which can be deemed as security incidents or threats. Deep learning solutions for anomaly detection has outperformed other traditional machine learning solutions. This review attempts to provide holistic benchmarking of the published deep learning solutions for videos anomaly detection since 2016. The paper identifies, the learning technique, datasets used and the overall model accuracy. Reviewed papers were organised into five deep learning methods namely; autoencoders, continual learning, transfer learning, reinforcement learning and ensemble learning. Current and emerging trends are discussed as well.
References
R. Yadav and M. Rai, "Advanced Intelligent Video Surveillance System (AIVSS): A Future Aspect," Research Gate, 2018.
W. Sultani, C. Chen and M. Shah, "Real-World Anomaly Detection in Surveillance Videos," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6479-6488, 2018.
A. Borner, "What is Deep Learning and How Does it Work? | Content Simplicity," 2019. [Online]. Available: https://contentsimplicity.com/what-is-deep-learning-and-how-does-it-work/.
J. Brownlee, "What is Deep Learning?," 16 August 2019. [Online]. Available: https://machinelearningmastery.com/what-is-deep-learning/.
M. U. Farooq, N. A. Khan and M. S. Ali, "Unsupervised Video Surveillance for Anomaly Detection of Street Traffic," (IJACSA) International Journal of Advanced Computer Science and Applications, pp. 270-275, 2017.
K. T. Nguyen, D. T. Dinh, M. N. Do and M. T. Tran, "Anomaly Detection in Traffic Surveillance Videos with GAN-based Future Frame Prediction," Proceedings of the 2020 International Conference on Multimedia, pp. 457-463, 2020.
A. Kushwaha, A. Mishra, K. Kamble, R. Janbhare and A. Pokhare, "Theft Detection using Machine Learning," IOSR Journal of Engineering (IOSRJEN), pp. 67-71, 2018.
K. Wiggers, "AI Guardsman uses computer vision to spot shoplifters," 26 June 2018. [Online]. Available: https://venturebeat.com/2018/06/26/ai-guardsman-uses-computer-vision-to-spot-shoplifters/.
Y. S. Chong and Y. H. Tay, "Abnormal Event Detection in Videos Using Spatiotemporal Autoencoder," arxiv, vol. 1701, no. 01546v1, 2017.
V. Lomte, S. Singh, S. Patil, S. Patil and D. Pahurkar, "A Survey on Real World Anomaly Detection in Live Video Surveillance Techniques," International Journal of Research in Engineering, Science and Management, vol. 2, no. 2, pp. 2581-5792, 2019.
B. Mohammadi, M. Fathy and M. Sabokrou, "Image/Video Deep Anomaly Detection: A Survey," Computing Research Repository (CoRR), vol. abs/2103.01739, 2021.
M. Singh, "A Survey on Video Anomaly Detection," International Journal of Engineering Research & Technology (IJERT), vol. 5, no. 10, 2017.
A. Ramchandran and A. K. Sangaiah, "Unsupervised deep learning system for local anomaly event detection in crowded scenes," Multimedia Tools and Applications, vol. 79, no. 47/48, p. 35275–35295, 2020.
K. K. Santhosh, D. P. Dogra and P. P. Roy, "Anomaly Detection in Road Traffic Using Visual Surveillance: A Survey," ACM Computing Surveys, vol. 53, no. 6, pp. 1-26, 2019.
Y. Zahid, M. A. Tahir and M. N. Durrani, "Ensemble Learning Using Bagging And Inception-V3 For Anomaly Detection In Surveillance Videos," in 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 2020.
D. Tran, L. Bourdev, R. Fergus, L. Torresani and M. Paluri, "Learning Spatiotemporal Features with 3D Convolutional Networks," IEEE International Conference on Computer Vision (ICCV), p. 4489–4497, 2015.
J. Carreira and A. Zisserman, "Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset," Computing Research Repository, vol. abs/1705.07750, 2018.
V. Meel, "YOLOv3: Real-Time Object Detection Algorithm (What’s New?)," viso.ai, 25 Feb 2021. [Online]. Available: https://viso.ai/deep-learning/yolov3-overview/. [Accessed 17 August 2021].
T. S. Nazare, R. F. de Mello and M. A. Ponti, "Are pre-trained CNNs good feature extractors for anomaly detection in surveillance videos?," eprint arXiv, no. 1811.08495v1, 2018.
S. Bansod and A. Nandedkar, "Transfer learning for video anomaly detection," Journal of Intelligent & Fuzzy Systems, vol. 36, no. 3, pp. 1967-1975, 2019.
L. P. Cinelli, "Anomaly Detection in Surveillance Videos Using Deep Resdiual Networks," Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2017.
K. Doshi and Y. Yilmaz, "Any-Shot Sequential Anomaly Detection in Surveillance Videos," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 934-935, 2020.
K. Doshi and Y. Yilmaz, "Online anomaly detection in surveillance videos with asymptotic bound on false alarm rate," Pattern Recognition, vol. 114, p. 107865, 2021.
W. Ullah, A. Ullah, T. Hussain, Z. A. Khan and S. W. Baik, "An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos," AI-Enabled Advanced Sensing for Human Action and Activity Recognition, vol. 21, 2021.
T.-N. Nguyen and J. Meunier, "Anomaly Detection in Video Sequence with Appearance-Motion Correspondence," in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
E. Duman and O. A. Erdem, "Anomaly Detection in Videos Using Optical Flow and Convolutional Autoencoder," IEEE Access, vol. 7, pp. 183914 - 183923, 2019.
B. Ramachandra, M. J. Jones and R. R. Vatsavai, "A Survey of Single-Scene," IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020.
Y. Zhao, B. Deng, C. Shen, Y. Liu, H. Lu and X.-S. Hua, "Spatio-Temporal AutoEncoder for Video Anomaly Detection," Proceedings of the 25th ACM international conference on Multimedia, pp. 1933-1941, 2017.
K. V. Pawar and V. Attar, "Deep learning approaches for video-based anomalous activity detection," World Wide Web, p. 22, 27 May 2018.
H. Wu, J. Shao, X. Xu, F. Shen and H. Shen, "A System for Spatiotemporal Anomaly Localization in Surveillance Videos," Proceedings of the 25th ACM international conference on Multimedia, pp. 1225-1226, 2017.
S. Bhakat and G. Ramakrishnan, "Anomaly Detection in Surveillance Videos," Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, p. 252–255, 2019.
M. Hasan, J. Choi, J. Neumann, A. K. Roy-Chowdhury and L. S. Davis, "Learning Temporal Regularity in Video Sequences," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 733-742, 2016.
M. F. M. M. Sabokrou, "Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder," Electronic Letters, vol. 52, no. 13, pp. 1122-1124, 2016.
T.-H. Vu, J. Boonaert, S. Ambellouis and A. Taleb-Ahmed, "Multi-Channel Generative Framework and Supervised Learning for Anomaly Detection in Surveillance Videos," Human Activity Recognition Based on Image Sensors and Deep Learning, vol. 21, no. 9, p. 3179, 2021.
K. Doshi and Y. Yilmaz, "Continual Learning for Anomaly Detection in Surveillance Videos," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 2020.
J. Torres, "A gentle introduction to Deep Reinforcement Learning," Towards Data Science, 15 May 2020. [Online]. Available: https://towardsdatascience.com/drl-01-a-gentle-introduction-to-deep-reinforcement-learning-405b79866bf4. [Accessed 18 August 2021].
S. Aberkane and M. Elarbi, "Deep Reinforcement Learning for Real-world Anomaly Detection in Surveillance Videos," in 2019 6th International Conference on Image and Signal Processing and their Applications (ISPA), Mostaganem, Algeria, 2019.
R. Chalapathy, A. K. Menon and S. Chawla, "Robust, Deep and Inductive Anomaly Detection," Machine Learning and Knowledge Discovery in Databases, vol. 10534, 2017.
K. Kavikuil and J. Amudha, "Leveraging Deep Learning for Anomaly Detection in Video Surveillance," First International Conference on Artificial Intelligence and Cognitive Computing, vol. 815, no. I, pp. 239-247, 2018.
K. Liu, M. Zhu, H. Fu, H. Ma and T.-S. Chua, "Enhancing Anomaly Detection in Surveillance Videos with Transfer Learning from Action Recognition," Proceedings of the 28th ACM International Conference on Multimedia, pp. 4664-4668, 2020.
L. P. Cinelli, L. A. Thomaz, A. F. d. Silva, E. A. B. d. Silva and S. L. Netto, "Foreground Segmentation for Anomaly Detection in Surveillance Videos Using Deep Residual Networks," XXXV Simposio Brasileiro de Telecomuniac, Oes e processamento de Sinais, pp. 3-6, 2017.
W. Ullah, A. Ullah, I. U. Haq, K. Muhammad, M. Sajjad and S. W. Baik, "CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks," Multimedia Tools and Applications, p. 16979–16995, 2021.
M.Murugesan and S.Thilagamani, "Efficient anomaly detection in surveillance videos based on multi-layer perception recurrent neural network," in Microprocessors and Microsystems, 2020.
V. A. Karishma Pawar, "Application of Deep Learning for Crowd Anomaly Detection from Surveillance Videos," in 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India, 2021.
Y. Zhao, B. Deng, C. Shen, Y. Liu, H. Lu and X.-S. Hua, "Spatio-Temporal AutoEncoder for Video Anomaly Detection," Proceedings of the 25th ACM international conference on Multimedia, pp. 1933-1941, 2017.
N. Nasaruddin, K. Muchtar, A. Afdhal and A. P. J. Dwiyantoro, "Deep anomaly detection through visual attention in surveillance videos," Journal of Big Data, vol. 7, no. 87, 2020.
A. Khaleghi and M. S. Moin, "Improved anomaly detection in surveillance videos based on a deep learning method," in 2018 8th Conference of AI & Robotics and 10th RoboCup Iranopen International Symposium (IRANOPEN), Qazvin, Iran, 2018.
UCSD, "UCSD Anomaly Detection Dataset," UCSD, 2014. [Online]. Available: http://www.svcl.ucsd.edu/projects/anomaly/dataset.html. [Accessed 10 May 2021].
C. Lu, J. Shi and J. Jia, "Avenue Dataset for Abnormal Event Detection," The Chinese Univeristy of Hong Kong, 2013. [Online]. Available: http://www.cse.cuhk.edu.hk/leojia/projects/detectabnormal/dataset.html. [Accessed 10 May 2021].
W. Liu, W. Luo, D. Lian and S. Gao, "Future Frame Prediction for Anomaly Detection -- A New Baseline," in 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA , 2018.
R. Mehran, A. Oyama and M. Shah, "Abnormal Crowd Behavior Detection using Social Force Model," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, 2009.
T. Kanstren, "A Look at Precision, Recall, and F1-Score," Towards Data Science, 09 September 2012. [Online]. Available: https://towardsdatascience.com/a-look-at-precision-recall-and-f1-score-36b5fd0dd3ec. [Accessed 18 August 2021].
H. M. Kun Liu, "Exploring Background-bias for Anomaly Detection in Surveillance Videos," Proceedings of the 27th ACM International Conference on Multimedia, pp. 1490-1499, 2019.
R. V. H. M. Colque, C. Caetano and M. T. L. d. Andrade, "Histograms of Optical Flow Orientation and Magnitude and Entropy to Detect Anomalous Events in Videos," IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, vol. 27, no. 3, pp. 673-682, 2017.
A. Sarkar, "Human Activity and Behavior Recognition in Videos. A Brief Review," 2014. [Online]. Available: https://www.grin.com/document/276054.
A. Kushwaha, A. Mishra, K. Kamble and R. Janbhare, "Theft-Detection using Motion Sensing Camera," International Journal of Innovative Science and Research Technology, pp. 90-97, 2017.
M. Sabokrou, M. Fayyaz, M. Klette and R. Fathy, "Deep-Cascade: Cascading 3D Deep Neural Networks for Fast Anomaly Detection and Localizaton in Crowded Scenes," IEEE Transactions on Image Processing, pp. 1992-2004, 2017.
M. Sabokrou, M. Fayyaz, M. Fathy, Z. Moayed and R. Klette, "Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes," Computer Vision and Image Understanding, pp. 1-25, 2018.
W. Badr, "Auto-Encoder: What Is It? And What Is It Used For? (Part 1)," towards data science, 22 April 2019. [Online]. Available: https://towardsdatascience.com/auto-encoder-what-is-it-and-what-is-it-used-for-part-1-3e5c6f017726. [Accessed 10 May 2021].
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 John Gatara Munyua, Geoffrey Mariga Wambugu, Stephen Thiiru Njenga
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The articles published in International Journal of Computer and Information Technology (IJCIT) is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
