Online view-invariant human action recognition using rgb-d spatio-temporal matrix.pdfVIP

Pattern Recognition 60 (2016) 215–226 Contents lists available at ScienceDirect Pattern Recognition journal homepage: /locate/pr Online view-invariant human action recognition using rgb-d spatiotemporal matrix Yen-Pin Hsu, Chengyin Liu, Tzu-Yang Chen, Li-Chen Fu n Department of Computer Science and Information Engineering National Taiwan University, Taiwan, ROC article info Article history: Received 10 June 2015 Received in revised form 2 April 2016 Accepted 3 May 2016 Available online 26 May 2016 Keywords: Action recognition View-invariant Self-similarity abstract We propose a novel approach to recognize action under view changes online with RGB-D camera. Perspective effects and camera motions have been considered as dif?cult problems in recognizing action that when two video sequences record a speci?c action from various camera views, the resulting appearances of actions would be entirely different. Consequently, if we simply apply feature extraction methods to the raw video, we will end up getting totally different features. Recent studies explored the stability of self-similarities for action sequence over time, an idea that put into practice in view-invariant action recognition. Instead of doing the extraction of spatio-temporal feature for every frame and using these feature vectors directly, our study uses the Euclidean distance between spatio-temporal feature vectors that are represented in a Spatio-Temporal Matrix (STM). To recognize the action, we describe the local tendency of the STM using pyramid-structural bag-of-words (BoW-Pyramid) and train a SVM as our classi?er. 2016 Published by Elsevier Ltd. 1. Introduction The ability to understand human is one of the central functions of modern computer vision systems. In the past few decades, considerable research efforts have been devoted to understanding humans in computer vision. Great advances have been achieved in human face detection [1,2], recognition [3], and pedestrian detection [4,5]. Human pose est