Modeling two-person segmentation and locomotion for stereoscopic action identification: A sustainable video surveillance system

Nida Khalid, Munkhjargal Gochoo, Ahmad Jalal, Kibum Kim

Research output: Contribution to journalArticlepeer-review

45 Citations (Scopus)


Due to the constantly increasing demand for automatic tracking and recognition systems, there is a need for more proficient, intelligent and sustainable human activity tracking. The main purpose of this study is to develop an accurate and sustainable human action tracking system that is capable of error-free identification of human movements irrespective of the environment in which those actions are performed. Therefore, in this paper we propose a stereoscopic Human Action Recognition (HAR) system based on the fusion of RGB (red, green, blue) and depth sensors. These sensors give an extra depth of information which enables the three-dimensional (3D) tracking of each and every movement performed by humans. Human actions are tracked according to four features, namely, (1) geodesic distance; (2) 3D Cartesian-plane features; (3) joints Motion Capture (MOCAP) features and (4) way-points trajectory generation. In order to represent these features in an optimized form, Particle Swarm Optimization (PSO) is applied. After optimization, a neuro-fuzzy classifier is used for classification and recognition. Extensive experimentation is performed on three challenging datasets: A Nanyang Technological University (NTU) RGB+D dataset; a UoL (University of Lincoln) 3D social activity dataset and a Collective Activity Dataset (CAD). Evaluation experiments on the proposed system proved that a fusion of vision sensors along with our unique features is an efficient approach towards developing a robust HAR system, having achieved a mean accuracy of 93.5% with the NTU RGB+D dataset, 92.2% with the UoL dataset and 89.6% with the Collective Activity dataset. The developed system can play a significant role in many computer vision-based applications, such as intelligent homes, offices and hospitals, and surveillance systems.

Original languageEnglish
Article number970
Pages (from-to)1-30
Number of pages30
JournalSustainability (Switzerland)
Issue number2
Publication statusPublished - Jan 2 2021


  • Geodesic distance
  • Human action recognition
  • Human locomotion
  • Neuro-fuzzy classifier
  • Particle swarm optimization
  • RGB-D sensors
  • Trajectory features

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Environmental Science (miscellaneous)
  • Geography, Planning and Development
  • Energy Engineering and Power Technology
  • Hardware and Architecture
  • Management, Monitoring, Policy and Law
  • Computer Networks and Communications
  • Renewable Energy, Sustainability and the Environment


Dive into the research topics of 'Modeling two-person segmentation and locomotion for stereoscopic action identification: A sustainable video surveillance system'. Together they form a unique fingerprint.

Cite this