STIP-GCN: Space-time interest points graph convolutional network for action recognition

Yenduri, Sravani and Chalavadi, Vishnu and Mohan, C Krishna (2022) STIP-GCN: Space-time interest points graph convolutional network for action recognition. In: 2022 International Joint Conference on Neural Networks, IJCNN 2022, 18 July 2022through 23 July 2022, Padua.

Text
Proceedings____.pdf - Published Version
Restricted to Registered users only
Download (2MB) | Request a copy

Abstract

Action recognition requires modelling the interactions between either human & human or human & objects. Re-cently, graph convolutional neural networks (GCNs) are exploited to effectively capture the structure of action by modelling the relationship among entities present in a video. However, most of the approaches depend on the effectiveness of object detection frameworks to detect the entities. In this paper, we propose a graph-based framework for action recognition to model the spatio-temporal interactions among the entities in a video without any object-level supervision. First, we obtain the salient space-time interest points (STIP) that contain rich information about the significant local variations in space and time by using the Harris 3D detector. In order to incorporate the local appearance and motion information of the entities, either low-level or deep features are extracted around these STIPs. Next, we build a graph by considering the extracted STIPs as nodes and are connected by spatial edges and temporal edges. These edges are determined based on a membership function that measures the similarity of entities associated with the STIPs. Finally, GCN is employed on the given graph to provide reasoning among different entities present in a video. We evaluate our method on three widely used datasets, namely, UCF-101, HMDB-51, SSV2 to demonstrate the efficacy of the proposed approach. © 2022 IEEE.

[error in script]

IITH Creators: