Center for Research in Comptuer Vision
Center for Research in Comptuer Vision



Activity Recognition in Groups of People


In this project we developed a novel approach to recognize the class of activities characterized by their rigidity in formation for example people parades, airplane flight formations or herds of animals. The central idea is to model the entire group as a collective rather than focusing on each individual separately. We model the formation as a 3D polygon with each corner representing a participating entity (see figure 1). Tracks from the entities are treated as tracks of feature points on the 3D polygon. Based on the rank of the track matrix we can determine if the 3D polygon under consideration behaves rigidly or undergoes non-rigid deformation..

Figure 1
(a) (b)
Figure 1: (a) An example of a parade scene. The rigidity of the formation (the red polygon) characterizes the parade activity. (b) The diagram gives an outline of our proposed method for classification of rigidity in formation.


Figure 2
Figure 2: Each row contains frames from a video sequence of multiple people walking. The frames are overlaid with tracks of the participating people. The first two sequences are detected as structured activities (parades) while the last two sequences are random crowds.


Related Publication


Saad M. Khan and Mubarak Shah, Detecting Group Activities using Rigidity of Formation, ACM Multimedia, Singapore 2005.

Back to Human Action and Activity Recognition Projects