Introduction

Human activity recognition is a rapidly growing research area with important application domains including security and anti-terrorist issues. Recently logic programming was recognized as a promising approach for dynamic visual scenes analysis (see surveys of logic-based recognition systems in [1,2]). The idea of the logic programming approach is in usage of logical rules for description and analysis of people activities. Knowledge about object co-ordinates and properties, scene geometry, and human body constraints is encoded in the form of certain rules in a logic programming language and is applied to the output of low-level object / feature detectors.

The distinctive feature of our approach to the visual surveillance logic programming is in the application of general-purpose concurrent object-oriented logic programming features, but not in the development of a new logical formalism. We use the Actor Prolog object-oriented logic language [3,4,2] for implementation of concurrent stages of video processing. A state-of-the-art Prolog-to-Java translator is used for efficient implementation of logical inference on video scenes.

Basic principles of video surveillance logic programming are described in Section 2. Development of the blob motion metrics for discrimination of running people, bicycles, and cars in the video scene is described in Section 3.

2 Basic principles of video surveillance logic programming Let us consider an example of logical inference on video. The input of a logic program written in Actor Prolog is a standard BEHAVE [5] sample. The video (see Fig. 1a) demonstrates a case of a street offence. Two groups of persons meet in the scope of the video camera, then one group attacks another one, they fight, then people run away.

We will solve the problem of anomalous human activity recognition using a logic program that describes a given scenario of complex people behaviour. The input data for the logic program will be supplied by low-level algorithms that trace objects in a video scene and estimate average speed in different segments of the trajectories [4,2]. This low-level processing is implemented in Java (not in Prolog) and includes extraction of foreground blobs, tracking of the blobs over time, detection of interactions between the blobs, creation of connected graphs of linked tracks of the blobs, and estimation of average speed of the blobs in separate segments of the tracks (see Fig. 2a). The input data include a special set of blob motion metrics to discriminate running pedestrians, bicycles, and cars during the logical inference.

The logic program checks the graph of tracks and looks for the following pattern of interaction among several persons: "If two or more persons meet somewhere in the scene and one of them runs after the end of the meeting, the program should consider this scenario as a kind of a running away and a probable case of a sudden attack or a theft." Thus, the program has to alarm if this kind of sub-graph is detected in the total connected graph of tracks. In this case, the program marks all persons in the inspected graph by yellow rectangles and outputs a warning in the middle of the screen (see Fig. 1b).

Development of the blob motion metrics

In contrast to the previously developed motion analysis methods of pedestrian detection that consider the periodicity and cyclic motion in the way humans movement [6], we have developed a set of blob motion statistical metrics based on the windowed coefficient of determination of the temporal changes of the length of the contour of the blob [7].

The coefficient of determination R 2 [8] indicates the proportionate amount of variation in the given response variable Y explained by the independent variable X in a linear regression model. In the general case, vehicles will be characterised by bigger values of the R 2 metrics than running persons, because the contour of the running person changes permanently in the course of his motion when he waves his arms and moves his legs (see Fig. 2b).

We use a windowed modification of the R 2 metrics, that is, the trajectory of a moving blob is characterised by a set of instantaneous values of the R 2 metrics computed in each point of the trajectory. Specifically, two statistical metrics that characterise the motion of the blob, namely, the mean of the wR 2 distribution and the bias-corrected skewness of the wR 2 distribution are used.

The properties of the mean(wR 2 ) and the skewness(wR 2 ) metrics can be illustrated by the example of the BEHAVE data set [5]. For that, we have computed the tracks of moving blobs in this data set by the blob extraction methods implemented in Actor Prolog [4,2]. The "speed-mean(wR 2 )" and the "speedskewness(wR 2 )" diagrams show clearly that these metrics allow discriminating fast moving persons and vehicles (Fig. 3). We have implemented the metrics in the Vision standard package of the Actor Prolog language and use them for experimenting with the intelligent visual surveillance. We have incorporated elements of fuzzy logical inference based on the blob motion statistics into the logic language by means of standard arithmetic means of the language. For example, the is a f ast object predicate for recognition of fast objects may take into account simultaneously two characteristics of blobs; namely, the average velocity of a blob and the length of a track. Combination of these two characteristics is performed by very simple fuzzy metrics described in terms of arithmetic functions. From the standpoint of the declarative semantics of the language, the procedure is a standard formula of the first-order predicate logic. Please see a detailed example of a logic program in [7].

We use a compilation from the Actor Prolog language to Java [2] to ensure necessary performance of logical inference as well as reliability, portability, and openness of the software, including the possibility of access to open source code libraries. At present, we can demonstrate that the Actor Prolog system is fast enough for real-time analyzing clips of the standard data set [5].

Conclusions

We have developed a research software platform based on the Actor Prolog concurrent object-oriented logic language and a state-of-the-art Prolog-to-Java translator for studying the intelligent visual surveillance. The platform includes the Actor Prolog logic programming system and an open source Java library of Actor Prolog built-in classes [9]. It is intended to facilitate the study of the intelligent monitoring of anomalous people activities, the logical description and analysis of people behaviour (see Web Site [10]). Special blob motion statistical metrics based on the windowed coefficient of determination wR 2 were developed. The blob motion statistics are collected by the low-level analysis procedures that are implemented in a specialised builtin class of the Actor Prolog system. Our studies have demonstrated that this approach is a prospective one for description and analysis of complex people behaviour, and is useful in recognition of anomalous people activities.