The article discusses the video processing methods necessary for the automation of processing the results of the experiments “Morris Water Maze”, “Open Field”, “Elevated Cross-shaped Maze”, which are used to study the behavior of laboratory mice depending on various external factors, as well as an experiment with daily tracking Rhesus macaque activity. To process the received information, the C++ programming language is used including the OpenCV 3.2 and Qt 5.2 libraries. Later in the paper new applications are discussed to further the research based on findings described here. For further use monkey observation is proposed. The choice was made based on similarity between discussed and proposed methods for video processing and automation of experiments. For reviewed methods their advantages and disadvantages are included in the work. For each of experiments “Morris Water Maze”, “Open Field”, “Elevated Crossshaped Maze” there is corresponding image sequence for anyone unfamiliar with the topic.
To acquire information concerning psychological condition of mammals in different unnatural environments by studying the difference in rodents’ behavior, Institute of Physiology Pavlova conducts different experiments. Those experiments include “Morris Water Maze”, “Open Field”, “Elevated Cross-shaped Maze”. Outside of rodent experiments, the university conducts experiment on mammals such as Rhesus macaque. For those big animals’ daily activity is recorded and anylised.
The open field maze is used as a simple test of emotionality. It is generally accepted that if the animal is in a suppressed emotional state, then it will be less mobile, compared with the psychologically normal.
During the Open Field experiment, the rodent is placed in the center of the maze and
its behavior is observed. As soon as the rodent enters a new square, it registers. An
example of the “Open Field” labyrinth is shown in Figure 1- a.
The method of background subtraction was chosen due to the motionlessness of the
background in the task. This method has several main points presented in Figure 2.
At the pre-processing stage, the frame must be prepared for the detection of moving
objects. To do this, it is necessary to reduce the frame size and transfer the frame from
the RGB system to the YUV system. After preprocessing, the background is subtracted
pixel by pixel from each frame [
In the “mix of Gaussian distributions” method , the background is the sum of the distributions . For this, a certain number of frames is taken, according to which the background will be determined and , which determines the minimum share of the background. Consider the method at time t. - Is a set of Gaussian distributions, each individual distribution of which is determined by two parameters: 2 - variance and – vector of average values. Also, a weight is calculated for each distribution , indicating how
well the distribution describes the background, considering iM=1i = 1. It should be noted that and - are numbers that describe the distribution, and the vector connecting each pixel with the selected distribution. It is believed that the selected distribution is closed if the distance between the points of (vT * v) Mahalanobis is more than three D2 (l ) = , v = l − , 2 where ̅ and ̅ are vectors.
Let’s sort the elements of find
B using 1.
according to the condition k k +1 . Next, let’s
We calculate the probability that belongs to the current background using only the first distribution.
b B = argbmin ( i )
i =1 P(lt ) = B ( * N(lt , m,t , m2,t * t )) m=1 N (l, , ) = 1 2 e v T *v
2 t +1 = t + (ot − t ) − * Ct t +1 = t + ot ( t +1
) * t t2+1 = t2 + ot ( ) * (t * t − t2 )
t +1 if P(lt (x)) , then we get this point lt (x) , belonging to the background.
The following calculations are carried out for each distribution from formula 2, formula 3 and formula 4. (1) (2) (3) (4) (5) (6) Implementation of Basic Computer Vision Methods for Analyzing the Results… 5 1 where =
. ot = 1 only for closed distribution with the highest weight t , in any other case ot = 0 . If there are no closed distributions in , then we add distribution to
t +1 = , t +1 = lt и t +1 = 0 , where 0 previously defined
constant. Next, we normalize the weights t and perform calculations for the new set
[
In this paper, to search for a grid in the Open Field labyrinth, a site in the Morris Water Maze, and a labyrinth in the Elevated Cross-shaped Maze, the Hough transform is used.
The Hough transform is a linear transform for defining lines in an image. The feature of the Hough transform is that the line can be represented as a point with coordinates m (slope coefficient) and b (intersection point with the ordinate axis) in the parameter space.
It is known that straight lines parallel to the ordinate axis have infinite value for the parameter m Therefore, in the Hough transform, the line is represented using the parameter r and By
understand the length of the normal to the line drawn from the origin. - is the angle between the radius vector of the point closest to the origin on the line and the abscissa. Therefore, the equation of the line in the Hough transform: cos sin
Therefore, you can connect the lines in the original image in the XY plane, the point with the coordinates r and n the Hough space , which is unique if [0, ] и r R , или [0, 2] и r 0 .
Through each point in space is possible to make an infinite number of straight lines if you specify the location of the point of using 0, 0 all lines passing through it correspond to the relationship: r() = x0* cos + y0 sin .
This equation corresponds to a sinusoidal line in the Hough space, which is unique for a given point and uniquely determines it. If these lines correspond to two points in the Hough space, then the point where they intersect corresponds to a straight line in the XОY plane.
A series of points that form a straight line define sinusoids that intersect at the
parameter point for that line. Thus, the problem of detecting points lying on one straight
line can be reduced to the problem of detecting intersecting curves [
The search algorithm for the center of mass of the rodent in the image implements the method of finding the center of the white spot in the image.
xc = yc = m n ( j *U (i, j)) i=1 j =1 n m ( U (i, j)) j =1 i=1 n m ( i*U (i, j)) j =1 i=1 n m ( U (i, j)) j =1 i=1 (8) (9)
Where U (i, j) – the brightness of the pixel, m , n – he number of rows and columns of the image.
As a result, the coordinates of the brightness center of mass of the image are
determined.[
Based on the analysis presented algorithms and developed math equations software was created to automate the processing of results of experiments. 4.1
Subprogram “Open field” At the input, the algorithm receives a video stream, starting from the frame at the beginning of the experiment. The frame consists of the cut-out area of interest of the original video file. The algorithm is divided into four stages: determining the background, searching for the labyrinth grid, tracking the movement of the rodent and determining the number of the square in which it is located.
At the first stage, using the “mixture of Gaussian distributions” algorithm, sorting through all the frames in turn, the background on the video is determined. The background is a place where the rodent is completely absent or completely invisible. An example of a detected background in Figure 3. In step two, stripes are detected in the background image that form the maze grid. Search is performed using the Hough transform. The user himself determines when the entire grid is found. Further, from the formed grid using the equation of intersection of lines, the quadrangular cells of the maze are determined. An example of a detected grid is shown in Figure 4.
After going to step three, the background is subtracted from each frame of the video stream. Then the frame is binarized with a threshold equal to the average background brightness, the image is filtered by the Gauss method with a 11x11 mask - this allows you to get rid of the remaining noise in the image. The result of background subtraction and filtering in Figure 5.
The next step is to search for all the contours in the image. A search is made for the largest contour and its center of mass. The detected point is the center of mass of the rodent. Then there is a search of the area from the predefined cells of the labyrinth into which the point enters, and on the current raw frame a specific cell of the labyrinth and the point of the center of mass are displayed. This algorithm applies to all subsequent frames.
In paragraph 4, the number of maze cells crossed by the rodent is determined, and
their characteristics are calculated along the length of the trajectory and speed. At the
last stage, the calculated parameters are written to a file, an example of which is shown
in Figure 6.[
In paragraph two, four intersecting lines are searched, and they will be considered a site. Site definition does not require user participation.
At stage 3, after determining the center of mass of the rodent, it is checked whether it enters the rectangle found earlier. If it enters, the loop exits.
Stage 4 is performed similarly to the "Open Field" experiment. The calculation of the distance, the speed of the rodent, and the time spent searching for the site. Data is written to a file, on which the operation of the algorithm ends. 5
To analyze the results, the algorithms were tested on five files with experiments, three of them - the “Open Field” maze, two - “Morris Water Maze”.
Testing revealed the following disadvantages: the displacement of the chamber
during the experiment leads to the inability to analyze this experiment (Figure 8 - a);
moving any objects between the camera and the maze leads to inaccuracies in determining
the position of the mouse (Figure 8 - b); errors in the determination of the labyrinth grid
or site due to the presence of distortion in the original image (Figure 8 - c, d). [
One of the experiments conducted by the Institute. Pavlova, is an analysis of the
cognitive abilities and daily activity of rhesus monkeys in the cell. At the input, the
program receives a video recording or online video stream that must be processed. The
object tracked on the studied videos is the rhesus monkey in the cell. Its daily activity,
special movements and habits must be entered into a text file obtained at the output of
the program. Figure 9a shows a freeze frame from the video, and figure 9b shows the
process of processing the frame with highlighting the position of the monkey on the
frame [
In conclusion, it should be noted that the work was carried out in the framework of cooperation between the Faculty of Applied Optics and the Institute of Physiology. Ivan Petrovich Pavlov of the Russian Academy of Sciences.