In this paper, various algorithms for the analysis of photoplethysmography signals are considered. The sequence of performing the intelligent analysis of the measured signals is presented. The study of heart rate variability was carried out, based on the assessment of the instantaneous frequency and the instantaneous period of the pulse wave. The estimates of heart rate and indicators of variation pulsometry were obtained.
An important subject area of data mining (ADM) is the development of medical
decision support systems (DSS) for recognizing a patient's diagnosis based on a
presented set of symptoms [
In solving the problem of identifying biological wave processes, an important role is played by the formation of the space of informative features based on the measured initial data. The basis for solving this problem is the use of artificial intelligence methodology in the form of a natural symbiosis of methods for structural and spectral analysis of the dynamics of non-stationary time series (TS).
Remote PhotoPlethysmography monitors changes in the intensity of light reflected from a subject's skin. An RGB video camera is used as a sensor, and the area of interest is most often the face, less often the palm or wrist.
The aim of this work is the intellectual analysis of heart rate variability (HRV) by various algorithms for processing photoplethysmography signals. It is important to note that HRV analysis refers to the pre-medical stage, i.e. does not make a diagnosis, but forms the basis for its statement. 2 2.1
As the initial data for the study, we took a database recorded by volunteers of the
Eindhoven University of Technology (Netherlands), and designed to study the methods
of photoplethysmography [
In this work, we analyze relatively short recordings of rPPG data - recording duration no more than 15 minutes at a frequency of 30 frames / sec. Such a measurement mode is typical for mass prophylactic examinations or during preliminary outpatient and clinical examinations, which allows registering the general state of health of the subject and finding out some significant indicators.
An area of the face is selected from the current frame - the so-called. the area of interest of certain areas of the skin of the face. Based on the results of measuring the degree of light scattering, the initial signals are formed in the form of TS. Each pixel in the image represents the red (R), green (G), and blue (B) color components. Thus, the face area can be represented as an array, the rows of which are frames, and the columns are color components. To obtain a pulse signal, which, in turn, does not depend on the spectral characteristics of a stationary illumination source and its brightness level, it is necessary to normalize each color channel by dividing the TS realizations by their average values (trends).
The paper considers promising methods for analyzing the dynamics of indicators of non-stationary TS - stabilization of the pulse and respiration after physical exercises. 2.2
To determine a person's health status, an rPPG signal is required, which can be
compared with a cardiogram or rhythmogram. This is due to the need to isolate the
sequence of RR intervals for HRV analysis. ADM models and algorithms are a reliable
methodological basis for the formation of an informative component, i.e. an aggregated
signal based on a set of RGB signals. The general algorithm for analyzing a
photoplethysmogram consists of the following steps:
─ For each frame, a face area is selected according to the Viola-Jones algorithm [
RGB non-stationary
TS is performed using
multiresolution analysis (MRA) in the discrete wavelet transform basis [
Hilbert-Huang transform. ─ Assess the indicators of heart rate variability. ─ Estimate the instantaneous phase and frequency of the pulse wave based on the
Allocation of the frequency range of the pulse wave using the AMD method. AMD extracts informative harmonic components from a wideband signal as follows: ( ) = (2 ) ∙ [ (2 ) 0( )] − (2 ) ∙ [ (2 ) 0( )] (1) (2) (3) (4)
Here Xf = (3R - 2G)/√13 and Yf = (-1.5R - G+ 1.5B)/√5.5 are two orthogonal chroma signals at the output of the FIR filter with a bandwidth equal to the frequency range of the pulse wave; σ is the operator of a robust estimate of the standart deviation (MSD).
In the POS algorithm, the unit vectors of the informative plane orthogonal to the skin
tone unit Z in RGB space are chosen as follows: X = (0, 1, -1)/√2 and Y = (-2, 1, 1) /
√6. Accordingly, the projections of the readings of the TS-centered color channels X =
(G - B) /√2 and Y = (-2R + G + B)/√6 onto this plane form an estimate of the pulse wave
[
( ) = 0( ) − ( )
Where x0(t) is the original signal; fb - boundary frequency; xf(t) - fast changing signal; xs(t) is a slowly changing signal.
Extraction of photoplethysmogram from TS color channels. CHROM algorithm
provides invariance of informative signals to the spectral composition of lighting as a
result of aggregation of components based on color difference, i.e. chrominance,
signals. The idea of the method consists in orthogonal projection of TS samples of color
channels onto the plane specified by the orts X = (3, -2, 0)/√13 and Y = (-1.5, -1,
1.5)/√5.5 in RGB-space, informative orthonormal basis (X, Y, Z) [
,
Where σ(X) and σ(Y) are robust standard deviations of nonstationary TS X and Y. Estimates of the instantaneous phase and frequency of the pulse signal based on the Hilbert - Huang transform. The Hilbert transform provides adaptive estimates of the instantaneous envelope, phase, and frequency of a signal. The total phase of the analytical signal is: ( ) = arg( ( )) = ̂( ) ( ) ( ) = d
dt ̂( ) ( ) = ̂′( ) ( ) − ̂( ) ′( ) ̂ 2( ) + 2( ) , The instantaneous frequency is defined as the time derivative of the total phase: Where ( ) is a pulse signal; ̂( ) - the signal obtained as a result of the Hilbert transformation of the pulse signal.
The energy of the Hilbert - Huang spectrum is the square of the pulse signal envelope ( ) = 2( ) + ̂2( ).
Assessment of indicators of heart rate variability. The RR time sequence of the pulse wave intervals is very informative. Based on this characteristic of the pulse wave, various HRV indicators are calculated, one of which is heart rate. It is defined as the number of samples for the total recording time. (5) (6) (7) HR = 601000 n NNi (мс) i=1 n
3 3.1
The TS extracted from the RGB video are subjected to multiresolution analysis (MRA).
In our computational experiments, the 40th order Debeschy wavelets were used as a
basis for the expansion. Ten levels of decomposition made it possible to form an expert
model of the structural components of non-stationary TS color channels of rPPG,
namely:
─ Trends (Figure 1) as the sum of the eighth, ninth, tenth detailing and tenth
approximating components of the MRA;
─ Respiratory, myogenic and other wave processes with frequencies not exceeding
0.667 Hz in the form of the sum of the slow part of the fifth, sixth and seventh
detailing components of the MRA;
─ Centered pulse waves of each color channel (Figure 2) in the frequency range from
0.667…4 Hz as the sum of the slow part [
Fig. 3. Projections of the 3D scattering diagram of the TS counts of the centered pulse waves onto the coordinate planes of the RGB space. Fig. 4. Fragment (10 s) of the pulse wave - a) and its PSD - b), averaged over the entire implementation of TS. Fig. 5. ACF of overlapping segments of the pulse wave - a) and their Thomson periodogram - b).
Algorithm
HR, bpm
Value, bpm Algorithm CHROM POS
It is clearly seen that as ns increases, the intervals between the zeros of the ACF increase, and the largest values of the PSD of the segments shift to the region of lower frequencies. 3.2
A simplified version of the heart rate assessment for a subject in a static state does not
require the restoration of the pulse waveform by the rPPG method. The frequency
range, including the fundamental heart rate harmonic, is 0.8 ... 1.6 Hz [
The shift of the heart rate to the region of high frequencies is associated with the peculiarity of the TS example chosen for the study.
The data in Table 3 demonstrate the dynamics of heart rate for 5 non-overlapping video segments.
HR, bpm 141 142 147 138 150 133 131 138 138
There is a significant correlation between the heart rate values, the instantaneous frequency values and their correspondence to the visual assessment of the subject's condition on video.
One of the significant disadvantages of estimating (1) using the Hilbert transform hilbert(pw) is, as is known, the presence of negative values of instantaneous frequencies. The physically non-interpretable result is due to errors in the calculation of the accumulated instantaneous phase (unwrap phase). In our opinion, this drawback can be easily eliminated by dividing the accumulated instantaneous phase into two components. The first is a continuous, differentiable, and monotonically nondecreasing function of time. The second component contains discontinuities of the first kind, i.e. is a non-decreasing "step function", thus taking into account the errors in the estimates of the accumulated instantaneous phase.
Within the framework of this approach, the continuous component of the
accumulated instantaneous phase is described using shape-preserving piecewise cubic
interpolation by Hermite polynomials [
а) b)
The solid line in Figure 6.a represents the nuclear robust locally weighted quadratic
regression of Cleveland [
Correct determination of HRV and heart rate indices requires a chronological sequence of RR intervals, similar to ECG data [14, 15]. Otherwise, there is a significant deviation of the indicators from the norm. To solve this problem, it is necessary to restore the shape of the pulse wave using the rPPG method.
The paper describes algorithms for extracting an informative component - a pulse signal from noisy data. For the pulse wave, its main characteristics, required for HRV analysis, were identified, and an example of analysis for one video sequence was given. In the example, both a complete record and its splitting were considered to identify the dynamics of indicators.
Based on the results obtained, it can be concluded that a structural decomposition is necessary in the analysis of rPPG signals, since interference at extraneous frequencies has a high-power relative to the required spectral range. This will allow restoring the shape of the pulse wave, obtaining a cardiogram, excluding contact with the patient.
It should be noted that the problem of reducing the degree of mixing of modes because of signal decomposition is essential for reconstructing the pulse waveform. 14. Sapunova, D. A.: Vascular rigidity and cardiovascular diseases. Medical Bulletin, 4-5, 581582 (2012). 15. Mathematical methods of heart rate variability analysis. In: Larin, V.V., Baevskiy R. M. (eds.) 1st All-Soviet Union Symposium, Science, Moscow (1968).