Model for Remote Physiological Signal Sensing

Abstract

This paper briefly introduces the solutions developed by our team, HFUT-VUT, for Remote Physiological Signal Sensing in the 4th Vision-based Remote Physiological Signal Sensing (RePSS) Challenge hosted at IJCAI 2025. Specifically, we present a difusion-based model for remote physiological signal sensing, named DifRePSS, inspired by difusion models' noise distillation abilities. The proposed model takes the multi-scale temporal map (MSTmap) and its temporal diference representation as input of an alternated spatial-temporal Transformer backbone model to extract the rPPG signal from Gaussian noise using a difusion process during training. The model then refines it using a reverse difusion process. To overcome the HR distribution imbalance, we propose an efective data augmentation strategy that synthetically expands the HR distribution. As a result, our solutions achieved a remarkable RMSE score of 11.89505 on the test set, securing the Champion of this challenge.

Keywords

• rPPG
• Difusion
• Transformer
• Data augmentation

1. Introduction

Remote physiological signal sensing (RePSS) [ 1, 2, 3, 4, 5, 6 ], particularly remote photoplethysmography (rPPG), has attracted increasing attention in recent years due to its contactless and low-cost nature, with broad applications in human-centric video understanding tasks [7, 8, 9, 10, 11, 12, 13, 14]. It enables heart rate (HR) and other vital signs to be estimated from facial videos by capturing subtle changes in skin color caused by blood volume variations. However, despite rapid advancements, robust and accurate rPPG estimation under real-world conditions remains challenging due to various factors such as head movement, illumination changes, and sensor noise.

Traditional hand-crafted algorithms [ 15, 16, 17, 18, 19] have attempted to suppress noise and isolate pulse signals using signal decomposition, color space transformation, or skin subspace modeling. More recently, deep learning-based approaches [ 20, 21, 22, 23, 5, 24, 25, 3, 26 ] have shown superior performance by learning end-to-end mappings from raw facial videos to physiological signals. Nevertheless, many of these models struggle to generalize in unconstrained environments, partly due to limited data diversity and a lack of robustness to noise and distribution shifts. In this work, we propose DifRePSS , a difusionbased model for remote physiological signal sensing, motivated by the generative capability and noise modeling strength of difusion models. Specifically, we design a conditional difusion framework to progressively recover clean rPPG signals from noise, guided by physiological priors extracted from facial videos. We introduce a multi-scale spatial-temporal map (MSTmap) and its temporal diference representation as inputs to our alternated spatial-temporal Transformer-based denoiser, which jointly models spatial and temporal dependencies in physiological features. Furthermore, to address the issue

CEUR Workshop

ISSN1613-0073 of imbalanced HR distribution in existing datasets, we propose a novel HR distribution augmentation strategy by temporally resampling MSTmaps to simulate diverse HR values. Our proposed approach demonstrates strong performance on the challenging VIPL-HR and OBF datasets, achieving state-ofthe-art results and winning the 4th RePSS Challenge.

The key contributions of this work are summarized as follows: • We propose a difusion-based framework for rPPG signal estimation, leveraging the DDIM mechanism for robust signal reconstruction. • We introduce a temporal diference representation to capture fine-grained physiological dynamics. • We introduce an alternated spatial-temporal Transformer denoiser to efectively model dependencies across space and time. • We propose a simple yet efective HR distribution augmentation strategy to improve model generalization across HR ranges. The proposed method achieves state-of-the-art performance and wins 1st place in the 4th RePSS Challenge.

2. Methodology

We consider the problem of estimating accurate rPPG signals from facial videos. Formally, given an input facial video ∈ ℝ × × ×3 , our goal is to predict the one-dimensional quasi-periodic rPPG signal ̃ ∈ ℝ , where , , and denote the frame number, height, and width of the video, respectively. As shown in Figure 1, we first send the facial video into Physiological Condition Extractor to extract the multi-scale spatial-temporal map (MSTmap) ∈ ℝ × × and its temporal diference representation Δ ∈ ℝ × × , where and correspond to the number of facial ROIs and channel dimensions, respectively. In the forward process, the ground truth rPPG signal ∈ ℝ is corrupted by adding Gaussian noise, yielding the noisy rPPG signal at the -th timestep. During the reverse process, the Gaussian noises are fed to Alternated Spatial-Temporal Denoiser, generating a denoised rPPG signal.

2.1. Physiological Condition Extractor

Since periodic pulse signals arise from subtle light reflections caused by blood volume changes in skin regions, non-skin pixels and facial geometric structures are typically regarded as noise relative to skin chrominance features. To suppress such noise and emphasize physiological signals, we transform raw facial videos into Multi-scale Spatial-Temporal Maps (MSTmaps), which have been widely adopted in prior works [ 23, 26, 3, 27 ]. MSTmaps efectively encode physiological spatial-temporal information from videos. To further extract dynamic physiological variations, we compute frame-wise diferences along the temporal dimension of the MSTmap, thereby capturing temporal changes related to the pulsatile signal. Given an input MSTmap ∈ ℝ × × , we compute the temporal diference as: Δ ,, = +1,, − ,, , for = 0, … , − 2. (1) To preserve the original dimensionality, we apply zero-padding or temporal interpolation to Δ , resulting in a temporal diference representation D ∈ ℝ × × . This representation accentuates the color lfuctuations induced by cardiovascular activity, providing a clearer physiological cue. c = Concat( , Δ ) ∈ ℝ × ×2 .

This fused physiological representation c captures both the chrominance patterns and their dynamic variations, serving as a robust input for subsequent physiological signal estimation. Forward Process. The Forward Process is an approximate posterior that follows the Markov chain to the original data 0. Followed by DDPM [28], the forward The entire framework is supervised by a standard Negative Pearson Correlation loss ℒ the inference stage, we first obtain an initial rPPG signal by sampling noise from a unit Gaussian. Then 0̃ is predicted by passing to the trained denoiser . Thereafter, 0̃ is used to generate the noisy rPPG signal ̃ ′ as inputs to the denoiser for the next timestep via DDIM, which can be formulated as: [26, 27]. At where , ′ are the current and next timesteps, respectively. The initial = . ∼ (0, I) is standard Gaussian noise independent of 0 and ′ = √ ′ ⋅ 0̃ + √1 − ′ − 2 ⋅ + , = ( − √ ⋅ 0̃)/√1 − , = √(1 − ′)/(1 − ) ⋅ √1 − / ′, =1 ( 1∶ | 0) = ∏ ( | −1 ), ( | −1 ) = ( |√

−1 , (1 − ) ), = √ ̄ 0 + √1 − ̄ ,

0̃ = ( , c , ), always increases as the sampling step increases.

Reverse Process.

In the training stage, is sent to a denoiser representation c and timestep to refine the rPPG signal 0̃ without noise:

where ∼ (0, ), ̄ = ∏=1 , = 1 − , is a noise schedule. We adopt the cosine-schedule, which conditioned on physiological

2.2. RePSS via Difusion Model

that gradually adds Gaussian noise (0, ) process can be defined as: where the scalars 1∶ are either predefined or learned variances, s.t. 1 > 1 > 2 > … > > 0. To simplify the training process, we sample arbitrarily:

Finally, we concatenate the original MSTmap and the temporal diference representation Δ along the channel dimension, forming the complete physiological condition representation: where is the noise at timestep . controls how stochastic the difusion process is. Then, we can regenerate 0̃ using ′ as inputs to the denoiser. This procedure will be iterated times. Since we start from at the beginning, the timestep of each iteration can be written as = ⋅ (1 − /) , ∈ [0, ) .

This process gradually adds noise to the data with high probability.

2.3. Alternated Spatial-Temporal Denoiser

Due to the promising information interaction and global aggregation capabilities of Transformers [29, 30, 31, 32, 33, 34, 35, 36], we implement the Alternated Spatial-Temporal Denoiser (ASTDenoiser) using a Transformer-like architecture. Specifically, assuming the input of ℎ layer is ∈ ℝ × × . In (2) (3) (4) (5) (6) (7) (8) (9) self-attention (MSA) to vectors of size : the spatial transformer, we obtain spatial relationship ∈ ℝ × ×

by directly applying multi-head ̂ = LayerNorm( + MSA( , , )),

= LayerNorm( ̂

+ FeedForward( ̂ )), is intermediate variable. LayerNorm(⋅) denotes layer normalization, FeedForward denotes a multi-layer feedforward network, MSA(Q, K, V) denotes the multi-head self-attention layer where Q, K, V serve as queries, keys and values. In the temporal transformer, we obtain temporal relationship where ̂

where ̂ ∈ ℝ × ×

by applying MSA to vectors of size : ̂

= LayerNorm((

= LayerNorm( ̂ )T + MSA((

+ FeedForward( ̂ )T, ( )), )T, ( )T)), is intermediate variable. (⋅)T denotes the transposition of matrices. Finally, we obtain the input of ( + 1)ℎ layer +1 = built to estimate the rPPG signal ∈ ℝ .

. After layer of the denoiser loop, a linear regression head is (10) (11) (12) (13)

2.4. HR distribution Augmentation

Existing physiological measurement datasets are typically limited in scale and sufer from imbalanced HR distributions, with most samples concentrated in [60, 90] beats per minute (bpm). This results in models overiftting to this dominant HR range while neglecting the tail ends of the HR range, particularly low and high HR values. However, normal human HR varies from approximately 40 to 180 bpm. To address this issue, we propose an efective data augmentation strategy that synthetically expands the HR distribution by directly applying upsampling and downsampling operations on MSTmaps.

Specifically, unlike traditional image-based data augmentation conducted in the spatial domain, our method leverages the fact that HR estimation depends on the the MSTmap by applying downsampling or upsampling to adjust the frequency of these color variations, efectively simulating higher or lower HRs compared to the original video, and generating new MSTmaps for model training. As shown in Figure 2, we analyze the HR distribution in the large-scale VIPL-HR dataset and observe that most ground-truth HRs fall between 60 and 90 bpm, with only a small fraction of samples having HRs below 60 bpm or above 90 bpm.

Based on this observation, we perform the following operations to augment the original data. For MSTmaps with ground-truth HR below 80 bpm, we create a copy and concatenate it with the original along the temporal dimension, resulting in a 600-frame MSTmap. We then apply a downsampling operation with a sampling rate of 0.5 along the temporal axis to generate a new 300-frame MSTmap. This efectively doubles the HR, as the frequency of temporal variation is doubled HRaug = 2 × HRorig.

Following the same principle, for MSTmaps with ground-truth HR above 80 bpm, we first extract the initial 150 frames, then apply an upsampling operation with a sampling rate of 2 along the temporal axis to obtain a new 300-frame MSTmap. Consequently, the HR is halved HRaug = 0.5 × HRorig. This data augmentation method enables us to construct a more diverse dataset with a broader HR distribution, covering both low and high HR ranges, thereby balancing the overall HR distribution.

3. Experiments

3.1. Experimental Setup

Datasets. Following the challenge requirements, we adopt the VIPL-HR dataset [37] to train and evaluate our model. VIPL-HR is a large-scale, multimodal, and challenging dataset for rPPG estimation. It contains recordings of 107 subjects under nine diferent scenarios, incorporating diverse head movements (e.g., stable, talking, and motion) and lighting conditions (e.g., lab, dark, bright), captured using three types of devices: Logitech C310 camera, RealSense F200, and smartphones. The dataset includes 3,130 visible-light facial video clips. The test set comprises the VIPL-HR [37] and OBF [ 2 ] datasets, each ofering paired RGBNIR videos for 100 subjects not included in the training set. For evaluation, the videos are segmented into 10-second clips. VIPL-HR captures a range of lighting conditions, while OBF features subjects with diverse skin tones. Figure 3: Compared with the NIR video in the test Evaluation Metric. The root mean squared set, the video in the training set is much error (RMSE) is employed as the evaluation darker, and even the facial contours are not metric to measure the discrepancy between clear, making it very dificult to capture the predicted heart rate pred and the ground- physiological signals. truth heart rate gt.

3.2. Implementation Details

During preprocessing, facial region-of-interest (ROI) areas are extracted using the landmark detection module from OpenFace. Following the setup in [23], we generate MSTmaps using a sliding window of 300 frames (equivalent to 10 seconds) with a stride of 15 frames (0.5 seconds). For the alternated spatialtemporal denoiser, we set the feature dimension to 128 and the number of layers to 6. The model is trained using the Adam optimizer with a learning rate of 1×10−4 and a batch size of 16. The maximum diffusion step is set to 1000. The probability of applying heart rate distribution augmentation is set to 0.5. During inference, the number of reverse difusion steps is set to 5. To estimate heart rate Table 1: The impact of diferent training/validation from the predicted rPPG signal, we follow pre- splits and video modalities on VIPL-HR. vious works [ 3, 4, 26 ] and apply a first-order FoTldrasin2,i3n,g4,5 VaFliodladt1ion MRoGdBal RM1S2E.2↓13(b6p3m) MNoIdRal RM1S7E.4↓63(b4p1m) Butterworth bandpass filter with a cutof fre- Folds 1,3,4,5 Fold 2 RGB 12.44439 NIR 18.44521 quency range of [0.66 Hz, 3.0 Hz], correspond- FFoollddss 11,,22,,43,,55 FFoolldd 43 RRGGBB 1122..1470066423 NNIIRR 1156..1944757011 ing to a heart rate range of [40, 180] beats per Folds 1,2,3,4 Fold 5 RGB 11.89505 NIR 14.59949 minute.

3.3. Experimental Results

We conducted experiments on the VIPL-HR dataset to evaluate diferent training/validation splits and video modalities (RGB and NIR). As shown in Table 1, the best performance was achieved using the training/validation split of Folds 1, 2, 3, 4 for training and Fold 5 for validation. We observed that mod- Table 2: The HR estimation results of the els trained on RGB videos consistently outperformed top-3 leaderboards on the test set those trained on NIR videos. To further investigate in the 4th RePSS. this performance gap, we visualized representative NIR Team Name Rank RMSE↓ (bpm) vinidFeiogsufrreo m3. bIotthis tehveidtreanintitnhgatanthdetetsrtaiinnginsgetsN, IaRs svhidoewons HFUxTjg-VrIoSUuTTps(cOuurs) 213 111122...837910875490605 are considerably darker than those in the test set, which severely compromises the already weak rPPG signals and ultimately degrades model performance. Due to the significant distributional discrepancy between the training and testing NIR videos, we chose to utilize only the RGB modality for all experiments. As shown in Table 2, we report the top-3 results on the test dataset in the 4th RePSS challenge. Compared to other teams, we can see that our team achieves 1st place (RMSE of 11.89505 bpm), which is higher than the 2nd place (RMSE of 12.31846 bpm) by 3.4%.

4. Conclusion

We presented DifRePSS , a difusion-based method for remote physiological signal sensing. By combining a conditional difusion framework with spatial-temporal representations and HR-aware augmentation, our model achieves robust rPPG estimation under real-world conditions. The proposed approach demonstrates state-of-the-art performance and ranked first in the 4th RePSS Challenge, highlighting the efectiveness of difusion models in this field.

Acknowledgments

This work is supported by National Key R&D Program of China (NO.2024YFB3311602), Natural Science Foundation of China (62272144), the Anhui Provincial Natural Science Foundation (2408085J040), and the Major Project of Anhui Provincial Science and Technology Breakthrough Program (202423k09020001), and the Fundamental Research Funds for the Central Universities (JZ2024HGTG0309, JZ2024AHST0337).

Declaration on Generative AI

During the preparation of this work, the authors used GPT-5 in order to: Grammar correction and language refinement. The authors independently reviewed, verified, and revised all generated content, and accept complete responsibility for the scientific validity and intellectual integrity of the publication. [7] D. Guo, K. Li, B. Hu, Y. Zhang, M. Wang, Benchmarking micro-action recognition: Dataset, methods, and applications, IEEE Transactions on Circuits and Systems for Video Technology 34 (2024) 6238–6252. [8] K. Li, D. Guo, G. Chen, C. Fan, J. Xu, Z. Wu, H. Fan, M. Wang, Prototypical calibrating ambiguous samples for micro-action recognition, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 2025, pp. 4815–4823. [9] J. Zhou, X. Shen, J. Wang, J. Zhang, W. Sun, J. Zhang, S. Birchfield, D. Guo, L. Kong, M. Wang, Y. Zhong, Audio-visual segmentation with semantics, International Journal of Computer Vision (2024) 1–21. [10] J. Zhou, J. Wang, J. Zhang, W. Sun, J. Zhang, S. Birchfield, D. Guo, L. Kong, M. Wang, Y. Zhong, Audio–visual segmentation, in: European Conference on Computer Vision (ECCV), 2022, pp. 386–403. [11] J. Zhou, L. Zheng, Y. Zhong, S. Hao, M. Wang, Positive sample propagation along the audiovisual event line, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 8436–8444. [12] J. Zhou, D. Guo, Y. Mao, Y. Zhong, X. Chang, M. Wang, Label-anticipated event disentanglement for audio-visual video parsing, in: European Conference on Computer Vision (ECCV), 2024, pp. 1–22. [13] Z. Li, D. Guo, J. Zhou, J. Zhang, M. Wang, Object-aware adaptive-positivity learning for audiovisual question answering, in: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2024, pp. 3306–3314. [14] Z. Li, J. Zhou, J. Zhang, S. Tang, K. Li, D. Guo, Patch-level sounding object tracking for audio-visual question answering, arXiv preprint arXiv:2412.10749 (2024). [15] R. Špetlík, V. Franc, J. Matas, Visual heart rate estimation with convolutional neural network, in:

Proceedings of the British Machine Vision Conference, 2018, pp. 3–6. [16] M.-Z. Poh, D. J. McDuf, R. W. Picard, Non-contact, automated cardiac pulse measurements using video imaging and blind source separation., Optics express 18 (2010) 10762–10774. [17] G. De Haan, V. Jeanne, Robust pulse rate from chrominance-based rppg, IEEE Transactions on

Biomedical Engineering 60 (2013) 2878–2886. [18] G. De Haan, A. Van Leest, Improved motion robustness of remote-ppg by using the blood volume pulse signature, Physiological Measurement 35 (2014) 1913–1913. [19] W. Wang, A. C. Den Brinker, S. Stuijk, G. De Haan, Algorithmic principles of remote ppg, IEEE

Transactions on Biomedical Engineering 64 (2016) 1479–1491. [20] W. Chen, D. McDuf, Deepphys: Video-based physiological measurement using convolutional attention networks, in: Proceedings of the European Conference on Computer Vision, 2018, pp. 349–365. [21] Z. Yu, W. Peng, X. Li, X. Hong, G. Zhao, Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 151–160. [22] Z. Yu, Y. Shen, J. Shi, H. Zhao, P. Torr, G. Zhao, Physformer: Facial video-based physiological measurement with temporal diference transformer, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4186–4196. [23] X. Niu, Z. Yu, H. Han, X. Li, S. Shan, G. Zhao, Video-based remote physiological measurement via cross-verified feature disentangling, in: Proceedings of the European Conference on Computer Vision, 2020, pp. 295–310. [24] Z. Sun, X. Li, Contrast-phys: Unsupervised video-based remote physiological measurement via spatiotemporal contrast, in: Proceedings of the European Conference on Computer Vision, 2022, pp. 492–510. [25] Z. Sun, X. Li, Contrast-phys+: Unsupervised and weakly-supervised video-based remote physiological measurement via spatiotemporal contrast, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024) 1–18. [26] W. Qian, K. Li, D. Guo, B. Hu, M. Wang, Cluster-phys: Facial clues clustering towards eficient remote physiological measurement, in: Proceedings of the 32nd ACM International Conference on Multimedia, 2024, p. 330–339. [27] W. Qian, G. Su, D. Guo, J. Zhou, X. Li, B. Hu, S. Tang, M. Wang, Physdif: Physiology-based dynamicity disentangled difusion model for remote physiological measurement, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 2025, pp. 6568–6576. [28] J. Ho, A. Jain, P. Abbeel, Denoising difusion probabilistic models, Advances in neural information processing systems 33 (2020) 6840–6851. [29] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin,

Attention is all you need, Advances in Neural Information Processing Systems 30 (2017). [30] K. Li, J. Li, D. Guo, X. Yang, M. Wang, Transformer-based visual grounding with cross-modality interaction, ACM Transactions on Multimedia Computing, Communications and Applications 19 (2023) 1–19. [31] K. Li, D. Guo, M. Wang, Vigt: proposal-free video grounding with a learnable token in the transformer, Science China Information Sciences 66 (2023) 202102. [32] K. Li, P. Liu, D. Guo, F. Wang, Z. Wu, H. Fan, M. Wang, Mmad: Multi-label micro-action detection in videos, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 13225–13236. [33] F. Wang, D. Guo, K. Li, M. Wang, Eulermormer: Robust eulerian motion magnification via dynamic ifltering within transformer, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 2024, pp. 5345–5353. [34] J. Gu, K. Li, F. Wang, Y. Wei, Z. Wu, h. Fan, M. Wang, Motion matters: Motion-guided modulation network for skeleton-based micro-action recognition, in: Proceedings of the 33rd ACM International Conference on Multimedia, 2025, p. 5461–5470. [35] K. Li, X. Peng, D. Guo, X. Yang, M. Wang, Repetitive action counting with hybrid temporal relation modeling, IEEE Transactions on Multimedia (2025). [36] Y. Zhu, K. Li, Z. Yang, Exploiting eficientsam and temporal coherence for audio-visual segmentation, IEEE Transactions on Multimedia 27 (2025) 2999–3008. [37] X. Niu, S. Shan, H. Han, X. Chen, Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation, IEEE Transactions on Image Processing 29 (2019) 2409–2423.

References

[1] X. Li , J. Chen , G. Zhao , M. Pietikainen , Remote heart rate measurement from face videos under realistic situations , in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2014 , pp. 4264 - 4271 .

[2] X. Li , I. Alikhani , J. Shi , T. Seppanen , J. Junttila , K. Majamaa-Voltti , M. Tulppo , G. Zhao, The obf database: A large face video database for remote physiological signal measurement and atrial ifbrillation detection , in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018 ), 2018 , pp. 242 - 249 .

[3] W. Qian , D. Guo , K. Li , X. Zhang , X. Tian , X. Yang , M. Wang , Dual-path tokenlearner for remote photoplethysmography-based physiological measurement with facial videos , IEEE Transactions on Computational Social Systems ( 2024 ).

[4] Q. Li , D. Guo , W. Qian , X. Tian , X. Sun , H. Zhao , M. Wang , Channel-wise interactive learning for remote heart rate estimation from facial video , IEEE Transactions on Circuits and Systems for Video Technology ( 2023 ).

[5] X. Liu , B. Hill , Z. Jiang , S. Patel , D. McDuf , Eficientphys: Enabling simple, fast and accurate camera-based cardiac measurement , in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2023 , pp. 5008 - 5017 .

[6] W. Qian , Q. Li , K. Li , X. Wang , X. Sun , M. Wang , D. Guo , Joint spatial-temporal modeling and contrastive learning for self-supervised heart rate measurement , arXiv preprint arXiv:2406.04942 ( 2024 ).