Improvements in difusion models have boosted the quality of image generation, which has led researchers, companies, and creators to focus on improving watermarking algorithms. This provision would make it possible to clearly identify the creators of generative art. The main challenges that modern watermarking algorithms face have to do with their ability to withstand attacks and encrypt many unique messages, such as user IDs. In this paper, we present METR: Message Enhanced Tree-Ring, which is an approach that aims to address these challenges. METR is built on the Tree-Ring watermarking algorithm, a technique that makes it possible to encode multiple distinct messages without compromising attack resilience or image quality. This ensures the suitability of this watermarking algorithm for any Difusion Model. In order to surpass the limitations on the quantity of encoded messages, we propose METR++, an enhanced version of METR. This approach, while limited to the Latent Difusion Model architecture, is designed to inject a virtually unlimited number of unique messages. We demonstrate its robustness to attacks and ability to encrypt many unique messages while preserving image quality, which makes METR and METR++ hold great potential for practical applications in real-world settings. Our code is available at https://github.com/deepvk/metr.
Nowadays, image generation is one of the major applications of computer vision technology. It is used
in various spheres, including entertainment [
With the growing popularity of image generation, the risks of it being used inappropriately or
maliciously are also increasing. These risks include policy [
Watermarking has been a widely used technique for image content protection long before the
emergence of generative models. The first works on the subject introduced algorithms that make
it possible to add watermarks to existing images [
X T
STF S
FIT eMsag: 01 . 1
mw X T WM adiusR aSmpling mopt”“Pr
IDM
EMTR Image PAI user ID: 10. servIn IDM
X T
I
TF les fI
ON MW
A diferent group of algorithms was proposed, which add watermarks to images during the generation
process [
In addition to being able to determine whether an image is generated or not, it is also important to
ifnd out the author of the generated image. This can be done by incorporating watermarks that contain
specific information, such as user IDs. Despite the high reliability and imperceptibility of Tree-Ring
watermarks, the algorithm cannot encrypt messages within the watermark, limiting its practical use for
certain real-world scenarios. One of the existing algorithms capable of encrypting messages, Stable
Signature [
In this paper, we propose METR, a watermarking algorithm based on Tree-Ring watermarking [
METR, METR++. It combines METR with Stable Signature [
x = √¯x0 + √
1 distribution (x|x− 1).
Difusion models [
The forward difusion process involves the gradual addition of noise to the initial data point x0: − ¯ , where ∼ (0, I), ¯ are values that parametrize variance at step for D (x) =
√ x − 1 − ¯ · (x, ) √¯
A point from the initial distribution can be approximated by employing a denoising process. While
reverse difusion was characterized as probabilistic in [
x .
obtain x. Meanwhile, x′() represents an estimate of x0 derived from the denoising of x.
0
The determinism of DDIM sampling [30] can be leveraged to trace back the initial noise from which
the image was generated. This process is called “inverse difusion” [
The Tree-Ring watermark, as presented in [
define () = 1 2 We then apply the following equation: = P( 2 ||, < |ℋ0) = 2 (), (1) where 2
||, denotes a non-central chi-squared random variable [31] with = 2 representing its cumulative distribution function [32]. Large p-values indicate the presence of a 1 2 ∑︀|=1| |WM|2, and watermark in the image, while small p-values indicate its absence.
The Latent Difusion Model concept, proposed in [
Drawing on the Latent Difusion Model concept that leverages VAE, the Stable Signature watermarking
method was proposed in [
The inference process for Stable Signature is simple, yet the watermarking algorithm requires a unique Variational Autoencoder (VAE) decoder to be trained for each specific message. In real-world scenarios, especially for services with millions of users where messages are often user IDs, the implementation of Stable Signature is not feasible.
iFne rtue
soL nelat
D 2
W iande-trp uopgr
W
To assess the robustness of watermarking
algorithms, it is common practice to not only
evaluate detection accuracy metrics, but also to
measure the resilience of watermark detection against
attacks [
3.1. METR
Figure 1 presents the pipelines for watermark generation and detection using METR. Similar to
TreeRing [
In METR, messages consist of binary sequences encoded using concentric circles with radii increasing from 1 to , where is defined as the watermark radius. This radius is a fixed hyperparameter representing the number of bits in the message. Therefore, it is possible to encode 2 messages. Each bit of the message is represented by a single circle, where the ones are assigned the value of and the zeros are given the value of − . is another crucial parameter that we call the message scaler. Figure 3 illustrates examples of concentric circles with both encrypted and decrypted messages. Algorithm 1 details the pseudocode for generating an image with message encryption using the METR algorithm.
Algorithm 1: METR image generation
Input: Scaler , radius , binary message
(e.g., user ID)
Output: Generated image 1 x ∼ (0, ); 2 xℱ ← Fourier(x ); 3 for = 1 to do 4 mask ← circle of radius ; 5 if [] = 1 then 6 xℱ [mask] ← ; 7 else 8 xℱ [mask] ← − ; 9 end 10 end 11 xwm ← Inverse Fourier(xℱ ); 12 Image ← DDIM Sampling(xwm); 13 return Image
Input: Image, radius , 0
Output: Predicted watermark 1 x′ ← DDIM Inversion(Image); 2 x′ℱ ← Fourier(x′ ); 3 if 2 ((x′ℱ )) < 0 // see Equation 1 then
return NO WM 4 5 end 6 ← []; 7 for = 1 to do 8 mask ← circle of radius ; 9 if x′ℱ [mask].() > 0 then 10 .(1); 11 else 12 .(0); 13 end 14 end 15 return
The message can be decoded by reverting the image to its noise using DDIM Inversion, followed by a transformation of the inverted image into the Fourier space. Then, we determine whether the image was watermarked or not by evaluating the p-value using Equation 1 and comparing it with a previously defined threshold 0.
The decryption process for a binary message requires determining the sign of the value for each circle, obtained by averaging all values across the circle.
The corresponding pseudocode for message decryption is shown in Algorithm 2. (b) = 60 (c) = 100 (d) = 140
In this section we describe how to select watermark radius and message scaler , which are the parameters of METR watermarking algorithm introduced in Section 3.1, and propose the metric Detection Resolution Metric for selecting the scale parameter.
The increase of the radius parameter leads to the increase of the number of the potential messages, and the enlarging scale makes the message more detectable. However, increasing each of the parameter also leads to the production of corrupted images marked by various visible artifacts. See Figure 4 and 1 and restored watermark: det(x0) = ||
∑︀|=1| |WM − Detect(x0)| here is the watermarked area, WM is a true watermark, x0 is the original image with a possible watermark and “Detect” function is Fourier transform of DDIM Inversion of its argument.
uopgr taioGenr
EAV iwth uTFine2 noise
X T X T
X T EMTR EMTR X T
IDM IDM a watermark x0 and an image with one x*0: det(x0, x*0) = (x0) − (x*0) (2)
A binary message is considered accurately detected if the inverse process error does not negate the value of . This means that − (x*0) < 0 should hold for watermarked images, and − (x0) > 0 for non-watermarked ones. Furthermore, it is essential that the detection resolution metric, which is the error contrast between watermarked and non-watermarked images, is a relatively large value, compared to . To specify this, we computed = det/ for situations with a perfect detection of message, and it occurred that = + , where and are linear fit parameters. It means that to prevent detection collisions, we need ≥ + or det/(2 + ) ≥ 1. During early experiments, we found out that the most reliable and are − 2.23 · 10− 3 and 0.653 respectively.
Since the error of the DDIM Inversion [30] process with a fixed model remains relatively constant with respect to the chosen image, we can assess the detection accuracy for a given value of using just a single pair of (x0, x*0). This way, we bring the selection of down to defining the possible range of its value and selecting the best one based on a subset of images. The algorithm consists of two steps. Initially, it measures the detection resolution using a generated image with and without watermark from the same noise. Then, if the criterion is satisfied, it assesses the image quality across the full test dataset. See Algorithm 3 for details.
Input: latent noise x ∼ (0, ), test dataset D = ⟨prompt, xtrue⟩, image quality threshold Θ, , , radius
Output: best message scaller 1 xℱ ← Fourier(x′ ); 2 min, max ← min(xℱ ), max(xℱ ); 3 for = min to = max; step do 4 x0 ← Generate(x ); 5 x*0 ← Generate(METR(x , , )); 6 0, *0 ← (0), (*0);
det(2+0,*0) ; Det_Res ← if 0 > and *0 < and Det_Res ≥ 1 then test ← Generate from prompts in with WM; ← Eval(test, ) // i.e., FID; if < Θ then
Return S
end
end
7
8
9
10
11
12
13
14
15 end
3.3. METR++
Although METR already provides support for encoding messages into images, the selected radius
still restricts their number. To overcome this, we developed METR++, an extension of the original
METR algorithm. METR++ is designed to significantly increase the original algorithm’s potential for
encoding various unique messages, while being limited in terms of models that it can be applied to. The
extended algorithm can only be used with Latent Difusion [
To encode multiple messages, they are first categorized into groups with a capacity of 2, a size that reflects the number of potential unique messages in the METR watermarking algorithm. Then, each group is assigned a distinct ID, and uses a specifically fine-tuned VAE decoder designed to encode the group’s ID as a Stable Signature watermark. Consequently, METR++ expands the capacity for encoding unique messages within METR, multiplying it by the number of specially fine-tuned VAE decoders for each group.
The identification process consists of two parts, that can be done in parallel. First is the group’s unique key decryption via the Stable Signature approach. Second is the METR message decoding, which identifies a user within a group. Decrypting the METR message requires carrying out the inverse difusion process on the image latent, that is obtained with a VAE encoder part of LDM, which weights stay intact.
To conclude, METR++ can be adapted to any Latent Difusion model and can encrypt approximately 2 · unique messages, where is METR’s watermark radius, and is the number of fine-tuned VAE decoders.
In this section, we describe our experimental setup and present the results of experiments conducted using METR and METR++.
In all experiments, we utilize the base version of Stable Difusion 2.1 [
For the baseline, we selected the Tree-Ring [
We compare the models based on three main
criteria: the accuracy of watermark detection, the
accuracy of watermarked message decryption, and
image quality, since we aim to ensure that the
encrypted message does not degrade the generated
image. The accuracy of watermark detection is
measured by False Positive Rate (FPR), True Positive
Rate (TPR), Area Under Curve (AUC) of Receiver
Operator Characteristic (ROC), TPR value when Figure 7: METR evaluation on image quality
FPR equals to 1%, denoted as “TPR@1%FPR”. The with diferent radii
quality of message detection is assessed using Bit Accuracy, Word Accuracy (the accuracy of fully
detected messages, as proposed in [
We utilized the MSCOCO-5000 dataset [39] to evaluate FID. This dataset consists of 5000 paired images and prompts. To assess watermark detection accuracy, message detection quality, and image quality using CLIP, the images were generated using a subset of 1000 randomly selected prompts from the Stable Difusion prompts [40].
In this subsection, we describe a set of experiments that compare METR with Tree-Ring. We focus on evaluating their robustness to white-box attacks, including ones that utilize generative models, as described in Section 2.4, watermark detection accuracy and the overall quality of the generated images.
To select proper METR parameters, we first searched for the optimal message scale with the algorithm described in Section 3.2 in range 60 ≤ ≤ 160 with multiple diferent radii. The resulting optimal value was always between 80 and 100. For the rest of our experiments, was set to 100 unless specified otherwise. Regarding message radius , we evaluated image quality for diferent radii on a subset of the MSCOCO-5000 dataset with 500 images. See Figure 7 for results.
It is clear that keeping the radius as small as possible benefits the quality of the resulting image. However, we argue that increasing the radius to 16 can still maintain acceptable overall quality. The benefit of doing so is that, this way, METR can encode up to 216 = 65536 messages. In our experiments, we set = 10, which allowed us to encode 1024 messages with almost no loss in quality.
Detection accuracy under image transformation attacks. The detection metrics for cases with
white-box attacks [
Detection accuracy under generative models’ attacks. We performed a difusion attack using
the base version of Stable-Difusion 2.1 [
In this section, we perform a series of experiments to evaluate the robustness of METR++ against
white-box attacks. We also compare its detection metrics with those of METR and the Stable Signature
watermark [
As previously detailed in Section 3.3, METR++ consists of the METR watermark and a fine-tuned VAE
decoder designed to encrypt a 48-bit Stable Signature [
The detection accuracy of METR++ consists of the detection accuracies for both the Stable Signature and METR watermarks. Table 2 presents the results of the detection accuracy evaluation. The term “METR: ft VAE” refers to the accuracy of detecting the METR watermark when processed through the METR++ pipeline. “Stable-Signature” refers to the accuracy of decrypting the referenced watermark using the METR++ pipeline. Lastly, “METR++ ” denotes the overall detection accuracy of the complete METR++ message, which includes both the METR and Stable Signature messages. The detection accuracy of the METR watermark remains unafected by the METR++ pipeline, as the image decoded by the fine-tuned decoder closely resembles the one generated by the original VAE decoder. Consequently, both the Bit and Word accuracy for METR watermark detection, as previously shown in Table 1, demonstrate robustness against most white-box attacks, aside from rotation and cropping. On the other hand, Stable Signature is vulnerable to white-box attacks, which in turn impacts the Word Accuracy of METR++. In Table 1, we highlighted a higher Word Accuracy between the METR and Stable Signature watermarks. It is important to note that the robustness of METR++, in terms of word accuracy, is limited to the lesser robustness of the two watermarks since it requires the correct decryption of both and in terms of bit accuracy it is highly correlated with Stable Signature part, because it has more bits, than METR part of METR++.
In terms of the potential number of messages, we use 58 bits for each message therefore, it is possible to encode 258 unique messages. When considering the practical application of either METR or METR++, it is important to evaluate the trade-of between the capability to encode numerous unique messages and the resilience of the watermarking method against white-box attacks.
In this study, we assessed the detection accuracy of METR and METR++ against several white-box attacks. METR demonstrated high resilience to most of the attacks, including state-of-the-art ones. We also propose researching methods to fine-tune the weights of the generative model to enhance the robustness of the METR watermarking algorithm. When it comes to METR++, this method showed low robustness to most attacks due to its word accuracy robustness being constrained by the lowest detection accuracy among the two messages. The Stable Signature watermark message is generally less robust to attacks than the METR watermark message. Therefore, to increase the encoding capacity of the METR watermarking algorithm, we suggest exploring alternative watermarking methods that could replace Stable Signature, or investigating Stable Signature modifications within METR++ that improve its detection accuracy under attacks.
Another area of research that needs further exploration is the impact of specific messages on the quality of the generated images. Our findings indicate that the quality of images generated using an average METR message is comparable to those produced with the Tree-Ring watermark. However, in real-world applications, it is essential to examine various combinations of binary values to ascertain whether certain messages with “extreme values” might significantly afect image quality. This consideration could be crucial for commercial projects aiming to ensure consistent image generation quality for all their users.
In this paper, we introduced the METR watermarking algorithm, which can be applied to any difusion model architecture with a non-probabilistic sampler (e.g., DDIM [30]). Building upon the Tree-Ring watermarking algorithm, METR retains its robustness in detection under white-box attacks and high image quality. The most significant advantage of METR is its ability to encrypt multiple unique messages without the need for model’s weights fine-tuning. This positions the proposed watermarking algorithm as one that can be considered the current leading watermarking algorithm in terms of robustness, image quality, and message encoding capacity. We also propose an extension of METR, named METR++, which is specifically tailored to Latent Difusion Models and requires additional ifne-tuning of a VAE decoder for each new user group. METR++ increases the potential number of encoded messages by the factor of fine-tuned VAE decoders. Comparing METR with its extension METR++ shows that the decision of which algorithm to use for practical applications should balance the total number of messages that can be encoded and the watermark’s overall robustness to attacks. Implementation of our work can be found at https://github.com/deepvk/metr.
arXiv:1807.09937. [26] K. A. Zhang, L. Xu, A. Cuesta-Infante, K. Veeramachaneni, Robust invisible video watermarking with attention, 2019. arXiv:1909.01285. [27] M. Tancik, B. Mildenhall, R. Ng, Stegastamp: Invisible hyperlinks in physical photographs, 2020.
arXiv:1904.05343. [28] Z. Jiang, J. Zhang, N. Z. Gong, Evading watermark based detection of ai-generated content, 2023.
arXiv:2305.03807. [29] Y. Liu, Z. Li, M. Backes, Y. Shen, Y. Zhang, Watermarking difusion model, 2023.
arXiv:2305.12502. [30] J. Song, C. Meng, S. Ermon, Denoising difusion implicit models, 2022. arXiv:2010.02502. [31] P. B. Patnaik, The non-central 2- and -distribution and their applications, Biometrika 36 (1949) 202–232. URL: https://www.jstor.org/stable/2332542. doi:10.2307/2332542. [32] P. Glasserman, Monte Carlo Methods in Financial Engineering, volume 53 of Stochastic Modelling and Applied Probability, Springer, New York, NY, 2003. URL: http://link.springer.com/10.1007/ 978-0-387-21617-1. doi:10.1007/978-0-387-21617-1. [33] S. Peng, Y. Chen, C. Wang, X. Jia, Intellectual property protection of difusion models via the watermark difusion process, 2023. arXiv:2306.03436. [34] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, N. Johnston, Variational image compression with a scale hyperprior, 2018. arXiv:1802.01436. [35] E. O. Brigham, R. E. Morrow, The fast fourier transform, IEEE Spectrum 4 (1967) 63–70. doi:10.
1109/MSPEC.1967.5217220. [36] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter, Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018. arXiv:1706.08500. [37] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, 2021. arXiv:2103.00020. [38] M. Cherti, R. Beaumont, R. Wightman, M. Wortsman, G. Ilharco, C. Gordon, C. Schuhmann, L. Schmidt, J. Jitsev, Reproducible scaling laws for contrastive language-image learning, 2022. arXiv:2212.07143. [39] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick, Microsoft coco: Common objects in context, in: European conference on computer vision, Springer, 2014, pp. 740–755. [40] Gustavosta, Stable difusion prompts, 2022. URL: https://huggingface.co/datasets/Gustavosta/ Stable-Difusion-Prompts.