Guaranteeing a certain level of user privacy in an arbitrary piece of text is a challenging issue. However, with this challenge comes the potential of unlocking access to vast data stores for training machine learning models and supporting data driven decisions. We address this problem through the lens of d -privacy, a generalization of Di erential Privacy to non Hamming distance metrics. In this work, we explore word representations in Hyperbolic space as a means of preserving privacy in text. We provide a proof satisfying d -privacy, then we de ne a probability distribution in Hyperbolic space and describe a way to sample from it in high dimensions. Privacy is provided by perturbing vector representations of words in high dimensional Hyperbolic space to obtain a semantic generalization. We conduct a series of experiments to demonstrate the tradeo between privacy and utility. Our privacy experiments illustrate protections against an authorship attribution algorithm while our utility experiments highlight the minimal impact of our perturbations on several downstream machine learning models. Compared to the Euclidean baseline, we observe > 20x greater guarantees on expected privacy against comparable worst case statistics.
Copyright ©2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Presented at the PrivateNLP 2020 Workshop on Privacy in Natural Language Processing Colocated with 13th ACM International WSDM Conference, 2020, in Houston, Texas, USA.
PrivateNLP ’20, February 7, 2020, Houston, TX, USA © 2020 SUMMARY • User’s goal: meet some specific need with respect to a query x • Agent’s goal: satisfy the user’s request • Question: what occurs when x is used to make other inferences • Mechanism: Modify the query to protect privacy whilst preserving semantics • Our approach:
Hyperbolic Metric Differential Privacy METRIC DIFFERENTIAL PRIVACY
P[M(x) 2 E] e" dh(x,x0)P [M (x0) 2 E] + . HYPERBOLIC WORD EMBEDDINGS