Introduction & Motivation

In public health, contact tracing is the process of identifying people who may have been exposed to an infected person, subsequent testing them for infection, and isolating or treating the infected [1]. Contact tracing performance criteria include infection suppression, protection of high-risk individuals, and cost-efficiency. These criteria are not necessarily aligned with each other, e.g., given testing capacity, infection suppression requires high priority testing for the potential super spreaders, while protection of high-risk individuals requires testing them with higher priority. Given the testing priorities, the existing Google/Apple Exposure Notification (GAEN) technology [2] can support an Exposure Notification System (ENS) by allowing public health authorities to quickly notify people for subsequent testing. GAEN is a framework and protocol specification developed by Apple Inc. and Google to facilitate digital contact tracing during the COVID-19 pandemic to augment more traditional contact tracing techniques using Android or iOS smartphones.

Extensive research on COVID-19 has revealed that while risk-aware, multi-criteria optimization of contact tracing has significant potential, realization of this potential requires deep knowledge of the infection propagation mechanisms, medical prognoses and treatment options for infected individuals with different risk profiles [3]. Even though COVID-19 originated almost five years ago, such knowledge is still lacking [4,5], which suggests that a contact tracing system should have the ability to collect and make sense of all up-to-date available information on infection. This can be achieved with the Certain equipment, instruments, software, or materials are identified in this paper in order to specify the experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement of any product or service by NIST, nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose. Envelope marbukh@nist.gov (V. Marbukh) support of a specialized Recommender System (RS). Given testing capacity, the RS should utilize the upto-date contact tracing, medical, and all other available relevant data to identify and through Exposure Notification System (ENS) notify individuals to be tested [2,6]. Due to incomplete information, the dynamic nature of the problem, and a large state and action space the RS should be supported by Deep Reinforcement Learning (DRL) for solving the corresponding Partially Observable Markov Decision Process (POMDP) [7,8]. POMDP describes the evolution of health status of each participating individual, where infectious status may not be observable and testing decisions are constrained by available testing capacity. Since a positive test result for some individual may reveal increased accumulated exposure for other individuals due to proximity to the newly discovered infection spreaders, the problem cannot be decoupled. These interdependencies significantly complicate the problem. The paper is organized as follows. Section 2 outlines operations and flow of information in the proposed RS supported contact tracing, and section 3 provides some technical details on accumulated exposure evaluation. 𝑖 . Note that participating individuals are likely to consent to revealing their health information since they would benefit from accounting for their risk factors, e.g., advanced age, preexisting conditions, etc. For not participating individuals, some relevant information, which does not require revealing individual identity, can be obtained without violating their privacy.

RS Supported Contact Tracing

RS is also fed the estimate of the infection reproduction number 𝑅(𝑡), i.e., the average number of new infections produced by one infected individual during his/her lifetime: R (𝑡) ≈ 𝑅(𝑡). Estimate R (𝑡) may combine information from EMS, the Health Care System, and possibly from other tracing mechanisms not shown in Figure 1, e.g., from manual tracing. Infection suppression requires keeping the infection reproduction number less than one: 𝑅(𝑡) < 1. Due to numerous uncertainties in the 𝑅(𝑡) estimation: R (𝑡) ≈ 𝑅(𝑡), the infection suppression condition is R (𝑡) ≤ 1 − 𝜀, where "safety margin" 𝜀 < 1 depends on the confidence level of the corresponding estimate. The reward of the RS supported Contact Tracing is quantified be the negative loss −𝐿(𝑡), where 𝐿(𝑡) = 𝐿 𝑒𝑐 (𝑡) + 𝐿 𝑠𝑐 (𝑡). Here economic loss due to lost productivity and cost of testing/treatment is 𝐿 𝑒𝑐 (𝑡), and "social cost" quantifying suffering and, most importantly, deaths due to the infection 𝐿 𝑠𝑐 (𝑡). The cost estimates are provided to RS by the Health Care System and Agencies collecting and processing economic data.

System evolution is described by POMDP 𝛿(𝑡) = (𝛿 𝑖 (𝑡)), where component 𝛿 𝑖 (𝑡) describes the health status of participating individual 𝑖 , i.e., "non-infected," "infected," "deceased." "Non-infected" and "infected" states may not be observable which makes process 𝛿(𝑡) partially observable. The decision to test a participating individual reveals his/her infected or not-infected status at a certain cost due to limited testing capacity. RL employs DRL to make testing decisions on the basis of

ℰ [0,𝑡] 𝑖 , 𝑥(1)

𝑖 , 𝑥

𝑖 . Constraints on the infection reproduction number can be incorporated through penalty function ℎ( R (𝑡)) which is flat for R (𝑡) ≤ 1 − 𝜀 and sharply increases for R (𝑡) > 1 − 𝜀.

Our conjecture is that (near) optimal notification strategy is threshold-based: individual 𝑖 should be notified at the first moment 𝑡 = 𝜃 𝑖 > 0 when this individual's accumulated exposure reaches threshold Ê 𝑖 :

𝜃 𝑖 = inf 𝑡≥0 {𝑡 ∶ ℰ [0,𝑡] 𝑖 ≥ Ê 𝑖 },(1)

where threshold Ê 𝑖 = Δ(ℰ , 𝑥) depends on the history of former testing decisions/results combined with medical and demographic data of these individuals. The function Δ(ℰ , 𝑥) can be evaluated by employing a Deep Supervised Learning (DSL) algorithm. Note that in practice, notification strategy may operate on the basis of a small number of risk groups [3], which may be defined and then redefined by an on-line clustering algorithm. Assumptions of homogeneity and large number of individuals within each group, simplifies optimization of group-specific thresholds in (1).

Accumulated Exposure

For each participating individual 𝑖, the contact tracing system identifies "accumulated exposure" to another participating individual 𝑗 during time interval [0, 𝑡] as follows:

ℰ [0,𝑡] 𝑖𝑗 = ∫ 𝑇 0 𝜙 [( d/𝑑 𝑖𝑗 (𝜏 )) 𝛼 ] 𝑑𝜏 ,(2)

where the corresponding instantaneous exposure rate 𝜙(𝑧) is an increasing function of 𝑧 > 0, the distance between individuals 𝑖 and 𝑗 at moment 𝜏 is 𝑑 𝑖𝑗 (𝜏 ), and d > 0, 𝛼 ≥ 1 are some parameters. Individual 𝑖 accumulated exposure to infection during time interval [0, 𝑡] is defined as the aggregated exposure to all known spreaders during this time interval:

ℰ [0,𝑡] 𝑖 = ∑ 𝑗 ∫ 𝑇 0 𝜋 𝑗 (𝜏 )𝜙 [( d/𝑑 𝑖𝑗 (𝜏 )) 𝛼 ] 𝑑𝜏 ,(3)

where 𝜋 𝑗 (𝜏 ) = 1 if individual 𝑗 is infected at moment 𝜏 and 𝜋 𝑗 (𝜏 ) = 0 otherwise. Consider some examples. As currently defined by the CDC [1], a high-risk COVID-19 exposure is a contact with a person who tests/tested positive for SARS-CoV-2 which takes place at a distance of less than two meters for a total of 15 minutes or more over a 24-hour period. In this case, d = 2 m, 𝛼 → ∞, 𝜙(𝑥) ≡ min(𝑥, 1), and thus an individual is assumed exposed if ℰ [0,𝑇 ] = ∫ 24 0 𝟙(𝑑(𝑡) − 2 m)𝑑𝑡 > 15 min, where 𝟙(𝑥) = 0 if 𝑥 ≤ 0 and 𝟙(𝑥) = 1 otherwise. In another example [9], d = 2 m, 𝜙(𝑥) ≡ 𝑥, and thus an individual is assumed exposed if ℰ [0,𝑇 ] = ∫ 24 0 (2/𝑑(𝑡)) 𝛼 𝑑𝑡 > 15 min. Finally note that available information on accumulated exposure to specific individuals can be used to identify "infection superspreaders" who otherwise could be unidentified, e.g., due to being asymptomatic or for any other reason. This can be done with known algorithms [10] on undirected exposure graph 𝐺 where nodes 𝑖 and 𝑗 are connected if ℰ𝑖𝑗 [0,𝑡] ≥ Ȇ [0,𝑡] , and Ȇ [0,𝑡] > 0 is a properly defined threshold.