<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Age-related Performance Decline Detected by a Virtual Reality Multidomain Cognitive Training Solution (Enhance VR)</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Victòria Brugada-Ramentol</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Maiya Morgan</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Hossein Jalali</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Amir Bozorgzadeh</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Virtuleap</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Portugal</string-name>
        </contrib>
      </contrib-group>
      <abstract>
        <p>Cognitive training systems aim to improve specific domains or global cognition by engaging users in cognitively demanding tasks. While screen-based applications can improve performance in the trained cognitive abilities, they often result in poor transferability to activities of daily living. These systems exclude the user's body and motor skills. Immersive Virtual Reality (IVR) systems, in contrast, present the user with body-related information, such as proprioceptive and visuomotor information, allowing for an immersive and embodied experience of the environment. This feature renders VR a very appealing tool for cognitive training and neurorehabilitation applications. Enhance VR is an IVR-based cognitive training and monitoring application that ofers short daily cognitive workouts. The games are designed to train and monitor specific cognitive domains (memory, task flexibility, information processing, orientation, attention, problem-solving, and motor control). Enhance VR enables the collection of a large and varied behavioral dataset. Our analysis showed that age is a determinant of performance, as measured by the Enhance VR scoring system, with score games decreasing with age. Overall, suggesting that the Enhance VR app can potentially capture informative datasets that predict individual cognitive variables.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Cognitive training</kwd>
        <kwd>Cognitive monitoring</kwd>
        <kwd>Immersive virtual reality</kwd>
        <kwd>Age-related decline</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Cognitive Training</title>
      <p>
        ever, screen-based CCT solutions have failed to show
transfer to untrained cognitive categories and tasks and
Cognitive decline, the gradual deterioration of cognitive ADLs [
        <xref ref-type="bibr" rid="ref3 ref8">3, 8</xref>
        ]. Furthermore, computer-based interventions
abilities and functioning, progresses in parallel with age showed improvements in cognition and non-cognitive
and becomes more pronounced in populations at risk of measures (e.g., mood), but failed to improve ADLs in
neurodegenerative diseases. The deterioration of cogni- people with dementia [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. A 12-week program showed
tive abilities has a direct influence on the execution of improved capacities in early-stage Alzheimer’s disease
activities of daily living (ADLs) and negatively afects (AD), without positive efects on ADLs [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
autonomy and well-being. A combination of a healthy The lack of ecological validity in screen-based CCT
diet, moderate exercise, and cognitive stimulation have could, in part, explain why these systems do not result in
been proposed as strategies to attempt to slow down the a transfer of benefits to ADLs and general cognition. VR
progression of cognitive decline [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. systems, on the other hand, yield ecologically valid
envi
      </p>
      <p>
        Computerized cognitive training (CCT) has the po- ronment scenarios with precise control over the
experitential to become a non-pharmacological intervention mental variables [
        <xref ref-type="bibr" rid="ref11 ref12">11, 12</xref>
        ]. VR has shown to be potentially
to maintain the cognitive functioning of cognitively im- relevant for cognitive training in the elderly population,
paired individuals. CCT has been shown to reduce cog- as a result of many diferent features, such as the
flexibilnitive decline in healthy aging [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], but with no clear ef- ity of the environments and the possibility to gather rich
fectiveness in delaying dificulties in instrumental ADLs. data, as a result of increased immersion [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Ultimately,
CCT programs have shown an improvement in trained improving cognitive functioning in healthy and at-risk
cognitive abilities (i.e., memory, reasoning, or speed-of- older adults and promoting the transfer of benefits.
processing) [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], which was maintained for 5 [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] or even
10 years [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. A meta-analysis found a small and
significant efect on cognitive function in healthy older adults
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Overall, CCT has shown promising, but
heterogeneous results in global cognitive functioning [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ].
How
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Virtual Reality Environments as</title>
      <p>a Cognitive Training and</p>
    </sec>
    <sec id="sec-3">
      <title>Monitoring Tool</title>
      <p>
        VR scenarios ofer many advantages over screen-based
cognitive training methodologies. Traditional
methodologies have to trade external validity (i.e., how the task
accurately represents the measure in real-life scenarios)
and the internal validity of conducting the tasks in a experience is a gamified and engaging representation of
controlled laboratory setting with controlled variables the neuropsychological validated tests. The Enhance VR
[
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. VR, on the other hand, ofers realistic scenarios is compatible with a wide range of commercially available
with a high degree of control over desirable experimental headsets.
variables [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], such as the stimuli, and the presence of The Enhance VR app consists of a library of games
distractors, among others. Additional advantages of VR that engages seven main cognitive categories. As of
Ocsystems include the possibility of creating situations that tober 2022, the library ofers 15 games divided across
are adapted to the individual’s needs or that are impossi- all of these skills: Pizza Builder, Whack-a-mole, Shufled
ble to recreate in real-life scenarios and collecting precise (Attention), React (Flexibility), Assembly and Harmonize
measurements of physical movements while sourcing (Information processing), Memory Wall, Maestro, Magic
large amounts of behavioral data [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. Deck (Memory), Stacker and Odd Egg (Problem-solving),
      </p>
      <p>
        Moreover, immersive VR (IVR) systems that are dis- Slinger and Balance (Motor Skills), and Orbital and Hide
played through a head-mounted display with embedded and Seek (Spatial Orientation) ([
        <xref ref-type="bibr" rid="ref21">21</xref>
        ], Table 1).
head-tracking, which updates the environment according The design of each individual Enhance VR game can be
to the participant’s movement, allow for a naturalistic compared to other workflows proposed for the design of
interaction with the environment. In these scenarios, the cognitive assessment exercises [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. Whenever the need
behavior of the environment matches the expected phys- for a specific cognitive category is identified, diferent
ical and motor consequences of the participant’s action. classical paradigms are selected and evaluated to
maxiThis is also referred to as the plausibility illusion. Fur- mize the training abilities in an IVR environment. Once
thermore, IVR environments engage the sensorimotor the candidate paradigm has been identified, the extensive
system enhancing the illusion of embodiment over a vir- literature review identifies task requirements. Namely,
tual body or body part [
        <xref ref-type="bibr" rid="ref15 ref16">15, 16</xref>
        ]. By providing immersive (1) the mechanics of the paradigm, (2) the actions that are
experiences of the scenarios, virtual environments pro- required from the participant, (3) the category and type
vide the feeling that the virtual world is the real world of stimuli, (4) the outcome measures that evaluate the
and provide the feeling of actually “being” in the vir- specific cognitive ability, and (5) identify the parameters
tual environment (i.e., placement illusion). The feeling that can be modified to increase the dificulty of the task.
of presence in a virtual environment has been shown to These parameters will define the design of the gamified
increase motivation for learning and attention to the task virtual reality exercise, developed by a team of game
[
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. designers, developers, artists, and quality assessment.
      </p>
      <p>
        Sensorimotor contingencies are, thus, important for The dificulty of the Enhance VR app is dependent
immersion. VR systems are particularly interesting for and controlled by adjusting the relevant parameters for
this matter, as they allow for the integration of proprio- each exercise. The Enhance VR app tests the baseline
ceptive, visual, and motor information [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. Ultimately, performance for each game by presenting a first longer
VR scenarios provide a high degree of realism that elicit session (hereafter, benchmark session). From there on,
naturalistic behaviors from participants [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. The natural- every session starts at the level where the user left of.
istic interaction with the environment provides a higher
degree of ecological validity than current screen-based
or pen-and-paper solutions. As a result, VR systems
become interesting tools for the training and rehabilitation
of cognitive functions in distinct populations [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ].
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. Enhance VR Enables the</title>
    </sec>
    <sec id="sec-5">
      <title>Collection of Behavioral Data</title>
      <p>for the Detection of Cognitive</p>
    </sec>
    <sec id="sec-6">
      <title>Performance</title>
    </sec>
    <sec id="sec-7">
      <title>3. Developing a Cognitive</title>
    </sec>
    <sec id="sec-8">
      <title>Training and Monitoring</title>
    </sec>
    <sec id="sec-9">
      <title>Solution in VR</title>
      <p>The Enhance VR app (https://virtuleap.com/download/)
is designed to be played regularly in the form of short
workouts. Each workout is composed of three randomly
chosen games. The mechanics of each game is motivated
by the mechanics of validated neuropsychological
principles. Every game stems from the collaboration between
scientists and game designers to ensure that the
mechanics of each test are maintained. At the same time, the
Individual progression is tracked by the Enhance VR
Performance Index (EPI), calculated as an aggregate of
weighted performance across all cognitive categories in
addition to scores for each game, the main cognitive
categories, and subcategories scores. The app collects
data on self-reported mood and sleeping hours at the start
of each workout. Upon registration, users are required
to input demographic data, such as age. The Enhance VR
app is compliant with GDPR and HIPAA regulations.</p>
      <p>
        IVR systems enable the collection of large and varied
behavioral datasets [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. As the participants engage with
the Enhance VR games, the system collects not only the
performance of the participants but also a large number
of game-related variables. Some of these variables
include motor outputs (e.g., movement coordinates or hand
preferences), and reaction times, among others. Thus,
the Enhance VR app collects behavioral data obtained
by game-specific events. These variables would provide
a sensitive description of the individual cognitive
status and allow for a self-to-self comparison of cognitive
changes. Thus, we propose that the behavioral data from
Enhance VR could help identify age-related cognitive
decline.
4.1. Enhance VR Scores are Afected by
      </p>
      <p>Age
The Enhance VR app is available commercially in VR
stores, such as HTC Vive Port and Oculus store. Data
collected from these users is anonymized. Users with
nonplausible age ranges (&gt;100 years old) or a combination
of age and education (e.g. those who reported being 12
years old with a Master’s degree) were removed from the
analysis. Age groups above 80 were excluded from the
analysis due to insuficient data points for all the games.</p>
      <p>The analysis was performed on a sample of 21,230 users.</p>
      <p>Furthermore, only data from the benchmark sessions
were selected.</p>
      <p>We observe a decreasing trend in the cognitive scores
measured by the Enhance VR app with increased age
groups. The highest median value was obtained by the
groups 11-20 and 21-30. The lowest median score was
observed in the 71-80 group (Figure 1).
ful co-design between scientific and development teams.</p>
      <p>Age-related cognitive decline can also be related to an
increasing prevalence of systemic disorders that are also
associated with cognitive impairment.</p>
      <p>
        The advantages of IVR are not only restricted to
cognitive training but also to provide immersive scenarios for
cognitive assessment [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ]. As a result of the naturalistic
interaction and realistic scenarios, VR cognitive
assess5. Discussion ment ofers the opportunity to accurately measure the
individual’s cognitive performance. Digital biomarkers
IVR scenarios ofer an ecologically valid environment are objective, quantifiable physiological and behavioral
that can potentially improve global cognitive functions data that are collected by digital devices. Previous
studand have the potential to transfer benefits to untrained ies have found a positive correlation between behavioral
cognitive abilities. The Enhance VR app takes advantage data obtained from gameplay patterns from specialized
of the immersive and ecological validity of VR environ- and of-the-shelf games such as mouse clicks and
cogniments to provide a structured and controlled setting for tive function [
        <xref ref-type="bibr" rid="ref25 ref26">25, 26</xref>
        ]. Thus, these data could be used to
cognitive training. The Enhance VR app ofers mentally monitor biological or pathogenic processes and assess
challenging exercises that aim to train specific cogni- responses to pharmacological interventions [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ].
tive skills presented in a structured manner and extracts The Enhance VR collects longitudinal behavioral
inforbehavioral measures of cognitive fitness [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]. mation that, when combined with supervised machine
      </p>
      <p>Increasing age has been previously related to a slow learning algorithms and clinically relevant data, could
decline in cognitive abilities. Our preliminary analysis provide novel non-invasive digital biomarkers of
cognishows that the Enhance VR app scoring system reflects tive status. Ultimately, the ability to identify patterns
an age-related trend of performance decline. Thus, sug- of impaired cognitive and motor performance at a
pregesting that the Enhance VR app has the potential to clinical stage could act as the earliest manifestations of
describe the individual profile. The next steps include the cognitive impairment and would be key in the
developanalysis of behavioral data. Evaluation of the Enhance ment of prevention and treatment strategies. Integrating
VR app will require confirmation of the face and content these digital biomarkers with current diagnostic methods
validity of the tests since the inclusion of a strong motor could provide an earlier, more sensitive, and specific
decomponent may add confounding variables despite care- tection of cognitive changes, as well as monitor disease
progression and act as proxies for the eficacy of
pharmacological interventions in clinical trials for
diseasemodifying drugs, as well as side efects.</p>
      <p>The potential application of Enhance VR is not
exclusive to age-related diseases cognitive impairment is a
hallmark of several diseases and conditions, including
ADHD, traumatic brain injury, multiple sclerosis, long
COVID, and even a common side efect of chemotherapy.</p>
      <p>The large diversity of areas covered by our games
enables the creation of curated playlists for these diferent
clinical indications, maximizing our reach and impact.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>T.</given-names>
            <surname>Ngandu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Lehtisalo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Solomon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Levälahti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Ahtiluoto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Antikainen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Bäckman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Hänninen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Jula</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Laatikainen</surname>
          </string-name>
          , et al.,
          <article-title>A 2 year multidomain intervention of diet, exercise, cognitive training, and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (finger): a randomised controlled trial</article-title>
          ,
          <source>The Lancet</source>
          <volume>385</volume>
          (
          <year>2015</year>
          )
          <fpage>2255</fpage>
          -
          <lpage>2263</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>C.</given-names>
            <surname>Hertzog</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. F.</given-names>
            <surname>Kramer</surname>
          </string-name>
          , R. S. Wilson, U. Lindenberger,
          <article-title>Enrichment efects on adult cognitive development: can the functional capacity of older adults be preserved and enhanced?, Psychological science in the public interest 9 (</article-title>
          <year>2008</year>
          )
          <fpage>1</fpage>
          -
          <lpage>65</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>K.</given-names>
            <surname>Ball</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. B.</given-names>
            <surname>Berch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. F.</given-names>
            <surname>Helmers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. B.</given-names>
            <surname>Jobe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. D.</given-names>
            <surname>Leveck</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Marsiske</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. N.</given-names>
            <surname>Morris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. W.</given-names>
            <surname>Rebok</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. M.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Tennstedt</surname>
          </string-name>
          , et al.,
          <article-title>Efects of cognitive training interventions with older adults: a randomized controlled trial</article-title>
          ,
          <source>Jama</source>
          <volume>288</volume>
          (
          <year>2002</year>
          )
          <fpage>2271</fpage>
          -
          <lpage>2281</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Willis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Tennstedt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Marsiske</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Ball</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Elias</surname>
          </string-name>
          ,
          <string-name>
            <surname>K. M. Koepke</surname>
            ,
            <given-names>J. N.</given-names>
          </string-name>
          <string-name>
            <surname>Morris</surname>
            ,
            <given-names>G. W.</given-names>
          </string-name>
          <string-name>
            <surname>Rebok</surname>
            ,
            <given-names>F. W.</given-names>
          </string-name>
          <string-name>
            <surname>Unverzagt</surname>
            ,
            <given-names>A. M.</given-names>
          </string-name>
          <string-name>
            <surname>Stoddard</surname>
          </string-name>
          , et al.,
          <article-title>Long-term efects of cognitive training on everyday functional outcomes in older adults</article-title>
          ,
          <source>Jama</source>
          <volume>296</volume>
          (
          <year>2006</year>
          )
          <fpage>2805</fpage>
          -
          <lpage>2814</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>G. W.</given-names>
            <surname>Rebok</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Ball</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. T.</given-names>
            <surname>Guey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. N.</given-names>
            <surname>Jones</surname>
          </string-name>
          , H.- Y. Kim,
          <string-name>
            <given-names>J. W.</given-names>
            <surname>King</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Marsiske</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. N.</given-names>
            <surname>Morris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Tennstedt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. W.</given-names>
            <surname>Unverzagt</surname>
          </string-name>
          , et al.,
          <article-title>Ten-year efects of the advanced cognitive training for independent and vital elderly cognitive training trial on cognition and everyday functioning in older adults</article-title>
          ,
          <source>Journal of the American Geriatrics Society</source>
          <volume>62</volume>
          (
          <year>2014</year>
          )
          <fpage>16</fpage>
          -
          <lpage>24</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A.</given-names>
            <surname>Lampit</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Hallock</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Valenzuela</surname>
          </string-name>
          ,
          <article-title>Computerized cognitive training in cognitively healthy older adults: a systematic review and meta-analysis of efect modifiers</article-title>
          ,
          <source>PLoS medicine 11</source>
          (
          <year>2014</year>
          )
          <article-title>e1001756</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lasaponara</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Marson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Doricchi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Cavallo</surname>
          </string-name>
          ,
          <article-title>A scoping review of cognitive training in neurodegenerative diseases via computerized and virtual reality tools: What we know so far</article-title>
          ,
          <source>Brain Sciences</source>
          <volume>11</volume>
          (
          <year>2021</year>
          )
          <fpage>528</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Owen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Hampshire</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Grahn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Stenton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Dajani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. S.</given-names>
            <surname>Burns</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. J.</given-names>
            <surname>Howard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. G.</given-names>
            <surname>Ballard</surname>
          </string-name>
          ,
          <article-title>Putting brain training to the test</article-title>
          ,
          <source>Nature</source>
          <volume>465</volume>
          (
          <year>2010</year>
          )
          <fpage>775</fpage>
          -
          <lpage>778</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>J. A.</given-names>
            <surname>García-Casal</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Loizeau</surname>
          </string-name>
          , E. Csipke, M. FrancoMartín,
          <string-name>
            <given-names>M. V.</given-names>
            <surname>Perea-Bartolomé</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Orrell</surname>
          </string-name>
          ,
          <article-title>Computer-based cognitive interventions for people living with dementia: a systematic literature review and meta-analysis</article-title>
          ,
          <source>Aging &amp; mental health 21</source>
          (
          <year>2017</year>
          )
          <fpage>454</fpage>
          -
          <lpage>467</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>B. S. A.</given-names>
            <surname>Polat</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Karadas</surname>
          </string-name>
          ,
          <article-title>Evaluation of a computerbased cognitive training program for early-stage alzheimer's disease</article-title>
          ,
          <source>Ann Clin Analyt Med</source>
          <volume>13</volume>
          (
          <year>2022</year>
          )
          <fpage>175</fpage>
          -
          <lpage>179</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>C. J.</given-names>
            <surname>Bohil</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Alicea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. A.</given-names>
            <surname>Biocca</surname>
          </string-name>
          ,
          <article-title>Virtual reality in neuroscience research and therapy</article-title>
          ,
          <source>Nature reviews neuroscience 12</source>
          (
          <year>2011</year>
          )
          <fpage>752</fpage>
          -
          <lpage>762</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>T. D. Parsons</surname>
          </string-name>
          ,
          <article-title>Virtual reality for enhanced ecological validity and experimental control in the clinical, afective and social neurosciences, Frontiers in human neuroscience 9 (</article-title>
          <year>2015</year>
          )
          <fpage>660</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>A. C. M. Bauer</surname>
            ,
            <given-names>G. Andringa,</given-names>
          </string-name>
          <article-title>The potential of immersive virtual reality for cognitive training in elderly</article-title>
          ,
          <source>Gerontology</source>
          <volume>66</volume>
          (
          <year>2020</year>
          )
          <fpage>614</fpage>
          -
          <lpage>623</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>A. J.</given-names>
            <surname>Martingano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Persky</surname>
          </string-name>
          ,
          <article-title>Virtual reality expands the toolkit for conducting health psychology research, Social and personality psychology compass 15 (</article-title>
          <year>2021</year>
          )
          <article-title>e12606</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>V.</given-names>
            <surname>Brugada-Ramentol</surname>
          </string-name>
          , I. Clemens, G. G. de Polavieja,
          <article-title>Active control as evidence in favor of sense of ownership in the moving virtual hand illusion</article-title>
          ,
          <source>Consciousness and cognition 71</source>
          (
          <year>2019</year>
          )
          <fpage>123</fpage>
          -
          <lpage>135</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>M. V.</given-names>
            <surname>Sanchez-Vives</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Spanlang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Frisoli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Bergamasco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Slater</surname>
          </string-name>
          ,
          <article-title>Virtual hand illusion induced by visuomotor correlations</article-title>
          ,
          <source>PloS one 5</source>
          (
          <year>2010</year>
          )
          <article-title>e10381</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>J. N.</given-names>
            <surname>Bailenson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Yee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Blascovich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. C.</given-names>
            <surname>Beall</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Lundblad</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Jin</surname>
          </string-name>
          ,
          <article-title>The use of immersive virtual reality in the learning sciences: Digital transformations of teachers, students, and social context</article-title>
          ,
          <source>The Journal of the Learning Sciences</source>
          <volume>17</volume>
          (
          <year>2008</year>
          )
          <fpage>102</fpage>
          -
          <lpage>141</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>M. V.</given-names>
            <surname>Sanchez-Vives</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Slater</surname>
          </string-name>
          ,
          <article-title>From presence to consciousness through virtual reality</article-title>
          ,
          <source>Nature Reviews Neuroscience</source>
          <volume>6</volume>
          (
          <year>2005</year>
          )
          <fpage>332</fpage>
          -
          <lpage>339</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>G.</given-names>
            <surname>Riva</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. K.</given-names>
            <surname>Wiederhold</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Mantovani</surname>
          </string-name>
          ,
          <article-title>Neuroscience of virtual reality: from virtual exposure to embodied medicine</article-title>
          ,
          <source>Cyberpsychology, behavior, and social networking 22</source>
          (
          <year>2019</year>
          )
          <fpage>82</fpage>
          -
          <lpage>96</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>D.</given-names>
            <surname>Perez-Marcos</surname>
          </string-name>
          ,
          <article-title>Virtual reality experiences, embodiment, videogames and their dimensions in neurorehabilitation</article-title>
          ,
          <source>Journal of neuroengineering and rehabilitation 15</source>
          (
          <year>2018</year>
          )
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>V.</given-names>
            <surname>Brugada-Ramentol</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bozorgzadeh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Jalali</surname>
          </string-name>
          ,
          <article-title>Enhance vr: A multisensory approach to cognitive training and monitoring</article-title>
          ,
          <source>Frontiers in Digital Health</source>
          <volume>4</volume>
          (
          <year>2022</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>S.</given-names>
            <surname>Krohn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Tromp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. M.</given-names>
            <surname>Quinque</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Belger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Klotzsche</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Rekers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Chojecki</surname>
          </string-name>
          , J. de Mooij,
          <string-name>
            <given-names>M.</given-names>
            <surname>Akbal</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>McCall</surname>
          </string-name>
          , et al.,
          <article-title>Multidimensional evaluation of virtual reality paradigms in clinical neuropsychology: Application of the vr-check framework</article-title>
          ,
          <source>Journal of Medical Internet Research</source>
          <volume>22</volume>
          (
          <year>2020</year>
          )
          <article-title>e16724</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>G.</given-names>
            <surname>Sala</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. D.</given-names>
            <surname>Aksayli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. S.</given-names>
            <surname>Tatlidil</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tatsumi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Gondo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Gobet</surname>
          </string-name>
          ,
          <article-title>Near and far transfer in cognitive training: A second-order meta-analysis</article-title>
          ,
          <source>Collabra: Psychology</source>
          <volume>5</volume>
          (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>P.</given-names>
            <surname>Kourtesis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Collina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. A.</given-names>
            <surname>Doumas</surname>
          </string-name>
          ,
          <string-name>
            <surname>S. E.</surname>
          </string-name>
          <article-title>MacPherson, Validation of the virtual reality everyday assessment lab (vr-eal): An immersive virtual reality neuropsychological battery with enhanced ecological validity</article-title>
          ,
          <source>Journal of the International Neuropsychological Society</source>
          <volume>27</volume>
          (
          <year>2021</year>
          )
          <fpage>181</fpage>
          -
          <lpage>196</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>P.</given-names>
            <surname>Dagum</surname>
          </string-name>
          ,
          <article-title>Digital biomarkers of cognitive function</article-title>
          ,
          <source>NPJ digital medicine 1</source>
          (
          <year>2018</year>
          )
          <fpage>10</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>R. L.</given-names>
            <surname>Mandryk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. V.</given-names>
            <surname>Birk</surname>
          </string-name>
          ,
          <article-title>The potential of game-based digital biomarkers for modeling mental health</article-title>
          ,
          <source>JMIR mental health 6</source>
          (
          <year>2019</year>
          )
          <article-title>e13485</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>S.</given-names>
            <surname>Vasudevan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Saha</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. E.</given-names>
            <surname>Tarver</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Patel</surname>
          </string-name>
          ,
          <article-title>Digital biomarkers: Convergence of digital health technologies and biomarkers</article-title>
          ,
          <source>NPJ digital medicine 5</source>
          (
          <year>2022</year>
          )
          <fpage>36</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>