<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Online sleep spindles detection with short and long time average ratio</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Felipe A. Torres</string-name>
          <email>felipe.torrese@sansano.usm.cl</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Patricio Orio</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>María José Escobar</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Centro Interdisciplnario de Neurociencia de Valparaíso</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Electronic Engineering, Universidad Técnica Federico Santa María</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Instituto de Neurociencia, Universidad de Valparaíso</institution>
        </aff>
      </contrib-group>
      <abstract>
        <p>Sleep spindles occurrence correlates with the consolidation of recently acquired information. The memory consolidation literature supports that there are more sleep spindles after a learning task. Thus, the detection of them does not only allow the classification of the N2 sleep stage, further provides a quantification value of memory replay and memory consolidation during sleep. Event detection is an important processing step performed in the analysis of diverse kinds of waveforms. The short and long term average ratio is the most widely used event detection approach to analyze passive seismic data and trigger the storing or discarding of data. Its popularity comes from its simplicity and the usage of a fixed threshold determined by the intention of the data usage and not based on the signal dynamics. This work explores the usage of this event detection approach on the online detection of sleep spindles. The advantages of the detection performance with this feature over using the same binary classification method using other fast calculation features come from its statistical properties. The classification features compared are the root mean square amplitude, relative spindle power, and the Teager-Kayser energy operator.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Sleep spindles are short-duration events (0.5-2 s) in the specific frequency range of 11-16 Hz that
occur during the NREM sleep and they are detectable in EEG registers. The principal characteristic
of the N2 sleep stage is the high presence of these events
        <xref ref-type="bibr" rid="ref8 ref9">Devuyst et al. (2011)</xref>
        . The ratio of
occurrence and other characteristics of the spindles could indicate some health or sleep disorders
        <xref ref-type="bibr" rid="ref6">(De Gennaro and Ferrara, 2003)</xref>
        , also is known that burst on the hippocampus, slow oscillations
and spindles has a time order in occurrence and are related to memory consolidation (Sara, 2017)
making the spindles a marker of the capacity of learning and memory processing
        <xref ref-type="bibr" rid="ref5">(Cairney et al.,
2015)</xref>
        . Then, find spindles with accuracy and precision could help to evaluate differences in the
sleep after learning or memory tasks and to detect pathologies. The human visual sleep scoring
usually employs the power in the spindle frequency range as a helper for the experts
        <xref ref-type="bibr" rid="ref14">(Purcell et al.,
2017)</xref>
        . Another manual method is the use of crowd-sourced annotations from non-experts
        <xref ref-type="bibr" rid="ref18">(Zhao
et al., 2017)</xref>
        . Event detection is an important processing step performed in waveform analysis. In
particular, in seismology from the appearance of digital signal acquisition systems, there was a
need to reduce the length of recordings to improve the storage and data transmission capabilities
of the electronic devices
        <xref ref-type="bibr" rid="ref1">Akram et al. (2019)</xref>
        .
      </p>
      <p>Classification algorithms have the assumption that there is some space of features where the
patterns are separated. Online or real-time applications require fast computation of these features.
Thus, it is a great achievement if this separation is notorious in a small dimensional space or even
in just one dimension.</p>
      <p>This work takes this last approach. The proposed algorithm does a binary classification (spindle
or not spindle) using only an extracted feature from the input signal. Then it also annotates the
time of occurrence of each spindle to make a detector.</p>
    </sec>
    <sec id="sec-2">
      <title>Related works</title>
      <p>
        This work uses a single feature from the EEG signal as many previous methods of spindles detection
        <xref ref-type="bibr" rid="ref8 ref9">(Devuyst et al., 2011)</xref>
        <xref ref-type="bibr" rid="ref13">(O’Reilly and Nielsen, 2015)</xref>
        . This work is also similar to works that perform
statistical analysis of spindles to obtain a Bayesian detection algorithm
        <xref ref-type="bibr" rid="ref4">(Babadi et al., 2012)</xref>
        or an
HMM-SVM detection scheme
        <xref ref-type="bibr" rid="ref11">(Mporas et al., 2013)</xref>
        . Other works used features of more
computational demand as wavelets
        <xref ref-type="bibr" rid="ref2">(Al-Salman et al., 2019)</xref>
        , a combination of features
        <xref ref-type="bibr" rid="ref10">(Liu et al., 2017)</xref>
        or
the non-negative matrix factorization (NMF) periodogram of the correntropy function
        <xref ref-type="bibr" rid="ref16">(Ulloa et al.,
2016)</xref>
        .
      </p>
      <p>
        The use of a public database is important to compare the results of different methods. In that
sense, this work use the same dataset used by
        <xref ref-type="bibr" rid="ref8 ref9">(Devuyst et al., 2011)</xref>
        , O’Reilly and Nielsen (2015),
        <xref ref-type="bibr" rid="ref10">(Liu et al., 2017)</xref>
        and
        <xref ref-type="bibr" rid="ref2">(Al-Salman et al., 2019)</xref>
        .
      </p>
    </sec>
    <sec id="sec-3">
      <title>Methods and Materials</title>
      <p>Data
The event detection methods were evaluated over a synthetic EEG signal and also in a real EEG
dataset. The baseline of the synthetic signal has a frequency spectrum slope of í f 3 typical of
slow-wave sleep. It is generated with a sum of sinusoidal signals:</p>
      <p>xEEG.t/ = 4918 ³nn==4299 .0:1n/13_.fs/ ^cos..2 .1 + . f /rf /.0:1n/t + .1 + rp.t///; (1)
where fs = 100Hz is the sampling frequency, f = 0:01 is the standard deviation given to the
central frequencies of the expression 1, rf is a random number, and rp.t/ is a random time series
generated from a normal Gaussian distribution. The baseline signal adds with a spindles signal:
xspindle.t/ = 16 ³k fspindle.t/ &lt; p.t * k/ xsigma.t/;
xsigma.t/ = ³m=^*1;0;1` .1 * ðmðda/cos.2 .12 + 0:5m/.1 + sf1/t/ + . ù1.2/ * ðmðda/cos.2 .24 + m/.1 + sf2/t/ ;
(2)
where da is the amplitude difference of the main lobe with the lateral lobes, sf1 and sf2 are random
deviations for the main frequency of 12Hz and its first harmonic for spindles. fspindle.k/ is a triangular
envelope defined by the expression (3) with a duration tspindle = 0:75s. Expression (4) defines the
probability of occurrence of spindles where U .t/ is a random time series uniformly distributed
between [0,1].</p>
      <p>h tspindleaspindle + aspindlet
fspindle.t/ = n 2
l
ntspindleaspindle * aspindlet t g tspindle_2
j
t &lt; tspindle_2 ;
(3)
nh1 U .t/ &gt; 0:998
p.t/ = nl0:01 0:99 &lt; U .t/ &lt; 0:998 ; (4)
nn0 U .t/ f 0:99
j
The tested synthetic EEG signal has 8 segments of 300s with different values for aspindle: 0.125, 0.250,
0.375, 0.5, 0.625, 0.750, 0.875, and 1.0.</p>
      <p>
        The real EEG signals come from the DREAMS dataset
        <xref ref-type="bibr" rid="ref8 ref9">(Devuyst and Dutoit, 2011)</xref>
        . This dataset
consists of eight registers of 30-minutes. The data were sampled at frequencies of 50, 100, and 200
Hz. The dataset includes the visual scoring of spindles from two experts. They marked the start
time and the duration of spindles. A duration of 1-second is annotated for many of the spindles
but they have another time length. This work uses annotations without modification from just one
expert.
      </p>
    </sec>
    <sec id="sec-4">
      <title>Spindles detection</title>
      <p>General Procedure
The event detection methods in this work use a general framework of a single dimension signal
and a single threshold to classify a sample as forming part of an event occurrence. Consecutive
samples classified as spindles must exceed the minimum duration to finally designate a section of
the input signal as an event. A time for possibles gaps is also included to actuate as replace of an
hysteresis mechanism.</p>
      <p>A common procedure before any step is to obtain the z-score value of the signal. Given that
there is not any assumption about the signal properties, the mean value and the standard deviation
are calculated at each sample, then there is the need of advance in various samples to achieve
consistency in these statistical estimators. The z-scored signal x.t/ is band-pass filtered in the sigma
band (11-16Hz) with a Chebyshev type I fourth order filter to obtain x .t/. The calculation of features
at each sample uses both signals. The detection follows the procedure show in Algorithm 1.
Algorithm 1 Spindle Detection Algorithm
Inputs: feature.t/, tmin, tgap
Outputs: thresholdfeature.t = tfinal/, is_spindle(t), count_spindles
1: if f eature.t/&gt;thresholdfeature.t/ then
2: if hold==False then
3: start=t;
4: end if
5: hold=True;
6: else if hold==True then
7: gap=gap + 1;
8: if gap&gt;tgap then
9: gap=0;
10: hold=False;
11: if t * start&gt;tmin then
12: count_spindles=count_spindles + 1;
13: is_spindle.start : t/=1;
14: end if
15: end if
16: end if
. The threshold could be time dependent</p>
      <sec id="sec-4-1">
        <title>Features calculation</title>
        <p>The short and long term ratio was compared with other features. The short-time and the ratio
were predefined for calculation of each feature. ST is the length in samples of selected short-time,
LT = ST is the length in samples of selected long-time. ST A0.n/ is the ratio feature using as
ratio
long-time all past samples, ST A1.n/ uses as short-time a single sample. ST A0.n/, ST A1.n/, and
T eager.n/ pass through a moving average filter of length ST .</p>
      </sec>
      <sec id="sec-4-2">
        <title>Root mean square (RMS) value</title>
        <p>RMS.n/ =
v ³nk=n*ST x .k/2</p>
        <p>ST
;
Short and long-time average ratio (STA/LTA)
ðx .k/ð
ST A0.n/ = 1 ³n
n k=0 ðx .k/ð
ðx .k/ð
ST A1.n/ = 1 ³n</p>
        <p>LT n=n*LT ðx .k/ð</p>
        <p>ST A_LT A.n/ = S11T ³³nnk=n*ST ðx .k/ð</p>
        <p>LT k=n*LT ðx .k/ð
;
(5)
(7)
(8)</p>
      </sec>
      <sec id="sec-4-3">
        <title>Teager-Kaiser energy operator</title>
        <p>T eager.n * 1/ = .x.n * 1/2 * .x.n * 2/x.n///;</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Detection metrics</title>
      <p>
        The detection metrics employed here are the same as used by
        <xref ref-type="bibr" rid="ref13">(O’Reilly and Nielsen, 2015)</xref>
        . An
additional metric of distribution separation, d , uses the probability distributions (pdf) of the feature
signals in the search for a threshold value.
      </p>
      <p>d = P ^y f ` * PP ^^yy ff `` ++ PP ^^yy &gt;&gt; `` * P ^y &gt; ` = 12+.P.P^y^yff ``**PP^y^yff `/`/ ;
(9)
where is the threshold value, y is the distribution of the detection feature when there is a spindle
and y the distribution of feature values when there is not a spindle. The best threshold, known the
probability distributions fy and fy, is which maximizes this metric. The maximum value is 1 and it is
achieved only if the pdf’s have disjointed domain and the threshold value is in the gap between
them.</p>
    </sec>
    <sec id="sec-6">
      <title>Results</title>
      <p>Statistical analysis
The probability density function (pdf) of samples of an EEG signal is assumed to be Gaussian and
reinforced by the histograms in Figure 1 C. Thus, the features calculations perform those operations
to a normal aleatory variable. Then, the root mean square (RMS) should have a Chi pdf, short and
long-time average (STA/LTA) and relative spindle power (RSP), as calculated here, should have a
Fractional Gamma pdf, and the Teager-Kaiser energy operator should have a Chi-squared pdf.</p>
      <p>The probability density function of samples of an EEG signal is Figure 1 D shows the histogram
of the values of the features calculated from spindles and not spindles samples. The first 3 seconds
of the synthetic signal were removed because the z-score is badly estimated for the first samples.
The mean and standard deviation are still not consistent. The first 30 seconds, the first epoch,
are removed from real signals for the same reason. The distance between the means of each
distribution and the length of the tails say something about the classification difficulty using a
single threshold. The metric d allows the selection of a better threshold. However, it needs caution
because the features depend on the length of the time window and the metrics are threshold
dependent (Figure 2). Figure 1 E shows the classification performance with different thresholds. The
most classical performance metrics have their best value at threshold values above the preferable
threshold selected with d .</p>
    </sec>
    <sec id="sec-7">
      <title>Detection results</title>
      <p>Figure 2 A presents ROC curves that are another perspective of the results of Figure 1 C. An
additional short-time value and an additional ratio were included to show the different behavior of
the detection performance. Figure 2 B shows the maximum value achieved for d occurring with
STA/LTA at the short-time window of 0.05 seconds and a ratio equal to 0.001.</p>
    </sec>
    <sec id="sec-8">
      <title>Discussion</title>
      <p>
        The proposed spindle detection method considers a single EEG channel and a single extracted
feature. This consideration could be a weakness in typical EEG studies where multiple channels
are recorded and more sophisticated analyses could be performed
        <xref ref-type="bibr" rid="ref12 ref2">(Mucarquer et al., 2019)</xref>
        . On
      </p>
      <p>
        G2
E 1
E
roe 0
c
-sz−1
liS10
e
d
n
p
10
le8
u
a
v
se6
tr
u
a
fe4
l
a
n
i 2
g
S
0
3.0
lsve22..05
u
a
e
r
tu1.5
a
e
f
la1.0
n
g
iS0.5
0.0436
6
e
u
lv5
a
s
e
r
t4
u
a
e
lf3
a
n
g
i2
S
1
0
0.30
e0.25
u
lva0.20
’sc0.15
i
r
t
e0.10
M
0.05
0.000.0
1.0
0.8
e
u
lva0.6
s
i’
c
tr0.4
e
M
0.2
the other hand, it is a necessary alternative in studies where recording devices of few channels are
available like the OpenBCI, Emotiv EPOC
        <xref ref-type="bibr" rid="ref17">(Xu and Zhong, 2018)</xref>
        or single channel as the Neurosky
MindWave device
        <xref ref-type="bibr" rid="ref15 ref2">(Torres et al., 2014; Avendaño et al., 2018)</xref>
        . In the framework of educational
research, these devices known as portable EEG technology (PEEGT) are not appropriate for their
single-use. These are the present alternatives to include electrophysiological data
        <xref ref-type="bibr" rid="ref17">(Xu and Zhong,
2018)</xref>
        in addition to other data and tools. Sleep research uses polysomnography for
electrophysiological data acquisition, but PEEGT could be an advantage to include more subject samples
        <xref ref-type="bibr" rid="ref2 ref7">(Debellemaniere et al., 2018)</xref>
        .
      </p>
      <p>The STA/LTA feature performs better with the smallest ratios for the real signals dataset. That
does not occur in the synthetic signals, where the Teager-Kaiser energy operator features performs
the better. This could indicate that there is a need for more pre-processing of the input signal to
remove noise and other physiological artifacts not considered in the construction of the synthetic
signals like ECG, EMG and, ocular movements. Without any other pre-processing another good
performance feature is RMS.</p>
      <p>The accuracy metric goes near to 1 with higher thresholds due to the best classification of
True Negatives samples that are much higher in quantity than True Positive samples. Interestingly,
Cohen- and F1 metrics have better values for higher thresholds than for the d metric (Figure 1
E). Furthermore, all metrics are consistent in performance between cases. The ROC curves allow
having another perspective than a simple value, although they represent the same information,
and clarifies why the common choice of the RMS feature over another feature in the task of spindle
detection. The maximum of the d metric occurs at a similar threshold value for the RMS feature in
any combination of short-times and ratios. The threshold value that gives the best d metric for
STA/LTA is more variable across cases (we do not show but the behavior of probability distributions
in Figure 1 D explain it).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Akram</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Peter</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eaton D</surname>
          </string-name>
          .
          <article-title>A k-mean characteristic function for optimizing STA/LTA based detection of microseismic events</article-title>
          . .
          <year>2019</year>
          ; .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Al-Salman</surname>
            <given-names>W</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            <given-names>Y</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wen</surname>
            <given-names>P</given-names>
          </string-name>
          .
          <article-title>Detecting sleep spindles in EEGs using wavelet fourier analysis and statistical features</article-title>
          .
          <source>Biomedical Signal Processing and Control</source>
          .
          <year>2019</year>
          ;
          <volume>48</volume>
          (February):
          <fpage>80</fpage>
          -
          <lpage>92</lpage>
          . https://doi.org/10.1016/j.bspc.
          <year>2018</year>
          .
          <volume>10</volume>
          . 004, doi: 10.1016/j.bspc.
          <year>2018</year>
          .
          <volume>10</volume>
          .004.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Avendaño</surname>
            <given-names>GO</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ballado</surname>
            <given-names>AH</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cruz</surname>
            <given-names>JCD</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bentir</surname>
            <given-names>SAP</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Camposanto</surname>
            <given-names>JCB</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Carreos</surname>
            <given-names>AL</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Domingo</surname>
            <given-names>LAB</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>M Garcia</given-names>
            <surname>KD. Sleep Onset Period Detection Using Slow Eyelid</surname>
          </string-name>
          <article-title>Movement (SEM) Through Eye Aspect Ratio with Electroencephalogram (EEG)</article-title>
          .
          <source>In: 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology,Communication and Control</source>
          , Environment and Management (HNICEM);
          <year>2018</year>
          . p.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          . doi:
          <volume>10</volume>
          .1109/HNICEM.
          <year>2018</year>
          .
          <volume>8666429</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Babadi</surname>
            <given-names>B</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Babadi</surname>
            <given-names>B</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McKinney</surname>
            <given-names>SM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tarokh</surname>
            <given-names>V</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ellenbogen</surname>
            <given-names>JM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ellenbogen</surname>
            <given-names>JM</given-names>
          </string-name>
          .
          <article-title>DiBa: A Data-Driven Bayesian Algorithm for Sleep Spindle Detection</article-title>
          .
          <source>IEEE Transactions on Biomedical Engineering</source>
          .
          <year>2012</year>
          ;
          <volume>59</volume>
          (
          <issue>2</issue>
          ):
          <fpage>483</fpage>
          -
          <lpage>493</lpage>
          . doi:
          <volume>10</volume>
          .1109/TBME.
          <year>2011</year>
          .
          <volume>2175225</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Cairney</surname>
            <given-names>SA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ashton</surname>
            <given-names>JE</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Roshchupkina</surname>
            <given-names>AA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sobczak</surname>
            <given-names>JM</given-names>
          </string-name>
          .
          <article-title>A dual role for sleep spindles in sleep-dependent memory consolidation?</article-title>
          <source>Journal of Neuroscience</source>
          .
          <year>2015</year>
          ;
          <volume>35</volume>
          (
          <issue>36</issue>
          ):
          <fpage>12328</fpage>
          -
          <lpage>12330</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>De Gennaro</surname>
            <given-names>L</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ferrara</surname>
            <given-names>M</given-names>
          </string-name>
          ,
          <article-title>Sleep spindles: an overview</article-title>
          . Elsevier;
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Debellemaniere</surname>
            <given-names>E</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chambon</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pinaud</surname>
            <given-names>C</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Thorey</surname>
            <given-names>V</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dehaene</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Léger</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chennaoui</surname>
            <given-names>M</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Arnal</surname>
            <given-names>PJ</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Galtier</surname>
            <given-names>MN</given-names>
          </string-name>
          .
          <article-title>Performance of an Ambulatory Dry-EEG Device for Auditory Closed-Loop Stimulation of Sleep Slow Oscillations in the Home Environment</article-title>
          . Frontiers in Human Neuroscience.
          <year>2018</year>
          ; https://doi.org/10.3389/fnhum.
          <year>2018</year>
          .
          <volume>00088</volume>
          , doi: 10.3389/fnhum.
          <year>2018</year>
          .
          <volume>00088</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Devuyst</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dutoit</surname>
            <given-names>T</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stenuit</surname>
            <given-names>P</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kerkhofs</surname>
            <given-names>M.</given-names>
          </string-name>
          <article-title>Automatic sleep spindles detection - Overview and development of a standard proposal assessment method</article-title>
          .
          <source>In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society</source>
          , EMBS;
          <year>2011</year>
          . p.
          <fpage>1713</fpage>
          -
          <lpage>1716</lpage>
          . doi:
          <volume>10</volume>
          .1109/IEMBS.
          <year>2011</year>
          .
          <volume>6090491</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Devuyst</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dutoit</surname>
            <given-names>T</given-names>
          </string-name>
          ,
          <string-name>
            <surname>The</surname>
            <given-names>DREAM</given-names>
          </string-name>
          <article-title>sLeep spindle database; 2011. www</article-title>
          .tcts.fpms.ac.be/~devuyst/Databases/ DatabaseSpindles/.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Liu</surname>
            <given-names>MY</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huang</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huang</surname>
            <given-names>NE</given-names>
          </string-name>
          .
          <article-title>Evaluating and Improving Automatic Sleep Spindle Detection by Using MultiObjective Evolutionary Algorithms</article-title>
          . Frontiers in human neuroscience.
          <year>2017</year>
          ;
          <volume>11</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Mporas</surname>
            <given-names>I</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Korvesis</surname>
            <given-names>P</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zacharaki</surname>
            <given-names>EI</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Megalooikonomou</surname>
            <given-names>V</given-names>
          </string-name>
          .
          <article-title>Sleep spindle detection in EEG signals combining HMMs and SVMs</article-title>
          .
          <source>Communications in Computer and Information Science</source>
          .
          <year>2013</year>
          ;
          <volume>384</volume>
          :
          <fpage>138</fpage>
          -
          <lpage>145</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>642</fpage>
          -41016-1_
          <fpage>15</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Mucarquer</surname>
            <given-names>JA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prado</surname>
            <given-names>P</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Escobar</surname>
            <given-names>MJ</given-names>
          </string-name>
          ,
          <string-name>
            <surname>El-Deredy</surname>
            <given-names>W</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zañartu</surname>
            <given-names>M. Improving</given-names>
          </string-name>
          <article-title>EEG Muscle Artifact Removal With an EMG Array</article-title>
          .
          <source>IEEE Transactions on Instrumentation and Measurement</source>
          .
          <year>2019</year>
          ; (1):
          <fpage>1</fpage>
          -
          <lpage>10</lpage>
          . doi:
          <volume>10</volume>
          .1109/tim.
          <year>2019</year>
          .
          <volume>2906967</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>O'Reilly</surname>
            <given-names>C</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nielsen</surname>
            <given-names>T.</given-names>
          </string-name>
          <article-title>Automatic sleep spindle detection: benchmarking with fine temporal resolution using open science tools</article-title>
          .
          <source>Frontiers in human neuroscience</source>
          .
          <source>2015; 9.</source>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Purcell</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Manoach</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Demanuele</surname>
            <given-names>C</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cade</surname>
            <given-names>B</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mariani</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cox</surname>
            <given-names>R</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Panagiotaropoulou</surname>
            <given-names>G</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saxena</surname>
            <given-names>R</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pan</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smoller</surname>
            <given-names>J</given-names>
          </string-name>
          , et al.
          <source>Characterizing sleep spindles in 11,630 individuals from the National Sleep Research Resource. Nature Communications</source>
          .
          <year>2017</year>
          ;
          <volume>8</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Torres</surname>
            <given-names>FA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sánchez</surname>
            <given-names>C</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Baus</surname>
            <given-names>KP</given-names>
          </string-name>
          .
          <article-title>Adquisición y análisis de señales cerebrales utilizando el dispositivo MindWave</article-title>
          .
          <source>Maskana</source>
          .
          <year>2014</year>
          ;
          <volume>0</volume>
          (
          <issue>0</issue>
          ):
          <fpage>83</fpage>
          -
          <lpage>93</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Ulloa</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Estevez</surname>
            <given-names>PA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huijse</surname>
            <given-names>P</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Held</surname>
            <given-names>CM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Perez</surname>
            <given-names>CA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chamorro</surname>
            <given-names>R</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Garrido</surname>
            <given-names>M</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Algarín</surname>
            <given-names>C</given-names>
          </string-name>
          , Peirano P.
          <article-title>Sleep-Spindle Identification on EEG Signals from Polysomnographic Recordings using Correntropy</article-title>
          .
          <source>Conf Proc IEEE Eng Med Biol Soc</source>
          <year>2016</year>
          .
          <year>2016</year>
          ; p.
          <fpage>3736</fpage>
          -
          <lpage>3739</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <surname>Xu</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhong</surname>
            <given-names>B.</given-names>
          </string-name>
          <article-title>Review on portable EEG technology in educational research</article-title>
          .
          <source>Computers in Human Behavior</source>
          .
          <year>2018</year>
          ;
          <volume>81</volume>
          :
          <fpage>340</fpage>
          -
          <lpage>349</lpage>
          . https://doi.org/10.1016/j.chb.
          <year>2017</year>
          .
          <volume>12</volume>
          .037, doi: 10.1016/j.chb.
          <year>2017</year>
          .
          <volume>12</volume>
          .037.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Zhao</surname>
            <given-names>R</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sun</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhang</surname>
            <given-names>X</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wu</surname>
            <given-names>H</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Liu</surname>
            <given-names>P</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yang</surname>
            <given-names>X</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Qin</surname>
            <given-names>W.</given-names>
          </string-name>
          <article-title>Sleep spindle detection based on non-experts: A validation study</article-title>
          .
          <source>PloS one</source>
          .
          <year>2017</year>
          ;
          <volume>12</volume>
          (
          <issue>5</issue>
          ):
          <fpage>e0177437</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>