Expert Disagreement in Sequential Labeling: A Case Study on Adjudication in Medical Time Series Analysis?

Abstract

Low inter-rater agreement is typical in various expert domains that rely in part on subjective evaluation criteria. Prior work has predominantly focused on expert disagreement with respect to individual cases in isolation. In this work, we report results from a case study on expert disagreement in sequential labeling tasks where the interpretation of one case can a ect the interpretation of subsequent or previous cases. Three board-certi ed sleep technologists participated in face-to-face adjudication sessions to resolve disagreement in the context of sleep stage classi cation. We collected 1,920 independent scoring decisions from each expert on the same dataset of eight 2-hour long multimodal medical time series recordings. From all disagreement cases (29% of the dataset), a representative subset of 30 cases was selected for adjudication and expert discussions were analyzed for sources of disagreement. We present our ndings from this case study and discuss future application scenarios of expert discussions for the training of non-expert crowdworkers.

Keywords

• Inter-rater disagreement Adjudication Sequence data

-

One of the most common use cases for crowdsourcing is the classi cation of objects into categories. While crowdsourced classi cation tasks traditionally focused on problems not requiring domain expertise, recent work suggests that crowdsourcing can also be e ective for expert-level classi cation. Examples of such expert tasks from the medical domain include the identi cation of low-level patterns in sleep-related biosignals [ 22 ], the annotation of retinal images [ 12 ], and medical relation extraction [ 3 ].

In many mission-critical expert domains including the interpretation of medical data, low inter-rater agreement rates are the norm [ 5, 11, 15, 16 ]. Expert disagreement, however, poses fundamental challenges to quality control procedures in crowdsourcing, and to the use of data labels in supervised machine ? Supported by NSERC CHRP (CHRP 478468-15) and CIHR CHRP (CPG-140200). learning, as it is not immediately obvious how cases at the inter-subjective decision boundary should be disambiguated if multiple equally-quali ed domain experts exhibit genuine disagreement.

Prior work has predominantly paid attention to the nature, sources and resolvability of expert disagreement on individual classi cation tasks in isolation [ 1, 4, 13, 21 ]. Many interpretation tasks, however, are sequential in nature, i.e., the interpretation of one case a ects the interpretation of subsequent or previous cases. For example, in text translation, the semantic interpretation of one phrase or sentence can a ect the translation of subsequent or previous phrases or sentences. Heidegger called this reciprocity of text and context the hermeneutic circle. Overall, sequential labeling makes up a large and diverse class of problems from numerous expert domains.

In this work, we present ndings from a case study on expert disagreement in the context of sleep stage classi cation, the expert task of mapping a sequence of xed-length pages of continuous multimodal medical time series (polysomnogram, see Figure 1) to a sequence of discrete sleep stages (hypnogram). Prior work has established that inter-rater agreement in sleep staging averages around 82.6% [ 17 ]. The objective of this case study is to identify various sources of expert disagreement in sleep stage classi cation and to investigate if and to what extent disagreement may be speci c to the sequential nature of the labeling task and underlying data.

To answer these questions, we collected 1,920 independent sleep scoring decisions from a committee of three board-certi ed sleep technologists. We then selected a representative subset of the resulting disagreement cases which were resolved through in-person adjudication among the members of the expert committee. The rest of this paper describes the related work, then details our study for collecting and analyzing the expert deliberation data, and concludes with a discussion of application scenarios for the training of non-expert crowdworkers. 2 2.1

Related Work

Ambiguity and Sources of Inter-rater Disagreement

Ambiguity, the quality of being open to more than one interpretation, and the phenomenon of expert disagreement are central to the justi cation of knowledge, and have been extensively discussed in the epistemic literature [ 1, 4, 13, 21 ]. An early theoretical investigation named three types of expert disagreement [ 13 ]: personality-based disagreement arising from the incompetence, ideology, or venality of experts, judgment-based disagreement arising from information gaps, or structural disagreement that arises because experts adopt di erent organizing principles or problem de nitions. Garbayo [ 4 ], on the other hand, distinguished a form of legitimate disagreement, that arises when experts can access the same evidence, but still diverge in interpretations, from verbal disagreement, i.e., misunderstanding among experts due to discrepancies in terminology.

Recent work in the eld of human-computer interaction (HCI) has explored the issue of disagreement in the context of crowdsourcing tasks. Gurari and Graumen [ 6 ] analyzed visual question answering tasks and found that disagreement can be attributed to ambiguous and subjective questions, insu cient or ambiguous visual evidence, di ering levels of annotator expertise, and vocabulary mismatch. Chang et al. [ 2 ] proposed to elicit help from the crowd for re nement of category de nitions, based on the nding that workers may disagree because of incomplete or ambiguous classi cation guidelines. Kairam and Heer [ 8 ] introduced a technique to identify clusters of workers with diverging, but legitimate interpretations of the same task. Their work shows that disagreement can arise from di erences in how liberally or conservatively workers interpret classi caion guidelines.

Our study revolves around the task of biomedical time series classi cation, a eld with typically low inter-scorer reliability. For example, Rosenberg and van Hout [ 17 ] conducted a large-scale study on inter-scorer reliability in sleep stage classi cation and found that average expert agreement is as low as 82.6%. In a comment on this study, Penzel et al. [ 15 ] explained that systematic studies on the inter-rater reliability of sleep automatically bring up the question of truth, claiming that the \true" state (i.e., sleep stage) is unknown and can only be approximated through aggregation of expert opinions. 2.2

Group Deliberation as a Method for Disambiguation

Group deliberation is an interactive form of decision making among humans which typically involves group members with con icting beliefs who try to reach consensus on a given question by presenting arguments, weighing evidence and reconsidering individual positions.

Several works explored factors that a ect the process and outcomes of group deliberation. Solomon [ 20 ] appreciates con ict as an important phenomenon of any fruitful deliberation process. He argues that dissent is both required and useful|as \dissenting positions are associated with particular data or insights that would be otherwise lost in consensus formation"|and criticizes procedures endowed with the a priori aim of reaching consensus. Instead, he advocates for a structured deliberation procedure that avoids the undesired e ects of groupthink [ 7 ] by actively encouraging dissent, organizing individual subgroups to deliberate on the same question, and ensuring diverse group compositions.

Kiesler and Sproull [ 9 ] found that time limits imposed on deliberation tend to polarize discussions and to decrease the number of arguments exchanged. The same work suggests the use of voting techniques or explicit decision protocols to structure the deliberation process.

Recent work by Schaekermann et al. [ 19 ] introduced a real-time deliberation framework to disambiguate edge cases in crowdsourced classi cation tasks drawing inspiration from some of these early design considerations. The same work also introduced a novel public deliberation dataset including all deliberation dialogues, original and revised classi cation decisions, and evidence regions from two di erent text classi cation tasks.

Navajas et al. [ 14 ] studied the e ectiveness of in-person group deliberation for general-knowledge questions reporting that averaging consensus decisions yielded better results than averaging individual responses. 2.3

Consensus Scoring in Medical Data Analysis

Group deliberation has also been proposed as a technique for disambiguating edge cases in the interpretation of medical data. Rajpurkar et al. [ 16 ] employed group deliberation among cardiologists to generate a high-quality validation data set in the context of arrythmia detection from electrocardiograms (ECGs). Their work revealed that a convolutional neural network trained on independent labels (i.e., labels collected without deliberation) exceeded the classi cation performance of individual cardiologists when benchmarked against the consensus validation set.

Krause et al. [ 11 ] compared majority vote to in-person deliberation as techniques for aggregating expert opinions for diagnosing eye diseases from photos of the eyeground. Compared to majority vote, in-person deliberation yielded substantially higher recall, suggesting the potential of group deliberation for mitigating underdiagnosis of diabetic retinopathy and diabetic macular edema. Krause et al. also showed that performing group deliberation on a small portion of the entire data set can make tuning of hyperparameters for deep learning models more e ective. The same consensus data set was later used by Guan et al. [ 5 ] to validate the classi cation performance of a novel machine learning approach involving the training of multiple grader-speci c models. They demonstrated that training and aggregating separate grader-speci c models can be more e ective than training a single prediction model on majority labels.

In the context of sleep stage classi cation, Penzel et al. [ 15 ] refer to the concept of in-person group deliberation as consensus scoring, concluding that an \optimal training for [...] sleep scorers is participation in consensus scoring rounds". In this work, we translate this idea to the non-expert domain suggesting a method to augment training procedures for crowdworkers through the use of edge-case examples and the associated expert discussion dialogues in the context of sleep stage classi cation. 3

Expert Deliberation Data Set

An in-person deliberation study was conducted with an expert committee of three board-certi ed sleep technologists at Sunnybrook Health Sciences Centre in Toronto to investigate the extent and potential sources of inter-rater disagreement in sleep stage classi cation, and the e ectiveness of group deliberation as a method for consensus formation. 3.1

Data Set

We prepared a data set of eight 2-hour-long PSG recording fragments. Each 2hour-long fragment contained a sequence of 240 30-second epochs of biosignal data, resulting in 1,920 (240 x 8) epochs for the entire data set. Half of the fragments were from healthy subjects, the other half from patients with Parkinson's disease. Both parts of the data set (Healthy and Parkinson) contained examples of di erent transition types. We included examples from four di erent transition types identi ed by Rosenberg et al. [ 17 ] as regions with typically low inter-rater agreement: the last epoch of stage Wake before sleep onset, the rst epoch of stage N2 after stage N1, the rst epoch of stage REM after stage N2, and transitions between stages N2 and N3. 3.2

Procedure

The full data set was rst scored independently by each sleep technologist, resulting in 5,760 individual scoring decisions, three for each of the 1,920 epochs. We then identi ed all epochs with disagreement among scorers and selected a subset of 30 epochs for in-person group deliberation. The selected disagreement epochs represented both disease states and all four transition types. All 30 epochs were discussed in person by the three scorers using a graphical scoring interface to facilitate detailed discussions about patterns present in the time series data. The experts participants were not explicitly required to reach unanimous consensus, and could instead choose to declare a case as irresolvable. We did not impose an explicit voting scheme or limit the amount of time available per discussion, but instead left the discussion dynamics open until all experts either agreed on one

Tech A Tech B Tech C Majority # Obs.

Tech B Tech C Majority sleep stage or declared a case as irresolvable. Unanimous decisions were reached for all 30 epochs through a process of verbal argumentation and re-interpretation of the patterns shown in the biosignal data The irresolvable option was never used. Discussions were recorded (screen capture and audio), transcribed and qualitatively coded for the di erent sources of disagreement. 3.3

Inter-rater Disagreement

We measured pairwise agreement between all scorers (Tech A, Tech B, Tech C), as well as the group labels as determined by majority vote (Majority) and the deliberation process (Deliberation). Agreement was measured by Cohen's kappa. Table 1 summarizes all agreement results. Pairwise agreement among scorers was moderate, ranging between 0.68 and 0.71 (N=1920). Agreement between individual scorers and the majority vote was high, between 0.84 and 0.87 (N=1920). For the epochs discussed in person, we measured pairwise agreement between the deliberation decision and individual scorers' decisions. Two of the three scorers showed weak agreement with deliberation outcomes (Cohen's kappa of 0.63 and 0.50, N=30), while the third scorer showed no systematic agreement with the deliberation outcomes (Cohen's kappa of 0.02, N=30). Agreement between the majority vote and deliberation decisions was low (Cohen's kappa of 0.54, N=30). 3.4

Sources of Disagreement

Initial qualititative coding of the expert discussions for 17 cases from two major transition types revealed a broad range of reasons why sleep technologists may disagree on the correct sleep stage label. Figures 2 and 3 compare the relative frequency of di erent sources of disagreement across two transition types (Last W before sleep and First N2 after N1 ) and across two disease states (Healthy and Parkinson) respectively. Overall, we identi ed two sources of disagreement which occurred with the highest frequency in both transition types and disease states. These were (a) the presence of multiple stages in one epoch causing disagreement about which stage was the dominant one, and (b) di erent con gurations of the graphical scoring interface in terms of amplitude scaling causing divergent interpretations of visual patterns in the signal.

Expert s Crowd Workers E1 + E2 + E3 + + - - Labeling (independent) + ? E1: Argument

E2: Argument E3: Argument

While these two sources of disagreement could persist on individual cases without the sequential context, we identi ed two other sources of disagreement that explicitly depend on the sequential nature of the labeling task and underlying data: { Number of scoring passes: for 3 out of 30 adjudicated cases, experts explicitly mentioned that their scoring decision depended on the number of passes they had taken on a particular recording. In other words, experts indicated that their interpretation of biosignals is often updated once certain patient-speci c patterns are observed towards the end of the recording. A subsequent re-interpretation (i.e., second scoring pass) would then allow experts to take into account observations they have made in the other parts of the data sequence in one of the earlier scoring passes. Disagreement could therefore arise if one expert had only performed one initial pass whereas other experts may have performed two or more passes. { Cascade from previous disagreement: 3 out of 30 adjudicated cases could be resolved automatically once the disagreement on one of the close-by preceding cases had been resolved. This dynamic was observed since evidence for speci c stages of sleep may sometimes be observed only at the transition point from one sleep stage to another. Consequently, disagreement may arise at a \critical" transition point and persist over multiple steps in the sequence. Once the disagreement at the transition point is resolved, the resolution can cascade to the subsequent steps until the next transition point.

These two sources of disagreement co-occurred once, meaning that 5 out of 30 adjudicated cases (17%) were associated with sources of disagreement that depend on the sequential nature of the labeling task and underlying data. 4

Discussion

In this work, we provided an initial investigation of expert disagreement in the context of sequential labeling tasks, studying the e ectiveness of in-person adjudication for resolving disagreement and for analyzing information about the original source of disagreement.

Our results suggest that majority vote is not necessarily a good proxy for group deliberation decisions in sleep staging. This nding provides some con dence in the usefulness of expert discussions for the purpose of resolving disagreement cases. Beyond that, our qualitative analysis of expert discussion dialogues uncovered a diverse set of di erent reasons why domain experts disagree in the context of sleep stage classi cation, most of which go beyond the notion of mere input mistakes.

Perhaps most importantly, we identi ed two sources of disagreement with a clear connection to the sequential nature of the labeling task and underlying data. This observation provides some support for our hypothesis that the reciprocity of data and context in sequential labeling may lead to unique forms of expert disagreement that are characteristic for sequential labeling tasks, where the interpretation of one case a ects the interpretation of subsequent or previous cases. One exciting avenue for future research is the problem of whether it is possible to detect the \critical" tasks that might set up a cascade of disagreement and potentially incorrect labels. Successful detection of such \critical" tasks would allow for a more cost-e ective use of expert resources by focusing disambiguation procedures on those cases and saving expert resources on other cases that may be resolved automatically.

Picking up on Penzel et al.'s comment on the nature of \truth" in sleep staging [ 15 ], some of the inherent di culty may arise because there exists a certain degree of both temporal and spatial continuity at transitions between states. In other words, despite the fact that any single neuron or cortical circuit may be thought of as existing in one state or another at any given moment, it is possible for local assemblies of neurons to take some time to transition from one state to another, and also that distant assemblies of neurons in di erent parts of the brain can exist in di erent states at the same time. These transitions may take minutes [ 18, 23 ] which encompasses several 30-second epochs. Thus, we hypothesize that some of the ambiguity stems from the need to force transitional states into one sleep stage category or another.

We posit that expert disagreement in complex tasks can be used as a signal to identify ambiguous edge cases, and as a driver for eliciting conclusive expert discussions to disambiguate such edge cases. For future work, we propose the idea that example-based training procedures for non-expert crowdworkers may bene t from the presentation of edge cases and their associated expert discussions. While expert disagreement may be one signal for the identi cation of edge cases, other techniques for the automatic selection of edge case examples, e.g., based on measures of machine uncertainty, have been proposed in prior work [ 10 ]. We believe that expert disagreement and the associated expert discussions open up interesting opportunities for optimizing example-based training procedures for human learners, e.g., to improve disambiguation skills and depth of understanding.

Figure 4 illustrates a high-level overview of some of these future directions. In summary, we hope to conduct research on augmenting example-based training procedures for non-expert crowdworkers using edge-cases and their associated expert discussions to help human learners develop more accurate classi cation strategies for expert-level tasks exhibiting a certain amount of ambiguity.

Another promising avenue for future work will be to explore the minimum \bandwidth" and e ective protocols of communication between experts needed to result in successful disambiguation in the context of sequential labeling settings like the one presented in this work. Comparisons may include di erent styles of expert communication ranging from online text-based asynchronous approaches, to in-person verbal real-time communication. 5

Conclusion

In this work, we reported results from a case study on expert disagreement in sequential labeling tasks where the interpretation of one case can a ect the interpretation of subsequent or previous cases. Three board-certi ed sleep technologists scored 1,920 cases in a sequential 5-class labeling task. Out of all disagreement cases, 30 cases were discussed and resolved through face-to-face adjudication. We identi ed various sources of disagreement that are speci c to the sequential nature of the underlying data and labeling procedure. Our work concluded with a discussion of promising application scenarios of expert discussions for the training of non-expert crowdworkers that we hope to explore in future work.

References

1. Beatty , J. , Moore , A. : Should We Aim for Consensus? Episteme 7 ( 3 ), 198214 ( 2010 ). https://doi.org/10.3366/E1742360010000948

2 . Chang , J.C. , Amershi , S. , Kamar , E.: Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets . In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17 . pp. 2334 { 2346 . ACM, ACM Press, New York, New York, USA ( 2017 ). https://doi.org/10.1145/3025453.3026044, http://dl.acm.org/citation.cfm?doid= 3025453 . 3026044

3. Dumitrache , A. , Aroyo , L. , Welty , C. : Crowdsourcing Ground Truth for Medical Relation Extraction . ACM Transactions on Interactive Intelligent Systems 8 ( 2 ), 1 { 20 (7 2018 ). https://doi.org/10.1145/3152889, http://dl.acm.org/citation.cfm?doid= 3232718 . 3152889

4. Garbayo , L. : Epistemic Considerations on Expert Disagreement, Normative Justication, and Inconsistency Regarding Multi-criteria Decision Making. Constraint Programming and Decision Making 539 , 35 { 45 ( 2014 ),

5. Guan , M. , Gulshan , V. , Dai , A. , Hinton , G.: Who said what: Modeling individual labelers improves classi cation . In: AAAI Conference on Arti cial Intelligence ( 2018 ), https://arxiv.org/pdf/1703.08774.pdf

6. Gurari , D. , Grauman , K. : CrowdVerge: Predicting If People Will Agree on the Answer to a Visual Question . In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17 . pp. 3511 { 3522 . ACM, ACM Press, New York, New York, USA ( 2017 ). https://doi.org/10.1145/3025453.3025781, http://dl.acm.org/citation.cfm?doid= 3025453 . 3025781

7. Jones , A.M.: Victims of Groupthink: A Psychological Study of Foreign Policy Decisions and Fiascoes . The ANNALS of the American Academy of Political and Social Science 407 ( 1 ), 179 { 180 (5 1973 ). https://doi.org/10.1177/000271627340700115, http://journals.sagepub.com/doi/10.1177/000271627340700115

8. Kairam , S. , Heer , J.: Parting Crowds: Characterizing Divergent Interpretations in Crowdsourced Annotation Tasks . In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing - CSCW '16 . pp. 1635 { 1646 . ACM Press, New York, New York, USA ( 2016 ). https://doi.org/10.1145/2818048.2820016, http://dl.acm.org/citation.cfm?doid= 2818048 . 2820016

9. Kiesler , S. , Sproull , L.: Group decision making and communication technology . Organizational Behavior and Human Decision Processes 52 ( 1 ), 96 { 123 (6 1992 ). https://doi.org/10.1016/ 0749 - 5978 ( 92 ) 90047 -B, http://linkinghub.elsevier.com/retrieve/pii/074959789290047B

10. Kim , J. , Park , J. , Lee , U. : EcoMeal: A Smart Tray for Promoting Healthy Dietary Habits . In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '16 . pp. 2165 { 2170 . ACM Press, New York, New York, USA ( 2016 ). https://doi.org/10.1145/2851581.2892310, http://dl.acm.org/citation.cfm?doid= 2851581 . 2892310

11. Krause , J. , Gulshan , V. , Rahimy , E. , Karth , P. , Widner , K. , Corrado , G.S. , Peng , L. , Webster , D.R. : Grader Variability and the Importance of Reference Standards for Evaluating Machine Learning Models for Diabetic Retinopathy . Ophthalmology (3 2018 ). https://doi.org/10.1016/j.ophtha. 2018 . 01 .034, http://arxiv.org/abs/1710.01711 http://linkinghub.elsevier.com/retrieve/pii/S0161642017326982

12. Mitry , D. , Zutis , K. , Dhillon , B. , Peto , T. , Hayat , S. , Khaw , K.T., Morgan , J.E. , Moncur , W. , Trucco , E. , Foster , P.J.: The Accuracy and Reliability of Crowdsource Annotations of Digital Retinal Images . Translational Vision Science & Technology 5 ( 5 ), 6 ( 2016 ). https://doi.org/10.1167/tvst.5. 5 .6, http://tvst.arvojournals.org/article.aspx?doi=10.1167/tvst.5. 5 . 6

13. Mumpower , J.L. , Stewart , T.R. : Expert Judgement and Expert Disagreement. Thinking & Reasoning 2 ( 2-3 ), 191 { 212 (7 1996 ). https://doi.org/10.1080/135467896394500, https://www.tandfonline.com/doi/full/10.1080/135467896394500

14. Navajas , J. , Niella , T. , Garbulsky , G. , Bahrami , B. , Sigman , M. : Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds . Nature Human Behaviour (1 2018 ). https://doi.org/10.1038/s41562-017-0273-4, http://www.nature.com/articles/s41562-017-0273-4

15. Penzel , T. , Zhang , X. , Fietze , I. : Inter-scorer reliability between sleep centers can teach us what to improve in the scoring rules . Journal of Clinical Sleep Medicine 9 ( 1 ), 81 { 87 ( 2013 )

16. Rajpurkar , P. , Hannun , A.Y. , Haghpanahi , M. , Bourn , C. , Ng , A.Y.: CardiologistLevel Arrhythmia Detection with Convolutional Neural Networks (7 2017 ), http://arxiv.org/abs/1707.01836

17. Rosenberg , R. S. , van Hout, S. : The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage Scoring . Journal of Clinical Sleep Medicine (1 2013 ). https://doi.org/10.5664/jcsm.2350, http://www.aasmnet.org/jcsm/ViewAbstract.aspx?pid= 28772

18. Saper , C.B. , Fuller , P.M. , Pedersen , N.P. , Lu , J. , Scammell , T.E.: Sleep State Switching. Neuron 68 ( 6 ), 1023 { 1042 (12 2010 ). https://doi.org/10.1016/j.neuron. 2010 . 11 .032, http://linkinghub.elsevier.com/retrieve/pii/S0896627310009748

19. Schaekermann , M. , Goh , J. , Larson , K. , Law , E.: Resolvable vs. Irresolvable Disagreement: A Study on Worker Deliberation in Crowd Work . In: Proceedings of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW'18) . New York City, NY ( 2018 ). https://doi.org/10.1145/3274423

20. Solomon , M. : Groupthink versus The Wisdom of Crowds : The Social Epistemology of Deliberation and Dissent . The Southern Journal of Philosophy 44 ( S1 ), 28 { 42 (3 2006 ). https://doi.org/10.1111/j.2041- 6962 . 2006 .tb00028.x, http://doi.wiley. com/10 .1111/j.2041- 6962 . 2006 .tb00028.x

21. Solomon , M.: The social epistemology of NIH consensus conferences . In: Establishing medical reality , pp. 167 { 177 . Springer ( 2007 )

22. Warby , S.C. , Wendt , S.L. , Welinder , P. , Munk , E.G.S. , Carrillo , O. , Sorensen , H.B.D. , Jennum , P. , Peppard , P.E. , Perona , P. , Mignot , E.: Sleep-spindle detection: crowdsourcing and evaluating performance of experts, non-experts and automated methods . Nature Methods 11 ( 4 ), 385 { 392 (2 2014 ). https://doi.org/10.1038/nmeth.2855, http://www.nature.com/doi nder/10.1038/nmeth.2855

23. Wright Jr, K.P. , Badia , P. , Wauquier , A. : Topographical and temporal patterns of brain activity during the transition from wakefulness to sleep . Sleep 18 ( 10 ), 880 { 889 ( 1995 )