Functional neuroimaging investigates the neural correlates of performing cognitive tasks. The empirical evidence in this field is constantly growing and gave rise to methods for assessment and integration of the results across diferent studies. A promising and suitable technique is the so-called mega-analysis. Performing mega-analysis is, however, challenging. It is a multi-step process which connects a researcher's implicit reasoning about information processing in the brain with complex analysis of heterogenous data. Although the process of mega-analysis is well understood, it comprises many concepts and queries that lack a formal definition. Therefore, it is dificult to choose a suitable data model, design a data schema, and implement the relevant queries. A prerequisite for a successful mega-analysis is a set of studies conforming to a carefully defined experimental setting. Finding such datasets is, however, a laborious and error-prone task of keywordbased literature search. To aid understanding of the underlying issues, we propose a conceptual model of mega-analysis. The model integrates a researcher's implicit knowledge with a systematic definition of relevant data. The nature of the data suggests a graph data model for efectively querying datasets. Consequently, we define a knowledge graph integrating the data associated with experimental setting, formally define the queries over the knowledge graph, and showcase their implementation in a graph database.
low researchers to share their data with each other. cessing the sex of male and female faces and does
OpenNeuro requires data to be stored according to this difer between men and women? The desired
BIDS, and currently houses more than 800 datasets. experimental settings for qualifying datasets are:
HED annotations are part of the BIDS specification. While recording a fMRI signal (acquisition
parameDespite these eforts, identifying the studies relevant ters), men and women (participant demographics)
to a mega-analysis remains a challenging task. were asked to identify the sex of faces presented in a
series of images (activity details). In this paper, we
Mega-analysis workflow. A mega-analysis typi- focus on the details of the activity the participants
cally investigates a particular cognitive process or were tasked to perform.
functionality of a brain region. The respective
research question usually involves contrasting two or Problem statement. The goal of this paper is a
more carefully defined experimental setting condi- querying framework for mega-analysis in functional
tions [
Ideally, the experimental settings should be nately, it currently faces three main challenges.
queried directly in the datasets. This, however, is 1. The experimental setting is not systematically
currently not possible due to poor data availability defined. A fundamental part of verifying the
releand annotation. Therefore, to find relevant datasets, vance of a dataset is matching its experimental
seta researcher starts with a keyword-based literature ting to the desired conditions for the mega-analysis.
search followed by data requests. Keyword search Despite this, experimental settings lack a formal
is error-prone because it strongly depends on the definition, which makes comparing studies dificult.
presence of keywords in an article text and is bi- The annotations provided in the data use arbitrary
ased by the choice of keywords [
Once the qualifying datasets are collected, they the study they conducted and not on possible future
undergo a signal analysis. The existing techniques analysis scenarios.
vary, and their choice depends on the research ques- 2. The data is heterogenous in its format. The
tion. The data acquired by fMRI can be interpreted commonly applied BIDS format [
To better illustrate the concepts throughout this narratively described in the corresponding
publicapaper, we introduce the following example. tions or they are poorly annotated with arbitrary
Example 1.1. Running example. A cognitive neuro- labels in the data. The custom and unstructured
scientist investigates the following research question: annotations vary between datasets and researchers,
Is there a diference in brain activation between pro- limiting the queries to imprecise keyword
matching. Curated taxonomies like HED [
Thus, these contributions demonstrate a proof- clude the creation of models that would break the of-concept for ontologically conceptualizing mega- axiomatization of UFO, thus allowing researchers analysis, translating the elements critical for dataset to explicate their domain specific knowledge within queries into a graph representation, and implement- syntactically and semantically valid models. ing a queriable prototype using Neo4j. The targeted The entities in our model are conceptualized as user group for subsequent large-scale implementa- kinds, i.e. basic types of objects that exist in the tions are cognitive neuroscientists in need of efec- real world and provide a uniform principle of identive and easy-to-use solutions for finding datasets tity for their instances. To represent the intrinsic suitable for mega-analysis. properties of kinds, we employ quality types and subkinds of quality types, i.e. functions that take elements in the extension of an object type and map 2. Modeling mega-analysis them to a respective quality structure. These quality structures form either one-dimensional (quality We present a conceptual model of a mega-analysis dimension) or multi-dimensional (quality domain) use case in Figure 1. It serves the purpose of clari- conceptual spaces. In OntoUML, quality structures fying relevant concepts in mega-analysis, their re- are represented as datatypes that organize the possilations, and value spaces to derive an appropriate ble values which can be attributed to the respective graph representation of the elements critical for quality types. The relationships between kinds are querying datasets (see Section 3). For building the modeled as material relations that are existentially conceptual model, we ontologically analyzed the dependent on both their bearer and an external process of mega-analysis in cognitive neuroscience entity, i.e. they link two kinds by establishing a ..* 1 <<kind>>
Predictor <<material>> defines <<material>> / comprises 1..* <<kind>>
Event <<subkind>> HED Grouping Graph Event Quality > > itraon luea tu v trcu sha s< < <<datatype>>
HED Grouping 1 1 Graph Dimension -g : HEDGraph <<kind>> 1..* Event File
1 > > f tnO tr e a n p poom sah c< < * .. 1 > > n o i tza isn ir tce ree raa inh h c<< ..*
1 <<quality>> Event Quality disjoint > > itrona leua tu v trcu sha s< <
1 <<datatype>> Duration Quality
Dimension -d : DurationValue <<subkind>> Temporal Event Quality disjoint > > itrona leau tu v trcu sha s< < Researcher's reasoning
Analysis
Data <<material>> input to <<material>> 1 <<kind>> 1..* answers
Research 1 Question <<kind>>
Signal 1..* Analysis litrae>> irsnem iltrae>> irsnem a te a te <<m /de <<m /de 1 1 1 1 1 1 1 <<subkind>> Event Duration
Quality <<subkind>> Event Onset Quality <<subkind>> Age Quality <<subkind>>
Sex Quality 1
1 1 1 semantically meaningful connection between them. can take.
As outlined in Figure 1, performing a
megaterization relations, i.e. they represent the features analysis necessitates to connect i) the researcher’s
intrinsic to the object type they existentially depend reasoning (top left) with ii.) the planned analysis
on. Qualities, in turn, are structured via structura- (top middle), iii.) the datasets that qualify for this
tion relations, connecting them to the datatypes analysis by means of a particular experimental
setthat define the space of possible values a quality ting (top right), and iv.) the diferent data types
n5
n8
n9
comprised in the datasets (bottom). Since events
are the essential building blocks of the
experimental setting, our current work focuses exclusively on
the connection between the researcher’s reasoning
and event data via a common, queriable datatype n3 n1 n7 n6
(colored entities). For a standardized description of
event data, we employ the HED taxonomy [
pends on whether it matches the desired experimen- Figure 2: HED Grouping Graph ℋ 1 tal setting. In terms of events, the experimental setting of a dataset is captured in the HED annotations.
In order to query the data for our mega-analysis we also define our conditions in HED annotations, thus establishing a common data format between the conditions derived from the researcher’s reasoning and HED annotated event data. dependent on the same information that we use for querying. Finding the conditions in a dataset is thus not only relevant during the searching stage of a mega-analysis, but also for the analysis part, which we aim to address in future work.
Example 2.1. The research question defined in our computationally derive the link between relevant running example (Example 1.1) determines one or conditions for a mega-analysis, specified by the remore contrasts that would qualify to assess this searcher’s reasoning, and events in a dataset (blue question, e.g. male faces presented with an instruc- relationship in Figure 1). Subsequently, we demontion to identify sex versus female faces presented strate an implementation of data and queries in a with an instruction to identify sex. In neuroimaging graph database. research, such a contrast is commonly defined as a function of two or more experimental setting conditions, in this case male face identification and female 3. Experimental setting conditions face identification . We can express these conditions in HED annotations strings using ((Face, Human- Our conceptual model indicates that datasets qualiagent, Female), (Task, (Discriminate, Sex))) and fying for a mega-analysis can be found by matching ((Face, Human-agent, Male), (Task, (Discriminate, the HED annotations of experimental setting conSex))). These condition qualities can be represented ditions and events (blue relationship in Figure 1). within the space of a HED grouping graph dimension HED annotations are stored in the data as long (see Section 3 for a formal definition). Experimental string values that are dificult to query. In fact, setting conditions, in turn, comprise the specific they can form arbitrarily nested groups of terms, events that need to be stored in the event files of a i.e., a graph. A graph of a single HED annotation dataset so that it qualifies for use in the researcher’s is called a HED grouping graph and the set of all mega-analysis. By projecting event qualities into terms from the HED taxonomy is depicted by HED. the same value space of the HED grouping graph Definition 3.1 (HED grouping graph). A HED groupdimension, we guarantee a mutual data type be- ing graph ℋ with nodes (ℋ ) and edges (ℋ ) is tween the events that are i.) present in a dataset a directed, connected, acyclic graph with exactly and ii.) required by the desired conditions for the one node without incoming edges, (ℋ ), called the mega-analysis, thus enabling efective querying. root. Nodes without outgoing edges are called leaves.
Although it is beyond the scope of the current paper, we incorporated the remaining parts of mega- Example 3.1. Figure 2 shows a HED grouping graph analysis as greyed-out entities in the model for com- ℋ 1 with the following leaf node labels: pleteness. For example, qualities and datatypes ( 1) = Face ( 2) = Rotated associated with the participants of datasets can determine additional aspects of the desired experi- ( 3) = Male ( 4) = Downward mental setting, e.g. with respect to a specific age ( 5) = Task ( 6) = Discriminate range. Since performing a full neuroimaging mega- ( 7) = Detect ( 8) = Sex analysis involves a complex, multi-step analysis, our conceptual model also includes a representation ( 9) = Press ( 10) = Push-button of the acquired signal with its qualities and value Using the HED grouping graphs, we define the spaces as these are pertinent to ultimately answer data concepts of our model (red entities in the topthe research question. Note that the analysis is right data section of Figure 1). • edges between the dataset node and all event annotations for the conditions that are relevant to ∈ ∧ ℋ ∈ ( )}. } and edges {( , ) | ∈ } ∪ {( , (ℋ )) | from Example 3.2 and ( ′) is the dataset graph Definition 3.2 (Event, event file, dataset) . An event is a triple of the form ( ( ), ( ), ℋ ( )), where ( ) ∈ R is the onset (the timepoint when the event started), ( ) ∈ R
+ is the duration of and ℋ ( ) is a HED grouping graph. An event file is a set of events and a dataset is a set of event files. which is composed of:
To facilitate returning datasets as the results of the queries, we introduce a dataset graph ( ) • a dataset node , • one event file node for every event file • all HED grouping graphs in the event files of ∈ , , ifle nodes, graphs. • edges between the event file nodes and the
root nodes of all respective HED grouping ∈ ︀⋃ ︀⋃ Definition 3.3
(Dataset graph). Let ( ) = ∈ ℋ ( ) be the union of all HED grouping graphs from an event file
in a dataset . The dataset graph ( ) is the union of all HED grouping graphs ( ) with additional nodes ∪ { |
∈
with a single event file
= {(1, 1.5, ℋ 1)}, where ℋ 1 is the HED {( , ), ( , (ℋ 1))} ∪ (ℋ 1). grouping graph in Figure 2.
graph ( ) has nodes { , } ∪ (ℋ 1) and edges
To perform a mega-analysis, the researcher deifnes a set of experimental setting conditions. For a dataset to qualify for the mega-analysis, it must contain at least one event for each of the specified conditions. We define an experimental setting condition as a HED grouping graph. Thus, the matching between events and conditions can be resolved over
collection S of dataset graphs containing a subgraph which matches exactly the HED grouping graph of the specified condition. In this paper we are we plan to make the condition query more flexible. query (S, ℋ ) is defined as follows:
dataset graphs and a condition ℋ , the condition (S, ℋ )={ | ( ) ∈ S∧ℋ is subgraph of ( )}
A condition query returns all datasets from a type data graph and the queries in Neo4j [
n8 n5
Example 3.3. In Example 2.1 we identified HED the research question of our running example. For demonstration purposes we consider a simplified version of one of these conditions, namely (Task, (Discriminate, Sex)). In other words, the researcher is interested in all datasets from a collection S that contain this particular condition.
The resulting datasets compose the input to the mega-analysis of interest. The HED grouping graph ℋ of the above condition is shown in Figure 4 and let S be the set { ( ), ( ′)}, where ( ) is the dataset graph in Figure 3. The leaf node labels of ( ), ( ′) and ℋ are as in Example 3.1. Then the condition query (S, ℋ ) = { } returns only the dataset , since ℋ is a subgraph of ( ), but not of ( ′).
mentation which enables cognitive neuroscientists
with efective querying for datasets relevant to the
desired mega-analysis. For this purpose, we
manually annotated 35 datasets from Openneuro[
Cypher queries.
to execute a condition query (cf. Definition 3.4) on the graph in Neo4j, we need to translate it into
ℋ , Algorithm 1 generates the respective Cypher query. For the pattern matching that is necessary to return the required datasets Algorithm 1 uses
verify whether the resulting datasets are appropriate without reading the related publications as well as modify the condition query when necessary.
1 begin 2 (ℋ ) ={MATCH (D:Dataset)} 3 foreach leaf ∈ (ℋ ) do 4 (ℋ ) = (ℋ ) ∪ {( : ( ))} 5 end 6 foreach (, ′) ∈ (ℋ ) do 7 (ℋ ) = (ℋ ) ∪ {( ) → ( ′)} 8 end 9 (ℋ ) = (ℋ ) ∪ {(D)-[*]->( (ℋ ))} 10 (ℋ ) = (ℋ ) ∪ {RETURN D } 11 return (ℋ ) 12 end
ferent studies in the field of cognitive neuroscience have, until now, mainly relied on aggregated results of previously performed analyses. Several systems have been designed to store and query such results.
annotation along with the development of systems for data storage. We list a selection of particularly influential systems and summarize their features in Table 1. These eforts, however, only ofer partial solutions with respect to our use case of efective data querying for mega-analysis. To enable a successful mega-analysis, we identify the following three aspects of system requirements.
be available. In contrast, aggregated data
derivaExample 4.1. Consider the condition HED grouping tives resulting from analysis, e.g., so-called peak
coordinates and statistical maps, are not suficient
tghraapthreℋ sideosf iEnxtahme pNleeo34.j3daantdabaasseet. oTfodeantaasbelte gtrhaepehxs- for mega-analysis [
tions must be available on an event level, not just Note that the node identifier , 5, 6, 8 and 68 to ensure precision of the query, but also to exeemerge from line 7 of Algorithm 1 and are arbitrary cute the analysis once datasets have been identified. Cypher variable names. Executing this query on Datasets that lack this level of annotation cannot our data graph in Neo4j returns three datasets. The be processed for the purpose of a mega-analysis. researcher can then perform mega-analysis proce- Queries: Ideally, a researcher can define custom dures with these three datasets and interpret the experimental setting conditions in a systematic way results of the analysis to answer the research ques- and find all qualifying datasets. Querying arbitrary, tion. This demonstrates that our approach allows non-standardized string labels requires guessing the the researcher to find all datasets that are rele- right label and potentially results in missing relevant to the research question. We achieve this vant datasets. Moreover, querying aspects other based on HED annotations at the level of individual than experimental setting conditions requires furevents, in contrast to other approaches (cf. Sec- ther investigation of the resulting datasets. tion 5), where researchers must limit themselves to In Table 1 we show that none of the existing using keywords or predefined labels to find relevant systems satisfies all requirements that we defined. datasets for mega-analysis. Another advantage is BrainMap, NeuroSynth, and NeuroVault focus on that researchers know the context of the query re- storing data derivatives. Brainmap and NeuroSynth sult, i.e. why a dataset is returned, and thus can
System
OpenNeuro [
Data original derivatives only derivatives only derivatives only original publications
Experimental setting HED (few datasets only) BrainMap taxonomy arbitrary labels keywords from publication texts ML classifier labels publication text
Queries store so-called peak coordinates, which are a sig- experimental setting is a form of a continuous narnificant reduction from original data. NeuroVault rative, e.g. a movie or an audio recording. Experistores statistical maps of the brain, which carry mental settings are automatically annotated using more information than peak coordinates, yet they various machine learning feature extraction techstill only represent derivations of the original data. niques which also predefine the available querying
Data stored in Brainmap has been manually ex- terms. NeuroScout is a valuable resource, but the
tracted and annotated from the literature. It con- data is a limited sample of what is collected in the
sists of over 4000 scientific publications and 21000 ifeld of cognitive neuroscience.
contrast analyses. It uses a custom annotation Although PubMed is a repository of scientific
pubschema [
NeuroSynth data has been extracted and anno- scribe them and explain why they are insuficient tated from the literature using automated text anal- for the use case of mega-analysis. NeuroSynth autoysis [22, 23]. It accumulates results in form of peak matically extracts terms from scientific publication coordinates from over 13000 publications. Users texts which are subsequently manually filtered for can query the data with single terms or their sets relevance in the field of neuroscience. Unfortunately, organized into topics. the publications primarily describe analyses, their
The data stored in NeuroVault is annotated man- results and how they contribute to the field. Inforually with arbitrary labels. There is no schema and mation about the data, in particular the events that the annotations often contain abbreviations and occurred during data acquisition, is often not prostudy specific terms. Additionally, the statistical vided, especially if it is not directly relevant for the maps are stored as part of a collection, which has a analysis. Consequently, the adequacy of datasets general description. Both the description and label returned by NeuroSynth is evaluated based on the can be queried using keyword search. original purpose of data collection, thus limiting the
OpenNeuro [
In this paper, we proposed a conceptual model for neuroimaging mega-analysis. We formally defined the parts of the model that are essential for finding qualifying datasets and implemented the resulting queries in Neo4j graph database. In a next step, we will demonstrate our solutions on a larger scale including more datasets and the remaining aspects of experimental settings. We plan to integrate the entire HED taxonomy into our knowledge graph and thus enrich the experimental setting annotations with otherwise implicit knowledge. An interesting extension of our work is to make the condition query more flexible by relaxing the subgraph constraint.
relevant datasets for a mega-analysis, but also to enable the execution of mega-analysis. To that extent, we will include the neuroimaging data, analysis workflows and their results into our framework, as highlighted in Section 2.
the Austrian Federal Ministry of Education, Science and Research (BMBWF) under grant number 2920 (Austrian NeuroCloud); the Federal State of
(Digital Neuroscience Initiative); the Austrian Science Fund (FWF) under grant number W1233-B (Doctoral College “Imaging the Mind”). We thank