We have been exploring direct representation for scientific research reports. Direct representation proposes that entire research reports can and should be highly structured. Moreover, we propose that collections of research reports can be interwoven into a rich semantic knowledgebase.
Causal models, whether explicit or implicit, are central to science. Scientific research articles would benefit from using highly structured models which support state changes and causal relations. We use "flows" as a generic term for sequences of transitions such as workflows. flowcharts, plans, mechanisms, and other causal sequences. Potentially, flows could be circular or have feedback loops.
Recently, we have focused on the comprehensive ontology SUMO [24] as the vocabulary for such models. One important feature of SUMO that distinguishes it from most other ontologies is the inclusion of rules. We also propose the adoption of object-oriented modeling [7] in place of traditional approaches to presenting and processing knowledgebases. We implement transitions between object states and apply linguistic models of "case roles" to describe them [9].
In previous work, we have proposed a broad framework for flows that can be applied across domains [6,10]. We have conducted several studies describing mechanisms and systems with structured, semantic vocabularies. Building on the modeling techniques in [9], we describe steps toward developing a rich model-oriented knowledgebase to support science. We describe policies for making these simulations plausible and useful. While our current work focuses on qualitative models, the approach should also support quantitative models.
There is a long tradition of research on scholarly publications (e.g., [1,30,32]). Structure has increasingly been added to descriptions of scientific research.
Taken to the logical conclusion, we propose that research reports should be totally structured. Structured research reports have many advantages. For instance, they can support interactive interfaces for visualizing and exploring the relationships among interlocking flows. Visualization of flows is related to timeline visualizations (e.g., [2]).
Several types of flows are already widely used in science.
Workflows are used to specify experimental procedures (e.g., [14]). Mechanisms are often central for describing complex phenomena [6,11]. However, before our work, Research Designs (e.g., [28]) as distinct from Research Procedures have not been explored as structured flows.
Beyond describing aspects of workflows and research phenomena directly, other parts of science research reports make claims and generalizations about phenomena. These can be characterized as a type of discourse [1,2,13,15,19,25,30]. We agree with [21] that research inferences cannot be based simply on formal logic. Rather, they follow a preponderance of evidence and consistency with other results.
Germ theory was a paradigm shift in biology. It was sparked by the development of the microscope and the resulting ability to see microbes. Louis Pasteur was a major proponent of germ theory, which was the notion that tiny organisms, invisible 1 While Pasteur's report was not as detailed as current research reports, it is straightforward and useful as an exemplar.
without the aid of a microscope, produced spoilage, fermentation, and some diseases.
One early controversy was whether microbes developed only from other microbes or whether they developed spontaneously. That is, whether existing organisms are needed to propagate new organisms and those existing organisms are carried by air currents. We focus on a version of Pasteur's classic experiments that explored spontaneous generation [23,26]. Pasteur's experiments are generally regarded as pivotal in confirming the importance of microbes and how they propagate. 1
In [3], we used Pasteur's germ theory experiments to illustrate the potential for applying direct representation to scientific research reports. In this paper, we return to that example and describe how several techniques proposed in our recent work can be implemented to produce unified scientific research reports. Our primary goal is the development of the underlying modeling framework for the organization and application of scientific knowledge. These models emphasize causal relationships (rather than classification) so we focus on what might be called transitional propositions. We also describe an interface for interacting with the models. 2In short, we propose that the best way to capture and describe the structure of scientific research is by considering multiple flows which are yoked. The claims from the research are propositions that can be coordinated in a knowledgebase.
While science uses systematic manipulations and/or observations, it also crucially depends on models about the phenomena under investigation.
We employ two major flows to capture these two aspects. [9,10].
While much of science is concerned with developing general principles, sciences such as geology and astronomy, as well as clinical medicine, deal more with particulars. Reasoned models can be developed for either general (abstract) principles or instances.
Traditional research papers follow the IMRD (Introduction, Methods, Results, Discussion)3 [32] framework. Swales [32] described the purpose of the Introduction of a research report as "creating a research space" (CARS). This includes defining a Research Question, Research Motivation, and Hypotheses.
Addressing the Research Question is the immediate goal of the research. Typically, it involves determining the existence, properties, mechanisms, processes, or applications associated with an entity or phenomenon. In some cases, the goal may simply be the replication of other research or addressing some criticisms that were raised about prior work. In this paper, we require that Research Questions can be answered with structured propositions. 4Examples of the Research Motivation might be practical (e.g., to find cures for a disease) or simply to acquire knowledge. Either way, it is an axiom, a given representing a valuation. Additional statements may link the Research Question to the Research Motivation.
The researcher then establishes plausible hypotheses by considering the factors potentially relevant to the Research Question by referring to established principles and previous research.
Based on the hypotheses, a Strategy is determined. The Strategy consists of the Research Design and Research Procedure. The Design is an overall framework for obtaining valid results. Independent and dependent variables are key parts of the Design. Typically, one of the hypotheses proposes some causal relationship between the independent and dependent variables.
The independent variable may be manipulated either directly or indirectly. In natural experiments, the researcher identifies a natural event that creates conditions suitable for the research. These may include cases from natural science, social science [8], and medical science (e.g., the effects of smoking on cancer). In field and laboratory experiments, the researcher takes specific actions to manipulate the test environment.
Standard Research Designs are so entrenched in some fields that many researchers are unaware of them. In other fields, a variety of research paradigms is used and their merits are debated. [28] is a well-known analysis of the issues with different research designs. It discusses a wide range of designs and provides a notation for describing them. Moreover, it compares the possible threats to valid inference using different research designs. While [28] is primarily based on field research with randomization such as is common in social science, it can and should be applied more generally.
It is highly desirable to have at least two conditions for comparison [28]. This is especially true when one group is a control group and there is randomization of participants across conditions. However, these recommendations are not followed when a second group is difficult or impossible to implement, or when the researcher believes that he/she knows about and has controlled for possible extraneous factors.
The Research Procedure is a script or plan for the researcher's actions. It applies methods and materials. Those are usually specific to the domain under investigation and may threaten the internal validity of the research if applied incorrectly.
There is considerable controversy about the role of hypotheses in scientific research. In cases such as Pasteur's experiment discussed below, the hypotheses are sharply drawn and are associated with a distinct, although not necessarily fully understood, mechanism. However, in other cases, a hypothesis may be nothing more than a hunch. 5 Our models are typically situated in a microworld 6which is a spatial region that provides the context for the interaction of objects involved in the phenomenon under investigation [12]. The 5 Perhaps it would be better to use the term "potential explanation" rather than hypothesis. For example, in [4] we examined [33], a modern biology paper dealing with the protein pathway related to Wallerian Degeneration. That paper cast a wide net and tested hypotheses which seemed unlikely to be relevant.
manipulations directly or indirectly change the state of the microworld and/or its contents. In other work (e.g., [7,8,10]) we allow complex microworlds; potentially, they could be subdivided and have different levels of temporal and spatial granularity.
As the research is conducted, the raw data can be structured and stored according to the semantic model.
The data can be manipulated and workflows for data transformations and statistical analyses can be included 7 along with the massaged data.
Using the data, we can make comparisons across the flows. These comparisons are the basis for claims. Claims are propositions. They have a truth value that expresses a judgment or opinion about some aspect of the research (e.g., the causal relationship between the independent and dependent variables). The primary comparison is set up by the Research Design. In Pasteur's study which we analyze below, the comparison is relatively simple. In other cases (e.g., [33]), the comparisons may involve complex objects and processes, and statistical tests that require additional flows.
Research must satisfy many constraints; many things can go wrong and invalidate the results. [28] identifies two major types of validity for research, internal and external validity 8 . Internal validity refers to problems with the Research Procedure and Methods, and whether they implemented the intended research conditions. The researcher may check on the effect of a novel or tricky manipulation. Such checks on the manipulation would also be described with flows.
[28] lists potential threats to validity for each research design. Structured research reports should include specific structures for handling each of these issues. For instance, the outcome summary could have a list of hypotheses and challenges to their validity.
External validity refers to the ability to generalize beyond the experiment. Some generalizations may be straightforward, but others would be based on conditions.
[28] describes criteria for generalizations.
Generalization may require referring to broader issues within the research area or in other areas.
We would like to model those broader contexts, but, in many cases, they are not currently part of any structured model base. Eventually, such a model base could be developed; until then we can sketch a temporary framework (see Section 3.4).
Explanations may simply state a general rule. They may also try to describe how the rule applies to a given situation. If pressed, a mechanism to support the rule might be given. For instance, if we were explaining why hot air balloons rise, we would assert the rule that "hot air rises" and then might go into a discussion of the molecular dynamics of gasses (see Section 4.1).
Farmers have considerable interest in understanding and controlling fermentation. The results of Pasteur's studies [23,26,31] are of practical importance for endeavors such as dairy, 9 No systematic randomization was done and there was no statistically significant sample, but the control groups suggest wine, beer, tofu, and soy sauce making, and for controlling infectious disease. In [3], we used Pasteur's research to explore the possibilities for highly structured research reports. In this paper, we take another step toward realizing that goal. We consider one of a series of related experiments by Pasteur. Specifically, we develop flows and an interface for presenting a structured description of one of Pasteur's germ theory experiments.
Pasteur put a nutrient broth in two sets of flasks. He boiled the broth and then sealed the neck of the flasks. He observed the flasks and eventually broke the neck open on one set of them. The flasks that remained sealed did not show microbe growth, while the flasks with the broken necks did.
We separate two main streams of activity in describing the experiments. The first is the Researcher Activity Model, which is what the researcher does based on the Design and Procedure. The second is the Outcomes Model, which is what happens, or could happen, in the environment under investigation. Although we distinguish them, the two streams are closely interlinked or yoked.
We focus on modeling the microworld and frame the experiment as a research design with two conditions. In the first condition, broth-filled flasks are sealed and then observed indefinitely. In the second condition, the flasks are sealed but eventually broken to demonstrate that spoilage occurs once external air reaches the broth. The critical test, between the sealed and broken-neck flasks, is determined by the Research Design and the manipulations. 9 By modern standards, Pasteur's description of the research is somewhat informal. For instance, although Pasteur mentions that he made multiple flasks, we do not know how many. For illustrative purposes, we have inferred details as needed to complete these examples.
the comparisons that can be made and that must be explicitly represented.
Figure 1 shows the Researcher Activity Model (left) and Outcomes Model (center). Each has two columns, for each of the two conditions. Also shown (right) are Actual results and the key comparison that indicates that H1 (Hypothesis1) is supported (lower right).
At the top of the interface, there are several options to control the features of the visualization. These include:
Toggle Method Details: Presents detailed descriptions of the procedures.
Toggle Model Details: Shows additional details of the models. Potentially, there would be unique IDs for each of the model entities and transitions and the ontological parents associated with each could be displayed [10]. Threats [to validity] and Alternative Explanations Inferences, Related Research, Applications, and Commentary
The interface was implemented with Python using the Tk graphics library. Development is ongoing; the current version is tailored to the specific example and does not include all the features needed for other research reports.
Because of the complex interaction of entities in the Microworld, developing the full hypothesis models required additions to our evolving ontology and modeling framework. While some air had live microbes suspended in it, the air in the sealed flask had no live microbes. Thus, the state of the air is correlated with its location and the history of that location.
We focus here on Hypothesis1 because it is much more specific than Hypothesis0. Hypothesis1 is justified by several claims:
Earlier research by Pasteur had confirmed (0). The other claims are largely consistent with common sense, though they could be tested more systematically as needed. However, even with extended testing, it is difficult to make an unassailable case [21].
A full executable flow model for Hypothesis1 would be analogous to the flow model in [10]. Note that a model for Hypothesis1 would need to include models of airflow in the microworld, ad hoc subregions for the air in the flasks, and multigranular models that describe transitions of individual microbes as well as collections of microbes.
Because they are yoked, any execution of the Hypothses1 model should execute the parallel Researcher Activity model.
Raw data and inferences based on those data can be collected and organized according to the models described here. In Pasteur's study, the key observation is whether spoilage develops once the flask neck is broken and microbe-laden air can enter. That is, the critical test for the Pasteur study supports Hypothesis1, that the living microbes carried by air currents lead to spoilage. 10 We did not model the Actual Outcomes in this case, but we could have because they could be different than the predictions of either of the Hypotheses.11
Based on accepting Hypothesis1, we can state two overall claims:
If we combine ( 6) with (8') we obtain (11).
Spoilage due to fermentation can be minimized by controlling the presence of microbes (11) This suggests the need for cleanliness to control contamination in the preparation of fermented products. Further, if we combine (3) with (11) we get (12), which is the basis of pasteurization.
Spoilage due to fermentation can be controlled by heating the nutrient medium (12) Joseph Lister generalized (3, 8', 13) to bacterial infections to study and promote the need for sterile surgery. Moreover, adding ( 14) yields (15).
Bacteria are a type of microbe (13) Antiseptics kill bacteria (14) Bacterial infection can be minimized by antiseptics (15) Given the importance of each of these inferences for humans, presumably additional work would be done. For instance, specific microbes and the details of conditions for growth could be studied for each medium.
The interface in Figure 1 is adequate for a straightforward experiment such as Pasteur's. However, many modern research papers are much more complex. For instance, [33] includes a description of developing a strain of Drosophila needed for the research. It then conducts a series of overlapping studies that makes a case for its conclusions although no one study provides a definitive test. In such a set of studies, a great many flows can be identified and modeled. The interface will need to be improved to provide better support for that complexity.
The model and interface should be able to reorganize the research report flows to fit the IMRD framework (see Section 2.2). An IMRD Methods section would include the Research Design, Procedure, Methods, and Materials. Each of these components should fit sub-structures or templates and be integrated into the overall IMRD framework.
Claims must be based on clear definitions [12]. We have proposed SUMO as an ontology. SUMO bases its rules on established definitions, but even these need to be expanded and refined.
Although we have related claims to natural language propositions, our structured approach does not require natural language. Moreover, the case roles may be more exactly defined for each transitional and its interaction with various objects.
In Section 2.6, we suggested that an explanation for a claim could present a rule and an underlying mechanism.
There is a broader sense of explanations that they should engage users in a way that promotes understanding. For instance, an extension of Figure 1 could support graphical guided tours as explanations. More elaborate explanations may be tutorial and can be based on pedagogical techniques.
Claims from research reports and general axioms could be collected into a comprehensive knowledgebase. Although comprehensive, such a knowledgebase would be fragmented, changing, and need to represent multiple viewpoints. Even for areas where there is considerable agreement, there are internally consistent areas of knowledge (e.g., Newtonian mechanics) that may be usefully modeled separately from their connection to broader models (e.g., quantum mechanics).
Any knowledgebase of claims will need a range of structured hedges to indicate the type of claim (conceptual/logical, empirical, etc.), level of confidence in the claim, and possible criticisms of it. We would use a preponderance-of-evidence criterion for the acceptance of claims.
To the extent that we want to do inference on these propositions, we will need to support both open and closed worlds [27] and temporal reasoning in a dynamic environment [18,22,29].
The knowledgebase of research reports and claims can be viewed as a digital library. In addition to structured research reports, the library could also include structured surveys and reviews. Such a library could be overlaid with services like those found in a text-based digital library such as metadata harvesting and search indexing. Because the contents are structured, daemons may be able to generate text versions of the reports and to identify redundancy and inconsistencies.
We emphasize propositions that make claims about state changes such as (8). In a knowledgebase these claims should be accompanied by metadata. The metadata should include basic details such as date and creator; they should also link to related claims. If the metadata are said to provide support for claims, the details of that support should be included.
There could be links across structured research reports that are analogous to citations [5]. Our focus is at the level of semantics rather than the characteristics of the documentation. Thus, rather than link authors, we link functionally and semantically related flows (e.g., about methods) that are shared across research reports. In addition, measures analogous to citation metrics and altmetrics could be developed for the strength of claims and the coherence of the knowledgebase [16].
Finally, (structured) annotations and commentary could be added. And administrative and editorial policies should be developed for managing the collection.
We have proposed using yoked flows to manage the complexity of scientific research reports and have presented a prototype of a user interface for exploring those flows.
In addition, we have discussed issues for how claims from empirical scientific research can be collected and coordinated. The discovery and evaluation of causal claims are common to other scientific paradigms [20]. While those other paradigms may have different procedures than empirical research, they are also based on flows. Even if we distinguish classification (e.g., identifying different types of microbes) as a scientific research activity, flows are still used and could be modeled.
While we have pointed out some promising directions, there is still challenging work to be done in populating and organizing a large knowledgebase of credible propositions.