The social science research process has traditionally required researchers to engage in a largely manual information seeking process and then manual analysis to extrapolate trends from past work into the study design process including hypotheses generation and variable declaration. Across several computational disciplines including probabilistic relational learning and machine reading, we see opportunity to advance and significantly positively change the social science research process in a world with more and more scientific textual data accruing on a yearly, if not, daily basis. Here we present an articulation of the problem we see with the nature of publishing scientific findings in largely unstructured natural language text along with our perspective for how both micro- and macro-reading methods can play a role together with the work being done on the scientific research cycle itself to drive better and more eficient research across all of science.
• Information systems → Information systems applications; Data mining; • Computing methodologies → Natural language processing; Information extraction;
The social science research process, and more generally, the
scientific research process is a general set of steps, forming a cycle, that
researchers within the social sciences generally take as they engage
in and conduct research in their sub-fields of interest. The process
usually starts in the model step (See Figure 1 for our working
definition of this process) with one or more questions of scientific inquiry
that a researcher wants to formally investigate where the research
begins considering prior literature and scafolding hypotheses; this
is seen as the start of a research cycle. These ’investigations‘ take
many forms (e.g. qualitative, quantitative, theoretical, conceptual)
and sub-types (e.g. causal, non-causal). Depending on the type of
investigation, for example, an experimental design with
hypotheses and analyses testing the efects of an independent variable on
a dependent variable, diferent levels of background context are
needed by the researcher to appropriately design such a study.
2
The research process itself, conceived and refined over hundreds
of years, typically allows for new research to be designed and
conducted by building of of past knowledge. It is however within the
past 60 years that the sheer magnitude of the scientific data being
observed and collected has resulted in an inability for researchers
to keep up and fully utilize it all. Perhaps as a symptom of this
or as the global workforce has slowly shifted away from physical
labor jobs towards those of science and engineering, the speed of
scientific literature growth every year has been rapidly increasing;
whereas, the amount of time researchers have to discover, digest,
and synthesize new research directions has not been increasing. [
Researchers are often times left between choosing what appears
within the first couple pages of their search platform’s results and
spending vast amounts of time trying to discover related terms (and
consequently, studies) that should likely be considered as a part of
their literature review and hypotheses and experiment planning
activities. Figure 2 is but one example of a bibliometric database’s
growth over the past several years; overall there is an increase from
year to year as more research publications are produced. Albeit, in
recent years there has been a push to create better bibliometric tools
and better citation search engines and recommendations systems
(e.g. [
Given for example several many research papers (e.g. 20-40 papers)
on a particular variable or construct of interest, averaging between
8-12 pages, the researcher may spend between 2 and 4 days
annotating and synthesizing what would amount to a meta-analysis over
the set of papers to find the information they need to perform the
necessary critical thinking that drives hypothesis formation (taking
place in the predict step). If instead there were semi-automated
processes that, together with the researcher, extracted: variables of
interest, relationships, and experimental trends1, then, some
significant amount of time could be saved from, among others, the
traditional literature review and analysis tasks that occur during a
research cycle; suddenly days of manual annotation and
relationship summarization are reduced to minutes or hours. This is in
fact an area where both macro- and micro-reading techniques can
play a significant role. During macro-reading activities, a collection
of research articles are skimmed to extract broad phenomena like
variables or methods used in specific articles (e.g. [
The amount of scientific data being generated is growing at a faster
rate every year and human ability to continue to suficiently include
and reason over these vast amounts of knowledge is already being
challenged. Gone are the days where research in sub-disciples grew
at a slow and steady rate and where researchers and their graduate
students could adequately review and synthesize findings as they
build on prior works. And whereas some would say that the amount
of data being generated bids a farewell to traditional scientific
methods and processes [
This material is based upon work supported by the Defense
Advanced Research Projects Agency (DARPA).
1We see here a need for the continued work related to design and development of
scientific research registrations processes and conceptual taxonomies (see e.g. [