In this paper, we discuss a data-intensive research methodology for the digital humanities. We highlight the di erences and commonalities between quantitative and qualitative research methodologies in relation to a data-intensive research process. We argue that issues of representativeness and reduction must be in focus for all phases of the process; from the status of texts as such, over their digitization to pre-processing and methodological exploration.
It is common among scholars in the eld of the humanities to emphasize the
inherent di erences between the methodologies of humanistic and natural-scienti c
research. Although the di erences between the humanities and the natural
sciences have been conceptualized in various ways { whether it has to do with
di erent elds of interest, ontologies, etcetera { commentators have tended to
warn that the humanities should not model itself after the methods of natural
science (see Kj rup 2009: 75{91 [
However, the merging of traditional humanities with quantitative approaches has opened new methodological venues that point towards the natural sciences rather than the interpretive, largely qualitative methodology associated with humanities. Digital humanities is to some extent a merging of two elds, the humanities and data science1. The aim of data science is to enable a high-level overview of our data and give us the possibility to discern patterns that could otherwise not be seen, and to grasp quantities and time spans of data that that we otherwise would have no hope to cover in a single lifetime. While traditional humanities has made use of relatively small amounts of text, data science has, in theory, the possibility to handle in nite amounts2. Still, data science develop methods that are limited to small scopes and single questions, while the qualitative humanities can see beyond individual texts and times, to answer multi-faceted questions related to cultural contexts, societal and identity conditions and change. Qualitative humanities can potentially capture the suggestive, sensory, existential, contradictory, ambivalent and ambiguous aspects stemming 1 While data science uses scienti c methods and processes to extract knowledge and insights from data of all kinds, in this paper, we consider only textual data. 2 The limiting factor is most often the amount of available, relevant texts. from the human imagination and experience. This is the domain expertise that the humanities bring into the meeting with data science.
The convergence between the humanities and data science brings with it clear
bene ts, including the possibilities to base conclusions on large and
representative samples of text. This paper focuses on methodological convergence.
Commentators have previously noted that, apart from o ering alternative
methodological and epistemological venues for traditional humanities, the expansion of
digital humanities has also given rise to great number of questions and issues
that need to be carefully considered from an epistemological perspective (see e.g.
Berry 2012 [
The overall purpose of the paper is to present a starting-point for addressing some speci c epistemological issues induced by the methodological convergence of humanities and data science. We discuss a number of interrelated issues and concepts concerning representativity and reduction, as they apply to method, data and results. Our ambition is not to be comprehensive, and each projects will inevitably be concerned with these issues to di erent degrees. We highlight some key features of the data-intensive humanities research process, in the hope to further an important discussion in, not only the digital humanities community, but also within the traditional humanities.
The disposition of the paper is as follows: We begin by discussing the dataintensive research process. A hypothesis-driven process is natural in data science, while the humanities are more often driven by a research question. In this section we connect the two, in order to clarify how research questions and hypotheses can work in data-intensive humanities research. Second, we discuss use of models as one instance of reduction. Third, we discuss the validation of results from the data-intensive process. Fourth, we discuss how the rise of digital humanities and quantitative methodologies places traditional humanistic approaches in another light. We conclude by emphasizing the current challenge for the digital humanities community in exploring, interpreting, validating, evaluating data science methods and, not least, discussing its further implications for the humanities. 2
To a great extent, digital humanities employs data science methods to gain insights into large scale, often diachronic, collections of digital text. While this has been done on other materials and for other purposes in data science, the aim in digital humanities is to generate humanities knowledge. This collaboration should, in the optimal case, o er the possibility for the humanities to base their conclusions on larger and representative samples of text, as well as o er possibilities to ask other kinds of research questions. This alternative methodology should generate knowledge that is sustainable and robust against time, and additional scrutiny of the same or additional sources. In addition, it should Research Question
Hypothesis results
Text mining method o er the data science new, broader kinds of problems to target their methods towards.
However, typical digital humanities projects are conducted with either a strong data science or humanities bias. The data science projects, on the one hand, are often conducted with a computer science, math, or language technology perspective where the interpretation and the understanding of the research questions are sacri ced at the expense of mining techniques and large quantities of data. The humanities projects, on the other hand, are often conducted on smaller scale data using methods that may not be the best suited for the problem, or data, at hand. 2.1
If we concentrate on the methodologies of systematic data-intensive research, it typically has a clear process and several important components. There is data, a text mining method and results. Motivating this are research questions and hypotheses. In the process of data-intensive research, there are two main methods for making use of large scale text. First, in an exploratory fashion to nd and formulate interesting hypotheses, that is, work departs from a general research question. Alternatively, one starts with a well-de ned hypothesis and employs large scale text to nd evidence to support or reject the hypothesis in a validating fashion. For both of the above, there is a research question involved that can correspond to one or several di erent hypotheses.
The process can be schematically illustrated as in Fig. 1. Both the exploratory and the validation paths follow the same process, but start at di erent points. The exploratory path moves from the research question to hypothesis via data and text mining method. The validation path has already been boiled down to one or several clearly de ned hypotheses and starts from there. Then data and methods can better be chosen with respect to the research question at hand. The exploratory path can be said to aim at discovering patterns, while the validation path is aimed at demonstrating or proving patterns.
In both paths, a text mining method is employed to generate (directly interpretable or aggregated) results from the text. 2.2
One of the great challenges of digital humanities is to reason about how results from text mining (or other kinds of data science) can be used to corroborate or reject a hypothesis, and as an extension, can contribute to the wider research question. This amounts to interpreting the results and \translating" them into conclusions about the original research question. Here is where the humanities' in-depth domain knowledge comes into play. However, let us compare three different starting points for a research process when it comes to the relationship between research question and hypothesis. 1. One research question and one hypothesis: A researcher is interested in how the general sentiment with regards to a concept, like a trade or technology, has changed over time. The research question focuses on \how", and data and method are designed so as to follow the exploratory path. If this results in a hypothesis about more precisely how notions changed, then this hypothesis \that" can be corroborated or refuted through the validation path with adjusted data and method. 2. One research question and several hypotheses: A researcher is interested in how a certain technology, means of transportation or communication, has a ected society. This research question needs to be broken down into several, and a number of them must be used to answer the question in full. Which sentiments were there? Which new behaviors were the result? Which facets of life were a ected? Suitable data and method need to be devised and by following the exploratory path, these questions can be reformulated as propositions: hypotheses, which are tested using the validating path. 3. Data and text mining method but no research question: We can envision the case when there is an interesting source of data but no clear research questions (for example, the digitized letters of an in uential author). A text mining method can be used to nd interesting patterns and signals to explore further. That is, we follow the exploratory path to nd a rewarding hypothesis. The focus is the data and the text mining method. Often times, a method like topic modeling is used as a way of getting an overview of di erent themes around a concept of interest. These topics can be explored and good hypotheses formulated in a more informed fashion. 2.3
In traditional humanities, the researcher is the bridge between results and interpretation. In data-intensive humanities research, the situation is slightly different. The typical result of a text mining method is not necessarily directly interpretable for the hypothesis, nor need the hypotheses be directly interpretable with respect to the research question. The process of moving between results and the research question is in itself a result and in need of evaluation.
To exemplify a \model of interpretation" (see more on models in the
context of humanities in section 3), we go outside of the digital humanities where
data-intensive research is tied to societal impact (SCB, [
Each individual criterion is measured by asking a sample of a countries' residents.To measure the criteria (hypotheses in our case), each person is asked a set of questions (corresponding to the text-mining methods). These questions are aimed at capturing the phenomena that we are interested in. One example question is Can you a ord at least two pair of shoes? The answers from each participant are then weighted according to group belonging, and groups with lower answering rates are weighted up to get a fair representation. This weighting is a part of the interpretation; di erent weightings provide di erent results and should be subject to evaluation and discussion. In addition, the number of criteria needed to de ne material poverty is also an essential part of the interpretation of results. All of these acts go into a model of interpretation. In this example, parts of the model are clearly de ned and can be subject to discussion and evaluation. We argue that all digital humanities-projects should make their model of interpretation clear, and preferably evaluate with respect to alternative models. 2.4
The process of formulating appropriate hypotheses to answer a research question
is almost always reductive: narrowing focus to particular aspects, sorting out in
order to clarify. In all cases, data is central. Data-intensive research can only
provide results for that which is represented in the data. Already at this point,
it is important to reason about representativity. Text is always a reduction of
the world, and only representative for a part of it. In historical text, like the
Google books corpus, men are almost ten times more likely to be mentioned
than women, until the beginning of the 20th century, when the two begin
moving towards the middle and nally meet somewhere in the 1980's ([
These conditions of text as data are general: to them come methodological reductions. Typical digital humanities applications have restricted access to text corpora, or reduce them for their speci c purposes and make use of collections of digitized newspapers, literature, or social media, to answer research questions. Reduction can be seen as the other side of the coin of representativity; typically, the more representativity, the less reduction. At this point, a rst methodological reduction is taking place. Then come other kinds of methodological reduction, as part of the method used. The model as such provides an illustrative example of methodological reduction and Fig. 1 is one example. Consequently, the usage of scienti c models within data-intensive approaches constitute another case of reduction. In the next section, we discuss the issue of scienti c models, and their function and implication for the digital humanities. 3
Models and their usage within science has been an object of debate within philosophy and theory of science for a long time, yet these discussions have almost exclusively been devoted to the usage and importance of models within the natural sciences and not within the humanities (e.g. Bailer-Jones 2009, 2003; Giere 2004; Morgan and Morrison 1999). Yet, several aspects within these discussions have implications for the purpose of this paper as the issue of models and modeling are an intrinsic reductive feature within the methods of data-intensive humanities.
Daniela Bailer-Jones de nes a scienti c model as a \description of a
phenomenon that facilitates access to that phenomenon" (2009: 1 [
Within the humanities, at least within mainstream humanities, the usage
of models is still rare (McCarty 2004: 255 [
In relation to the function and usage of models within natural science, models
within digital humanities seem to attain the same kind of functions and usage.
For example, Willard McCarty points up on how models o er both possibilities
for an increased access and manipulation, as well as a way for representation of
phenomena: `By "modeling" I mean the heuristic process of constructing and
manipulating models; a "model" I take to be either a representation of models
within something for purposes of study, or a design for realizing something new'
(McCarty 2004: 255 [
We can say that models and modeling within digital humanities carry a reductive potential; that is, at the same time as models - by simpli cation o er a number of advantages (they both represent and o er possibilities for manipulation of phenomena of interest), they reduce the phenomena that we want to investigate. Consequently, models do carry an intrinsic reductive power that we have to be aware of; of what is the model representative? 4
In all phases of the data-intensive humanities research process, reductions take place. The rst, basic step of reduction is that our texts cannot be fully representative for all phenomena that we want to study. Next, our selection of the texts further reduces our base. The digitization of the text is, as explained above, a model of the original text and thus a reduction in itself. Depending on the quality of the digitization, more or less of the textual information is intact. Finally, the pre-processing and the text mining methods additionally reduce the original text in several ways.
Large scale texts cannot be studied by taking all aspects and words into account. It resembles the situation with creating a dictionary of language usage. Not all words, nor every single usage of a word, can be included in the dictionary, without making the dictionary as large as the text itself, and thus rendering it useless. Instead, a dictionary generalizes the usage. In text mining, the generalization is done by focusing on certain aspects of a text, or certain parts, or both. A typical method for focusing in large scale text mining is to keep only the most frequent words (typically ranging from the 10 000 to the 250 000 most frequent word forms), or to keep words of certain part-of-speech (nouns, noun phrases, verbs, adjectives etc). Stopwords and function words (words that carry little meaning but are very frequent) are often ltered out to speed up the process and to increase the chances of the methods to nd relevant information. In each ltering step, a reduction is made.
In the example: \... Cecilia was a very clever woman, and a most skillful counter-plotter to adversity." we keep \Cecilia was / clever woman / skillful counter-plotter / adversity"after ltering function words and stopwords. Only \Cecilia", \woman" , and \counter-plotter" are kept if we focus on nouns and only \was" if we keep verbs. Additional words might be removed if we lter on frequency as well.
The original information is thus reduced at the preprocessing step. In addition, the text-mining method itself also performs a reduction. To see patterns of di erent kinds, we cannot view all of the words at the same time. Methods like topic modeling, clustering and word representations of di erent kinds lead to a set of results. These results need not include all text. For example, topic modeling can result in a set of topics that are not representative for each text on which the topic modeling was performed. Again, a reduction is performed. Additionally, the accounts need not include all results: often, the number is predetermined. Finally, the results are interpreted in support or refutation of one or several hypotheses, or as indications pertaining to a wider research question.
From the results of text mining, we have to draw conclusions that contribute to the knowledge of one or another aspect of the world. At this stage, we need to reason about representativity as the opposite to reduction. We began by discussing how texts are representative of a part of the world, and di erent texts have di erent representativeness. Social media texts for example, have a much higher representativeness of western young people, than of middle east elderly or women. Next, the reduction of the text mining method a ects the representativity of the result. How much of the original text are the results valid for? Finally, the same question should be asked once we reach the conclusion step; these conclusions that we draw, for which part of the world are they valid; men or women, rich or poor, young or old? { the parameters are numerous. The limit is set by the representativity of data; results can not be considered, with su ciently high con dence, to be representative for more than the original text was; if there were few women in the original text, then the results cannot fully re ect women.
Generally, we can see a trade-o between reduction and representativity which we illustrate in Fig. 2. We rarely start with full representativity, and we rarely end with full reduction (or we would not be left with anything at all). The relation between the two need not be linear; representativity does not have to decline at the same rate as the amount of information is reduced. Good text mining methods attempt to reduce information with a lower loss, similar to compression methods of images or videos; we can reduce the amount of information hypothesis method data preprocessing digitization selection m o d e l that is being sent, while keeping a fairly high quality of color and movement. While improvements are made at a quick pace, current data science techniques are almost always associated with loss of representativity.
This loss of representativity means that our results are a small window out of which we view our large-scale and possibly long-term text. Our window gives us access to incomplete pictures of the view, and di erent positioning of the window will result in di erent views. The image that can be viewed corresponds to the text(s) that we have chosen as our basis. The di erent positioning of windows corresponds to the method and the preprocessing that has been chosen. Creating topic models using nouns or verbs, using 10 or 100 topics, looking at the rst 10 or 500 most likely words, are all choices that result in very di erent windows out of which we view the same text that a ect the conclusions that we draw.
In the mathematical world, this insight is well known. Assume that we want to nd the highest point on (or the value that maximizes) a curve, for example corresponding to pro t. The curve has two peaks, the right one is higher than the left one, and in between there is a low point. If we start at the leftmost end of the curve, and walk upwards as long as there is an upwards, we will end up in the peak that is the lowest of the two (a local optimum). If our criterion for continuing is that we have to keep moving upwards, this criterion is met at the peak, and we will stop walking. With a limited window size, we will not be able to see the other, higher peak. If we instead start in the right end of the curve, using the exact same methodology, we will end up in the higher of the two peaks (a global optimum). Again, without knowing that we have found the highest peak, because of our limited view out of the window. If we look through the rst window, we see the local optimum and might draw one set of conclusions. If we instead look through the second window, we might draw completely di erent conclusions (for example, that the bene ts are high enough to continue with the development of a product, or not).
Any scholarly approach demands reduction in order to be precise, and in the humanities, assumptions about society and mankind traditionally govern these perspectives in more conspicuous ways than in the hard sciences. This has considerable implications when new kinds and volumes of data are introduced and historical transformations are modeled. In particular, the question of representativity and validation is a ected by the question of quantitative and qualitative methods.
For a critical example of methodological issues inherent in a traditional
humanities approach, we can turn to the study of lm genres. In recent years there
has been a revisionist trend in lm genre studies. Commentators have argued
that many traditional accounts of popular lm genres are inaccurate and that
the research has been driven primarily by critical and theoretical agendas, rather
than by a commitment to wide historical analysis. Not only have scholars all too
often limited their interest to a handful canonized classics or works by
wellknown lmmakers, but they have also tended to substitute \assumptions and
generalizations for detailed empirical research" (Neale 2000: [
A similar situation can be found within literature studies. Here, Franco
Moretti's proclamation about distant reading as a way of reinterpreting literary
history can serve as an illustration of both the enrichments and the
complications induced by methods in which the empirical material is analysed through
various quantitative methods (summarized in Moretti 2013 [
In the same vein, Matthew Jockers, a forceful advocate of quantitative
literary analysis consequently delimits studies to the quantitative in his
Macroanalysis (Jockers 2013 [
As Ted Underwood comments (2016 [
There are more ways than this to achieve a productive convergence of
quantitative and qualitative methods, but we may conclude that this kind of approach
has at least two distinct bene ts. On the one hand, it connects in a very natural
way to traditional humanities, as it provides powerful ways to prepare qualitative
studies with quantitative tools. On the other hand, it alleviates the demand for
representativity, as the quantitative results need not claim to be exhaustive, but
rather lead to careful explication of patterns, structures, lines of reasoning and
the like { and as in traditional humanities, such explication can be made without
strict demands of representativity. So, the proposition is that traditional micro
levels cannot only be replaced by macro levels, but that meso levels are also
needed, and above all, that it must be possible to freely move between the levels
and between the quantitative and the qualitative: adaptive reading. It is here
that the humanities can o er the sensibility of the vague and elusive cultural
contextualization. Machine-aided reading also makes it possible to explore the
dynamics of literature in other ways. Studying only the rst edition of a work is
fruitful in many ways, but we may now trace the changes of a work through all
3 An entirely di erent, highly interesting exploration of reduction as a method is
presented by Allison (2018) [
Somewhat paradoxically, questions of completeness and representativity are often (and rightly) leveled in criticism of digital humanities, but only rarely discussed in traditional studies. There is a distinct need to clarify how collection, preprocessing and exploration have been performed in digital humanities studies { but there is also a distinct need for the corresponding methodological accuracy in a great number of traditional studies. One reason that this has been neglected is the acknowledging that a traditional, qualitative study cannot be entirely representative when the material is too large { however, the traditional approach has also been that of course the study cannot claim to be entirely complete, but it can claim to show distinct developments and tendencies at the same time as there may be other examples that do not correspond. On the one hand, this is precisely the reason why qualitative digital humanities more easily can handle the problem of representativity and completeness. On the other hand, there is a need for concise clari cation of how a study was performed, in order for it to be properly assessed. Here, we believe that digital humanities have the power to set fruitful examples and incite methodological development even in those parts of the humanities which do not focus on digital materials and methods. 6
With this paper, we hope to further a discussion of nding good ways of employing a data-intensive research methodology for the humanities. We have highlighted the di erences and commonalities between the quantitative and qualitative in relation to a data-intensive research process.
Even though every project has its own conditions and characteristics, we argue that the data-intensive digital humanities must focus on representativeness and reduction in all phases of the process; from the status of texts as such, over their digitization to pre-processing and methodological exploration. Because the aim is to generate sustainable knowledge in the humanities, special attention must be given to the interpretation of hypotheses with respect to a research question. The merger of quantitative and qualitative methods emphasizes the need to validate on di erent levels of the research process and to maintain transparency. We conclude that the methodological convergence between the humanities and data science has the potential to raise methodological awareness in the more traditional and non-digital humanities, which only rarely or to a limited extent deals with questions of, among other things, reduction and representativity.