The objective is to transform indicator-coded variables extracted from open-ended questions by using semantic technologies rather than manual coding, in order to compare two forecast models and evaluate which yields the best results. Experiments are carried out within the domain of customer satisfaction. For this purpose, a leading Austrian financial institution provides a large sample of survey data. In the questionnaire, respondents are asked explicitly about their overall customer satisfaction. This dependent variable is to be predicted by use of statistical analysis techniques. For the process of model building, a procedure is required for generating coding categories in order to enable the use of unstructured verbatim data on critical incidents. One approach is based on traditional methods, while the other uses ontologies and reasoning.
Semantic technologies enable new forms of collaboration beyond the boundaries of applications. Through this inventive kind of technology, computers are able to put information to a wider variety of uses and provide a novel basis for reasoning, and this results in a direct effect on quality.
Quantitative research methods can be supplemented with semantic technologies to
make use of unstructured data from qualitative research. Companies might use the
integration of such procedures in knowledge management systems as a strategic
advantage [
The coding of open-ended responses to survey questions by applying semantic technologies to utilize the information in the data analysis process is a new, and interdisciplinary area of application. Hardly any explicit literature referring to this topic exists. Combining “data and text mining” in business forecasting is also something which has only relatively recently been demonstrated experimentally. There have so far been no documented or published evaluations on the application of semantic technology in the coding of open questions, or its integrated usage within the scope of data mining utilized in building predictive modeling methods. Although applications for such combinations have been presented by well-known software companies, as solutions with high potential for the near future, the scientific evaluation of the integration of semantic technologies in forecast analytics is missing.
Terms of reference and state of the field vary between the borders of standard quantitative and qualitative research procedures.
In the field of quantitative research methods, there is a wide variety of statistical and
mathematical analysis procedures to choose from. The concept of “data mining” [
In analytics and interpretation within qualitative research procedures, computer-aided
techniques are becoming increasingly more used in order to analyse and interpret
methodically evaluated data such as text files [
Transcribed text files from qualitative interviews, such as expert interviews, are
becoming more commonly analysed through QDA (qualitative data analysis)
applications in correlation with concepts like “the stages of open, axial and selective
coding in the grounded theory”, “thematic framework” or the “application of the most
essential technique in objective hermeneutics” [
Responses to open questions determined by a specific survey design are regularly coded (founded on similar concepts used in QDA applications) to take advantage of statistical analysis. ESOMAR (European Society for Opinion and Marketing
Semantic Technologies used in the Data Analysis Process 3 Research) defines the process of coding as: “The organising of responses into categories and the assignment of a unique numerical code to each response prior to data entry.”
This transformation into quantitative components is subsequently used for analysis and can lead, in principle, based on the resultant additional information, to an improvement in predictive modelling. Manual coding efforts are nevertheless time consuming and subject to human error when coders develop certain rules for classifying ambiguous cases. Part of the conceived work is to evaluate a new method for improving this coding procedure through the application of semantic technologies.
The aim of the thesis is to develop and deploy an application by assembling opensource components capable of handling both the mathematical and statistical evaluation of analytical methods, as well as the classification of answers to openended questions. This will be carried out by the use of semantic technologies, in contrast to the use of manual classification and coding. In order to demonstrate a comparison between these experimental results, the quantification of any differences will be shown empirically.
Most procedures for automatic and quantitative content analysis track the occurrence
of specific words in n-dimensional space [
Rather than solely use these mathematical and statistical procedures, this paper will focus on the extended possibilities offered by semantic technologies, such as the classification of open answers to “meaningful higher-level unities” through reasoning by rules and ontologies in combination with Natural Language Processing methods.
Automatic word sense disambiguation using a “light-weight ontology” like GermaNet
[
The classification process should be extended to pre-existing knowledge in the
form of upper-level ontologies like e.g. DOLCE, PROTON, SUMO [
For this purpose, a leading Austrian financial institution will provide a large sample of survey data. An extensive dataset with 21,146 interviews from 35 customer satisfaction projects/waves are available for analysis. The critical incident technique was applied using questionnaires. This resulted in 8,242 positive comments and 9,854 negative comments, which can potentially be used for the coding.
In the questionnaire, respondents were asked explicitly about their overall level of customer satisfaction. This dependent variable is to be predicted through statistical analysis techniques. The objective of this research is the theoretical foundation, analysis and development of a coding process for forecast models with semantic methodologies.
These experiments have not been performed as of yet, and the results therefore remain to be determined. The concept for the work has, however, been developed. The means of obtaining the data have been established and the authorisation granted - a confidential agreement has been signed and several discussions have been held regarding the data-quality, as well as the procedure itself.
The next steps will involve: development of the code-plan; performing the coding process with several coders; and applying quality assurance. Data understanding, data preparation and performing the modelling tasks are to follow subsequently. The work’s conclusion will provide an answer to the research question of whether or not it is possible to substitute and/or enhance manual coding through the use of semantic technologies.