QUESTO is an ontology formalizing questionnaires and related entities, such as the processes of administering the questionnaire or capturing the answers. It is built according to the OBO Foundry methodology and is a component of an ontological model that aims to enable interoperability between various clinical data sources in the context of a Learning Health System. This article presents the main entities of QUESTO and provides an example of its application: a relational data model is generated from the ontology and used to retrieve data from public health questionnaires stored in a REDCap database.
Clinical questionnaires are commonly used in the medical domain to collect data for patient care and research. Computerized questionnaires are becoming increasingly prevalent, thanks to the widespread use of platforms such as REDcap, which includes a secure web-based application and a database system for building and managing online surveys [1].
These questionnaires could be especially helpful in Learning Health Systems (LHS). Such systems analyze health information generated from patient care in order to better understand a situation, develop new clinical practices and transfer them back to patient care through knowledge transfer tools such as decision support systems [2]. Furthermore, in a pandemic context, LHS can be particularly beneficial by providing a rapid integration loop from data capture to data analysis, instead of relying solely on the traditional, dedicated data capture through study-specific forms.
LHS rely on access to a wide range of health data - including questionnaires - usually scattered across numerous heterogeneous information systems. However, in the absence of standardization of data and practices, these clinical sources are not easily integrated. This is the well documented “Tower of Babel problem” [3]. In recent years, open source, applied ontologies have emerged as a reliable solution to this problem. Applied ontologies can support a common, source-independent representation of clinical information in order to allow interoperability between various sources and contexts (care delivery, research…) instead of data models developed for more context specific operations [4].
As part of the ontology-based Learning platform in health research and social
services PARS3 (“Plateforme apprenante en recherche en santé et en services sociaux”
https://griis.ca/en/solutions/pars3 ), we have developed ontologies for domains like drug
prescriptions (PDRO [
While these ontologies enable access to data stored in clinical systems like electronic
medical records, we did not have the ontological blocks to represent questionnaires like
those handled by REDCap. There are already associations between semantic resources
and REDcap databases (see for example [
This paper presents the creation of an ontology for representing questionnaire:
QUESTO. Our ontology has been inspired by a previous work from Bona et al. about an
ontology of patient questionnaires [
Like our prior ontologies PDRO and LABO, QUESTO has been developed in
accordance to the OBO foundry methodology [
The ontology was used in the context of a health research introduction course taken
by every third-year student of the Faculty of Medicine at the University of Sherbrooke.
As part of their training, the students conducted phone interviews, administering a public
health questionnaire to selected citizens from the region about their lifestyle and health
needs. The information collected from the telephone interviews was captured using a
centralized database using REDcap. Each student designed a research question, using
two to four variables from those in the questionnaire. A subset of the data fitting these
variable choices was then extracted and loaded in a virtual server specific to each student.
In order to do so, a relational model has been created from the QUESTO ontology [
Before specifying the main classes constituting this ontology, it is important to define what a questionnaire is, detail its different constitutive parts, and describe the form answering process, i.e. the list of subsequent steps that will lead to the recording of the answers of an individual to a questionnaire.
The Merriam-Webster dictionary defines a form as “a printed or typed document with
blank spaces for insertion of required or requested information” [
The Ontology of Biomedical Investigation (OBI) [
Therefore, we introduced the classes QUESTO:Form and QUESTO:Questionnaire, with the latter being a subclass of the former. Both are subclasses of IAO:Document that is defined as: “A collection of information content entities intended to be understood together as a whole.” We propose the following Aristotelian definitions (we will communicate with the OBI team in order to harmonize them): • QUESTO:Form =def.“A document that contains a set of questions. It may also contain allowed answers to some of those questions, and specifications about how to record and store the answers.” • QUESTO:Questionnaire =def.“A form that is intended to be answered by a human respondent.” The mention of an “human respondent” in the definition of QUESTO:Questionnaire implies that some forms may not be answered by human. For example, a login form on a web site could be answered automatically by a password manager service.
A questionnaire not only includes a set of questions with a choice of possible answers in some cases, but also indications on how to ask the question, how to record the answer or what to do if there is no answer. Let’s consider the example presented on Figure 1. It consists of two questions in a questionnaire administered in the following way: one person asks the questions directly or by telephone to the respondent and writes down the answers on paper version. The paper questionnaires are then centralized and the answers are entered into a database by a third party. In addition to the questions, the questionnaire therefore includes instructions on how to report them and how to store them in the database.
Question1 asks for a Canadian postal code associated to the respondent’s primary residence and requires that the answer be reported according to the Canadian postal format: “letter digit letter space digit letter digit”. Another indication about the required format is given by the presence of underscores in the answer field. In addition, in the absence of an answer by the respondent or if the respondent does not have a primary residence in Canada, the following responses should be reported respectively: ‘A9A 9A9’ or ‘Z9Z 9Z9’. Finally, the answer to be entered in the database should not include the ‘space’ character.
Question2 asks whether the respondent is under the care of a family doctor. It provides a choice of answer with boxes to be checked by the person who report the answer. Moreover, a specific content should be entered in the database according to the box checked (ex. ‘01’ for ‘yes’).
QUESTO introduces classes to represent these components and bind them in a mereological structure in the following class, that we are going to present now: Question and reporting and storing specification.
This entity is the cornerstone of our ontology. It includes, for a given question, the question per se, possible answers (if any), and all the additional information about how to ask, report and record the answer. Since it directs how to administer a question, we define it as a subclass of IAO:Directive information entity:
Questioning and reporting and storing specification=def.“A directive information entity that provides specifications about how to: 1) ask a question to a respondent; 2) report this answer; 3) store this answer.”
A question and reporting storing specification has as parts two other entities: an Extended question representation, which specifies the question and the constraints on its possible answers, and an Answering and reporting and storing item specification, which specifies constraints on the choice of answers, their reporting and storing.
This class includes the part of the question that will be directly asked to the respondent, i.e. the question with its possible answers. It is also a subclass of IAO:Directive information entity:
Extended question representation =def. “A directive information entity that specifies a question to be asked to a respondent with possible answers and/or answer format constraints.”
representation, which is the question itself, e.g.: “Do you have a family physician?” in
Restricted question representation =def. “A directive information entity that specifies a question to be asked to a respondent. It does not include the possible answers nor answer format constraints.”
Question2 includes a list of possible answers (‘Yes’, ‘No’, ‘Does not know’ and ‘does not answer’), whereas Question1 merely includes an indication about the expected format. In both cases, these constraints on answers constitute the answer specifications:
Answer specification =def. “A directive information entity that specifies constraints on the answer to be given by a respondent to a question.”
As highlighted by the previous examples, an Answer specification may be a selection of possible answers or an indication of the format allowed for the answer. We accordingly introduce Answer content specification (e.g. ‘Yes’ in Question2) and
characteristic specification. Note that only possible answers by the respondent are taken into account here. Thus, for Question2, ‘01’, ‘02’, ‘08’ and ‘09’ are not part of the
For a specific answer, we consider all possible answer specifications as being a part of an Answer item specification:
Answer item specification =def. “An answer specification that is composed of all answer characteristic specifications that constraint a specific answer that can be given by a respondent to a question.”
Moreover, all the answer item specifications of a question are regrouped in an
Answer item group specification =def. “An answer specification that is composed of all possible answer item specifications constraining a possible answer by a respondent to a question.” The Answer item group specification for Question2 includes the following four Answer content specifications: ‘Yes’, ‘No’, ‘Does not know’ and ‘Does not answer’.
While the Extended question representation specifies the constraints on how to ask a question, the Answering-reporting-storing (ARS) item group specification specifies how to record and store the answers. It is defined as follows: “A directive information entity that is, for a given question, the composition of all its answer item specifications, as well as its associated answer reporting and storing item specifications, if it has any.”
For example, the ARS item group specification related to Question2 includes the following four ARS item specifications ‘Yes … 01’, ‘No … 02’, ‘Does not know … 08’ and ‘Does not answer … 09’. We consider the group specification as a collection of items, and therefore use the mereological property RO:has member to link them.
ARS item specification is defined as follows: “A directive information entity that specifies constraints pertaining to an answer item specification, as well as its associated answer reporting and storing item specification if it has any.”
In addition to the constraints at the answer level, already specified in the Answer item specification, we introduce two other levels of constraints that occur when recording the response and storing it. Both are represented by the corresponding specifications for reporting or storing an answer: • •
Answer reporting item specification =def. “An answer reporting specification that is composed of all answer reporting characteristic specifications that constraint the reporting of a specific answer that can be given by a respondent to a question.” In Question2, ‘Check the appropriate box’ is an Answer reporting item specification that specifies to report the answer by checking the box corresponding to the answer given by the respondent.
Answer storing item specification =def. “An answer storing specification that is composed of all answer storing characteristic specifications that constraint the storing of a specific answer that can be given by a respondent to a question.” In
to store the content ‘01’ when the answer to the question is ‘Yes’.
Here again, all the reporting and storing item specifications of a question are regrouped in the subsequent Answer reporting item group specification and Answer storing item group specification. Thus, the answer reporting item group specification for
• The answer reporting format specification: 'letter digit letter space digit letter digit'; • The answer reporting content specifications: ‘A9A 9A9’ and 'Z9Z 9Z9' (in case the answer is respectively ‘Does not know’ or 'Does not have a primary residence in Canada').
Additionally, the answer storing item group specification for Question1 includes the following answer storing item specification: • The answer storing format specification: 'letter digit letter digit letter digit without a space' (which directs e.g. the process that takes as input the reported answer ‘A9A 9A9’ and gives as output the stored answer ‘A9A9A9’, and is motivated by the fact that a space is not pertinent when storing the content).
This process is illustrated in Figure 2 and will be detailed below.
The process of answering a questionnaire involves several agents as participants. The central agent is the respondent, i.e. the person to whom the questions are asked. The respondent is not necessarily the focus of the questions: he or she may be a third party completing the form for another person who may not be able to answer the questions herself (e.g. a patient in intensive care or a newborn baby), or who is not available to do so at the time (e.g. “what is the nationality of your mother?”)
Along with the respondent, one or more other agents may participate in the process by asking him questions, collecting his answers and recording them. These subsequent sub-processes can be carried out by agents that are not necessarily humans. For example, a question can be asked by a person orally interrogating the respondent or transmitted visually or orally to the respondent by a computer. Consequently, one can define classes of various roles and role-bearing agents (merely defined as subclasses of BFO:Material entity): •
Questionnaire respondent =def. “A material entity that has a questionnaire respondent role.” • Questionnaire respondent role =def. “A role borne by an agent that is manifested by the agent providing an answer to a question from a questionnaire.” • Questionnaire reporting agent =def. “A material entity that has a role of reporting an answer from a respondent to a question from a questionnaire.” • Questionnaire reporting agent role =def. “A role borne by an agent that is manifested by the agent reporting an answer from a respondent to a question from a questionnaire.” • Questionnaire storing agent =def. “A material entity that has a role of storing a reported answer from a respondent to a question from a questionnaire.” • Questionnaire storing agent role =def. “A role borne by an agent that is manifested by the agent storing a reported answer from a respondent to a question from a questionnaire.”
answering a question, but not in all cases (e.g. a mail-in questionnaire). The questionnaire answering process takes as input a questionnaire and produces the following output document that includes the answers to the questions and associated metadata:
Questionnaire answers and metadata report =def. “A document that is comprised of the stored answers to some questions from a questionnaire as well as other data collected during the answering process. It includes information such as identifiers (for the questions, the questionnaire and the respondent) and timestamps.”
From the initial questionnaire to the questionnaire answers and metadata report, various informational entities are generated at each key stage of the process: answer given by the respondent, reported answer, and stored answer. Those are captured by the following entities: • Respondent answer to a question=def. “An information content entity that is an answer by a respondent to a question from a questionnaire constrained by the corresponding answer specification (if there is any).” For example, a respondent in a phone survey could answer Question2 with the following sentence: ‘Indeed I have a family physician’, but the Respondent answer to a question will be: ’Yes’. • Reported answer to a question=def. “An information content entity that is the report of an answer by a respondent to a question from a questionnaire.” This capture can be the direct transcription of the answer, or a specific information in the absence of an answer, for example ‘A9A 9A9’ in Question1 if the respondent does not answer. • Stored answer to a question=def. “An information content entity that is the stored representation of an answer to a question from a questionnaire.” It can be the transcription of an answer in another format (e.g. ‘Z9Z9Z9’, without space, for ‘Z9Z 9Z9’ in Question1), but it can also the direct replication of the respondent answer and/or the reported answer.
For now, we did not classify those entities under IAO:Data item, which is “intended
to be a truthful statement about something” [
Each of these informational entities can be the input and the output of sub-processes of the process Answering a questionnaire=def. “A process of a human answering some questions from a questionnaire.”, which is the process described above as a whole. It has a Questionnaire as input and a Questionnaire answers and metadata report as output. It has as parts several subprocesses, that are also instances of subclasses of BFO:Process, as listed below: • Answering a question =def. “A process of a human answering a question from a questionnaire.” • Reporting an answer =def. “A process of reporting an answer to a question from a questionnaire.” • Storing a reported answer =def. “A process of storing a reported answer to a question from a questionnaire”. The output of this process can be a code associated with the given answer, the transcription of the answer in a different format, etc.” Note that these labels are more general than the intended meaning of the classes, as they have been chosen for ease of reading. Input and output relations are formalized with the object properties OBI:has specified input and OBI:has specified output.
Main classes of QUESTO are hierarchized in the taxonomy outlined in Figure 3.
187 medicine students took the health research introduction course and administered the questionnaire to more than 1,600 local citizens. All data collected were centralized in a database developed with REDCap.
Our ontology was then processed in our PARS3 environment to automatically generate a relational model. This model was then mapped to the relational model of the REDCap database.
Once the mapping was done, we were able to automatically generate 187 anonymized data subsets from the REDCap database, one for each student, according to the questions they selected. Each extracted dataset was then loaded in separate virtual servers containing the statistical software RStudio (https://rstudio.com). Every student was able to log in only to his or her server, therefore having access only to the data he or she had selected for statistical analysis.
The QUESTO ontology was created as an expansion to the work of Bona et al. [
Along with other ontologies such as PDRO and LABO, QUESTO takes part in an ontological model to enable interoperability between various clinical data sources in a context of learning health system. QUESTO allows the creation of queries that are not directly related to a specific database schema. Its practical application for collecting data from a clinical questionnaire designed and stored in a REDcap system provided a validation of this model. QUESTO enabled the secure and efficient extraction of data out of a REDCap database to a RStudio environment on a personal virtual server (one per student), thereby ensuring that each student would see only the required information. The queries were expressed in neutral terms using classes mentioned above, and the PARS3 system translated them for local execution on the REDCap database. Returned data was structured according to the QUESTO structure and semantic. This ensures that the enduser (in this case, a medical student) has access to a non-ambiguous semantic for the extracted data without having to rely on contextual knowledge, a priori knowledge of the source system or even plain old guessing. It has proven to be an effective way for students to complete their courses and will be used for future cohort of students.
In addition, given that QUESTO does not include anything specific for the experiment described above, we can anticipate that our model would allow us to capture data from any other REDCap-managed questionnaire or similar systems.
However, while this work focused on the ontological representation of the structure
of a questionnaire and associated processes of answering, reporting and storing, future
work should investigate the semantic dimension of the questions and answers. As a
matter of fact, the current state of QUESTO enabled us to give access to the required
information in context of the project previously mentioned, but it implies that the
requester knows what questions are of interest. QUESTO allows us to represent the
constituent elements of the questions and the processes related to them, but we do not
have yet the ability to represent what the questions are about, such as a geographical
location in Question1 or the presence of a family physician in Question2. Further work
is required to this effect, as many open questions remain regarding aboutness of ICEs
[
Finally, future work on this ontology of questionnaire will take into account a newly
proposed mereology of informational entities in clinical documents [