- Someone seeking entry to an access controlled facility or through a border control point may face an in person interview. Questions that may be asked in such an interview may depend on the context and vary in detail. One of the issues that interviewers face is to ask relevant questions that would enable them to either accept or reject entrance. Repeating questions asked at entry point interviews may render them useless because most interviewees may come prepared to answer common questions. As a solution, we present an interactive voice response system that can generate a random set of questions that are contextually relevant, of the appropriate level of difficulty and not repeated in successive question answer sessions. Furthermore our system will have the ability to limit the number of questions based on the available time, degree of difficulty of generated questions or the desired subject concentration. Our solution uses Item Response Theory to select questions from a large item bank generated by inferences over multiple distributed ontologies.
I. INTRODUCTION
Physical control points such as human guarded gates, border control points and visa counters provide entry into facilities or geographical regions to those that can be admitted legitimately. Legitimacy is usually determined by rules, regulations or policies known to entry control personnel whose duty is to ensure that these policies are enforced while admitting people. In order to do so, they hold an interview, in which an aspiring entrant is asked a series of questions, and possibly show some documents and demonstrate some knowledge about the contents of the documents or attributes contained in them. Successful interviews should have questions that are relevant, of a reasonable level of difficulty (i.e. not too difficult or common knowledge) and not to have been asked in prior interviews for the same purpose without drawing accusations of bias from rejected entrants. Ideally, a successful interview should accommodate differences in accents and provide assurance that it is unbiased against similar attributes.
Given the recent success of interactive voice response (IVR) systems such as auto attendants, satellite navigation, and personal assistants such as Apple’s Siri, Google’s Voice, Microsoft’s Speech, we investigated the possibility of specializing IVR systems for access control such as: Visa interviews, entry point interviews, biometric enrollment interviews, password reset, etc.
Although IVR systems have come a long way in recognizing human voice, and responding to human requests as if responses come from another human, most of the existing IVR systems are pre-programmed with questions and their acceptable answers, and consequently have limited capability in satisfying the Use Case at hand.
The first minor limitation of current IVR systems comes from the fact that, the human starts and drives the conversation. The second limitation is that most IVR systems have a finite number of pre-programmed conversations. Therefore the set of questions generated by such a system are the same for every conversation. This limitation may expose the set of questions so that aspiring entrants may come with prepared question-answer pairs, even if the subject matter of the questions may be unfamiliar to them. Consequently, having the ability to select questions from a large pool may resolve this limitation. The third limitation is that when selecting a random set of questions from a large pool, the set of questions asked may not have the desired overall level of difficulty to challenge the user. Solving this issue is relevant because all aspiring entrants expect to have a fair interview. The forth limitation is that questions must be able to discriminate between someone that knows the subject matter from someone who guesses an answer.
As a solution we created an ontological inference
based IVR system that uses item response theory (IRT) to
select the questions [
In previous work, we introduced a policy-based IVR
system for use in access control to resources [
Usually ontologies have a large number of axioms and assert even more facts when using reasoners. Consequently, blindly converting such an axiom base to human-machine dialogue would result in very long conversations with many disadvantages. The first is that human users would become frustrated of being subjected to long machine driven interrogations, and thereby reducing the usability of the system. The second is that long conversations take longer time to arrive at an accept/reject decision, and likely to create long queues at points of service, such as Airports and guarded doors. In addition, having a line of people behind one person in close proximity may leak private information of the interviewee. Also, others may quickly learn the set of questions and answers that would get them mistakenly authorized, thereby gaining unauthorized access.
We use IRT, which provides the basis for selecting
tests from large number of potential questions.
Psychmotricans in social sciences and standardized test
preparation organizations such as the Educational Testing
Services that administer standardized test examinations
like SAT, MCAT, GMAT etc. have developed
methodologies to measure an examinee’s trust or
credibility from answers provided to a series of questions.
In traditional tests, the ability of the examinee is
calculated by adding up the scores of correct answers.
Currently, Computerized Adaptive Testing (CAT) that
relies on IRT has been used to better estimate an
examinee’s ability. It has also been shown that the use of
CAT/IRT reduces the number of questions necessary to
reach a credible estimation of the examinee’s ability by
50%. CAT/IRT can be used to control the number and
order of questions to be generated based on examinee’s
previous answers [
Our goal in this work is to demonstrate and build an access control system using dialogues of questions and answers generated from a suitable collection of ontologies. Table I shows a sample dialogue that is generated from our research. Our prototype automated IVR system can help immigration enforcement at a border control point making a decision to permit or deny a person asking for entry. Through a dialogue of questions and answers, the interviewee will be assigned a numerical score that will then serve as a threshold in the decision making process. This score is calculated using IRT, which takes into account the correctness of the user’s responses and the weight of the individual questions.
The rest of the paper is written as follows. Section II describes an ontological use case, Section III describes the response theory. Section IV describes the system architecture. Section V describes our implementation. Section VI is about experimental results and section VII concludes the paper.
II. Motivating Use Case
In this section, we describe an example ontology used in our work to generate efficient dialogues of questions and answers that are used in assigning a numerical value to an interviewee’s ability or trust level .
Person: defines humans in general and has subclasses like; International Student and Friend. Event: defines an event that has a location, date, time and type like terrorist attack International Student: is a person who is on an F-1 or J-1 Visa type University: defines a university. Some of its current members are MIT and GMU City: defines a city like Boston Country: defines a country like USA, Russia, Dagestan, Kazakhstan, etc.
State: defines a state like Massachusetts Visa: defines visa types like F-1 and J-1 student visas and maybe others.
This ontology represents many kinds of data classes and relationships between these major classes and individuals. For example, we define the “Boston Marathon Bombing” as a “Terrorist At tack” that happened in “Boston”, which is a city in “Massachusetts” state. Another fact is that “Dzhokhar Tsarnaev” is an “Event Character” in the “Boston Marathon Bombing” “Terrorist Attack”. Also we have an “International Student” who is a friend to “Eve nt Character” in the “Boston Marathon Bombing”.
We use this ontology in our work because it serves as a good example showing the strength of our system. First, it shows the possibility of generating valuable questions from asserted or inferred facts. Second, it enables the implementation of the theory under consideration (to be discussed later in the background section) to generate efficient and secure dialogs that are used in: (1) making entry control decisions, (2) assigning numerical values to ability or trust in the shortest time possible and (3) load distribution among interviewers and diverting people for further investigation.
The use of ontology in such an application provides many benefits. The most important amongst them is reasoning. Using a reasoner we are able to derive facts from asserted ones. These facts are used to generate questions to measure the knowledge or ability level of an interviewee on a subject under questioning. In IRT, better item selection and ability estimation happens when a large set of items is available to draw questions from. Using ontology, the large number of derivable facts provides us with the ability to increase the number of questions, and also control the quality and difficulty of questions.
Although there are many reasoners such as FaCT++,
JFact, Pellet, RacerPro, we use HermiT [
One such fact derived from asserted ones in our ontology, is finding the friends that hold a student visa of a person involved in a terrorist attack. To explain this, we have “ dzhokhar is friend of Dias”, “Dias is friend of Azamat”, “Dias has F -1 visa”, “Azamat has a J-1 visa”, “dzhokhar is an “Event Character” in the “Boston Marathon Bombing”, “Boston Marathon Bombing” is a “Terrorist Attack”. Thus we infer (using the HermiT reasoner) that Azamat and Dias are the friends of the Boston Bomber and therefore need to be questioned at any entry point. We use this chain of derivations to generate specific questions from them.
Reasoners and the explanations that they provide are very important components in our work to generate relevant and critical questions from ontology that measure knowledge and estimate ability from a response in order to grant access or assign trust. In the example above, the reasoner provided an explanation of the inference using 11 axioms. We use such a number in defining the difficulty of questions generated from such inferences, as Fig. 2. A sample explanation of an inferred axiom in Protégé using the HermiT reasoner will be explained in section V. Fig. 2 shows the HermiT reasoner explanation of our inferred fact.
III. BACKGROUND
The main purpose of an IVR system is to interact with
humans using a voice stream. An IVR environment
consists of a markup language to specify voice dialogues,
a voice recognition engine, a voice browser and auxiliary
services that allow a computer to interact with humans
using voice and Dual Tone Multi-Frequency (DTMF)
tones with a keypad enabling hands-free interactions
between a user and a host machine [
IRT, sometimes called latent trait theory is popular
among psychometricians for testing individuals, and a
score assigned to an individual in IRT is said to measure
his latent trait or ability. Mathematically, IRT provides a
characterization of what happens when an individual
meets an item, such as an exam or an interview. In IRT,
each person is characterized by a proficiency parameter
that represents his ability, mostly denoted by ( ) in
literature. Each item is characterized by a collection of
parameters mainly, its difficulty (b), discrimination (a)
and guessing factor (c). When an examinee answers a
question, IRT uses the examinee’s proficiency level and
the item’s parameters to predict the probability of the
person answering the item correctly. The probability of
answering a question correctly according to IRT in a
three-parameter model is shown in (1), where e is the
constant 2.718, b is the difficulty parameter, a is the
discrimination parameter, c is the guessing value and is
the ability level [
= + (1 − ) ( ) (1)
In IRT, test items are selected to yield the highest information content about the examinee by presenting items with difficulty parameter values that are closer to his ability value. This reduces time by asking fewer and relevant questions rather wider range ones while satisfying content considerations such as items or rules that are critical for a decision of access or scoring.
In order to determine the difficulty and discrimination
parameters of a test item, IRT uses Bayesian estimates,
maximum likelihood estimates or similar methods (MLE)
[
In IRT, responses to questions are dichotomously
scored. That is, a correct answer gets a score of “1” and
an incorrect answer gets a score of “0”. The list of such
results consist an item response vector. To estimate the
examinee’s ability, I RT utilizes maximum likelihood
estimates (MLE) using an iterative process involving a
priori value of the ability, the item parameters and the
response vector as shown in (2). Here, is the estimated
ability within iteration s. is the discrimination
parameter of item i, where i=1,2,...,N. is the response
of the examine (1/0 for correct/incorrect). is the
probability of correct response from (1).
probability of incorrect response = 1-
[
Access control policies specify which subjects may
access which resources under some specified conditions
[
, and if the user provides one, the system’s voice recognition software attempts to recognize the input and checks the correctness of the answer. Based 94 facts raising the total number of axioms and candidate request and sends back one of the following responses: to generate questions to 273. accept, reject, error, or unable to evaluate.
IV. USING IRT TO MANAGE AND CONTROL
Fig. 3 shows the overall architecture of our system. We use derived or axiomatic facts of the ontology to create questions asked by our IVR system. Given that a large number of facts can be derived from our ontology, but only few questions can be asked during an interview, we use IRT to select the facts that are used to generate questions.
without human involvement by combing English words or phrases such as “Does” or “ Is-a” with ones chosen from the ontology of (subject, property, object) triples.
The expectation is a dichotomous answer of either (yes, no) or (true, false). The ontological property names such as “isa”, “has -something” are prime candidates for creating true/false questions. Our system transforms the question We use IRT to manage and control dialogue questions generated from a large pool of ontologically derived facts in a way that shortens the length of dialogues while keeping the maximum accuracy in estimating the user’s trust. The IRT-based estimated (θ) represents the trust or confidence of the system in the person answering the questions in order to make an access decision.
We have used the OWL annotation property to assign IRT parameters to axioms. Annotations were selected in order to keep the semantics of the original ontology and structure intact. We annotate every asserted axiom in the ontology with IRT parameters, which are: difficulty (b), discrimination (a) and guessing (c). Currently, we assume all asserted axioms have the same default degree of difficulty and
discrimination values of 1. The code snippet in Fig. 4 illustrates our annotation using Java with OWL API. An improvement to this approach would be to assign different values for difficulty and discrimination by using domain experts. )); OWLAnnotationProperty irtDifficultyAP = AnnotationProperty(IRI.create("#irt_difficulty" OWLAnnotation irtAnnotation = df.getOWLAnnotation( for (OWLAxiom axiom : axioms) { irtDifficultyAP , df.getOWLLiteral(1.0));
OWLAxiom axiom2 = axiom.getAnnotatedAxiom
(Collections.singleton(irtAnnotation)); manager.addAxiom(ontology, axiom2); }
We weigh inferred facts more during the estimation process. We are calculating these parameter values from the number of explanation axioms used in individually inferred fact.
Our current scheme each of difficulty value assignment is shown in Table II; where higher values or weights are assigned according to the number of explanation axioms used to infer a fact, and consequently the question generated from it is considered to be more difficult than one generated from an asserted fact. Fig. 5 illustrates a code snippet for inferred axiom annotation.
In our current work and for testing purposes we use a
default value of “1.0” for discrimination and “0.0” for
guessing, which practically neutralizes them leaving the
difficulty parameter as the sole factor in estimating ability
using equation 2. However, our solution and algorithm are
based on the IRT two-parameter model, which relies on
the item’s difficulty and discrimination parame
ters. Fig. 6
shows our algorithm to estimate ability based on equation
2 [
Set<OWLAxiom> explanation =
explanationGenerator.getExplanation(axiom);
//Annotate inferred axioms using the number of
explanation
OWLAxiom tempAxiom =
Annotation));
axiom.getAnnotatedAxiom(Collections.singleton(irt
manager.addAxiom(inferredOntology, tempAxiom);
characteristics of the axiom and not on the number or the
percentage
of
correctly
answered
questions
as in
traditional testing. The ability estimate produced by our
implementation also comes with a standard error (SE)
value that is a measure of the accuracy of the estimate.
Equation (3) presents the formula used for standard error
calculation [
Higher standard error indicates that the estimate is not very accurate, while lower values indicate higher confidence in the estimation. This too can be used as a means to discontinue the dialogue or use an alternate decision method.
V. IMPLEMENTING THE ONTOLOGY-BASED IVR
SYSTEM FOR ENTRY CONTROL
Here, we present a prototype of our system showing the major components. It is not yet validated as a deployable system, but it works for the sample use case.
ay[i]); Algorithm 1: IRT Ability estimation Input:a priori theta, Difficulty, Discrimination, Answer Output: posteriori theta, standard error /* calculate theta and standard error*/ 1:for (counter < items.length) do 2: itemDifficulty=parseFloat(difficultyArray[i]); 3:itemDiscrimination=parseFloat(discriminationArr 4:answer=parseFloat(answerArray[i]); 5:probTheta=calculateProbability(itemDiscriminati on,aTheta,itemDifficulty); // equation 1 6:thetaSplus1= claculateTheta(probTheta, thetaS); //equation 2 9:return thetaSplus1;
We use the Voxeo’s Prophecy local server as our voice platform for voice recognition and to run the dialogues. Java, Java Server Pages (JSP), and Java Script (JS) are used to implement the architecture modules and to implement IRT procedures used to estimates the user’s ability/trust scores.
Voxeo’s Prophecy is a comprehensive IVR and
standards-based platform [
In our work, we start with ontology, annotate every axiom with an “irt_difficulty” property of value “1”. T hen we use this ontology in the HermiT reasoner to infer implicit axioms and their explanations. The inferred facts are themselves annotated with “irt_difficulty” property and values calculated by factoring the number of explanation axioms using the schema stated in Table II.
For example, when annotating the inferred fact “the friends of the Boston Attack Bomber”, which has an explanation that includes 11 axioms shown in Fig. 2, the irt_difficulty annotation would be “3.0”;; which is the highest value on the scale of IRT difficulty parameter values in Table II. We assume that answering a question generated from a high-valued fact is a difficult task. Consequently, if the answer to a question derived from this fact is correct, the ability estimate would be impacted more positively than a correct, but easy one and more negatively if the opposite happens. An example is the asserted axiom that “Boston is located in Massachusetts”. Because this is an asserted fact, it is annotated with value “1.0”;; which makes a question generated from it an easy one and thus not affecting the ability estimate greatly.
This process is basically generating the item bank in
CAT/IRT terminology. Each item in the item bank
contains a question, an answer and IRT parameters. In
addition to saving it as ontology in any of the supported
formats, this item bank can also be supported by using a
more specialized CAT/IRT platform like Cambridge
University’s Concerto [
<form id="Begin"> <block> <prompt bargein="true">
Welcome to the United States. To accelerate your entry, we will appreciate your responses to some questions to verify your identity and eligibility </prompt> <assign name="xacmlResource" expr="’point of entry"/> <goto next="#Resource"/></block> </form>
Fig. 7. A sample Homeland security VoiceXML greeting form The conversation starts with a menu in VoiceXML hosted on the local Voxeo Prophecy web server. The voice browser connects to the web server and converts text to speech and speech to text. Fig. 7 shows a sample VoiceXML code.
Fig. 8 shows our algorithm integrating ontology, IVR and IRT. This algorithm was successfully implemented using JavaScript and Java Server Pages (JSP) embedded in VoiceXML pages. The main steps are as follows: Load the ontology and parse the XML into Document Object Model (DOM).
Extract the axiom’s triplet (subject,p roperty, object) Extract the axiom’s IRT difficulty value from the annotation Establish a VoiceXML “For” loop that synthesizes a question from string or text values to speech (TTS). The question consists of an auxiliary verb, object, property and subject to test the correctness of an axiom.
The system waits for a response. If there is one it converts it to text and recognizes it. If it adheres to grammar then a value is assigned as an answer. If there was no answer then VXML re-prompts the question up to a programmed number of times. If exceeded then an appropriate VXML is executed. The vector of binary answers is used to estimate the IRT ability.
The loop continues until a threshold of or the maximum number of questions is reached.
The IRT ability estimation algorithm, as illustrated in Fig. 6, takes the variables: answer vector, a priori , difficulty, discrimination and calculates a posteriori .
If the answer is correct (“yes” or “true”), a value of “1” is assigned. If not, a “0” is assigned.
The last posteriori in the loop is the estimated user’s ability and can be compared to a threshold value set by an administrator. Access is granted if ( > ℎ ℎ ) and denied otherwise. Algorithm 2: dialogue access evaluation Input: a priori theta, Difficulty, Discrimination, Answer Output: access control decision /* make access control decision from ontology*/ 1: domDocument=parse(ontology); // DOM 2: subjectArray=getAxiomSubject(axiom); 3: propertyArray=getAxiomProperty(axiom); 4: objectArray=getAxiomObject(axiom); 5: difficultyArray=getAxiomDifficulty(axiom); 6: /*use voiceXML , JSP to generate dialog*/ 7: for (counter < items.length) do 8: <vxml:Prompt> ‘[auxiliary verb]’ +propertyArray[i]+ “ ” + objectArray[i] +“ ”+ subjectArray[i]; 9: <vxml:Field>= user_utterance; 10: response[i] =
Field.voiceRecognition(user_utterance); 11: if response[i]= ‘Yes’o r‘ true’ 12: resultVector[i]=1; 13: else 14: resultVector[i]=0; 15: endfor; 16: theta = IRT_algorithm(resultVector,
difficulty, discrimination,aPrioriTheta); 17: if theta > thetaThreshold 18: permit; 19: else 20: deny;
EXPERIMENTAL RESULTS
Our implementation shows that efficient dialogs could
be generated from ontologies that have been enhanced
with IRT attributes. The successful implementation of the
IRT in dialogues of questions and answers shortens the
number of questions necessary to reach an accurate
estimation of subject’s ability, knowledge or trust by at
least 50% as it has already been proved by the IRT
literature [
VII.
CONCLUSION
We have designed and implemented a novel IVR
system that can dynamically generate efficient interactive
voice dialogs from ontologies for entry control. We have
used IRT to generate shorter dialogues between the
system and a human speaker. IRT is useful in
compensating for inaccurate voice recognition of answers
during dialogs or accidental mistakes. Our entry control
decisions are made based on an estimation of a level of
trust in a subject derived from the importance or
relevance of axioms in ontology. The use of IRT also
enables the reordering of questions with the purpose of
preserving privacy in IVR systems. With the advancement
in the fields of mobile, cloud and cloud based voice
recognition such systems become important in defence
and physical security applications [