The increasing rate of information pollution on the Web requires novel solutions to tackle that. Question Answering (QA) interfaces are simplified and user-friendly interfaces to access information on the Web. However, similar to other AI applications, they are black boxes which do not manifest the details of the learning or reasoning steps for augmenting an answer. The Explainable Question Answering (XQA) system can alleviate the pain of information pollution where it provides transparency to the underlying computational model and exposes an interface enabling the end-user to access and validate provenance, validity, context, circulation, interpretation, and feedbacks of information. This position paper sheds light on the core concepts, expectations, and challenges in favor of the following questions (i) What is an XQA system?, (ii) Why do we need XQA?, (iii) When do we need XQA? (iv) How to represent the explanations? (iv) How to evaluate XQA systems?
the Web requires novel solutions to tackle. In fact there major deficiencies in the area of computation, information, and Web science as follows: (i) Information disorder on the Web: content is shared and spread on the Web without any accountability (e.g., bots [6–9] or manipulative politicians [10] posts fake news). The misinformation is easily spread on social networks [11]. Although tech companies try to identify misinformation using AI techniques, it is not sufficient [12–14]. In fact, the root of this problem lies in the fact that the Web infrastructure might need newer standards and protocols for sharing, organizing and managing content (ii) The incompetence of the Information Retrieval (IR) and Question Answering (QA) models and interfaces: the IR systems are limited to the bag-of-the-words semantics and QA systems mostly deal with factoid questions. In fact, they fail to take into account the other aspects of the content such as provenance, context, temporal and locative dimensions and feedbacks from the crowd during the spread of content. In addition, they fail to 1) provide transparency about their exploitation and ranking mechanisms, 2) discriminate trustworthy content and sources from untrustworthy ones, 3) identify manipulative or misleading context, and 4) reveal provenance.
Question Answering (QA) applications are a
subcategory of Artificial Intelligence (AI) applications where
for a given question, an adequate answer(s) is provided
to the end-user regardless of concerns related to the
structure and semantics of the underlying data. The
spectrum of QA implementations varies from
statistical approaches (
approaches
exploited might range from Knowledge Graphs (KG)
holding a solid semantics as well as structure to
unstructured corpora (free text) or consolidation of both.
Apart from the implementation details and the
background data, roughly speaking, the research
community introduced the following categories of QA
systems:
• Ad-hoc QA: advocates simple and short questions
and typically relies on one single KG or Corpus.
• Hybrid QA: requires federating knowledge from
heterogeneous sources
Why not som ething else? why do you fail? why do you succeed? H owwhednocIacnorirtercutsatnyoeur?ror? Question Answering System
Search Engine the question of why? Why have been a particular answer chosen? Why were the rest of theAcbanstdriadcattes dis- [P1a]rRagetruarpnhtoA,ORlyemtupruns tios tOhelyomnplyusa:lbum by the alternaregarded? Why did the QA system fail to answer? tive rock band Malfunkshun. [2] It was released after whether it is the fault of tEhxeistmingoqdueels,tiqonuaanlsiwtyeroinfg d(QaAta),doatrasets fail the band had broken up and after lead singer Andrew lack of data? The truth isttohtaratinthQeAesxyisstetimnsgtoQpAerfsoyrmstceommsplex rea- Wood (later of Mother Love Bone) had died of a drug similar to other AI applicasWotienoiinnngtsroaadnrudecpearHobvOlidaTecPOkexTbpQolaAxn,a(atsioneneewsFfdoiagrt-aasnestwweirtsh. coovemrpdiolseed itnhe19so9n0g.s[3a]ndStroenleeaGseodssthareda,lobfumPeaornl hJaisml,abheald, ure 1) meaning they do 1n13okt Wprikoivpeiddiea-baanseyd squuepsptioonr-tainnswger pairs Loosegroove Records. fact (explanation) about wthiteh rfoeuprrkeesyenfetaetudreas:ns(1w) ethrewquitehstions re- Paragraph B, Mother Love Bone: respect to the trustworthqinuieressfinrdaitneg taondtrheeasosnoiungrcoeveorfmiunlt-iple sup- [4] Mother Love Bone was an American rock band that formation, the confidencep/orrteinligabdoilciutmyernatstetotoatnhsweecr;h o(2s)enthe ques- fwoarsmeadctiivne Sferaotmtle1,9W87ashtoin1g9to9n0.in[61]98F7r.on[tm5]anThAendbraenwd answer, and the chain oftiornesasaroendinivgersoeranledanronticnognsstrtaeinpesd to any Wood’s personality and compositions helped to catapult led to predict the final anprsew-eexirs.tiFngorkneoxwalmedpgeleb,aFseisguorrekn1owledge the group to the top of the burgeoning late 1980s/early sshiodwes tehfaft ethcetusoefr seanndtsisptcbohhrieteimonqgatusfi;eacsc(t3tssi)or?wen’qeu‘iptrworeohdvtafihdotererseQieanssAtoennitscneyhg-s,lee-avlelolwsuinpg- f1“o9Ar9ep0pstlheSe”e,astcththlueesdmuelunesddiicnrgseclteehanesee.gr[oo7fu] ptWh’seoohbdoapndedise’dsofodsneulbycuctdeasaysl.sbub[m8e]-, tem. If the answer is repQreAsseynstteemds itonreaaswonawyitshimstriolnagrsutopervision The album was finally released a few months later. the interface of Google, tahnednextphlaeinetnhedp-uresdeicrtiomnsi;g(h4)t wheaovfefer a new Q: What was the former band of the member of Mother a mixed feeling as to whettyhpeerosf/fahcetocida ncormeplyariosonn tqhuiesstaionn-s to test Love Bone who died just before the release of “Apple”? swer or how and why sucaQhnAdanpseyarsftnoermsmws’neearcbeiilssistcyarhytoocsoeexmntrpaaacrmtisrooenlne.vgWanet sfhaocwts SAu:pMpaolrftuinnkgsfhaucnts: 1, 2, 4, 6, 7 numerous candidates? that HOTPOTQA is challenging for the latest
The rising challenges rQeAgasrydstienmgs,tahned cthreedsuipbpiolirttyin,grfealcit-s enableFigureF2ig:uAren1e:xAamnepxlaemfprloemof (tYheanmguletit-haolp. 2q0u1es8ti)onshinere the ability, and validity of themosdtaeltset-ooifm-tphroev-earpterQforAmasnycsetaenmd smake ex- HOTPOTQA. We also highlight the supporting gfaicwvtes nin quesare of high importance, epslpaienacbialellpyreodincticornist.ical domains tsiuopnpQorbatliurneegiltaifslaitcecsdt,sw.hniechceasresaalrsyo ptaortaonftshwedeartatshete. such as life-science involved with human life. The Explainable Question A1nswInetrriondgu(cXtiQonA) systems are an emerging area which tries to address the shortcomings First, some datasets mainly focus on testing the of the existing QA syTshteemabsi.liTtyhteo rpeecrefonrtmarretiacsloeni(nYganangd infereunpce witahbilditiyscorfimreainsoantiinnggwiinthfoinr masaitnigolne pwarhagicrhaphisorbiased et al. 2018) publishedovaerdnaattuaraslelatncgouangteaiinsianngimppaoirrtasntoafspect ofbians-ed odoncruamceen,tg,oernsdinegr,lea-ghoep,ertehasnoinciintyg,. rFeolrigeixoanm,psloe,cial or question/answer alontgelwligiethncteh. eTshueptpasokrtoifngqufeasctitosnoafnthsweering (QpAo)liticainl rSaQnukADof (pRuajbpluirskhaerretanald., t2a0r1g6e)tqeudesutisoenrs (aBreuranyi corpus where an inferepnrocveidmesecahqaunainstmifiaobvleeranthdeombjelcetdivteoway to 2te0s1t7). d(Gesuignnnedintgo b2e0a1n7s)wreariesdegsivseixn afusinndglaempaernatgaralpchompethe answer. Figure 2 isthaenreaxsaomninpgleabtailkiteynoffrinotmellitgheentosryisgte-ms. To ttehniscy qausethsteiocnonsterxetg, aarnddinmgosXt AofI tahsefqoulleoswtiosn:s can in inal article (Yang et ale.n2d,0a1 8fe)w.Tlahregea-sscsaulemQpAtiodantabseeths ihnadve been 1p.roW-hyfadcitdbethaensAwIesreydstbeymmdaotchthinagt?the question with this data set is that thepoqsueeds,twiohnicshrsepqaurkireed msigunlitfii-chanotppsrotogress in2t.hiWshyadsiindglneostenthteencAeIinsythsatetmpardagoraspohm.Aetshainregsuellts,eit? conclude the answer, wdihreiccthionis. nHootwtehveerc,aesxeistainllgthdaetatsimetseh.ave limita- has fallen short at testing systems’ ability to reason Besides, this kind of rtieopnrsethseatnhtiantdioernfsurmtheigrhadtvnanoctebmeenatnsof mac3h.inWeheonvedriadlatrhgeerAcoInstyexstt.eTmrivsiuacQcAee(dJo?shi et al., 2017) ideal form for XQA; forreeasxoanminpgloev,ewrnhaetuthraelrlarnegpuraegsee, nestpinecgially in 4te.stW-heannddiSdeatrhcheQAAI (sDyustnenmetfaali.l,?2017) create a more solely the supportinginfgacQtsAissysstuemffisc’iaebnilti?tyhtoowperrfoerlmiabmluelti-hop5re.aW-hecnhadlloeensgitnhgesAettIinsgysbtye musigngivienfeonrmouatgiohncroetnrifiedvaelnce in are the supporting fascotsn?ingW, hwohepreutbhleisshyesdtemthheams?toArenadson with int-he dtoecciosliloenct tmhautltyipoleu dcoacnumtreunstts?to form the conhow credible is the publisher? And furmthoerer mthoanreo,nreed-ocumen6t. toHowtecxatngitvheen existing question-answer pairs. Nevgarding the interface, ifasorrrnimvoeattaitothntehteeakannedns-wufresor.emr overwhelmed ertheless, mAoIstsyosfttehmeqcuoersrtieocntsacnanerbreoarn?swered if s/he wants to go through all the supporting facts? Is In thbey amreatachoinfgXtQheAq, uwesetioandowpitthtahefeswe qneuaersbtyiosnens-; hownot there a more user-fr⇤iTehnedselyautahporpscroonatrcibhutefdoreqruealplyr.eTsheenor-der of autehvore-r, wteenacpespilnyosnue ffisincgielenptamraogdraipfihc,awtihoicnhsisasli mfoiltelodwass: tation? ship† WisodrekciddoendetwhrhoeunghWdWicCerwolalsinagt.CMU. 1. WhyitddiodesthneotQreAqusiryestmeomrecchoomopsleext hreiassaonnisnwge(er.?g.,
(XAI) is expected to be transparent, accountable and 2. Why did not the QA system answer something else? fair (Sample). If QA is biased (bad QA), it will come 3. W23h6e9n did the QA system succeed? Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–2380
Brussels, Belgium, October 31 - November 4, 2018. c 2018 Association for Computational Linguistics
expectations and challenges in favor of the questions (i) What is an Explainable Question Answering (XQA) system?, (ii) Why do we need XQA?, (iii) When do we need XQA? (iv) How to represent the explanations? (iv)
two layers i.e., model and interface for XQA similar
to XAI
Definition 1 (Explainable Question Answering) XQA is a system relying on an explainable computational model for exploiting the answer and then utilizes an explainable interface to represent the answer(s) along with the explanation(s) to the end-user.
of XQA as (i) explainable computational model and
(ii) explainable interface. In the following we discuss
these two components in more details:
Explainable Computational Model. Whatever
computational model employed in XQA system, (e.g.,
learning-based model, schema-driven approach,
reasoning approach, heuristic approach, rule-based
approach, or a mixture of various models) it has to
explain all intermediate and final choices meaning the
rationale behind the decisions should be
transparent, fair, and accountable (Sample). The responsible
QA system distinguishes misinformation,
disinformation, mal-information, and true facts
Explainable Interface. The explainable interface
introduced in
Information Disorder Era. The growth rate of mis-,
dis-, mal- information on the Web is getting
dramatically worsened
a decision upon the given answer, XQA matters since it enables the end user to make a decision with trust.
contrary, in the domain of health, having the explanations is demanding otherwise the health care providers can not entirely rely on the answers disposed by the system.
We illustrate the life cycle of information on the Web in Figure 4 which can be published as a stack of the metadata. Each piece of information has a publishing source. Further, genuine information might be framed or manipulated in a context. Then, the information might be spread on social media. Concerning its circulation on social media or the Web, it might be annotated or commented on by the crowd.
We feature the explainable QA interface with respect to its life cycle as it should enable the end-user to 1) access context, 2) find the provenance of information, 3) I can access context I can find the provenance of information I can do fact-checking I can do source-checking I can check the credibility of the source I can detect manipulated information I can report mis-, dis-, mal- information I can access annotations (feedbacks) of the crowd about information I can see the circulation of information trusted
not trusted do fact-checking, 4) do source-checking, 5) check the credibility of the source, 6) detect manipulated information, 7) report mis-, dis-, mal- information, 8) access annotations (feedbacks) of the crowd, 9) reveal the circulation history.
The evaluation of an XQA system has to check its performance from both qualitative and quantitative perspectives. The Human-Computer Interaction (HCI) community already targeted various aspects of the human-centered design and evaluation challenges of black-box systems. However, the QA systems received the least attention comparing to other AI applications such as recommender systems. Regarding XQA, the qualitative measures can be (i) adequate justification: thus the end user feels that she is aware of the reasoning steps of the computational model, (ii) confidence: the user can trust the system, and place the willing for the continuation of interactions, (iii) understandability: educates the user as how the system infers or what are the causes of failures and unexpected answers, and (iv) user involvement: encourages the user to engage in the process of QA such as question rewriting. On the other hand, the quantitative measures are concerned with the questions such as "How effective is the approach for generating explanations?". For example, it measures the effectiveness in terms of the preciseness of the explanations. However, this area is still an open research area that requires the research community introduce metrics, criteria, and benchmarks for evaluating various features of XQA systems.
In this paper, we discussed the concepts, expectations, and challenges of XQA. The expectation is that the future generation of QA systems (or search engines) rely on computational explainable models and interact with the end-user via the explainable user interface. The explainable computational models are transparent, fair and accountable. Also, the explainable interfaces enable the end-user to interact with features for sourcechecking, fact-checking and also accessing to context and circulation history. In addition, the explainable interfaces allow the end-user to report mis-, dis-, mal- information.
We are at the beginning of a long-term agenda to mature this vision and furthermore provide standards and solutions. The phenomena of information pollution is a dark side of the Web which will endanger our society, democracy, justice service and health care. We hope that the XQA will be the attention of the research community in the next couple of years.
mation Disorder: Toward an interdisciplinary framework for research and policymaking. https://shorensteincenter.org/information-disorderframework-for-research-and-policymaking/.
memory networks for visual and textual question answering. In International conference on machine learning.
Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.