Requirement analysis for social acceptance of AI medical interview - from view of quality in use Natsuko Noda Shin'ichi Fukuzumi Shibaura Institute of Technology RIKEN Tokyo, Japan Tokyo, Japan nnoda@shibaura-it.ac.jp shin-ichi.fukuzumi@riken.jp Abstract— The purpose of this study is to clarify the extract the requirements on AI medical interview services requirements that must be met for AI-based medical interview referring to the concept of quality in use in ISO/IEC to be accepted by humans and society. In this study, we set up 25010:2011. personas of users who have various ideas, and create usage cases of AI medical interview service based on the personas. From the II. EXTRACTION OF FEELINGS AND THOUGHTS FOR AI created usage cases, we extract and analyze the feelings and MEDICAL INTERVIEW thoughts of the personas in response to each question and usage flow of the AI medical interview service. From this analysis, we In this section, we first assume a free AI medical interview clarify requirements on AI medical interview services referring service [3] and create user personas for it. We create not only to the concept of quality in use in ISO/IEC 25010:2011. Using just one persona but several personas. Although we survey just this standard, we were able to extract the requirements of one specific AI medical interview service in this study, our "usability," "reliability," and "acceptability" that have been goal is to obtain requirements for general AI-based medical not enough considered in the past. In this paper, we introduce services. Those services should be used by various users, that the requirements acquired from the analysis. cannot be represent by one persona. Thus, we create several personas that could cover broader range of those users. We Keywords— quality in use, AI systems, persona, requirement assume that each persona has a symptom, comes to a hospital analysis outpatient clinic for the first time, uses the AI medical interview service in the hospital, and receives information on I. INTRODUCTION possible symptoms and recommended departments. Next, we Currently, AI technology is expected to be used in the analyze how the user feels and thinks about the AI interview medical field in a wide variety of ways, such as medical image service based on the assumed usage case. Table 1 shows some diagnosis by AI-based image processing and medical record examples of questions asked by the AI medical interview analysis by natural language processing. As one of the medical service and the users' feelings and thoughts toward it. applications of AI, AI-based medical interview services are being implemented to reduce opportunities for human-to- human contact in order to prevent infection and to improve Table 1 Examples of questions of AI medical interview and operational efficiency in hospitals. extracted feelings and thoughts However, it is difficult to draw a line between the Question by the Feelings and/or thoughts Persona responsibilities of the doctors and the AI-based services. In the service case of the first visit to the hospital, the relationship between Please indicate Since they use AI, can't No.1 the user and the AI-system is simple because there is no your age and they just use my patient judgment by the doctor. In the case of the second visit, the gender. registration card and skip diagnosis and treatment by the doctors that the user has visited this question? before will occur, and that makes the relationship between the user and the AI complex and blurs the lines of responsibility Please indicate If I were genderless, what (many) between AI and doctors. Therefore, it is necessary to make your age and could I answer? requirements for the acceptance of such AI diagnosis services gender. in society. Please provide I'm not comfortable No.2 To make such requirements, it is necessary to consider not your occupation typing the answers to such only the analysis from the physician's point of view, but also and address. private questions. the system that has contact with the patient and others [1]. And as pointed out in [2], it is necessary to analyze not only from Please provide I can answer this kind of (many) the viewpoint of the service developer but also from the your occupation private question without viewpoint of the users. and address. any hesitation with an AI. The purpose of this study is to clarify the requirements that Are you mentally It's uncomfortable to be No.3 must be met for AI-based medical interview to be accepted by depressed? asked such question when humans and society. I'm depressed because of my injury at a critical We set up personas of users with a variety of possible ideas, time. and create usage cases for an AI medical interview service based on the personas. In this study, patients are limited to Are you mentally If I talk to the AI about No.3 first-time patients; that means these personas have the first depressed? mental issues, can the AI contact with the AI medical interview service. Then, from the understand it? created usage cases, we extract and analyze the feelings and thoughts of the personas in response to each question and the flow of use of the AI medical interview service. Finally, we Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Have you stopped I'm uncomfortable being No.6 going out on a asked such a misguided daily basis? question when I'm bringing my child here because he has a fever. Do you have any This is a question that has No.7 fear of weight gain not been applied to the or dietary patients I support. restrictions? Difficult to support. Do you have any I'm here because I have no No.8 dietary appetite, and this question restrictions? is offensive. In this study, we call this set of question and corresponding Fig. 1 Classification table of usage data feeling/thought pairs as usage data. Note that there can be more than one feeling/thought for a single question by AI, so the set of the pairs of question and feeling/thought in the usage Based on the above classification method, a classification data contains more than one pair of question and table of usage data is created, as shown in Figure 1. Each circle feeling/thought for the same question. Each pair of question shows a usage data and the number in it means the and feeling/thought, i.e. each usage data is numbered for identification number of the usage data. identification. Note that this number is just for identification and the order of the numbers has no meaning. The classification is based on a two-axis matrix analysis from two perspectives: the perspective of "strength of III. CLASSIFICATION OF FEELINGS, THOUGHTS AND relevance of AI in extracted feeling and thought" of the usage IMPORTANCE data, and the perspective of "easiness of answering question." On the horizontal axis, the degree of specificity of "feelings As a preliminary step for clarifying the requirements for and thoughts" as AI interview is classified. and on the vertical AI medical interview services, we classify the extracted usage axis, the degree of ease of answering "questions" is classified. data. To extract the requirements, we classify the usage data with high importance in the medical diagnosis and those with According to this classification, the lower right the data is low importance. in the table, the more important it is considered to be for requirement acquisition for AI medical interview services A two-axis matrix diagram is used to classify the data from from the following reasons. First, the data is in the right side two perspectives: the perspective of "extracted feelings and is strongly related to the AI services. Because we try to extract thoughts" and the perspective of "questions" from which the requirements for AI medical interview services, not for feelings and thoughts are extracted. medical interview services generally, the right side data is • Horizontal axis: Extracted feelings and thoughts. more important. Secondly, to be accepted widely in society, In the horizontal axis, the criterion is whether or not services should have high quality in use. However, difficult the extracted feelings and thoughts are unique to questions to answer that the services ask contribute to making systems and services using AI. For example, feelings the services harder to use. Therefore, the lower down the table and thoughts such as "It's easy to answer private the data is, the more carefully it has to be analyzed. questions with the AI." can be regarded as unique to IV. REQUIREMENTS ACQUISITION AI-based services. On the other hand, feelings and thoughts such as "If I were genderless, what could I In this study, we refer to the quality in use incorporated in answer?" may be general thought to all medical ISO/IEC 25010:2011 [4] in order to clarify the requirements interviews. This categorization is used to classify for AI medical interviews from the perspective of use by important usage data regarding feelings and thoughts patients, attendants, etc. that should be considered in the context of AI medical Based on the classification table shown in Figure 1, we interview service. find requirements for AI medical interview services. First, we • Vertical axis: Questions of the extraction source. examine the "feelings and thoughts" of the usage data and On the vertical axis, the criterion is whether the extract the issues for AI medical interviews. The issues are question from which feelings and thoughts are compared with the elements of the quality model of quality in extracted is easy to answer in the medical interview. use, and the quality elements related to the issues are identified. For example, questions that deal with information on Then, the requirements to be satisfied by AI medical demographic attributes may be easy to answer. On the interviews are extracted for these quality elements. Figure 2 other hand, a question such as "have you been in a shows the extracted requirements, categorized according to dense space or a poorly ventilated room recently?" is the importance of the usage data. slightly less easy to answer, because it deals with information on the patient's factual behavior, which may depend on the user's memory and feelings. Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Table 2 Extracted requirements From high importance usage data From medium importance usage From low importance usage data data Usability - A number of metaphors and - Provide a good tutorial or guide. - The total time from start to result requirements onomatopoeias should be output should be shortened. available to convey the pain. - Provide a rationale for the - Use UI and background music results. that does not cause anxiety. - Use an objective method of answering the question. - Similar or peripheral questions should be grouped together. - Consider the surrounding environment and make the UI easy to handle private information. - Make the UI concise and easy to use. Reliability - The output results should be - Assume errors in patient - The response method should be requirements comparable to the doctor's judgment and input of responses. such that it results in accurate diagnosis results. - Identify the patient's input of information. - To cover all diseases. unrecognized symptoms and cover - Indicate the need for questions. the symptoms. - Include a phase to ask for - Make a comprehensive medical history and medication assessment of Covid-19 based on history. occupation, commuting method, - Provide strict protection of and whether or not he/she information. participates in events. - As an infection control measure, use voice UI and other methods to reduce the number of contacts. Acceptability - Tell the user that the purpose is - The presentation of questions - Consideration should be given to requirements screening. that assume input support by a various ideas of gender to the - It should be like spoken, not caregiver should also be prepared. extent that it is not medically mechanical. - To be linked with other systems problematic. - Observing the patient's state of for the prevention of the spread of tension in real time (for example, Covid-19 infection (e.g. cocoa). by measuring the heart rate), - Clearly inform the user of the change the system's behavior. intended use of the input information. - Understand the characteristics of the patient and the background of the visit. As a result, we were able to capture the impact of "use" requirements. In the future, we will consider more diverse (quality in use) from the patient's perspective and the medical usage scenarios to make the requirements more practical. side's perspective, and incorporate them as requirements. In addition, since the system requirements for improving the Acknowledgments: This paper is based on the research quality were presented in relation to "use," they were not and discussions conducted by Mr. Junpei Sakura (2020 simply functional requirements, but system requirements from graduate of Shibaura Institute of Technology) under our the user's perspective. Although it is difficult to express these guidance. things as product or system specifications, they can be REFERENCES effective requirements for acceptance in society. [1] Fukuzumi, S. , et al. Extraction of new guideline items for ELSI-like perspectives in AI-based services, Proc. 32nd National Conference of V. CONCLUSION the Japanese Society for Artificial Intelligence (2018). (in Japanese) The demand for the use of AI in the medical field is rapidly [2] Ema, Yusa; Nagakura, Katsue; Fujita, Takusen. Trust in Medical AI increasing. Against this backdrop, we examined the from Physician Survey, Proc.34th National Conference of the Japanese requirements for AI to be accepted in society, with AI medical Society for Artificial Intelligence (2020). (in Japanese) interviews as the first target. [3] Ubie AI consultation Ubie Corporation, https://ubie.app (referred Oct. 30, 2021) Referring ISO/IEC 25010:2011, we were able to capture [4] Fukuzumi, S., Wada, N. and Hirasawa, N.: "Quality in Use -Issues and the impact of "use" (quality in use) from both the patient's and proposal-", proceedings of IWESQ2020 (2020) the medical side's perspectives and incorporate it into the Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).