A Tool for Covid-19 Symptom Evaluation Developed on Microsoft Xamarin Platform Nikos Petrellis Department of Electrical and Computer Engineering, University of Peloponnese, Patra, Greece; e-mail: npetrellis@uop.gr Abstract. The recent Covid-19 pandemic outbreak has forced the whole world in an emergency situation. The primary health care infrastructure has difficulty in handling all the cases of patients that ask remotely for help. The diagnosis of patients that need to have further medical examinations is performed in a fuzzy manner and the medical reactors needed are in shortage. To this end, a novel tool is proposed in this paper that could assess the patient symptoms and give appropriate instructions to the patient or for more safety, forward the data collected from the patient to qualified medical staff. More advanced features offered by the developed tool include geolocation and sound processing for cough and respiratory classification. It has been developed in Microsoft © Xamarin platform and thus, it could be offered as an Android or iOS mobile application. Keywords: Covid-19; Symptom assessment; Xamarin; mobile app. 1 Introduction Human diseases can be diagnosed either from their symptoms or more precisely from molecular tests. In the first case, the patient clinical view is examined by medical staff while blood, urinary, saliva, tissues, etc. can be analyzed in a molecular level [1]. The condition of a patient can be even remotely monitored using for example, temperature, respiratory, blood pressure, glucose, skin perspiration sensors. More advanced tests such as electrocardiogram, electromyogram, etc., can also be performed [2]. Image processing techniques [3] can either be employed for the classification of the disease symptoms or for molecular tests. Magnetic Resonance Imaging (MRI) [4] can be used for various diseases relevant to the brain such as glioma, dementia, Alzheimer, cancer, etc. Skin disorders can also be recognized from image processing. Melanoma is the most important skin disease [5] but several others including mycosis, warts, papillomas, eczema, acne, vitiligo, etc., can also be recognized by images displaying skin lesions [6-7]. These images are usually represented in Red-Green-Blue (RGB) color space although different spaces such as spherical color coordinates and L*a*b have also been employed to separate the Regions of Interest (ROI) with higher precision [8-9]. Smart phone implementations for skin disorder diagnosis have also 79 been reported [10]. Other diseases where image processing is applied include mammograms for the detection of breast cancer [11], images of blood vessels for cardiovascular diseases [12], retinal examination [13], etc. Sound processing has been reported in some disease diagnosis applications. For example, ultrasound Doppler systems are routinely used for the diagnosis of cardiovascular diseases [14]. The authors of [15] describe how audible tones with varying pitch are perceived by patients with Alzheimer Disease. A review of audio processing algorithms applicable for digital stethoscopes is presented in [16]. Covid-19 (often called simply Corona) virus is currently one of the major threats for thousands if not millions of lives throughout the world. No commonly accepted medicine is available yet for curing the often-fatal pneumonia caused by this virus although some drugs used for other infections like malaria or some cocktails of drugs seem to be effective. A large effort is given by many research teams to produce a vaccine that would equip vulnerable populations with antibodies capable of fighting the Covid-19 virus. However, many features of this lethal virus have not been revealed yet and its behavior as well as its consequences are still unknown to a large extent. The most effective way of protecting the population from the Covid-19 virus is the drastic isolation of each person in his home. Whole countries are locked down to support this isolation with tremendous effects in the economy worldwide. Staying home for a long time, without performing creative activities and without being able to socialize also causes severe psychological and physical health issues that in turn pose additional overhead to the health services offered by each country. The available Intensive Care Units (ICUs) and breathing equipment needed are not sufficient for the support of ill people infected by Covid-19, even in developed countries like Italy and Spain. An important issue is the treatment of individuals potentially infected by Covid-19 at the primary health care units. If an ill person visits with no precautions such a primary health care unit to check whether he or she is infected by Covid-19, there is a large chance to spread this virus to most of the visitors and the medical staff of this unit due to the extremely high contagious nature of Covid-19. For this reason, the citizens are instructed to treat all symptoms common with flu or cold as Covid-19 cases preferably staying at home for two weeks, even if the symptoms vanish earlier. People are generally discouraged from visiting any health care unit or meet a doctor in person. They are urged to call appropriate health care services and describe their symptoms and their clinical condition in general. It is obvious, that the medical staff that is responsible for the communication with these patients cannot remotely diagnose whether they are potentially infected by Covid-19. They merely assess the symptoms of the patient based on his description that may not be precise. Then, if they judge that the condition of the patient may get worse, a costly transport of the patient is arranged to a hospital with all the necessary precautions, in order to perform a precise molecular examination that will formally confirm whether the patient is infected by Covid-19 or not. In the first case he will probably be moved to an isolated hospital room with negative pressure where he will be monitored until full recovery. In case of pneumonia he will need breathing equipment or ICU support. From the procedure described above, it is obvious that a critical point is the decision on whether a patient should be transported to the primary health care unit or not. This decision depends heavily on a) the communication between the patient and the doctor, 80 b) the accuracy of the symptom description that the patient will provide, c) the experience of the responding doctor, d) potential missing updates of the protocol for Covid-19 treatment and many other subjective factors. Moreover, the network traffic congestion between the patient and the health care call center may cause critical delay in the communication. The mobile platform proposed in this paper (Coronario) aims to assist this procedure reducing the traffic of these call centers and providing a more objective opinion about whether a patient should follow for a hospital treatment. Coronario asks the end user to fill a questionnaire concerning the symptoms that he may have. It also takes into account the geographical location of the user and the possibility that he has come in touch with other persons already infected. A more advanced feature is based on sound processing. The cough or the breathing of the patient can be recorded and analyzed by Coronario application in order to decide more accurately on whether the user needs further medical tests or treatment. The incorporated medical protocol that would combine with different weights a) the data given by the questionnaire, b) the position and c) the sound analysis results, should support online update in order to follow always the latest directions in the Covid-19 treatment. It has to be stressed that the Coronario platform does not intend to substitute the valid opinion of the qualified medical staff nor will provide itself a diagnosis directly to the user. It can rather be used as a complementary tool supervised by medical experts. This paper is structured as follows. In section 2 the implementation of the Coronario platform is described in detail. More specifically, the implementation and the rationale behind the controls of each page in the application is explained. The incorporation of a sample medical protocol and the potential extensions and uses of the Coronario application are discussed in Section 3. 2 The Architecture of the Coronario Application The Coronario application has been implemented in Microsoft Xamarin platform that allows its deployment as a smart phone application for Android or iOS operating systems or as a desktop/laptop/tablet application. It has been developed in C# and Visual Studio 2019 while the following NuGet packages/libraries have been used: a) SkiaSharp and SkiaSharp.Views for bitmap handling (for the representation of a sound file in frequency domain), b) Xam.Plugin.SimpleAudioPlayer for playing sound files, c) Xamarin.Plugin.FilePicker for the selection of a file stored in the file system of the target device. The initial page of the application is shown in Fig. 1a. The user selects the user interface language that will be used throughout the rest of the application pages and clicks the button Next to proceed with the questionnaire, the geolocation and the sound processing as will described in the following subsections. 81 2.1 The Questionnaire with the Symptoms The checkboxes with the symptoms that the end user has to check appear in the 2nd page of the Coronario application shown in Fig. 1b. (a) (b) Fig. 1. Selection of the User Interface language (a) and the symptoms’ questionnaire (b). Table 1. Symptoms of Covid-19, Common Cold, Flu and Allergies. Symptom Covid-19 Common Cold Flu Allergies Fever Common Rare Common Sometimes Dry Cough Common Mild Common Sometimes Breath Short. Common No No Common Headaches Sometimes Rare Common Sometimes Aches/pains Sometimes Common Common No Sore throat Sometimes Common Common No Fatigue Sometimes Sometimes Common Sometimes Diarrhea Rare No Sometimes No Running Nose Rare Common Sometimes Common Sneezing No Common No Common Taste Difficulty Sometimes No No No Smell Sometimes Sometimes No No Difficulty 82 In the questionnaire of Fig. 1b the user checks the symptoms that he may have. After the Covid-19 outbreak tables that compare the frequency these symptoms appear, have been published to help people understand whether their symptoms match Covid-19, common cold, flu, or allergies. This comparison is displayed in Table 1. The estimated frequency that these symptoms appear may change dynamically. For example, some Covid-19 patients complained about the difficulty they had in the odor or the taste. Thus, although this symptom was not initially taken into consideration, recently it is also considered as a typical Covid-19 one. Moreover, gastrointestinal problems have been recently reported as Covid-19 symptoms although at the beginning of the pandemic outbreak these symptoms were assumed irrelevant with this virus. The weight of each symptom is obviously not the same. Moreover, the weights of the symptoms used for the classification of a clinical view in one of the four categories listed in Table 1 (Covid- 19, common cold, flu, allergies) may change dynamically as more accurate statistics are developed worldwide. The list of questions can also change dynamically according to any new findings from the study of the virus behavior. (a) (b) Fig. 2. Estimation of the current user position and the region where he resides (a) display on the map of the user position (b). 2.2 Geolocation Support The next page of the application is shown in Fig. 2a. The user can exploit the GPS of his mobile phone to retrieve the coordinates of his position and display his location 83 on Google Maps © as shown in Fig. 2b. The target of this geolocation service is to find evidence about how risky the region is where the user lives, based on the latest number of Covid-19 cases reported. This kind of information would be very useful if it concerned a restricted area e.g. the prefecture of the current user position. This would be feasible if local databases were available with detailed information about the Covid- 19 cases. Since this information is not yet available, a website with global data was used in the present version to get the total number of cases reported for each country. This website is: https://www.worldometers.info/coronavirus/ . Fig. 3. Selection of a sound file (a) and the display of its spectrum (b). 2.3 Sound Processing The most representative symptoms of Covid-19 are the dry cough and a short of breath or difficulty in breathing. The analysis of the cough sound can give valuable information about the condition of the lungs. A recorded cough sound is analyzed in the frequency domain in the Coronario application searching for representative frequencies that may suggest Covid-19 infection. Of course, an extensive study should be performed for a successful classification of these sounds and this will be part of our future work. In the present implementation the input sound is segmented in groups of 1024 samples and Fast Fourier Transform (FFT) is applied to get the spectrum of these segments. The FFT output frames are visualized with different colors. 84 The page where the sound file is selected is shown in Fig. 3a. With the button Open the user can select a recorded sound file. The sound can be played and analyzed with the corresponding buttons. Check boxes indicate if the sound is cough or breathing sound. The spectrum of a specific cough sound is shown in the next page (Fig. 3b). Specific signatures can be recognized even by the naked eye in the specific spectrum. The respiratory function can be analyzed either in frequency domain using the same analysis as the one described for cough sound or in time domain using data from a respiratory sensor as will be described in the next section. The respiratory is a pseudo- periodic function and the distance between the peaks can lead to useful conclusions about whether the user is in a short of breath or more generally if he has difficulty in breathing. 3 Discussion - Extensions The information collected from all the application pages described earlier can be used to reach a decision on whether further examinations are needed to confirm the infection from Covid-19. The diagnosis cannot be based on the symptoms only thus, the question that has to be answered is if the symptoms match adequately Covid-19 or another disease. A different score can be estimated for each one of the infections listed in Table 1 (Covid-19, common cold, flu, allergies). The score of each infection is estimated by adding the weights of the symptoms checked in the questionnaire. The symptoms can have different weights and the same symptom can participate with different weight in each disease. Similarly, the current spread of each infection (not only the Covid-19 one) as determined by the geographical location of the user can modify its score. The processing of cough or respiratory sounds can be used with pattern matching techniques or other classification methods (neural networks, supervised vector machines, decision trees, random forests, k-nearest neighbors, etc.). The result of these methods can be the classification of the recorded sound as one of the supported infections or the modification of their scores. Simple observations of several sound spectrums show that productive cough has more intense low frequency components than dry sound, but extensive experimentation has to be carried out in order to select an appropriate pattern recognition strategy. The output of the tool can be a descriptive analysis notifying the user about the significance of each symptom and its relevance with Covid-19. Based on this analysis a suggestion can be given about the necessity of further examinations. However, such a suggestion has to be approved by qualified medical staff that will take into consideration the results of the analysis performed by the Coronario tool. In any case it has to be stressed once again that this tool does not intend to substitute the formal diagnosis performed by the supervisor doctor but rather act as a complementary tool for fast remote monitoring of patients. Several extensions can be incorporated in the next version of the Coronario tool. First of all, new controls will be added to the symptoms and geolocation pages for the exploitation of information about risky contacts that the user may have had and for the tracking of the places that he has visited or the people that he has met. A connection to cloud services will be supported for: a) user authentication for secure communication 85 with medical staff-supervisor, b) dynamic update of the medical protocol used by the application and c) for the exploitation of additional data generated by sensors connected to the cloud. Cooperation with eHealth infrastructure will be supported. More specifically, temperature sensors, glucose and body pressure sensors, respiratory and body position sensors, electrocardiogram (ECG), electromyogram (EMG), etc., will be connected to the cloud for the remote monitoring of fever, breathing difficulties and sensitive population like diabetics, patients with heart diseases etc. The application pages will be improved in the following: a) the check boxes in the questionnaire will be replaced by sliders so that the user can describe in an analog way the intensity of each symptom, b) new fields will be added for the entry of information about the time the symptoms started and c) aesthetic and ergonomic improvement of the application pages. The application will also be extended for monitoring other similar diseases/infections such as common cold, flu, H1N1, SARS, etc. 4 Conclusions A cross platform mobile application called Coronario has been developed for the assessment of the early symptoms of Covid-19. The target of this tool is to reduce the overhead of the primary health care units, by suggesting the cases of patients that should undergo further tests (e.g., for Covid-19) or treatment. The tool input is a questionnaire, geolocation information and cough or respiratory sound analysis. Future work will focus on incorporating dynamically, appropriate medical protocols as well as on the extension of the functionality of the developed tool and the infections covered. A connection to external eHealth sensors and cloud will be also be supported. References 1. Georgakopoulou, K., Spathis, C., Petrellis, N. and Birbas, A. (2016) A Capacitive to digital Converter with Automatic Range Adaptation. IEEE Trans. On Instrumentation and Measurements, 65(2), p. 336-345, DOI: 10.1109/TIM.2015.2498538. 2. Patel, S., Park, H., Bonato, P., Chan, L. and Rodgers, M. (2012) A review of wearable sensors and systems with application in rehabilitation. Journal of Neuro Engineering and Rehabilitation, p. 9-21. https://doi.org/10.1186/1743-0003-9-21. 3. Dougherty, G. Digital Image Processing for Medical Applications; Cambridge University Press: Cambridge, UK, 2009. 4. Hasan, A., Meziane, F., Aspin, R. and Jalab, H. (2016) Segmentation of Brain Tumors in MRI Images Using Three-Dimensional Active Contour without Edge. MDPI Symmetry, 8, 132. doi:10.3390/sym8110132 5. Elgamal, M. (2013) Automatic Skin Cancer Images Classification. Int. J. Adv. Comput. Sci. Appl., 4, p. 1–8. 86 6. Kabari, l.G. and Bakpo, F.S. (2009) Diagnosing skin diseases using an artificial neural network. Proc. of the IEEE Int. Conf. of Adaptive Science & Technology, Accra, Ghana, DOI: 10.1109/ICASTECH.2009.5409725. 7. Petrellis, N. (2018) Using Color Signatures for the Classification of Skin Disorders. Proc. of the IEEE Int. Conf. on Modern Circuits and Systems Technology (MOCAST), Thessaloniki, Greece. 8. Ganster, H., Pinz, A., Röhrer, R., Wildling, E., Binder, M. and Kittler, H. (2001) Automated melanoma recognition. IEEE Trans. Med. Imaging, 20, p. 233–239. http://dx.doi.org/10.1109/42.918473. 9. Ubale, A.V. and Paikrao, P.L. (2019) Detection and Classification of Skin Diseases using Different Color Phase Models. International Research Journal of Engineering and Technology (IRJET), 06(7) 10. Wadhawan, T., Situ, N., Lancaster, K., Yuan, X. and Zouridakis, G. (2011) SkinScanc : A Portable Library for Melanoma Detection on Handheld Devices. Proc. of the IEEE Int. Symp. On Biomedical Imaging, Chicago IL, USA. 11. Martins, L.D.O., Junior, G.B., Silva, A.C. and Paiva A.C.D. (2009) Gattass, M. Detection of Masses in Digital Mammograms using K-means and Support Vector Machine. Electronic Letters on Computer Vision and Image Analysis, 8(2), p. 39- 50. 12. Danilov, A., Pryamonosov R. and Yurova, A. (2016) Image Segmentation for Cardiovascular Biomedical Applications at Different Scales Alexander. MDPI Computation, 4, 35. doi:10.3390/computation4030035. 13. Guo, Y., Budak, U., Sengür, A. and Smarandache, F. (2017) A Retinal Vessel Detection Approach Based on Shearlet Transform and Indeterminacy Filtering on Fundus Images. MDPI Symmetry, 9, 235. doi:10.3390/sym9100235 14. Ricci, S., Dallai, A., Boni, E. et al. (2008) Embedded System for Real-Time Digital Processing of Medical Ultrasound Doppler Signals. EURASIP J. Adv. Signal Process, 418235. https://doi.org/10.1155/2008/418235 15. Golden, H.L., Agustus, J.L., Nicholas, J. M., Schott, J.M., Crutch, S.J., Mancini, L. and Warren, J.D. (2016) Functional neuroanatomy of spatial sound processing in Alzheimer's disease. Neurobiol aginig, 39, p. 154-164. 16. De Lima Hedayioglu, F., Tavares Coimbra, M. and Da Silva Mattos, S. (2009). A survey of audio processing algorithms for digital stethoscopes. In Proc. of the International Conference on Health Informatics, p. 425-429. DOI: 10.5220/0001512104250429 87