A User-Centered Evaluation Framework for the SeaLife Semantic Web Browsers Gayo Diallo 0 1 2 Gemma Madle 0 1 2 Gawesh Jawaheer 0 1 2 Patty Kostkova 0 1 2 Dimitra 0 1 2 Alexopoulou 0 1 2 Robert Stevens 0 1 2 Simon Jupp 0 1 2 Khaled Khelif 0 1 2 INRIA Sophia Antipolis Méditerranée 0 1 2 France 0 1 2 Bioinformatics Group, Biotechnological Centre T U Dresden , Dresden , Germany City eHealth Research Centre, City University of London , London , UK School of Computer Science, University of Manchester , Manchester , UK

1 http://www.biotec.tu-dresden.de/sealife 2 http://cohse.cs.manchester.ac.uk/ 3 http://www-sop.inria.fr/edelweiss/software/corese 4 http://www.gopubmed.com/ 5 http://www.neli.org.uk 6 http://www.ncbi.nlm.nih.gov/pubmed/

 -

The SeaLife project1 is investigating semantic browsing and its application to the life science domain. SeaLife’s main objective is to develop the notion of context-based information integration by extending three existing semantic web browsers (SWB) to link the existing Web to the eScience infrastructure. The browsers under development are COHSE2, Corese3 and GoPubMed4, all of which already offer some semantic browsing functionality. This poster presents a user-centered evaluation framework for the SeaLife browsers. There is a need for a common framework and we will focus on evaluating the end-user perception of the SWB and how a SeaLife browser helps them gather desired information. The criteria used in the evaluation is related to the user experience and the impact the semantic browsers have on two case studies involving the NeLI portal5 and PubMed6. They will help to confirm or contradict the following hypotheses: i) having a SWB reduces the length of the pathway taken or time taken to find information or perform a task; ii) having a SWB reduces the number of links to be explored and the ability to explore information the user may not have been aware of; iii) users think that semantic links/ranking is a good help and this facilitate the use of the system; iv) users always prefer using the system with the semantic browser capability. Users range from infectious disease practitioners to molecular biologists, who will perform a set of tasks in both workshop and online-based evaluations. Three sources will provide the necessary data: i) Web server logs collected automatically as users navigate the portals; ii) a task performance questionnaire constituted by a set of tasks to be performed; iii) a user satisfaction and attitude questionnaire about the system with and without the semantic browsers.