Local and global dimensions of a clinical skills e-book development project Laura Smith1, Rebecca O’Rourke1, Memory Van Beek1, Suzanne Bickerdike1, Rania Alkhadragy2 1 Institute of Medical Education, University of Leeds, Leeds, United Kingdom 2 Medical Education Department, Suez Canal University, Ismailia, Egypt L.E.Smith@leeds.ac.uk Abstract This qualitative study develops an account of how, when, where and why Leeds the undergraduate medical students engage with an online curriculum mapped clinical skills intended to provide a quality assured resource for placement supervisors to support teaching and assessment. The research population was medical undergraduates entering Year 4 . Descriptive statistics and thematic analysis of a short survey of closed and open questions was compared with usage analytics and documentary analysis of the development brief. Preliminary analysis shows 96% of students who completed the questionnaire used the eBook. Students use all features but to varying extents. The eBook is not used as intended to prepare for and support placement learning and assessment but predominantly used for assessment and exam preparation. Keywords: qualitative, clinical skills, clinical placement, e-book, learner engagement. 1. Introduction There is growing interest in exploring the use of eLearning (1) and mobile technologies in undergraduate medical education to support the acquisition and development of clinical skills (2, 3). This paper reports an initiative in which an open access clinical skills e-book [CS e-book] was developed by the Clinical Skills Education Team [CSET] at Leeds Institute of Medical Education to support initial acquisition and subsequent application and refinement of clinical skills during placement and beyond. The paper begins with the rationale for developing the CS e-book, including the digital and pedagogic decisions informing its design and deployment, before presenting interim findings from a mixed methods research project that explores how, when, where and why students engage with this resource. Initially designed to explore the extent to which the developers’ intentions had been met, the study raises questions relevant to the wider deployment of digital solutions for learning. It also surfaced questions about how resources developed for use in specific local contexts circulate and are re-used globally. 2. Context The acquisition and practice of clinical skills is central to the undergraduate medical curriculum. Students encounter them in a variety of ways: observation in practice, simulation-based practice in dedicated practice centres, self-directed learning utilising written and/or multi-media resources and the application of this knowledge in supervised practice during placements in. The clinical skills which medical undergraduates need are organised by body systems and characterised by early and varied patient contact through clinical placements of increasing length and responsibility. Consequently, the skills which medical undergraduates develop also increase in complexity as they progress from their first to fifth year. This continued involvement in learning and consolidating clinical skills requires independent learning from the student, prefiguring the lifelong learning they will engage with as professionals. The CSET supports several initiatives to promote such learning, often centred on preparation for the Objective Structured Clinical Examination [OSCE]. These include opportunities for unsupervised practice and peer learning and assessment. Learning clinical skills is not only knowledge acquisition but also involves translating this knowledge into practice under the direction of clinical supervisors during placements. A number of challenges arose from each of these processes that led, in 2013, to the creation of the CS e-Book. The first challenge was the frequency with which students’ independent learning involved consulting unregulated web-based resources, sometimes contrary to national guidelines, inaccurate or out of date. There was potential for disseminating this incorrect information during informal and peer learning. . The second challenge was ensuring clinical supervisors, responsible for teaching and assessing students in practice, had access to materials clearly setting out the required procedures and standards students had to achieve. Clinical supervisors cannot access the university’s virtual learning environment so the e-book was developed as a web- based open access resource. 3. Digital Solution The CS e-book is a self-directed learning tool that can be accessed on a mobile phone, tablet or PC. The resource was developed by staff from the CSET working with clinical placement providers and educational technologists. Pedagogy rather than technology led the initiative. The technology input is deceptively simple: embedded videos, interactive diagrams and text within a swipe left / swipe right format and hyperlinks from chapter headings and sub-heading and to launch videos and images. The emphasis is on content and demonstration. Interactivity is carefully designed to keep page lengths manageable on mobiles and tablets. Collaboration with placement providers was an important aspect of ensuring the CS e-book was fit for purpose and it was hoped involvement in developing the resource would encourage its adoption in the clinical supervision relationship. 4. Practical deployment The resource is mapped to the undergraduate medical curriculum, divided into chapters for Year 1, 2, and 3 with each broken down into the 7 or 9 key clinical skills respectively, plus chapters on clinical examination, history taking and the crash trolley (adult). Each chapter follows a similar structure: video in which each skill is demonstrated; full written description of the skill; checklist outlining the kit along with photographs of the equipment needed. Content is regularly updated and new chapters added. 5. Methodology The CS e-Book has been available for five years and collection of usage data is built in to the site. However, this data raises as many questions as it answers and so it was agreed to run a mixed-method evaluation to explore how undergraduates actually engaged with the resource and the extent to which the designers’ aims had been realised.. The research population was medical undergraduates entering Year 4 of Leeds university’s MBChB who completed a short survey of closed and open questions. Descriptive statistics and thematic analysis of their responses were compared with usage analytics and a documentary analysis of the development brief. A second phase of the study will interview students about the ways in which they use the e-book to embed learning. One aim for the evaluation project is to create baseline data for future analysis of the CS e-Book. The project received ethical approval from the University of Leeds School of Medicine Research Ethics Committee in 2017. 6. Findings There was a 57% return rate ; of these, 96% had used the CS e-book. 6.1 Frequency of Use Google analytic data from the web pages show the usage patterns from all users worldwide. They show spikes in usage during the university’s revision and examination periods and on skills assessment days. However, this data is not trackable to individuals. The survey responses, and follow-up interviews, make it possible to drill down into the detail of these broad patterns. They show the resource is used by the majority of registered medical undergraduates but with variation in frequency. About a third of the respondents gave a figure of 10 or lower; about half a figure of up to 100. The remainder reported using the resource “Many times a week”, “Very often before OSCEs” and, in one case, “Several times a day during revision”. It was expected revision and the exam period would figure highly and they did . Other periods of more frequent use, e.g. to prepare for the start of the year, the teaching of new skills or starting a placement were significantly lower (in single figures). This highlights the need for further work to achieve the full range of use envisaged, particularly refreshing knowledge prior to performing a clinical skill , practicing clinical skills and during placements . 6.2 Where participants access the CS e-Book The designers had intended the CS e-Book to be accessible, and used, in a variety of settings especially clinical placements. Although students did access the CS e-book in clinical practice this option received the lowest score and they were most likely to use it at home or the university . These options were not mutually exclusive – participants were invited to record all that applied. A supplementary question asked where participants most often used the CS e-Book. Home was by far the most popular location , with the Clinical Practice Centre and Placements reported as 2nd and 3rd choices. For students who did consult the CS e-Book during placements in clinical practice a supplementary question ascertained who they were with when they did so. This showed an overwhelming preference for using the CS e-Book privately and with colleagues . A minority used it with patients suggesting issues around perception of confidence and expertise are in play. However, the equally low number using it with their supervisors highlights the need for further work to realise this intended purpose. 6.3 Devices used to access the CS e-Book Participants were asked about all the devices they used to access the CS e-Book and to identify the one they used most frequently. Responses confirmed findings that medical undergraduates predominantly saw the CS e-Book as being for private use, linked to revision, rather than a resource to support placements and supervision in clinical practice. When asked about all the devices they used, the most popular were: personal laptop or pc , personal mobile phone and personal tablet . When asked which device they used most frequently the personal laptop or pc was significantly more popular than other devices. Personal mobile phone was the second most frequently used device with personal tablets or iPads significantly higher than public devices: the practice centre iPad and hospital pc . Students using mobile phones were most likely to access the e-book in clinical practice. 6.4 How has the CS e-Book been used? Questions established not only what content had been accessed but how participants had accessed that content. Offering a combination of media was designed to support its use at different stages of development: acquiring skills and preparing to acquire them, preparing for clinical placement, applying knowledge in practice and revising for assessment. The different mediums support these different purposes, e.g. checklists for refreshing knowledge prior to application, videos provide an overview and introduction to the skill. These choices also acknowledged preferences in learning, offering visual alternatives to the written word. The data shows students use all features built into the eBook Data analytics show technical/procedural skills are more commonly viewed than examination skills (which could reflect examination skills having been added recently). Individuals reported an overwhelming preference for the combination of all three modes suggesting a more dialogic approach is needed to understand how exactly medical undergraduates use the CS e-Book to develop, reinforce and apply their learning about clinical skills. This will be explored in interviews during phase 2. 6.5 Why participants use the Clinical Skills e-Book Participants could choose from a list of statements to report all their reasons for using the CS e-Book– see figure 1. Revision and exam preparation top the list and confirm the power of assessment to both drive and define the learning experience of medical undergraduates. However, the broader purposes envisaged - consolidating skills , preparation for placement and clarification - are recognised and valued, albeit marginally Why do students use the clinical skills ebook? Curious after hearing friends discuss it 10 Look ahead to see what I will be learning in the… 18 Clarify something I encountered on placement 46 Preparation for placement 58 Videos help as I am a visual learner 76 Practice Skills I Struggle with 78 Prepare for upcoming Clinical Skills Session 78 Prepare for Formative OSCE 80 Consolidate Taught Skills 93 Prepare for Summative OSCE 104 0 20 40 60 80 100 120 Figure 1 Reasons medical undergraduates use the CS e-book 7. Open Challenges The study found that the resource was not used as intended. Leeds students use the eBook in a legitimate way- predictably driven by assessment and exam preparation – but one which counters the primary aim of the designers to develop a resource supporting placement and placement supervision. The eBook usage demonstrates a need is met but only by speaking to students will we know if this is active and developmental, providing a foundation for lifelong self-directed learning, or the high achieving student seeking reassurance. A further challenge surfaced by the evaluation is the need for further work to promote the CS e-Book to clinical supervisors and provide guidance as to how best they can draw on its resources during placements. Whilst we continue to investigate local users is it also timely to reflect whether we should also be exploring, and perhaps guiding, how this resource is used on a global scale. Data analytics from 2015 to date show the Clinical Skills eBook has been viewed across 88 countries. There are limited channels for feedback which would enable the designers to understand how it is being used in these very different contexts. This challenge has technical and pedagogical dimensions. Technically, the question is whether it is possible, or desirable, to build some form of evaluation into the resource – perhaps a social media based network to gather and exchange information about how the resource is being used. Pedagogically, the challenge is greater. Martimianakis and Hafferty (5) identify three distinct discourses of medical education in a globalized world: the universal global physician, the culturally versed global physician and the global physician advocate. These are distinguished by the extent to which they believe it possible or desirable to achieve universal standards of medical competency and training. The discourse of the universal global physician assumes someone can be trained anywhere in the world using a set of universally applicable standards of competency. It contrasts with the discourse of the culturally versed global physician, which assumes the acquisition of culturally specific knowledge and training through exposure and experience, and discourse of the global physician advocate, whose bio-medical knowledge is complemented by knowledge of the economic, cultural and political determinants of health and who advocates for social change in the medical context. The CS e-book, within the specific medical undergraduate population it was designed for and the global contexts within which it circulates, is both produced by these discourses and contributes to their reproduction. References 1. Bloomfield JG. Jones A. Using e-learning to support clinical skills acquisition: Exploring the experiences and perceptions of graduate first-year pre- registration nursing students – A mixed method study. Nurse Education Today 33 (2013): 1605 – 1611 2. Clay CA. Exploring the use of mobile technologies for the acquisition of clinical skills. Nurse Education Today 31 (2011): 582 – 586 3. Forbes H, Oprescu FI, Downer T, Phillips NM, McTier L, Lord B, Barr N, Alla K, Bright P, Dayton J, Simbag V, Visser I. Use of videos to support teaching and learning of clinical skills in nursing education: A review. Nurse Education Today 42 (2016): 53 – 56 4. McGrath D, Crowley L, Rao S, Toomey M, Hannigan A, Murphy L, Dunne CP. Outcomes of Irish graduate entry medical student engagement with self- directed learning of clinical skills BioMed Central - Medical Education (2015) 15:21 DOI 10.1186/s12909-015-0301-x 5. Martimianakis, M and Hafferty, F The world as the new local clinic: A critical analysis of three discourses. Social Science & Medicine 87 (2013) 31-38