As the development and implementation of new learning analytics and other digital tools in higher education appear to be on a continual rise, we examined the attitudes toward new tools, and particularly what afects the adoption and experienced usefulness of new tools. An increasing amount of knowledge has been accumulated on aspects of learning analytics solutions that afect impact and user experiences, but the efects of the processes surrounding the deployment of new solutions appear less clear. We aimed to discover what factors higher education staf find useful when required to take a new tool into use, and what factors can hinder the learning and use of new learning analytics tools. Results indicated that deployments often fail to account for user-characteristics, and that deployment processes should be more tailored, accounting for users' skills, roles, and tasks. HE staf indicated that new LA tools often lack adequate support, communication, instructions, and considerations of user needs, and are not able to communicate clear use-cases and expected value. These identified shortcomings provide good lessons for future learning analytics deployment projects, urging developers of analytics tools to invest time and efort into smooth deployment and support, so that the tools can have an impact in higher education institutions. Improvement at a relatively easy-to-deliver -level, such as good tailoring of instructions and communication of practical value, could improve the acceptance and use of learning analytics, and thus also impact on the intended learning or teaching targets of the learning analytics solution.
Online environments, learning analytics (LA) and other digital tools have become an integral part of the various learning, teaching, research, and administrative processes in higher education (HE). Institutions are increasingly utilising various LA solutions and other digital tools to better understand and support students and teachers (Larrabee Sonderlund, Hughes, & Smith, 2019; learning analytics solutions and digital tools, there appears to be little evidence in literature of NORDIC LASI 2021 © 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). wide use of these tools (Viberg et al., 2018). Additionally, there is a need to better understand the HE institutions’ staf perspectives on LA deployment. For example, Howell et al. (2017) identified, that teachers and advisors may need clarification on the purposes of using LA tools as a support for guidance and teaching practices. These findings indicate a need to better understand the mechanisms of adoption process of the digital tool, including the role of diferent background variables explaining the adoption experience of the users in diferent roles in educational institution.
We examined the factors that both teachers and non-academic staf in Finnish HE institutions associate with the deployment of new tools. In Finland, some trials and implementations of LA tools in HE have been done, but wider deployment of diferent tools and the culture of using LA in diferent educational practices is still emerging. The key aim was to identify good and bad practices associated with introduction of new digital and LA tools, as experienced by the expected user-groups, as these users have a key role in guiding and developing the future processes where learning analytics tools will be implemented. In order to examine deployment conditions in a broad sense, rather than specifying the exact types of learning analytics or tools of interest, we investigated the conditions surrounding deployment and learning of new LA and digital tools in general, rather than in relation with a specific new LA tool, to identify common factors. We attempt to primarily cover the experienced challenges and good practices surrounding implementations of LA and digital tools, for the purpose of supporting future LA deployment processes, so that the potential of new tools and solutions might be actualized more efectively. This approach is intended to supplement the previous research that has identified aspects of the LA tools that afect user experiences and attitudes in HE (e.g., Viberg et al., 2018), with a view on how those aspects could be better supported with good deployment practices. This aim was achieved through a questionnaire aimed at HE staf, investigating their experiences of LA and other digital tool deployment in relation to their user characteristics, such as basic ICT self-eficacy and general use of ICT (Hatlevik, 2017), and background of working in higher education. ICT self-eficacy is an important concept in understanding HE stafs’ ICT use. It refers to HE employees’ beliefs about their capabilities to utilise ICT for teaching and other work-related practices (Krumsvik, 2011). The assumption that underlies behind the concept of self-eficacy is that when people are confident about being able to complete a task, they can be more willing to direct their efort for the task (Bandura, 2006). Previous studies have shown that teachers with low ICT self-eficacy were the less frequent users of technology in their work (Hammond, Reynolds & Ingram, 2011). Additionally, it has been shown that the intention to use technology can be influenced by self-eficacy and technological complexity (Teo, 2014). In the context of LA implementation, ICT self-eficacy may play an important role for users’ willingness to adopt new digital tools.
We were interested in whether ICT self-eficacy and use of ICT, together with roles in HE would afect how the conditions surrounding the deployment of new tools are experienced, and how the importance and efectiveness of diferent forms of training materials for new tools are perceived. We also examined participants’ views with a qualitative approach regarding what good practices for LA and digital tool deployment they identify, and what challenges they face when required to adopt new tools and solutions.
The key motivation of this study was to identify such aspects of supporting academic staf that help them adopt new LA tools into their daily educational practices. The practical goal is to gain information on how we might approach new deployment processes to make them more approachable and user-friendly for the academic staf. This knowledge will improve the opportunities of LA tools to reach their potential impact in HE, by providing lower thresholds for adoption of LA tools into wider use. To investigate these, four key questions were formulated: 1. How do HE work-background, basic ICT self-eficacy and use of ICT afect how adoption of new learning analytics solutions and digital tools is experienced?
2. How do HE work-background, basic ICT self-eficacy and use of ICT afect how important training materials are for new tool adoption?
3. What are the key aspects that training materials and support services should have in order to be helpful when adopting and using new digital tools?
4. What are the key challenges associated with adopting new learning analytics solutions and digital tools?
Some reviews have indicated relatively little impact of LA solutions on learning or teaching, despite their identified potential to improve learning, support or teaching (Larrabee Sonderlund et al., 2019; Viberg et al., 2018). There are also clear indications that user characteristics afect the impact of LA solutions (e.g., Hatlevik, 2017), as do aspects of the LA tools themselves. Regarding LA design, it appears that when impact of LA tools and solutions is evaluated, the important factors are focused on the visual aspects of the tool and general usability, (e.g., Park & Jo, 2019), as well as the functions and design of the tool (e.g., Charleer, Klerkx, Duval, De Laet, & Verbert, 2016; Herodotou, Rienties, Boroowa, Zdrahal, & Hlosta, 2019), but more understanding of the practices related to the implementation and use of the tools should be achieved. Also investigating how new LA and other digital tools should be deployed in order for the key stakeholders to find the tools useful, and to help the users apply the solutions in ways relevant to them, could contribute to the impact of the tools. Coupled with an understanding of how digital skills and related characteristics afect the experience with new digital tools, we may be able to facilitate changes in diferent educational processes involving new ways of working with LA better support learning, teaching and support processes.
Implementation of LA tools requires the development of practices and processes related to the planned use of the designed tools. With a clear understanding of the experienced conditions surrounding typical deployment processes, developing good deployment processes could be an opportune way to impact the experiences of and the attitudes towards adopting new LA tools. Although some previous findings have indicated that involving intended user groups in LA development in higher education has a positive impact on the users’ experiences (Lallimo & Sjöblom, 2019), it is unclear what deployment factors afect LA tool adoption and impact. The tools and solutions that implement diferent kinds of LA methods generally involve various predictive or descriptive models and/or visualisations, categorisations and supervised and unsupervised learning methods (Klein, Lester, Rangwala, & Johri, 2019; Larrabee Sonderlund et al., 2019). This may produce challenges for those in higher education who are less used to using ICT tools. Previous research has identified a gap between the capabilities of LA solutions and user needs (Klein et al., 2019), amplifying the potential problems in introducing new tools, and reducing the possibilities of the new solutions improving teaching, learning and support services. Additionally, it has been demonstrated that for teachers, digital competence, ICT self-eficacy and general use of ICT are interrelated (Hatlevik, 2017), which in turn may suggest a relation with readiness to adopt new ICT tools. Given the quickly developing nature of LA, it is likely that continuous efort is required from HE institutions in training the afected stakeholders in order for new tools to be used efectively. It has already been identified in some specific circumstances, such as teacher education, that continual development of digital competences and ICT self-eficacy is important, given the increasing digitalisation (Gudmundsdottir & Hatlevik, 2018).
Ninety-nine people from HE institutions in Finland completed the questionnaire. Majority of the participants were from universities (83), with the rest from applied universities, and 68 had positions that involved teaching, and majority (78) had at least some pedagogical training. Thirty-two (32.32%) of the participants were from science, technology and mathematics (STEM) -fields. Participating in the study was voluntary and data were collected anonymously.
Data were collected with an online questionnaire that was circulated at higher education events and networks, as well as internal information channels. The participants were informed about the purpose of the questionnaire. The questionnaire consisted of three parts. First participants were asked to provide information about their roles in HE (teaching/administrative role, STEM/non-STEM field, university/applied university), and to answer Likert-scales measuring their basic ICT self-eficacy and use of ICT. The questions were based on previously used questionnaires (Hatlevik, 2017) and translated and modified for current purposes. Questions are presented in the Appendix. Next, participants were asked to think about their experiences of adopting to LA and other digital tools. Regarding new LA and digital tool deployment, the participants were asked about their attitudes and experiences taking new LA and digital tools into use and about their experiences of training material usability in for digital tool adoption with five Likert-scale questions. The participants were also asked about the type of support they ifnd useful when taking a new tool into use with a multiple-choice question with an opportunity to provide other open-ended answers. The third part contained two open-ended questions for thematic analysis (Braun & Clarke, 2012), concerning what training materials should contain in order to be useful, and what factors typically make the learning and using a new digital tool challenging.
As the new LA and digital tool experience was expected to be simultaneously afected by
changes to the background variables of teaching, institution, and field, together with basic ICT
self-eficacy and use of ICT, it was predicted with a multiple regression. A significant regression
equation was found, F(
Experienced importance of training materials was expected to be afected by the same
independent variables. A similar multiple regression was used to investigate the experienced
importance of training material usability and availability, indicating a significant regression
equation, F(
Regarding the usefulness of diferent forms of support, out of the provided options quick guides (with 79.8% of participants finding them useful), video instructions (72.7%), and available support staf (72.7%) were found to be useful by most participants. FAQs (57.8%), instruction session (53.3%), written instructions (43.4%) and discussion forums (42.4%) also received support, while only 16.2% considered learning games to be useful.
Two qualitative questions were analysed in depth, employing a thematic analysis. Two coders analysed the answers, and identified similar themes for both important training material aspects and key challenges. Regarding the aspects that make training materials for new LA tools useful, participants systematically identified several key themes, relating to training materials’ contents and qualities: 1) pictures to make instruction more concrete, 2) both quick guide and more elaborate instructions, 3) examples of how the tool works for specific tasks, 4) role-specific stepby-step guides, 5) good structure, and 6) communication channels between users, developers and support.
First, participants commonly mentioned a need for pictures and videos to accompany explanations, and to make the instructions more concrete, and to ease the translation from instruction to the tool itself. This was often associated with a description of need to better relate instructions and actual tool use.
“Requires clarity, good relations between images, text, functions, structure, functionality and systematicity.”
“Videos based on good scripts. Instructions must not be jumpy or illogical. Clear images and the possibility of zooming in closer.”
Another key theme was a need for both a quick guide for the basic functionalities of the tool, and also for longer detailed instructions that allow the user to go into finer details according to their interests. This was often related to the theme of a need for better examples of how the tool can be applied to diferent academic or pedagogical tasks, and what kind of results to expect in relation to functionalities. These issues were discussed commonly in the context of understanding how the tools could or should be used in relation to specific tasks that the user must undertake, depending on their role, as well as a need for relevant progression examples of how the user might expand their knowledge once they understood the basics.
“A basic users’ guide for the tool, but in particular good use cases, how the tools have been utilised before.”
“Written instructions should contain various good keywords, to make it easier to find the information you need. A good search engine connected to all materials. Clear pictures of the user interface. Example cases to give the user an idea about all of the things that can be done with the diferent properties of the tool. A possibility to comment on all instructions, so that they can be updated and supplemented and you can get tips from other users’ problems. Good self-learn online materials, that you can complete while taking the tool into use. For example step-by-step instructions for diferent functionalities, with dummy-versions for practicing, while simultaneously taking the real tool to use.”
One of key themes was a need for a clear, logical progression, with the answers often including descriptions of unfulfilled needs for role-specific step-by-step guides as well as for clear language free of jargon, and a good thematic structure. A communicative aspect was also commonly mentioned, indicating a need for clear communication channels between users, developers and support, both in terms of the tool and any instructional material, so that both the tools and the relevant materials would suit the processes they were designed to support.
“Usually instructions are not on the right level, and do not show how the tool should be used in my role. Instructions often have a lot of inessential details, and they can make the use of the tool sound too complicated.”
“It would be useful to know some background and bases about why this [tool] is important. Attention is usually directed too quickly to learning a new tool and performing some actions, and it is forgotten why these things are done.”
“A support person should always be named. It is important to have someone you can contact and ask for advice, as videos or written instructions cannot be trusted to contain all the information regarding specific roles you have or tasks you do.”
Second, five key challenges were identified from the answers. Here, the answers were relatively similar to the answers above regarding what good materials and support should contain, now detailing them from a perspective that they often do not receive the kind of support they would find helpful. Many participants also identified the LA tools themselves as a key challenge (1), along with either the quality or non-existence of support or instructional materials (2). These were often associated with experiences that a lack of communication made the learning and use of the new digital tools challenging (3). Many participants also explicitly linked these answers to their own answers to the earlier question.
“Inadequate instructions, dificult user interface.” “Communication is missing or too delayed. Organised information and communicated benefits are essential. Not enough time for familiarisation. Poor instructions.”
Participants also commonly found that the actual usability of the tools to support the teaching,
academic advising and administrative and other tasks was challenging, and that the tools could
not be tailored to suit their needs (4). In particular, many participants indicated that the new
tools generally meant a higher workload with new tasks added to the requirements of their role,
rather than the tools supporting them in their existing tasks (
“There should be a real need for the tool, based on actual professional requirements and tasks, rather than administrative or organisational pressure to use tools.”
“Unsuitability of tools and systems to real needs and tasks. Poor instructions. Tools that seem incomplete/untested.”
“Not enough time. No available evaluations of tools, no readily available, genuine estimates about how useful the tool would be in my own work. No good instructions or support.”
The first two hypotheses regarding variables afecting experiences of using new LA tools and the experienced importance of training materials were only partially supported. The key quantitative results indicated that participants’ general use of ICT, influenced how participants experienced the deployment of new LA solutions and digital tools, with higher indicated ICT use related to more positive experiences. The findings also indicated that lower basic ICT self-eficacy was related to heightened need for training materials. Together these findings indicate that user characteristics associated with ICT use and ICT self-eficacy have an impact on adopting new LA solutions and digital tools, and that accounting for them might improve the acceptance and use of new solutions. More specifically, the results suggest that general readiness to use ICT tools provides the users with more readiness to adopt new LA tools, while less fluent ICT user groups would benefit from being in the focus when deployment support and instructions are prepared, so that they could receive the additional support they are likely to need. This also indicates that when requiring users to adopt new tools, assessing their familiarity with ICT use and adoption and their ICT self-eficacy would aid in preparing the appropriate deployment processes and support for users. Along the same lines, when deployment of new LA solutions is targeted for user groups where lower ICT self-eficacy might be expected, focus and resources should be directed at providing good, targeted instructional material and support, as these groups in particular are likely to require support in order to benefit from new LA tools. Additionally, it is also important notion for educational institutions to consider, whether high quality training materials could be used to improve academic stafs’ ICT self-eficacy beliefs and therefore support diferent user groups to fluently adopt new technologies in institutions.
However, the hypothesised predictive efect of HE background variables was not supported, where the chosen HE work background variables did not account for a significant amount of either experiences with new LA tools, or how instructional materials are viewed. This may indicate that, in general, diferent groups within the higher education context, regardless of whether the users are teachers or not, in STEM or in other fields, their experiences might be rather similar. At the same time, this speaks to the qualitative analysis that indicated that both LA tools and the associated instruction materials and support should be specifiable according to roles and tasks. These findings together appear to suggest that this is usually not the case, making the experiences of diferent groups similar in the aspects that they require more specificity. The general message from the open answers was a common shared experience of new LA tools being deployed for them to use without providing enough support, and without good demonstrations of how these new tools should be adopted to suit their roles and needs.
For the third research question about key aspects of good training materials, the thematic analyses identified several factors: need for pictures and videos, quick guides and detailed, task-specific instructions, role-specific examples and step-by-step guides, good structure, and communication between users and developers. These would appear obvious, yet findings regarding the fourth research question about challenges to adopting new tools indicate that many of these aspects are often lacking or missing when new LA tools are deployed. Regarding the types of training materials, particularly videos, quick guide and support persons were found to be potentially most useful, linking to the qualitative findings, which indicated that often training materials are missing, unsuitable or unclear. The participants also expressed that the contents of training materials should be easily related to the use of the actual tool, with examples to demonstrate how the new LA tool supports the tasks that the user needs to do, based on the kind of teaching, advising, administration or learning support processes that they undertake. Participants often mentioned that instructional material needs to be clearer and better structured, and that it should be specifiable according to roles and tasks. Communication aspects were also highlighted, both in terms of support and in informing the new users of the intended purpose and value of using the new LA tools. Interestingly, very similar topics were mentioned both when participants were asked about what training materials should contain, and when they were asked about the challenges of adopting new LA tools. This appears to indicate that often the deployment of new tools and solutions are lacking the critical support aspects that users identify. These aspects also ofer key issues to solve when aiming for a successful deployment of a new LA tool in higher education. If the adoption and use of new solutions and tools can be boosted and supported by addressing the users’ key concerns by providing adequate and well-tailored support, we might expect an increase in both the usage and the impact of new solutions. Some negative experiences with new LA tools may also be circumvented more efectively, if teachers and other expected user groups for new tools are invited to participate early on in the planning and development processes for new tools, rather than first experiencing them when the new tool or solution is implemented. It has been shown previously that student-centered LA development processes improve the students’ experiences of new learning analytics tools (Lallimo & Sjöblom, 2019), and this could be expected to generalise to other higher education stakeholders as well.
Perhaps the most dificult theme that was identified was time requirements associated with adopting and using new LA solutions and digital tools. Generally, new tools require both learning time, and the experiences generally indicate that the continued use of new tools also adds to the workload and require additional time, rather than helping in the essential tasks associated with either teaching, learning support or administrative processes. Time requirement concerns have also been identified before (Hatlevik, 2017), and are likely one of the key issues that must be addressed. Concerns about the additional time cost and workload made by the requirements to use new LA tools likely tie in with the other concerns that were raised regarding usability, the suitability of the tools for the actual teaching, learning and learning support tasks, and lack of appropriate instructions and support, and may be alleviated if they are addressed. Workload concerns do, however, also suggest that a critical evaluation of new LA tools and their values and requirements in higher education is in order, so we may avoid a negative impact of additional workload associated with LA solutions and digital tools, in addition to addressing the specific concerns on how to lower the workload caused by each individual tool. We do propose, however, that time requirement problems may be partially solved by providing users with clear and task-appropriate instructional material, that aids them both in adopting the tool quicker, and in making the use of the tool more efective and useful for their primary teaching, learning support and administrative tasks.
Stakeholder-centric approach has been previously found to be an appropriate, efective and user-friendly approach for providing a deeper insight into how value can be created for users when developing learning analytics concepts and solutions (Klein et al., 2019; Lallimo & Sjöblom, 2019; Silvola, Näykki, Kaveri, & Muukkonen, 2021). The current findings indicate that a similar approach could be useful also when planning and implementing the deployment of LA tools and solutions, and could further the actualisation of the solutions’ potential for supporting teaching and learning and other processes in higher education. The current findings on the importance of communication are also in line with the previously identified perceptions that LA may be beneficial, if collaboration exists amongst the HE stakeholders: teachers, students, and their institution (Howell, Roberts, Seaman, & Gibson, 2018). Simultaneously, the findings suggest that HE institutions may benefit from providing ICT skills training in general, as it may positively influence the staf’s ability to adopt new LA and other digital tools into use, given that use of ICT was shown to positively afect experiences with learning and using new LA tools. Engaging and exciting staf to adopt new methods and tools is essential for the tool to have an impact in the learning, teaching and learning support environments and experiences in higher education (Zilvinskis, Willis, & Borden, 2017). We propose that if we can be more sensitive to the needs of the intended user groups, both in terms of the LA tools and solutions as well as the surrounding implementation conditions, e.g., training, support, communication, and integration to the primary functions of teaching, learning and learning support, we can improve the attitudes towards and the adoption of new LA and digital solutions. This will also increase the opportunity for the tool to have recognisable impact in HE institutions.
This work was funded by the Finnish Ministry of Education and Culture, [grant number OKM/272/523/2017].
Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association.
Bandura, A. (2006). Guide for constructing self-eficacy scales. In F. Pajares, & T. Urdan (Eds.), Adolescence and education: Vol. 5. Self-eficacy and adolescence (pp. 307–337). Greenwich, CT: Information Age.
Charleer, S., Klerkx, J., Duval, E., De Laet, T., & Verbert, K. (2016). Creating efective learning analytics dashboards: Lessons learnt. In European conference on technology enhanced learning (42-56). Springer, Cham.
Gudmundsdottir, G. B., & Hatlevik, O. E. (2018). Newly qualified teachers’ professional digital competence: implications for teacher education. European Journal of Teacher Education, 41(2), 214-231.
Hammond, M., Reynolds, L., & Ingram, J. (2011). How and why do student teachers use ICT? Journal of Computer Assisted Learning, 27, 191–203.
Hatlevik, O. E. (2017). Examining the relationship between teachers’ self-eficacy, their digital
competence, strategies to evaluate information, and use of ICT at school. Scandinavian Journal
of Educational Research, 61(
Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., & Hlosta, M. (2019). A large-scale
implementation of predictive learning analytics in higher education: the teachers’ role and
perspective. Educational Technology Research and Development, 67(
Howell, J. A., Roberts, L. D., Seaman, K., & Gibson, D. C. (2018). Are we on our way to becoming a “helicopter university”? Academics’ views on learning analytics. Technology, Knowledge and Learning, 23(1), 1-20.
Klein, C., Lester, J., Rangwala, H., & Johri, A. (2019). Technological barriers and incentives to learning analytics adoption in higher education: in-sights from users. Journal of Computing in Higher Education, 31(3), 604-625.
Krumsvik, R. J. (2011). Digital competence in Norwegian teacher education and schools. Högre Utbilding, 1, 39–51.
Lallimo, J. & Sjöblom, A. (2020). Student-centered development of learning analytics at a higher education institution. Companion Proceedings of the 10th International Conference on Learning Analytics & Knowledge, 176-177.
Larrabee Sønderlund, A., Hughes, E., & Smith, J. (2019). The eficacy of learning analytics
interventions in higher education: A systematic review. British Journal of Educational Technology,
50(
Park, Y., & Jo, I. H. (2019). Factors that afect the success of learning analytics dashboards.
Educational Technology Research and Development, 67(
Silvola, A., Näykki, P., Kaveri, A., & Muukkonen, H. (2021). Expectations for supporting student engagement with learning analytics: An academic path perspective. Computers & Education, 168, 104192.
Teo, T. (2014). Unpacking teachers’ acceptance of technology: Tests of measurement invariance and latent mean diferences. Computers & Education, 75, 127–135.
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98-110.
Zilvinskis, J., Willis III, J., & Borden, V. M. (2017). An overview of learning analytics. New Directions for Higher Education, 2017(179), 9-17.
A. Questionnaire items for basic ICT self-eficacy (1), use of ICT (2), experience relating to deployment of new LA/digital tools (3), forms of support (4) and experienced needs for materials and challenges (5 & 6) 1. Evaluate the following statements about digital skills (scale: 1 (completely disagree) – 7 (complete agree)) • I can use spreadsheets to draw figures • I can download and install programs • I can edit digital photos and graphics • I can form a database 2. Evaluate the following statements about use of technology (scale: 1 (completely disagree) – 7 (complete agree)) • I find it fluent to use information technology for administrative tasks • I find it fluent to use information technology as a part of teaching during lectures and exercises • I find it fluent to use complex statistical figures • I find it fluent to start using new information technology tools 3. Evaluate the following statements about new LA/digital tools: (scale: 1 (completely disagree) – 7 (complete agree))
General experience with new tools: • I find it inspiring that there are new tools available to support advising / teaching • The necessary amount of support is usually ofer with new tools • In general, the time required to learn new tools is reasonable in relation to the benefits of the tool Training materials: • Availability of instructional material is important for fluent use of new tools • The usability of training materials (ease of following instructions, relevance) is important for smooth implementation 4. What kind of deployment instructions / support services are / would be helpful (binary yes/no) • Instructional video • Thorough written instructions • Quick guide for use • Frequently asked questions • Learning game • Personal / group instruction sessions • Support person available to assist • Forum for users to share their experiences • Other, what?