=Paper=
{{Paper
|id=Vol-2960/paper6
|storemode=property
|title=Knowledge-Based Recommendations for Climbers (Short paper)
|pdfUrl=https://ceur-ws.org/Vol-2960/paper6.pdf
|volume=Vol-2960
|authors=Iustina Ivanova,Marina Andric,Francesco Ricci
|dblpUrl=https://dblp.org/rec/conf/recsys/IvanovaA021
}}
==Knowledge-Based Recommendations for Climbers (Short paper)==
https://ceur-ws.org/Vol-2960/paper6.pdf
Knowledge-Based Recommendations for Climbers
Iustina Ivanova, Marina Andrić and Francesco Ricci
Free University of Bozen-Bolzano, Piazza Università 1, 39100, Bolzano, Italy
Abstract
Climbing is a popular sport and recreational activity. Unfortunately, there is a lack of technologies for supporting climbers in
choosing what climbing route to climb next. We introduce a project aimed at developing a Climbing Recommender System
for suggesting routes that are suited for training and practicing sport climbing. We model a climber by relying on both
explicit and implicit feedback. Implicit feedback is acquired by an automatic activity recognition component (in climbing
gyms), while explicit feedback is acquired by means of a mobile application. We also present a recommendation approach
based on the prediction of the subjective evaluation of climbing routes’ difficulty. In fact, often climbers perceive the difficulty
of a route differently from the official grade. The prediction method is based on the analysis of how climbers deviate their
assessment of routes’ difficulty from the official difficulty grade, and it generates explanations for the predictions.
Keywords
recommender system, climbing, knowledge-based, explanation
1. Introduction there is no implemented methodology to collect data on
a big scale. Moreover, it is unclear what the real needs of
Climbing guidebooks provide climbing enthusiasts with climbers are and how the human-computer interaction
important information about available climbing routes. of the CRS should be designed.
Actually, new climbing routes are continuously devel- Recommender systems (RS) for climbers have not yet
oped, either indoor (climbing gyms) [1] or outdoor [2], been proposed in the scientific literature. RSs for similar
and this large set of alternative routes makes it difficult sports, such as running [3, 4, 5], hiking, or trekking [6, 7],
for any climber, either novice or expert, to make the right have instead been developed. Hiking is probably the
choice. There is a clear need for Climbing Recommender most similar sport; here the difficulty of a hike, the po-
Systems (CRS) that could support climbers in choosing tential risk of hiking above the hiker’s technical skills
the most suitable next routes. Ideally, such a system and physical condition, the weather conditions, and the
should manage all the aspects considered by climbers hiking group composition, are important aspects as well.
in their choices. The recommended routes should be: Calbimonte et al. [8, 9] have proposed an RS for hiking
suitable for training, challenging but within their capa- trails, where recommendations are adapted to the hiker’s
bilities, enjoyable, compatible with the contextual sit- profile, describing the current physical level and pref-
uation (weather conditions), and good for a group of erences, which are obtained by explicitly querying the
climbers. Building such a system requires knowledge user. Similarly, the work of Vías et al. [10] presents a
about climbers’ preferences, their physical and technical simple approach for recommending hiking routes based
level, the group’s characteristics, and, importantly, the on search criteria, such as, hike’s difficulty, and dura-
climbing routes’ characteristics, such as, difficulty grade, tion. The core approach of these works lies in building
location, and safety level. This can be acquired and de- a knowledge-rich user and item profile and then recom-
livered by means of a well-designed human-computer mending hikes that match the characteristics of the hike
(mobile) interaction where climbers can leave explicit to the explicitly formulated needs of the hiker. More
feedback about the routes they climbed. Moreover, sen- advanced RS, but for runners, are able to leverage the
sor data (body and device sensors) should be used to athlete’s physical level that is measured via activity track-
implicitly detect climbers’ activities and performance. ing sensors [11, 12, 13, 14]. These technologies have not
Unfortunately, current sensing technologies for climb- yet been exploited in sport climbing, as there is a lack of
ing are at an early stage of development, and furthermore, effective and easily available activity recognition tech-
3rd Edition of Knowledge-aware and Conversational Recommender nologies and devices for this sport [15, 16, 17, 18].
Systems (KaRS) & 5th Edition of Recommendation in Complex Clearly, there is a need for novel research in this area,
Environments (ComplexRec) Joint Workshop @ RecSys 2021, as current hardware and software technologies are not
September 27–1 October 2021, Amsterdam, Netherlands sufficient for semi-automatically building any form of
Envelope-Open iivanova@unibz.it (I. Ivanova); maandric@unibz.it (M. Andrić); climber’s profile. This is a prerequisite for suggesting
fricci@unibz.it (F. Ricci)
Orcid 0000-0001-7116-9338 (I. Ivanova); 0000-0002-0877-7063
suitable climbing routes on the basis of the climber’s
(M. Andrić); 0000-0002-9421-8566 (F. Ricci) needs and capabilities. We here first describe the required
© 2021 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
profiling of climbers, and then we introduce a CRS for
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
outdoor climbing routes suggestion. assigned by the climber to the route. This prediction can
be used to recommend to the climber routes that are in
the right difficulty range for her. In order to address this
2. Routes Recommendation task, we have applied data mining techniques to explicit
feedback data collected by means of the Vertical-Life mo-
We describe the results we have obtained in developing a
bile app where climbers can rate, grade, and comment
knowledge-based CRS. Recommendations are generated
on routes that they have climbed. Our work is motivated
by understanding users’ characteristics (profile), which
by Draper [24], who reports that climbers often have
are obtained by semi-automatically analyzing their be-
different opinions about the difficulty grade of the same
havior [19]. We consider two types of sources of climber-
route. The app offers climbers the possibility to express
related knowledge, which are acquired by means of either
their disagreement with the ‘official’ climbing grade of a
implicit or explicit feedback [20]. Implicit feedback is
route, as it is given in the guidebook. We have adopted a
collected through sensor data analysis, in order to auto-
knowledge-based approach to understanding the reasons
matically detect the climber’s performed activities, i.e.,
of such disagreements. We have compared the predic-
which route was climbed and how the climber performed
tions, for the subjective evaluations of routes’ difficulty,
(e.g., the duration, or the effort made). Explicit feedback
generated by a regression model, based on features aimed
relates instead to climbers’ manual input, typically de-
at capturing why climbers disagree with the official grade,
scribing the experience of climbing a route, such as, the
to a standard collaborative filtering algorithm.
safety of the route, or the perceived difficulty. In order
to collect this type of data, we are collaborating with
Vertical-Life (www.vertical-life.info), which offers a mo- 2.1. Route Grade Prediction
bile application for climbers. We plan to augment their We focus our investigation on ascents performed on
application with recommendation technologies. routes located in mountains and crags in the lead style;
Related to implicit feedback, we have developed some however, the approach described below can be extended
initial solutions and technologies for activity detection, to ascents performed in climbing gyms and other climb-
which are currently designed only for indoor lead climb- ing disciplines, such as bouldering.
ing. We have employed a ‘smart quickdraw sensor’ [21]. The data set of climbers’ entries into the mobile app
A standard quickdraw is a piece of climbing equipment about their ascents is summarized in Table 1. The data set
used by climbers to allow the climbing rope to run freely is restricted to the most frequently climbed route grades.
through protection such as a bolt anchors, hence it is Grades are represented with integers ranging from 6 (5a)
used for securing the climber at a specific point on the to 22 (7c). Observed deviations of climbers’ grades with
climbing route. A smart quickdraw is a regular one aug- respect to routes’ official grades range from -3 to 3. It is
mented with an accelerometer. The movements of the important to note that, on average, the climber’s grade
rope, used for securing the climber, are propagated to the coincides with route’s official grade in 92% of ascents.
quickdraw. Then, they are detected by the sensor and
sent wirelessly to a computer for data analysis. By using
this ‘smart quickdraw sensor’, we were able to detect with Table 1
Data set of climbers’ ascents description. ’% climber grades’
an accuracy of 93% the activity of ‘rope pulling’, which
refers to the percentage of ascents where the climbers gave
happens shortly after the climber finishes an ascent and an explicit grade evaluation that differs from the official one.
when she removes the rope from the wall. Recognizing
rope pulling is instrumental in measuring the number of Outdoor climbs
ascents made on a line equipped with this sensor, and it # climbs 157,576
can also be employed to distinguish beginners from ex- # climbers 2,624
pert climbers [22]. More sophisticated data analysis has # routes 10,738
also shown that the energy of the quickdraw movement % climber grades 8%
can be used for climber’s performance measurement. In
addition to this, we employed video cameras to detect The first personalized grade prediction method, which
when the climber is lowered to the ground after her as- we call ‘knowledge-based’ uses a linear regression model,
cent is finished [23]. We plan to adopt a similar solution for which we generated features representing climbers-
also for outdoor climbing. routes interactions. The assumption is that the official
Explicit feedback data, includes instead the subjective difficulty of the routes, as defined by the route setters may
evaluations of ascended routes’ difficulty grades, which also depend on their skills, while a climber’s perceived
are often registered by climbers after they have climbed difficulty of the routes may depend on the climber’s phys-
a route. By using this data we aim at predicting the sub- ical level and on contextual conditions (e.g., the season).
jectively perceived difficulty of a route, which would be Therefore, we included the time factor, as the outdoor
routes can change with the time (rocks might deteriorate, the climber-assigned grade of a route coincides with the
equipment might break): we generate similar features official grade of the route. Hence, predicting correctly
which take into consideration the specified time inter- the grades assigned by climbers when they deviate from
val limit. As a result, we predict the climber’s perceived the official grade is challenging.
grade as a linear dependent variable from the identified It is also worth noting that a major disadvantage of
features. the SVD rating prediction model is that users and items
Assume that we have a target climber 𝑐, and a route 𝑟. are here represented in a joint latent factor space, which
is hardly useful for explaining the predictions [26, 27]. In
⋀
We are interested in the prediction 𝑔𝑟𝑎𝑑𝑒(𝑐, 𝑟, 𝑡) of how
climber 𝑐 would grade the route 𝑟 at a particular point in fact, in this domain it is pivotal to properly convince the
time 𝑡. The predictive features that we have introduced climber about the reliability of the provided information:
are as follows: there are important safety issues to consider.
We have measured the root-mean-square error (RMSE)
• 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟): the ‘official’ route difficulty grade. of the proposed models (linear regression knowledge-
• 𝑚𝑑(𝑟, 𝑡): given the grade assessments of climbers based and SVD based collaborative filtering) and com-
who previously climbed the target route, before pared them to the baseline. The results show that both
time 𝑡, we compute the average deviation of these models give a lower error than the baseline, meaning
grades from the route’s official grade 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟). that they are capable of correctly using climbers’ feed-
• 𝑚𝑑 𝑌 (𝑟, 𝑡): this feature is a variant of feature back about difficulty level of routes (see Table 2). In the
𝑚𝑑(𝑟, 𝑡) such that only the final year and a half of table, ‘per user RMSE’ indicates the RMSE computed for
data collection is included for feature computa- each user and then averaged, while ‘RMSE’ is the global
tion. The feature is meant to capture change in average of all prediction errors.
the difficulty of an ascent that may result from de-
terioration of the rock over time due to frequent Table 2
climbing activity (the so-called polished routes) Performance of perceived difficulty grade prediction on the
or recent maintenance work. data set of climbers’ ascents on outdoor routes.
• 𝑐𝑚𝑑 𝑀 (𝑐, 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟), 𝑡): given the grade assess- Model Outdoor routes
ments of the target climber 𝑐, before time 𝑡, for RMSE per user RMSE
routes of the same official grade 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟), this
feature expresses the mean deviation of these baseline 0.339 0.191 (± 0.306)
Lin. Regression KB 0.317 0.176 (± 0.284)
grades from 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟) within a three-month pe-
SVD CF 0.322 0.174 (± 0.284)
riod in the final year of data collection. This fea-
ture is supposed to capture the influence of envi-
ronmental conditions on a climber’s perception of
route difficulty. For example, climbing outdoors 2.2. Explanations and GUI
in the summer months is typically considered as
harder than climbing in the spring. The knowledge-based model is employed to prototype
a novel GUI of the Vertical-Life climbing application,
As we have noted above, climbers do not often devi- which in addition to the official grade shows the pre-
ate with their subjective evaluations of route difficulty dicted climber’s perceived grade. Additionally, we have
from the official evaluations. Hence, a strong baseline employed the coefficients of the linear regression model
method for predicting 𝑔𝑟𝑎𝑑𝑒(𝑐, 𝑟, 𝑡) is actually using di- to generate explanations to the climbers for the predicted
rectly 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟). grades of the selected routes. The linear regression model
In addition to this baseline prediction method, we com- along with its coefficients is shown in the following equa-
pare the predictions generated by the above-mentioned tion:
linear regression knowledge-based model with those ⋀
𝑔𝑟𝑎𝑑𝑒(𝑐, 𝑟, 𝑡) = 0.027 + 0.998 ⋅ 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟) + 0.410 ⋅ 𝑚𝑑(𝑟, 𝑡)
computed by a standard collaborative filtering (CF) al-
gorithm. In the application of collaborative filtering, we + 1.051 ⋅ 𝑚𝑑 𝑌 (𝑟, 𝑡) + 0.279 ⋅ 𝑐𝑚𝑑 𝑀 (𝑐, 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟), 𝑡)
model the grade prediction task as a special rating predic- Clearly the official grade 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟) has the largest im-
tion problem, where the rating of a route is the deviation portance, but also the other features, related to the grad-
of the climber’s grade from the official grade, namely: ing behavior of the climbers, do have important weights
𝑔𝑟𝑎𝑑𝑒(𝑐, 𝑟, 𝑡) − 𝑜𝑔𝑟𝑎𝑑𝑒(𝑟). We use matrix factorization, in the model. The explanation sentence, which is com-
namely singular value decomposition (SVD), to generate menting why the predicted grade is different from the
such predictions [25]. We note that the rating matrix official grade, is generated by considering, case by case,
is in this case very sparse (0.994 sparsity) and with a the features that, for a given route and climber combina-
prevalence of 0 ratings: these are the evaluations where tion, have the largest positive (or negative) impact on the
Figure 1: Explanation of the ‘perceived’ grade, shown in the system GUI prototype. The left figure shows routes’ information
and includes ‘Perceived grade’ in addition to the real ‘Grade’; the right figure includes the pop-up window that appears when
the climber points to the ‘Perceived grade’ for route 04.
predicted grade deviation. Such an explanation may be with the explicit feedback collected from the mobile ap-
useful if a climber would like to better understand why plication. In fact, one related research question is how
the predicted difficulty of the route is different from the the climber’s skill level acquired by the implicit feedback
official grade. Potentially, this can lead to fewer accidents, can be used to improve the subjective route difficulty
as the climber may choose routes that better match her grade predictions that we have computed by relying only
capabilities. on explicit feedback.
Figure 1 shows an explanation example sentence given Secondly, as we have mentioned in the introduction,
in the prototype app to the climber when she points to climbing route recommendations may support diverse
the perceived grade column by touching the screen on users’ needs. One important aspect is to identify routes
route 04. that fit a specified training plan or to give explicit feed-
back to climbers about their ‘mistakes’ in trying the
wrong routes. In fact, the training aspect is very im-
3. Discussion and Conclusion portant, as many climbers find that being able to track
and achieve gradual progress is a crucial motivation for
There are some clear limitations of the proposed CRS
them. One possible approach to this problem is trying
that we plan to address in the future.
to intelligently revise or complete an existing climber’s
Firstly, we have developed sensor data analysis tech-
training program [28, 29, 30, 31], which the climber typi-
nologies for automatic activity recognition in climbing
cally stores in the app.
gyms, but we have not yet developed a similar solution
Thirdly, the implemented explanation component can
for outdoor climbing. Moreover, knowledge extracted
be improved in order to give a more convincing expla-
from low-level sensor data, e.g., the average speed of a
nation, not only of the predicted difficulty, but also of
climber during an ascent, has not yet been integrated
the recommended route [32]. Moreover, for motivational 2021 CHI Conference on Human Factors in Com-
and training purposes, climbers sometimes repeat the puting Systems, 2021, pp. 1–6.
routes which they tried, and specific explanations should [4] J. Berndsen, B. Smyth, A. Lawlor, Fit to run: Per-
be generated in these cases [33]. For instance, the sys- sonalised recommendations for marathon training,
tem may argue: ‘This lead climbing route was climbed in: Fourteenth ACM Conference on Recommender
by you with 3 stops, try it again with fewer stops this Systems, ACM, 2020.
time’. In fact, the specific rationale of a recommendation [5] C. Feely, B. Caulfield, A. Lawlor, B. Smyth, Provid-
should be made clear to the climber. As a matter of fact, ing explainable race-time predictions and training
some routes are more enjoyable and should be recom- plan recommendations to marathon runners, in:
mended for climbers’ satisfaction; other routes are more Fourteenth ACM Conference on Recommender Sys-
important for training and motivation; other routes are tems, 2020, pp. 539–544.
relevant because they may better satisfy the needs of the [6] M. Boerger, Hiking suggestions and planner, 2014.
group of climbers the target user belongs to. US Patent 8,688,374.
Finally, we must properly evaluate the proposed sys- [7] Y. Takama, W. Sasaki, T. Okumura, C.-C. Yu, L.-H.
tem prototype, and understand whether such a CRS Chen, H. Ishikawa, Walking route recommendation
would be suitable and interesting for climbers. For this system for taking a walk as health promotion, in:
purpose, we have created an online survey [34] to collect 2015 IEEE/WIC/ACM International Conference on
climbers’ opinions on the proposed CRS. Web Intelligence and Intelligent Agent Technology
In conclusion, in this paper, we have presented raw (WI-IAT), volume 1, IEEE, 2015, pp. 556–559.
components and preliminary results that will be in- [8] J.-P. Calbimonte, N. Zappellaz, E. Hébert, M. Si-
tegrated into a novel CRS. We want to create a rich mon, N. Délétroz, R. Hilfiker, A. Cotting, Santour:
knowledge-based climber’s profile taking into consid- towards personalized recommendation of hiking
eration climber’s preferences, current physical level, be- trails to health profiles, in: International Con-
havior and skills. Such knowledge should be extracted ference on Web Engineering, Springer, 2018, pp.
from log data of the interaction of climbers with the 238–250.
routes that they have tried and evaluated. By better ex- [9] J.-P. Calbimonte, S. Martin, D. Calvaresi, A. Cotting,
ploiting the bulk of knowledge contained in electronic A platform for difficulty assessment and recommen-
guidebooks and climbers’ diaries, we aim at increasing dation of hiking trails, in: Information and Commu-
climbers’ satisfaction but also their safety, as climbers nication Technologies in Tourism 2021, Springer,
will be supported to choose routes that are more aligned 2021, pp. 109–122.
with their skills and expectations. [10] J. Vías, J. Rolland, M. L. Gómez, C. Ocaña, A. Luque,
Recommendation system to determine suitable
and viable hiking routes: a prototype application
Acknowledgments in Sierra de las Nieves Nature Reserve (southern
Spain), Journal of Geographical Systems 20 (2018)
This work has been partly supported by the project
275–294.
“Sensors and data for the analysis of sports activities
[11] J. Berndsen, A. Lawlor, B. Smyth, Running with
(SALSA)”, funded by the EFRE-FESR programme 2014-
recommendation., in: HealthRecSys@ RecSys, 2017,
2020 (CUP: I56C19000110009). The authors thank An-
pp. 18–21.
drea Janes, Ben Lepesant and the Vertical-Life (https:
[12] B. Loepp, J. Ziegler, Recommending running routes:
//www.vertical-life.info/) company for the data provided
framework and demonstrator, in: Workshop on
for this research.
Recommendation in Complex Scenarios, 2018.
[13] B. Smyth, P. Cunningham, A novel recommender
References system for helping marathoners to achieve a new
personal-best, in: Proceedings of the Eleventh ACM
[1] R. Ferrero Camoletto, D. Marcelli, Keeping it natu- Conference on Recommender Systems, 2017, pp.
ral? challenging indoorization in Italian rock climb- 116–120.
ing, Annals of Leisure Research 23 (2020) 34–51. [14] B. Smyth, M. C. Willemsen, Predicting the personal-
[2] G. M. Marrosu, T. Balvis, Environmental impact best times of speed skaters using case-based reason-
assessment in climbing activities: A new method ing, in: International Conference on Case-Based
to develop a sustainable tourism in geological and Reasoning, Springer, 2020, pp. 112–126.
nature reserves, Geoheritage 12 (2020) 1–16. [15] C. Ladha, N. Y. Hammerla, P. Olivier, T. Plötz,
[3] S. Shreepriya, C. Legras, S. Clinchant, J. Willam- ClimbAX, in: Proceedings of the 2013 ACM interna-
owski, Evaluating an itinerary recommendation tional joint conference on Pervasive and ubiquitous
algorithm for runners, in: Extended Abstracts of the computing, ACM, 2013.
[16] J. Pansiot, R. C. King, D. G. McIlwraith, B. P. L. field, 2008.
Lo, Guang-Zhong Yang, Climbsn: Climber perfor- [30] M. L. Michailov, Workload characteristic, perfor-
mance monitoring with bsn, in: 2008 5th Interna- mance limiting factors and methods for strength
tional Summer School and Symposium on Medical and endurance training in rock climbing, Medicina
Devices and Biosensors, 2008, pp. 33–36. Sportiva 18 (2014) 97–106.
[17] A. Tonoli, R. Galluzzi, E. C. Zenerino, D. Boero, [31] T. O. Bompa, C. Buzzichelli, Periodization-: theory
Fall identification in rock climbing using wearable and methodology of training, Human kinetics, 2019.
device, Proceedings of the Institution of Mechanical [32] F. Ricci, L. Rokach, B. Shapira, Recommender
Engineers, Part P: Journal of Sports Engineering systems: Introduction and challenges, in: Rec-
and Technology 230 (2016) 171–179. ommender systems handbook, Springer, 2015, pp.
[18] A. Bonfitto, A. Tonoli, S. Feraco, E. C. Zenerino, 1–34.
R. Galluzzi, Pattern recognition neural classifier for [33] K. Tsukuda, M. Goto, Explainable recommenda-
fall detection in rock climbing, Proceedings of the tion for repeat consumption, in: Fourteenth ACM
Institution of Mechanical Engineers, Part P: Journal Conference on Recommender Systems, 2020, pp.
of Sports Engineering and Technology 233 (2019) 462–467.
478–488. [34] S. team, Poll for climbers ‘predicted perceived climb-
[19] M. De Gemmis, P. Lops, C. Musto, F. Narducci, G. Se- ing difficulty grades’, 2021. URL: https://forms.gle/
meraro, Semantics-aware content-based recom- YhKaCM3aqN8hAY557.
mender systems, in: Recommender systems hand-
book, Springer, 2015, pp. 119–159.
[20] I. Ivanova, Climber behavior modeling and rec-
ommendation, in: Proceedings of the 29th ACM
Conference on User Modeling, Adaptation and Per-
sonalization, 2021, pp. 298–303.
[21] I. Award, Winner | OUTDOOR Digi-
tal Services, https://www.ispo.com/en/
awards/ispo-award/2019/outdoor/winners/
vertical-life-smart-quickdraw, 2019. [Online;
accessed 22-July-2021].
[22] I. Ivanova, M. Andric, A. Janes, F. Ricci, F. Zini,
Climbing activity recognition and measurement
with sensor data analysis, in: Companion Pub-
lication of the 2020 International Conference on
Multimodal Interaction, 2020, pp. 245–249.
[23] I. Ivanova, M. Andrić, S. Moaveninejad, A. Janes,
F. Ricci, Video and sensor-based rope pulling de-
tection in sport climbing, in: Proceedings of the
3rd International Workshop on Multimedia Content
Analysis in Sports, 2020, pp. 53–60.
[24] N. Draper, Climbing grades: Systems and subjectiv-
ity, The Science of Climbing and Mountaineering
(2016) 227.
[25] X. Ning, C. Desrosiers, G. Karypis, A comprehen-
sive survey of neighborhood-based recommenda-
tion methods, Recommender systems handbook
(2015) 37–76.
[26] N. Tintarev, J. Masthoff, Explaining recommenda-
tions: Design and evaluation, in: Recommender
systems handbook, Springer, 2015, pp. 353–382.
[27] Y. Koren, R. Bell, Advances in collaborative filtering
(2015) 77–118.
[28] E. Horst, How to climb 5.12, Rowman & Littlefield,
2011.
[29] E. Horst, Training for climbing: The definitive guide
to improving your performance, Rowman & Little-