=Paper=
{{Paper
|id=Vol-1953/healthRecSys17_paper_10
|storemode=property
|title=Neighborhood-based Collaborative Filtering for Therapy Decision Support
|pdfUrl=https://ceur-ws.org/Vol-1953/healthRecSys17_paper_10.pdf
|volume=Vol-1953
|authors=Felix Gräßer,Stefanie Beckert,Denise Küster,Susanne Abraham,Hagen Malberg,Jochen Schmitt,Sebastian Zaunseder
|dblpUrl=https://dblp.org/rec/conf/recsys/GrasserBKAMSZ17
}}
==Neighborhood-based Collaborative Filtering for Therapy Decision Support==
https://ceur-ws.org/Vol-1953/healthRecSys17_paper_10.pdf
Neighborhood-based Collaborative Filtering for Therapy
Decision Support
Felix Gräßer, Hagen Malberg Stefanie Beckert, Denise Susanne Abraham
and Sebastian Zaunseder Küster and Jochen Schmitt Klinik und Poliklinik für
Institut für Biomedizinische Technik Zentrum für Evidenzbasierte Dermatologie Universitätsklinikum
Technische Universität Dresden Gesundheitsversorgung Dresden
Dresden, Germany Universitätsklinikum Dresden Dresden, Germany
felix.graesser@tu-dresden.de Dresden, Germany susanne.abraham@
jochen.schmitt@ uniklinikum-dresden.de
uniklinikum-dresden.de
ABSTRACT ACM Reference Format:
Clinical decision support systems (CDSS) providing assistance for Felix Gräßer, Stefanie Beckert, Denise Küster, Susanne Abraham, Hagen
Malberg, Jochen Schmitt and Sebastian Zaunseder. 2017. Neighborhood-
diagnosis and treatment decisions are expected to play an increas-
based Collaborative Filtering for Therapy Decision Support. In Proceedings
ingly important role in future healthcare. Especially data-driven of the Second International Workshop on Health Recommender Systems co-
approaches employing data mining and machine learning tech- located with ACM RecSys 2017, Como, Italy, August 2017 (RecSys’17), 5 pages.
niques to exploit the large volume of daily captured clinical data
promise to open up new perspectives. Particularly in e-commerce
Recommender Systems (RSs) have evolved considerably over the
1 INTRODUCTION
last years, yielding extremely sophisticated and specialized meth-
ods. In healthcare, however, such algorithms have not found wide Clinical decision support systems (CDSS) aim to assist health profes-
application although offering wide opportunities. Within this work sionals with the clinical decision-making tasks. Providing diagnosis
the idea of RSs, namely neighborhood-based Collaborative Filtering or treatment recommendations can foster personalization and con-
(CF), is transferred to the domain of CDSS aiming at helping to find tribute to improve quality and efficiency of patient care. CDSS
an optimal personalized therapy for a given patient and time, i.e. approaches are typically distinguished between data-driven and
consultation under consideration. Particular focus of this work is to knowledge-based approaches.
adapt neighborhood-based CF methods to exploit high-dimensional Knowledge-based systems rely on rule-based expert knowledge
clinical data. To leverage trust and reduce risk of the proposed only. Implementing and updating of such manually encoded evidence-
system, an exclusively data-driven approach is extended by a set based guidelines, however, puts a challenging bottleneck on providers
of evidence-based contraindication rules excluding inappropriate of CDSS. Furthermore, individual patients’ characteristics com-
therapies from the recommendation list. The proposed therapy monly differ from the strict inclusion criteria on which evidence is
recommendation system is practically evaluated on an exemplary based on, i.e. a clinical study was conducted on, which may result
clinical dataset. Its underlying conceptual framework, however, is in differing therapy responses. Thus, a patient-specific treatment,
intended to be transferable to other diseases and medical disciplines. i.e. an individually optimal therapy option cannot be provided on
the basis of guidelines derived from clinical studies only.
Data-driven CDSS, on the other hand, are supposed to automati-
CCS CONCEPTS cally extract information from clinical data and facilitate automatic
• Information systems → Expert systems; • Applied comput- adaptability to evolving databases. In this way, data-driven CDSS
ing → Health care information systems; Health informatics; are expected to be capable of exploiting the collective clinical expe-
• Human-centered computing → Collaborative and social com- rience represented by large-scale databases to improve quality and
puting theory, concepts and paradigms; increase personalization level of automatically generated therapy
recommendations. Thus, data-driven CDSS are a promising alter-
KEYWORDS native and are expected to open up new perspectives in medicine.
However, clinical data is often characterized by uncertainties and
Clinical Decision Support System (CDSS), Health Recommender
incompleteness (sparsity), high dimensionality and complex inter-
System, Therapy Recommender System, Collaborative Filtering,
dependencies [23][1], which places high demands on applicable
Recommender System Evaluation
methods. Traditional methods from machine learning, i.e. artificial
neural network classifier (ANN) or complex classifier ensembles
have proven to be very effective in learning patterns from large-
Second International Workshop on Health Recommender Systems co-located with ACM scale databases. However, the stated properties of medical data
RecSys 2017, August 2017, Como, Italy make the application of such algorithms challenging. Moreover, an
© 2017. Copyright for the individual papers remains with the authors. Copying per- essential requirement for acceptance of data-driven CDSS among
mitted for private and academic purposes. This volume is published and copyrighted
by its editors. health professionals are interpretabilty and comprehensibility of
the produced results. The blackbox-behaviour of typical machine
HealthRecSys’17, August 2017, Como, Italy F. Gräßer et al.
learning methods restricts insight into the classification process. idea of using CF for therapy decision support are a nursing care
This can be assumed to be one key factor hindering a wider spread plan recommender [5] and an approach recommending wellness
of data-mining and machine learning applications in the context of treatment [14].
CDSS to date.
Particularly in e-commerce applications, Recommender Systems 3 METHODOLOGY AND EVALUATION
(RSs), namely neighbourhood-based approaches as Collaborative Within this work, we propose and evaluate an exemplary therapy
Filtering (CF) algorithms, have gained increasing popularity within decision support system targeting therapy recommendations for
the preceding years. In this context, RSs support customers to in- patients suffering from the autoimmune skin disease psoriasis. The
dividually identify most interesting products from a wide range developed therapy decision support system aims at recommending
of possible options by predicting a user’s preference for products the potentially most effective systemic therapy out of M = 21
very effectively[20]. Incorporating only a modest number of near- therapy options for a given patient and consultation.
est neighbors into the computation can provide transparency on
the process of recommendation generation. Additionally, by ap- 3.1 Data Characteristics
plying a suitable similarity metric, such algorithms can cope with
heterogeneity and sparsity. Therefore, to address both, challenges The exemplary data at hand comprises excerpts from health records
related to the data characteristics and comprehensibility issue, we that were collected in the Clinic and Polyclinic for Dermatology,
proposed transferring the idea of CF algorithms into the domain University Hospital Dresden. The collected database comprises
of CDSS [9]. The overall objective of the proposed therapy recom- 1111 consultations from 213 patients suffering from various types
mender system is to find an optimal personalized therapy for a of psoriasis. For each sample, i.e. each consultation in the collected
consultation under consideration by converting estimations of a database, patient related attributes containing demographic data,
patient’s therapy response into recommendations. comorbidities and state of health as well as information on current
Within this contribution our previous approach is extended by a and previous treatments are contained. All relevant therapies ap-
set of rules derived from evidence-based absolute contraindication plied to a patient up to the consultation under consideration are
criteria in order to increase trust into and reduce risk of the demon- summarized under previous treatments, whereas therapies which
strated overall system. The exemplary therapy decision support were applied within the last two weeks preceding the respective
system is developed and evaluated targeting therapy recommen- consultation are collected under current treatments.
dations for patients suffering from the autoimmune skin disease For both, previous and current therapies, up to three different out-
psoriasis. Within this clinical application the developed therapy de- come indicators are given, namely a therapy effectiveness indica-
cision support system aims at recommending the potentially most tor (good, medium, bad) representing the subjective assessment,
effective systemic therapy for a given patient and consultation. The an objective health state improvement indicator (Psoriasis Area
recommender system’s underlying conceptual framework, how- and Severity Index [6]) and occurrence of adverse effects (yes, no).
ever, is intended to allow transferring the developed ideas to other Overall therapy effectiveness is modeled using a weighted sum of
diseases and medical disciplines. those three parameters as introduced in [8, 9] ranging from 0 (bad
The paper at hand is organized as follows. After presenting works response) to 1 (good response). Thus, ground truth, i.e. actually
related to CDSS for therapy decision support in general and systems applied therapy along with outcome for a given consultation, is
making use of CF techniques in particular, the available data, the derived from the succeeding consultations therapy response.
applied evaluation procedure and the RS algorithm are described, Table 1 and 2 summarize patient attributes and therapy information,
respectively. Finally, we present the results of the proposed method respectively. All attributes are supplied with scale of measurement,
and summarize our findings leading to future works in the field. range of values and availability relative to all consultations. In
case of comorbidities and therapies the availability is related to all
applied comorbidities or therapies, respectively.
2 RELATED WORK
Research on expert systems in clinical context date back to the 70ies. 3.2 Evaluation Procedure
Various approaches were published, deriving therapy decision sup- The quality of RSs is typically evaluated concerning accuracy met-
port from computerized medical guidelines [4, 12]. However, as rics for preference prediction performance as Root Mean Square
stated beforehand, knowledge-based approaches suffer from consid- Error (RMSE) and decision support metrics for ranked lists of items
erable efforts during development and updating of the underlying derived from information retrieval research as precision and recall
set of rules and are not always generally applicable. Proposed data- [10]. Generally, quality is evaluated offline and retrospectively based
driven approaches on the other hand, typically apply machine learn- on a test dataset comprising ratings on previously consumed items.
ing algorithms to derive therapy recommendations [17] or range In the context of therapy RSs this implies that evaluation metrics
from majority voting [15], systems based on association rules [2] to are computed on the actually applied therapy associated to a con-
approaches applying case-based reasoning [14]. In spite of gaining sultation for which outcome is known. However, the focus of a
increasing popularity in other domains, the use of CF techniques clinical recommender system should not only be to meet the ther-
is very limited in the context of CDSS. CF in the medical context apy decision of the attending physician but finding therapies with
was proposed in scientific works [1, 22] but studies on clinical data possibly good outcome and rejecting bad ones. Therefore, an ad-
are rare. There are few works applying CF algorithms for disease ditional output-driven precision metric precisiono @N is used in
risk or mortality prediction [3, 11, 13]. Work loosely related to the this study as introduced in [16] and previously demonstrated in
Neighborhood-based Collaborative Filtering for Therapy Decision Support HealthRecSys’17, August 2017, Como, Italy
[9]. Precisiono @N is computed for each evaluated consultation on Table 1: Patient describing attributes
the top-N recommendations matching the actually applied therapy
only. That means, precision is defined as the ratio of recommenda- Attribute Scale Range Availability %
tions having good outcome (true-positive T P) and all cases match-
Patient Data
ing the actually applied therapy and having good or bad outcome
(true-positive T P and false-positive F P). Employing an effectiveness Year of Birth interval 1931 - 1998 100
indicator threshold, data was divided into instances showing good Gender nominal 1,2 100
outcome (effectiveness > 0.5) and the remaining ones. Weight interval 50 - 165 50
Consequently, precisiono @N can be improved by increasing the
number of good outcome recommendations and rejecting bad out- Size interval 99 - 204 36
come recommendations from the top-N recommendation list. Planned Child nominal 1,2,3 100
To make most out of the already limited amount of data, the pro- Year of First Diagnosis interval 1950 - 2014 90
posed algorithm is evaluated using a leave one out cross validation
(LOOCV) on the entire dataset. Multiple consultations from the Family Anamnesis ordinal 1,2,3 50
patient for which outcome prediction and recommendations are Type of Psoriasis nominal 1,2,3,4,5,6 100
evaluated are excluded from the training dataset during evaluation. Comorbidities
Comorbidity nominal 1,2,3,...,34 -
3.3 Demographic-enhanced Collaborative
Status ordinal 1,2,3 100
Filtering Recommender
The CF algorithm applied in this contribution and initially pre- Under Treatment dichotomous 0,1 45
sented in [8, 9] uses both, information on therapy history, i.e. pre- State of Health
viously applied therapies and associated therapy response, along PASI Score interval 0 - 43 69
with all information on a patient’s type of disease, comorbidities
and demographic data to represent consultations in the database.
Furthermore, the attribute vector was extended to incorporate addi- Table 2: Therapy describing attributes
tional information on disease progression and associated therapies.
To that end, attributes from the temporal sequence of state of health Attribute Scale Range Availability %
and applied therapies, i.e. lag features from preceding consultations, Systemic Therapy nominal 1,2,3,...,15 -
are added. The underlying assumption is that therapy history to-
gether with the stated patient and disease progress describing data Effectiveness ordinal 1,2,3 98
carries sufficient information to reliably compare consultations. ∆PASI interval -37 - 25 42
Additionally, by not relying on therapy history solely, the cold-start Adverse Effect dichotomous 0,1 100
limitation can be overcome in cases where only limited or no in-
formation on therapy history is available. The overall objective
is to make prediction on patients having similar therapy history confidence in automatically generated therapy recommendations
and characteristics. Therefore, similarity is computed between a the risk of inaccurate or even health endangering recommendations
vector representation of the consultation under consideration and must be minimized.
representations of all other consultations in the database. Attributes For this purpose we implemented a set of exclusion rules based
representing consultations are (i) of various level of measurement on European S3-Guidelines on the systemic treatment of psoriasis
and differ in range and are (ii) only intermittently available. A [18] as summarized in table 3. Therapy options which are included
similarity metric capable of coping with both, missing values and in the recommendations provided by the CF algorithms and are
varying levels of measurement, is the Gower Similarity Coefficient affected by an exclusion criterion are removed from the list. Addi-
[7], which is applied in this work. Here, the level of measurement of tionally, therapies showing good outcome in a patient’s previous
the individual attributes is respected for each attribute comparison consultation were moved to the top of the list. Finally, from the
using a data type-specific similarity coefficients. Therapy outcome modified recommendation list the top-3 entries are presented to
predictions are estimated based on a weighted sum of the k nearest the user.
consultations to a consultation under consideration. In a subse-
quent recommendation step a list of therapy options, ordered by 4 EVALUATION RESULTS
outcome prediction, is available for further processing.
Both precision@3 and outcome-driven precisiono @3 of the CF-
based algorithm are highly dependent on the neighborhood size k
3.4 Exclusion rules incorporated into the outcome prediction computation (see figure
Particularly in the context of CDSS, trust plays a crucial role to 2). Outcome prediction accuracy and recommendation precision
leverage acceptance and applicability of such systems. However, show extrema in the neighborhood of around k = 10...20. Integrat-
in contrast to other domains of RS applications, e.g. e-commerce ing too many neighbors, i.e. increasing k, provokes a performance
applications, particular in the area of health and medicine failures in decline due to noise influencing prediction accuracy and thus rec-
recommendations can accompany high risk. Therefore, to increase ommendation precision. In contrast, outcome-driven precisiono @3
HealthRecSys’17, August 2017, Como, Italy F. Gräßer et al.
Table 3: Contraindications and exclusion rules. 1
Precision@3, Precisiono @3
Contraindication Excluded Therapy
arterial hypertonia Ciclosporin 0.8
renal disease Ciclosporin
hepatic disease Methotrexat
0.6
cancerous disease any Biologics
planned child Methotrexat, Acitretin
psoriasis arthritis Fumaderm, any UV therapies 0.4
0 20 40 60 80 100
Acitrecin, Ciclosporin
k
0.08 Figure 2: Overall precision@3 of therapy recommenda-
tions and outcome-driven precisiono @3. precision@3 and
precisiono @3 for the CF output ( , ) and with addi-
0.07 tional exclusion rules applied ( , ) are shown.
RMSE
0.06 1
consultation coverage
0.99
0.05
0 20 40 60 80 100
k
0.98
Figure 1: RMSE ( ) computed between effectiveness esti-
mated by the CF recommender and effectiveness of actually 0.97
applied therapies. 0 20 40 60 80 100
k
seems to benefit from a somewhat larger neighborhood. However, Figure 3: Consultation coverage, i.e. ratio of overall consul-
when comparing precisiono @3 for different k it must be kept in tations for which recommendations could be provided. Con-
mind that precisiono @N highly depends precision@N . sultation coverage for the CF output ( ) and with addi-
Adding additional rules clearly increases recommendation preci- tional exclusion rules applied ( ) are shown.
sion and outcome-driven precision. Continuing successful therapies
has favorable effect on recommendation precision. Furthermore,
therapy options which were successfully applied at neighboring pa- similarity metric heavily affects the obtained results. Therefore,
tients, i.e. located in the top-3 list, can be contraindicated therapies future efforts will concentrate on those aspects, namely feature
for a patient and consultation under consideration. Eliminating selection methods [19] and metric learning algorithms [21, 24], to
them from the recommendation list moves non-contraindicated further improve the proposed CF performance and contribute to
actually applied therapies to the top instead. create a basis for applicability and acceptance of suchlike CDSS.
For small neighborhoods, the coverage of possible therapy options Besides the used features, it is shown that the neighborhood size k
can be too low to facilitate recommendations for a consultation plays a vital role in terms of outcome prediction accuracy and rec-
under consideration leading to low overall consultation coverage. ommendation quality. For the exemplary application an appropriate
Removing recommendations from the recommendation list affects neighborhood size was determined by cross validation. However,
the coverage additionally as shown in 3. Consequently, adding ad- this size can neither be expected to provide the best results in case
ditional exclusion rules demands for an increased neighborhood k of an extended dataset nor it can be readily transferred to other
to facilitate satisfactory consultation coverage. problems. Aspects related to the neighborhood size will be further
investigated in future studies, particularly incorporating more data.
5 CONCLUSION AND FUTURE WORKS In fact, one major limitation of this work is the rather small database
Our analyses show that the proposed prototype combining a data- our studies are based on. Therefore, future work will address apply-
driven CF approach with evidence-based knowledge can provide ing the proposed methods to more comprehensive datasets which
reliable personalized therapy recommendations. However, the selec- we assume will improve the recommendation quality significantly.
tion and extraction of appropriate attributes and applying a suitable Particularly, data provided by additional dermatologists needs to be
Neighborhood-based Collaborative Filtering for Therapy Decision Support HealthRecSys’17, August 2017, Como, Italy
incorporated into the database to prevent learning a limited number Update 2015 âĂŞ Short version âĂŞ EDF in cooperation with EADV and IPC.
of experts recommendations and improve generalization capability. Journal of the European Academy of Dermatology and Venereology 29, 12 (2015),
2277–2294.
As a consequence, another aspect which needs to be investigated [19] P. Pudil, F. Ferri, J. Novovicova, and J. Kittler. 1994. Floating search method for
in future works to both, improve recommendation quality and cope feature selection with non monotonic criterion functions. Pattern Recognition 2
(1994), 279–283.
with scalability issues when applied to large-scale data, is identi- [20] Xiaoyuan Su and Taghi M. Khoshgoftaar. 2009. A Survey of Collaborative Fil-
fying clusters in the database. This is intended to be done offline tering Techniques. Advances in Artificial Intelligence 3, Section 3 (2009), 1–19.
prior to actual recommendation generation. arXiv:421425
[21] Kilian Q. Weinberger and Lawrence K. Saul. 2009. Distance metric learning for
large margin nearest neighbor classification. The Journal of Machine Learning
Research 10 (2009), 207–244.
[22] Martin Wiesner and Daniel Pfeifer. 2014. Health recommender systems: Con-
ACKNOWLEDGMENTS cepts, requirements, technical basics and challenges. International Journal of
This work is part of the project Therapieempfehlungssystem which Environmental Research and Public Health 11, 3 (2014), 2580–2607.
[23] David Windridge and Miroslaw Bober. 2014. A Kernel-Based Framework for
is funded by the Roland Ernst Stiftung für Gesundheitswesen. Medical Big-Data Analytics. Interactive Knowledge Discovery and Data Mining in
Biomedical Informatics: State-of-the-Art and Future Challenges (2014), 197–208.
[24] Eric P. Xing, Andrew Y. Ng, Michael I. Jordan, and Stuart J. Russell. 2002. Distance
REFERENCES Metric Learning with Application to Clustering with Side-Information. Advances
[1] André Calero Valdez, Martina Ziefle, Katrien Verbert, Alexander Felfernig, and in Neural Information Processing Systems (NIPS) 15 (2002), 505–512.
Andreas Holzinger. 2016. Recommender Systems for Health Informatics: State-
of-the-Art and Future Perspectives. Springer International Publishing, Cham,
391–414.
[2] Jonathan H. Chen and Russ B. Altman. 2014. Automated Physician Order Rec-
ommendations and Outcome Predictions by Data-Mining Electronic Medical
Records. AMIA Summits on Translational Science proceedings (2014), 206–210.
[3] Darcy Davis, Nitesh V. Chawla, Nicholas Christakis, and Albert László Barabási.
2010. Time to CARE: A collaborative engine for practical disease prediction.
Data Mining and Knowledge Discovery 20, 3 (2010), 388–415.
[4] Paul A. De Clercq, Johannes A. Blom, Hendrikus H. M. Korsten, and Arie Hasman.
2004. Approaches for creating computer-interpretable guidelines that facilitate
decision support. Artificial Intelligence in Medicine 31, 1 (2004), 1–27.
[5] Lian Duan, W. Nick. Street, and E. Xu. 2011. Healthcare information systems:
data mining methods in the creation of a clinical recommender system. Enterprise
Information Systems 5, 2 (2011), 169–181.
[6] T. Fredriksson and U. Pettersson. 1978. Severe Psoriasis - Oral Therapy with a
New Retinoid. Dermatologica 157, 4 (1978), 238–244.
[7] J. C. Gower. 1971. A General Coefficient of Similarity and Some of Its Properties.
Biometrics 27, 4 (1971), 857–871.
[8] Felix Gräßer, Stefanie Beckert, Denise Küster, Jochen Schmitt, Susanne Abraham,
Hagen Malberg, and Sebastian Zaunseder. 2016. Application of recommender
system methods for therapy decision support. 2016 IEEE 18th International
Conference on e-Health Networking, Applications and Services (Healthcom) (2016),
1–6.
[9] Felix Gräßer, Stefanie Beckert, Denise Küster, Jochen Schmitt, Susanne Abraham,
Hagen Malberg, and Sebastian Zaunseder. 2017. Therapy Decision Support Based
on Recommender System Methods. Journal of Healthcare Engineering 2017 (2017).
[10] Asela Gunawardana and Guy Shani. 2009. A Survey of Accuracy Evaluation
Metrics of Recommendation Tasks. J. Mach. Learn. Res. 10 (Dec. 2009), 2935–2962.
[11] Shahzaib Hassan and Zeeshan Syed. 2010. From Netflix to Heart Attacks: Collab-
orative Filtering in Medical Datasets. Proceedings of the 1st ACM International
Health Informatics Symposium (2010), 128–134.
[12] Ellen Kilsdonk, Linda W. Peute, Rinke J. Riezebos, Leontien C. Kremer, and
Monique W M Jaspers. 2013. From an expert-driven paper guideline to a user-
centred decision support system: A usability comparison study. Artificial Intelli-
gence in Medicine 59, 1 (2013), 5–13.
[13] Maytiyanin Komkhao, Jie Lu, and Lichen Zhang. 2012. Determining Pattern
Similarity in a Medical Recommender System. Data and Knowledge Engineering
7696 (2012), 103–114.
[14] Thean Pheng Lim and Wahidah Husain. 2013. Recommender System for Person-
alised Wellness Therapy. International Journal of Advanced Computer Science and
Applications 4, 9 (2013), 54–60.
[15] Haifeng Liu, Guotong Xie, Jing Mei, Weijia Shen, Wen Sun, and Xiang Li. 2013. An
Efficacy Driven Approach for Medication Recommendation in Type 2 Diabetes
Treatment Using Data Mining Techniques. Medinfo 2013: Proceedings of the 14th
World Congress on Medical and Health Informatics, Pts 1 and 2 192 (2013), 1071.
[16] Jing Mei, Haifeng Liu, Xiang Li, Yiqin Yu, and Guotong Xie. 2015. Outcome-
driven Evaluation Metrics for Treatment Recommendation Systems. Studies in
health technology and informatics 210 (2015), 190–194.
[17] Geert Meyfroidt, Fabian GÃČÂijiza, Jan Ramon, and Maurice Bruynooghe. 2009.
Machine learning techniques to examine large patient databases. Best Practice
and Research Clinical Anaesthesiology 23, 1 (2009), 127 – 143.
[18] A. Nast, P. Gisondi, A.D. Ormerod, P. Saiag, C. Smith, P.I. Spuls, P. Arenberger,
H. Bachelez, J. Barker, E. Dauden, E.M. de Jong, E. Feist, A. Jacobs, R. Jobling, L.
KemÃľny, M. Maccarone, U. Mrowietz, K.A. Papp, C. Paul, K. Reich, S. Rosumeck,
T. Talme, H.B. Thio, P. van de Kerkhof, R.N. Werner, and N. Yawalkar. 2015.
European S3-Guidelines on the systemic treatment of psoriasis vulgaris âĂŞ