=Paper=
{{Paper
|id=None
|storemode=property
|title=Choicla: Intelligent Decision Support for Groups of Users in the Context of Personnel Decisions
|pdfUrl=https://ceur-ws.org/Vol-1253/paper5.pdf
|volume=Vol-1253
}}
==Choicla: Intelligent Decision Support for Groups of Users in the Context of Personnel Decisions==
https://ceur-ws.org/Vol-1253/paper5.pdf
Choicla: Intelligent Decision Support for Groups of Users
in the Context of Personnel Decisions
Martin Stettinger Alexander Felfernig
Institute for Software Institute for Software
Technology Technology
Inffeldgasse 16b Inffeldgasse 16b
A-8010, Graz, Austria A-8010, Graz, Austria
mstettinger@ist.tugraz.at afelfern@ist.tugraz.at
ABSTRACT the decision can also be negatively influenced. Decision pro-
Group recommendation technologies have been successfully cesses are often not open in the sense that it is impossible to
applied in domains such as interactive television, music, and easily integrate new decision alternatives or change the in-
tourist destinations. Existing technologies are focusing on dividual preferences within the scope of a decision process -
specific domains and do not offer the possibility of support- both aspects can lead to low-quality decision outcomes (see
ing different kinds of decision scenarios. The Choicla group [13]). In many cases, the criteria for the decision remain
decision support environment advances the state of the art unclear since there is no explanation of the outcome of ”the
by supporting decision scenarios in a domain-independent final decision”. All these mentioned threats can negatively
fashion. In this paper we present an overview of the Choicla influence the quality of group decisions.
environment and exemplify it’s application in the context of One major goal of the Choicla environment is to facilitate
personnel decisions. group decision making and improve the overall quality of de-
cision outcomes. The idea of this environment is to support
definitions of different types of decision tasks in a domain-
Categories and Subject Descriptors independent fashion while taking into account the above
D.2 [Software and its engineering]: Software creation mentioned risk factors. In order to achieve this goal, Choicla
and management; H.5 [Information Interfaces and Pre- builds upon different group recommendation algorithms [11]
sentation]: Modelling Environments which are used for determining alternative solutions for the
participants of a group decision process.
General Terms One example of the application of Choicla is to support
Algorithms; Human Factors; Experimentation groups of users in context of personnel decisions with the
aim of achieving a more structured, fair, and transparent
way of job interviews as well as to find the most suitable
Keywords candidate for the job advertisement. Other typical scenarios
Recommender Systems, Group Recommendation, Group De- for the application of Choicla technologies are the decision
cision Making, Personnel Decisions about which restaurant to select for a dinner or - in a sci-
entific community - a decision regarding the selection of the
1. INTRODUCTION destination of next year’s conference.
Decisions in everyday life often come up in groups, for ex- The remainder of this paper is organized as follows. In
ample, a decision about the destination for the next holi- Section 2 we provide insights to (1) the Choicla modelling
days or a decision about which restaurant to choose for a process where participants can design decision tasks from
dinner. Knowledge about the preferences of other users in scratch and (2) the intelligent management of already cre-
early phases of a decision process can lead to sub-optimal ated decision apps. In the Section 3 we give an overview of
decision outcomes [12]. Missing explanations can lead to a the personnel decision scenario. We then discuss related &
lower level of trust in recommendations [2]. So-called an- future work (Section 4) and thereafter conclude the paper
choring effects [6] are responsible for decisions which are (Section 5).
biased by the voting of the first preference-articulating per-
son. If single persons have to take a decision in place of 2. CHOICLA DECISION SUPPORT
persons who are not available for a meeting, the outcome of Because decision scenarios differ from each other in their
process design, a variety of parameters is needed to spec-
ify all relevant properties of a decision task. We will now
discuss basic features (parameters) which can be configured
(modelled) by the creator of a decision task. In this context
we refer to the example features depicted in Figure 1.
2.1 Design of Decision Apps
IntRS 2014, October 6, 2014, Silicon Valley, CA, USA. Because decision scenarios differ from each other, some de-
Copyright 2014 by the author(s). cision scenarios rely on a preselected decision heuristic that
Figure 1: Choicla: definition of a decision task. Basic settings & further configurable features.
defines the criteria for taking the decision, for example, a If the scope is ”public”, the decision task is accessible for
group decides to use majority voting for deciding about the all users – this is typically the case in the context of so-
next restaurant or cinema visit. The design of decision tasks called Micro-Polls. The decision quality can be influenced
(the underlying process) can be interpreted as a configura- if the individual preferences of the other participants are
tion problem (see [17]). The achieved flexibility of making visible during the decision process (see [3] and [7]). There
the process design of a decision task configurable is needed exist decision scenarios where all participants profit from
due to the heterogeneity of decision problems. This way the the knowledge of who entered which rating. If, for example,
Choicla components are organized as a kind of a software the decision task is to find a date for a business meeting
product line that is open in terms of the implementation it is essential to find a date where all managers can attend
(generation) of problem-specific decision applications. the meeting and therefore it is important to know the indi-
vidual preferences of the participants. On the other hand
Explanations. Explanations can have an important role there are decision scenarios where full preference visibility
in decision tasks since they are able to increase the trust of can lead to disadvantages for some participants but some
users in the outcome of a decision process [2]. When design- kind of transparency of the individual preferences is helpful
ing a decision task in Choicla, explanations can be selected to achieve a reasonable decision. In such cases a summary
as a feature of the decision process. If this feature is selected, of all given preferences is a feasible way to support decision
the administrator of a decision task has to enter some ex- makers (participants). A summary prevents the participants
planatory text, if not, the entering of such a text remains from statistical inferences to the individual preferences but
just an option. still can help participants who are unsure about how to rate.
Administration of Decision Alternatives. The admin- Recommendation Support. In the context of group de-
istration of decision alternatives within the scope of a de- cision tasks, an essential aspect is the aggregation function
cision task can be supported in different ways. First, only (recommendation heuristic). In a group decision process ag-
the initiator of a decision task is allowed to add alternatives gregation functions can help to foster consensus. User stud-
– this could be desired if a person is interested in knowing ies show that these functions also help to increase the degree
the opinions of his/her friends about a concrete set of al- of the perceived decision quality (see, for example [3]). Indi-
ternatives (e.g., alternative candidates for the next family vidual user preferences can be aggregated in many different
car). Another related scenario are so-called ”Micro-Polls” ways and there exists no default heuristic which fits for every
where the initiator is only interested in knowing the pref- decision scenario. To provide a support for groups of users
erence distribution of a larger group of users. Second, in in different decision scenarios, the selection of recommenda-
some scenarios it is important that all decision makers can tion heuristics is a key feature which has to be configured
add alternatives during the decision task by themselves – by the initiator of a decision task. Due to space limitations
a common example of such a scenario is the group-based we only describe selected aggregation heuristics below. Mas-
decision regarding a holiday destination or a hotel [7]. In thoff [11] gives an overview of basic aggregation heuristics
such a context, each participant should be allowed to add such as Majority Vote (MAJ), Average Vote (AVV), Least
relevant alternatives. The support of group-based personnel Misery (LMIS), and Most Pleasure (MPLS) which are also
decisions can be seen as an example scenario of the third available in the Choicla environment.
case (only external users can add alternatives) – in this con- Group Distance (GD) (see Formula 1) returns the value d
text it should be possible that candidates apply for a certain as group recommendation which causes the lowest overall
job position (the application itself is interpreted as the addi- change of the individual user preferences where eval(u, s)
tion of a new alternative to the decision task). The selection denotes the rating for a solution s defined by user u.
of the next conference location where proposers can submit X
their material is another example. GD(s) = minarg(d∈{1..5}) ( |eval(u, s) − d|) (1)
u∈U sers
Preference Visibility. The scope ”private” allows only
invited users to participate, i.e., the decision task is only ac- Ensemble Voting can be seen as an example of a meta-
cessible for invited users and not accessible for other users. aggregation function included in Choicla. Ensemble Voting
(see Formula 2) determines the majority of the results of
the individual voting strategies H = {MAJ, AVV, LMIS, gets subjective. In such a case the assessment criteria of
MPLS, GD} where eval(h, s) denotes the result of an indi- the candidates change and no ”fair” and objective decision
vidual voting strategy for a solution s. can be made. Another important factor is that in most
[ cases personnel decisions come up in groups of users which
EN S(s) = maxarg(d∈{1..5}) (#( eval(h, s) = d)) (2) means that often more than one person is affected by the
h∈H
hiring procedure.
To prevent groups from unsystematic reviews, Choicla of-
2.2 Choicla Decision Apps fers a structured and fair way to evaluate candidates of a
After the design process has been finished, the creator of the
job position. Figure 3 shows the evaluation of the candi-
decision task as well as all invited participants (after accept-
dates in context of our working example (new receptionist)
ing the invitation) see a corresponding decision app directly
for a particular decision maker.
on the personal home screen (see Figure 2).
Figure 2: Choicla: Home screen of a registered user.
The symbols within the tiles trigger actions which
can be performed in the current state of the decision
app. Possible actions are (from left to right): con-
figuration, evaluation (only possible if the decision Figure 3: Choicla: example of individual ratings.
app is publicly available over the store), and delete. Each user can take a look at the current recommen-
The tab DecisionApp Store contains publicly available de- dation and adapt his/her preferences if needed.
cision apps which can be searched and installed on the per- To keep the screen understandable, only the line with the
sonal Home Screen. This method prevents a creation from aggregated information of a candidate is visible - by clicking
scratch every time for frequent decision tasks such as, for ex- on this line, several dimensions including their actual rat-
ample, scheduling decision tasks. In such a case the decision ings show up for the corresponding candidate (only visible
process can be triggered right after the download of a deci- for first candidate in Figure 3). In order to avoid misun-
sion app. This reuse technique has the potential to reduce derstandings in context of evaluation the sliders of the first
the entry barrier for using Choicla and keep the interaction candidate are automatically displayed if the screen is loaded.
simple – especially for people who want to start a decision Due to the fact that depending on the advertised job posi-
process quickly. The tab Create DecisionApp allows a user tion different assessment criteria are needed, the dimensions
to design a completely new decision app from scratch. on which a candidate can be evaluated can be chosen by
Due to the fact that many decision tasks occur regularly – the creator of a decision task. If we look at the example in
for example, a group of friends go for dinner once a month Figure 3 we can see that for the ”New Receptionist” the di-
– a concept is needed to manage a potentially large num- mensions English skills, Communication, Friendliness, and
ber of decision tasks. To keep the potentially large number Punctuality are chosen.
of decision tasks manageable, every decision app consists of In situations where there are candidates for whom not all
a variable number of instances. A concrete instance of a criteria (dimensions) have been evaluated or there exists a
decision app can be accessed within the corresponding de- discrepancy between individual evaluations, special mark-
cision app - all instances of a concrete decision app will be ers are used to point out open issues. This approach cre-
loaded when the decision app is opened. The created in- ates need for closure (see, e.g., [15]), i.e., users are addition-
stance of the example depicted in Figure 1 is accessible in ally motivated to make the candidate evaluations complete
the ”Personnel-decision” app (see Figure 2). This mecha- and consistent.
nism offers the possibility of an exact documentation of all If a candidate should be excluded from the application pro-
past decisions and is also a basis for supporting recurring cedure in early phases (e.g., some criteria are not met), this
decision tasks. can be achieved by using the ”Manage Candidates” button (a
new menu shows up). The early exclusion of an unsuitable
3. CHOICLA PERSONNEL DECISIONS candidate supports more clarity since only the ”relevant”
3.1 Users View candidates are displayed.
Personnel decisions are often influenced by various factors. The tab Group Preference presents the current group rec-
Such factors are, for example, if a candidate has physi- ommendation, after a predefined number (the threshold) of
cal handicaps, in most cases no concrete structure is fol- participants articulated their preferences. This threshold
lowed during the job interview and the evaluation often prevents from statistical inferences to the individual pref-
erences of other participants (only in combination with a mation by a single person - in most cases a secretary - to
”private” decision scope - see Section 2). The group rec- the applicants.
ommendation in context of personnel decisions is based on
the MAUT-principle (multi-attribute-utility-theory [1]). A
group recommendation based on the MAUT-principle (see
4. RELATED & FUTURE WORK
There exist a couple of online tools supporting decision sce-
Formula 3) returns the average value of all individual MAUT
narios. Rodriguez et al. [16] describes a system called Smar-
values of all participants as group recommendation for one
tocracy. Smartocracy is a decision support tool which sup-
candidate (solution s). A group member’s individual MAUT
ports the definition of tasks in terms of issues or questions
value represents the weighted average of all personal ratings
and corresponding solutions. The recommendation (solu-
of the dimensions of an alternative. This means that the at-
tion selection) is based on exploiting information from an
tribute values are subjective and the weights are fixed which
underlying social network which is used to rank alternative
is different in a typical MAUT scenario.
P solutions. Dotmocracy 1 includes a method for collecting and
X d∈s eval(u, d) ∗ weight(d)
visualizing the preferences of a large group of users. It is
M AU T (s) = (3) related to the idea of participatory decision making – it’s
u∈U sers
|dimensions|
major outcome is a graph type visualization of the group-
If we look at the individual ratings in Figure 3 we notice the immanent preferences. Doodle2 is an internet calendar tool
values 8, 5, 8, and 5 for the dimensions. For simplification with the focus on coordinating appointments. VERN [19]
purposes we assume in our example that all dimensions have is (very similar to doodle) a tool that supports the identi-
the same weight (wd1 = wd2 = wd3 = wd4 = 5). Due to fication of meeting times. VERN is based on the idea of
Formula 3, the individual MAUT value for the actual user unconstrained democracy where individuals are enabled to
of the first alternative is 32.5. To present the evaluation of freely propose alternative dates themselves. A major ad-
a solution (candidate) within a five star scale, these values vantage of Choicla 3 compared to these tools is that users
have to be normed. of Choicla are able to customize their decision processes de-
pending on the application domain and can also focus on
specific tasks. Furthermore, the mentioned tools provide no
3.2 Candidates View concepts which help to improve the overall quality of group
All previous described options and screens can only be ac- decisions, for example, in terms of integrating explanations,
cessed by the decision makers of the decision task itself and recommendations for groups, and consistency management
can of course not be seen by the applicants of the job posi- for user preferences.
tion. During the design phase of a decision task the input Recommendation approaches in the line of Choicla are also
fields (e.g., name, age, and application text) which are then presented in Sangeetha et al. [8] and Malinowski et al. [10].
visible by the applicants during the application process can Sangeetha et al. [8] introduce recommendation approaches
be defined. Figure 4 shows the view of an applicant in our that support people-to-people recommendation (detection
running example ”New Receptionist”. of latent relationships between similar users) whereas Ma-
linowski et al. [10] discuss approaches (based on fitness
measures) that support the pre-selection of candidates for
existing teams (groups). In contrast, Choicla focuses on
supporting a group decision where parameters such as the
fit of a candidate with an existing group are represented in
terms of MAUT dimensions.
Our future work will focus on the analysis of further applica-
tion domains for the Choicla technologies. Our vision is to
make the design (implementation) of group decision tasks as
simple and straightforward as possible. The resulting deci-
sion task should be easy to handle for users and make group
decisions in general more efficient. Our focus will also be
on the analysis of decision phenomena within the scope of
group decision processes. Phenomena such as decoy effects
[5], [18] and anchoring effects [6] have been well studied for
single-user cases, however, in group-based decision scenarios
no studies have been conducted.
Biases can be induced if a system is open in the sense that
new decision alternatives can be added during the decision
process. However, such a feature is imperative in cases where
all possible decision alternatives are not available from the
Figure 4: Choicla: example of the entering of appli- beginning. The group preferences can also be influenced by
cation data. Each applicant can insert his/her per- the order of the incoming individual preferences due to the
sonal data needed for the advertised job position. fact that the participants of a group will perceive already
selected alternatives more attractive than new options [14].
All the added information of the candidates is then prepared
1
and accessible for the decision makers during the assessment dotmocracy.org.
2
phase - see Figure 3. This way of adding solutions to a de- doodle.com.
3
cision process shifts the burden of entering candidate infor- www.choicla.com.
If consensus out of discussion is reached in early phases, lit- of Consumer Research, 9(1):90–98, 1982.
erature shows that this consensus is cognitive resistant to [6] K. Jacowitz and D. Kahneman. Measures of
changes. That means that additional information which is Anchoring in Estimation Tasks. Personality and Social
added later in a decision process will be adapted to already Psychology Bulletin, 21(1):1161–1166, 1995.
defined consensus and due to this it is very unlikely that [7] A. Jameson. More than the sum of its members:
another alternative is chosen [9]. Such a phenomenon can challenges for group recommender systems. In
be explained by the assimilating effect which is ascribable Proceedings of the working conference on Advanced
to the dissonance theory [4]. The assimilating effect states visual interfaces, AVI ’04, pages 48–54, New York,
that individuals are motivated to reduce psychological in- NY, USA, 2004. ACM.
congruity or discrepancy that is very likely to arise if new [8] S. Kutty, L. Chen, and R. Nayak. A people-to-people
information is added to a present perception [14]. A high recommendation system using tensor space models. In
group cohesion intensifies this effect, because within such a Proceedings of the 27th Annual ACM Symposium on
group the fear of exclusion is higher (see [9]). Future versions Applied Computing, SAC ’12, pages 187–192, New
of Choicla will reduce this effect by providing a special way York, NY, USA, 2012. ACM.
of preference visibility which, for example, only shows the [9] E. Lind, L. Kray, and L. Thompson. Primacy effects
preferences of other users for those participants who com- in justice judgments: Testing predictions from fairness
pleted their individual ratings of the alternatives. Another heuristic theory. Organizational Behavior and Human
research direction in this context is if such mechanisms can Decision Processes, 85(2):189 – 210, 2001.
increase the willingness of participants to articulate their
[10] J. Malinowski, T. Weitzel, and T. Keim. Decision
real preferences. A further issue for future work is to figure
support for team staffing: An automated relational
out which group recommendations help to achieve consensus
recommendation approach. Decision Support Systems,
more quickly. Finally, we will develop further group recom-
45(3):429 – 447, 2008. Special Issue Clusters.
mendation heuristics which help to achieve a high level of
fairness (in the long run). [11] J. Masthoff. Group Recommender Systems:
We want to emphasize that one of our major goals is to make Combining Individual Models. Recommender Systems
the Choicla datasets available to the research community in Handbook, pages 677–702, 2011.
an anonymized fashion for experimentation purposes. [12] A. Mojzisch and S. Schulz-Hardt. Knowing other’s
preferences degrades the quality of group decisions.
5. CONCLUSIONS Journal of Personality & Social Psychology,
98(5):794–808, 2010.
In this paper we gave a short introduction to Choicla which
supports the flexible design and execution of different types [13] E. Molin, H. Oppewal, and H. Timmermans. Modeling
of group decision tasks with a focus on personnel decisions. Group Preferences Using a Decompositional
With the help of Choicla it is possible to achieve more trans- Preference Approach. Group Decision and
parent, fair, and structured personnel decisions. Compared Negotiation, 6:339–350, 1997.
to existing group decision support approaches, Choicla pro- [14] M. Neale, L. Ross, and J. Curhan. Dynamic
vides an end user modelling environment which supports an Valuation: Preference Changes in the Context of
easy development and execution of group decision tasks. We Face-to-face Negotiation. Journal of Experimental
also discussed further research directions which can help to Social Psychology, 40(2):142–151, 2004.
extend the available functionality of the Choicla environment. [15] G. Ninaus, A. Felfernig, M. Stettinger, S. Reiterer,
G. Leitner, L. Weninger, and W. Schanil. Intelligent
6. REFERENCES techniques for software requirements engineering.
[1] J. Dyer. Maut - multiattribute utility theory. In European Conference on Artificial Intelligence,
Multiple Criteria Decision Analysis: State of the Art Prestigious Applications of Intelligent Systems (PAIS),
Surveys, volume 78 of International Series in to appear 2014.
Operations Research & Management Science, pages [16] M. Rodriguez, D. Steinbock, J. Watkins,
265–292. Springer New York, 2005. C. Gershenson, J. Bollen, V. Grey, and B. deGraf.
[2] A. Felfernig, B. Gula, and E. Teppan. Smartocracy: Social networks for collective decision
Knowledge-based Recommender Technologies for making. In HICSS 2007, page 90, Waikoloa, Big
Marketing and Sales. International Journal of Pattern Island, HI, USA, 2007. IEEE.
Recognition and Artificial Intelligence (IJPRAI), [17] M. Stettinger, A. Felfernig, G. Ninaus, M. Jeran,
21(2):1–22, 2006. S. Reiterer, and G. Leitner. Configuring decision
[3] A. Felfernig, C. Zehentner, G. Ninaus, H. Grabner, tasks. Workshop on Configuration, Novi Sad, pages
W. Maalej, D. Pagano, L. Weninger, and 17–21, 2014.
F. Reinfrank. Group decision support for requirements [18] E. Teppan and A. Felfernig. Asymmetric dominance-
negotiation. In L. Ardissono and T. Kuflik, editors, and compromise effects in the financial services
Advances in User Modeling, volume 7138 of Lecture domain. In Commerce and Enterprise Computing,
Notes in Computer Science, pages 105–116. Springer 2009. CEC ’09. IEEE Conference on, pages 57–64,
Berlin Heidelberg, 2012. July 2009.
[4] L. Festinger. A Theory of Cognitive Dissonance. [19] S. Yardi, B. Hill, and S. Chan. VERN: Facilitating
Stanford University Press, June 1957. Democratic group Decision Making Online. In
[5] J. Huber, J. Payne, and C. Puto. Adding International ACM SIGGROUP Conference on
Asymmetrically Dominated Alternatives: Violations of Supporting Group Work (GROUP 2005), pages
Regularity and the Similarity Hypothesis. The Journal 116–119, Sanibel Island, Florida, USA, 2005. ACM.