User eXperience (UX) is a well known term, but there is a divergence of understanding its meaning between UX professionals and end users. Even among UX professionals definitions and attributes vary from source to source. Therefore it is not surprising that the importance of the different UX attributes between UX professionals and end users also vary greatly. The differences are even bigger, if the personal characteristics are taken into account. This might lead to a situation where the UX designer does not know what an end user of the software would like to have, also vice versa; an end user does not necessarily appreciate what the UX designer has done. This problematic situation forms the basis of the currently ongoing study and the preliminary results from the conducted UX surveys are presented here.
Despite user experience (UX) is an important aspect in
virtually every modern day business activity and its
importance has been acknowledged by scholars around the
globe, general consensus about the meaning and
understanding of it, is still missing [
Results presented in this paper are based on a two phase survey study. First part was conducted with 20 UX professionals, which were chosen by browsing IEEE and ACM digital libraries as well as the Google Scholar with a keyword ‘user experience’. The second part; end user survey was sent to about 15000 university students. Accurate amount of students who received the survey cannot be given since emailing was done by the universities IT support and the submission list was automatically generated from the students.
End user responses were received from the Lappeenranta
University of Technology and the University of Oulu.
Other Universities in Finland were also asked to participate
but those did not participate due to different reasons. UX
professionals and end users answered to a virtually same
survey, but UX professionals had more open questions and
a possibility to suggest new UX attributes for the survey.
The professional survey form the qualitative part of the
data and end user survey form the quantitative part. Later
on the qualitative part will be enhanced with professional
interviews. The results gained from the conducted surveys
are compared against each other and also against the result
gained by the authors of [
Table 1 presents the common characteristics of the current respondents of our surveys. End users are from multiple different disciplines like IT, business administration, energy, environment, chemistry, medicine, economics and humanities. • • • •
Most of the end user respondents (72%) are from 20 to 29 years old and 78% of end users have 3-4 level computer ability. Multiple different nationalities were found from the responses, but 95% of all end user respondents are Finnish so the effect of nationality cannot be evaluated, this is also true for professional respondents, 50% are Finnish. Ability to use computers was divided from very poor (1) to a rock solid professional (6). Levels three and four, which contain the majority of the respondents, were defined as follow: 3 - I can use, install and update programs 4 - I can develop / maintain minor programs, web sites, etc.
Despite large amount of the respondents are 20-29 years old Finnish, this sampling should give somehow reliable picture of what the most potential end users of common applications appreciate. In the future, the effect of nationality and other age groups, like under aged, will be taken into account and those results will be compared with the present ones.
The conducted surveys consist of three different categories;
personal characteristics, UX definitions and UX attributes.
Characteristics are all commonly studied in the field of
social sciences and psychology e.g. [
Second part of the survey repeats the UX definitions part
from the baseline survey, but with a modest differences.
One definition was dropped out since it was too closely
related to a company and its’ services and products (D1 in
a baseline survey). Some definitions on the other hand
were added to the survey, e.g. the new ISO 9241-210
definition [
Last part of our survey is using a 5-point evaluation scale, where 1 is the best and 5 is the worst. This same classification is also used in the presented tables and figures. The task was to evaluate 21 plain UX attributes in addition with a possibility to answer “I don’t know” or “I don’t understand”. Respondents were asked to: “Evaluate the importance of the following UX attributes to you when using the software”. Earlier in the survey they were asked to pick some software that they should think while answering to the questions. With this procedure respondents had a possibility to approach UX attributes from “What is best for me?” point of view. This had hopefully led to more reliable results than asking only about the importance of general UX attributes.
Clause formatted UX statements from the baseline survey were left out since those were considered to be too UX designer-oriented for end users to understand and answer properly, for instance, clause like “We cannot design UX, but we can design for UX”. The conducted surveys can be found in the following links:
1,33 1,58 1,33 1,33 1,67 1,75 1,58 1,58 Professional s avg. • https://www.webropol.com/P.aspx?id=356910&cid=5 6046243 (UX professional) • https://www.webropol.com/P.aspx?id=404764&cid=1 17863430 (End user) 1436 end user responses were given to the survey, but 76 responses were dropped out from the final analysis. Most common reason for a rejection of an answer from these analyses was a row of empty values. Second reason for a rejection was a row of I don’t know / I don’t understand values. As an example this rate for aesthetics in all answers was 17,9% and 9,4% for user interface.
Results presented here do not go into details. Thorough results will be published after in depth analyzing.
The most important finding from the definition part is that
end users do not seem to appreciate definitions (over 25% I
don’t care answers). This finding is also reinforced by a
quite equal amount of support for every listed definition
among end users. UX professionals included in this survey
on the other hand were fairly consent about the same two
definitions, d3 and d4 (over 70%) as the respondents in the
baseline survey (50%). It is also noteworthy that d5
remained without support in baseline survey as well as in
our professional survey. Baseline survey also showed that
there are great differences between UX professionals in
academia and in industry [
If the averages of top rated attributes from UX professionals are compared against the same attribute averages by all end users, differences are notable as Table 2 presents.
On the other hand if top rated attributes by end users are compared against the same attributes from professionals, differences are fairly small as can be seen from Table 3. Results show that end users do not necessarily agree with UX professionals in all attributes but professionals seem to agree with end users. Generally it seems that UX professionals consider the whole picture which includes environmental and emotional aspects as well, while an end user shows more interest towards something concrete like the actual device or software and its properties.
In the baseline survey [
First if the average of all attributes among all respondents was compared, difference is only 0,24 (professionals 2,15, end users 2,39), so from that viewpoint divergence is not an issue. When we moved on to more detailed results, differences were found.
End user results between males and females are close to each other when the average of all 21 attributes is considered (2,38 vs. 2,40). If same comparison is done to UX professionals, results are (males 2,32 vs. females 1,98), but only three male respondents are included so the result 23,75% 21,25% 17,00% 12,75%
9,00% 7,25% 3,75% 3,00%
End users avg. 2,28 2,43 2,01 1,84 End users avg. 1,31 1,46 1,43 1,46 might not be reliable. When attributes are considered individually, interesting differences can be found. End user male seems to be more oriented towards UI and interactive features than end user female, but in UX experts survey the results were opposite as Figure 1 shows. In figure (e) means end user and (p) means professional.
Males
Females
User interface (p) When age is used as the divider, again the average of all end user attributes is almost the same for all age groups (2,33-2,42). Divergence among professionals is a bit larger (1,98-2,19). Interesting and linear differences are found when individual attributes are considered. Older respondents in both groups seem to appreciate general usability and user interface attributes more than younger respondents. Younger respondents on the other hand in both groups are a bit more positive about pleasure and coolness than older ones. Clearly age is an important affecting factor regardless of the experience in UX. As the final part the effect of ability to user computers is studied. Respondents were asked to categorize themselves with a 6-step evaluation, where 1 was very poor and 6 was a rock solid professional. Average behaves the same way as before and similar linear effect as in the age was founded in some attributes. As an example, interaction and environment are presented in figure 1. In both cases the upper and longer line is the end user graph and the lower one is the UX professional graph.
3,5 n itco 3 a r ten2,5 if eo 2 c n tra1,5 o p Im 1 <=2 tn3,5 e m ron 3 i v en2,5 e fsu 2 o ce1,5 n a t ro 1 3 4 5 6 Ipm <=2 3 4 5 AbilFityigtouursee c2o:mEpfufteerct of ability to uAbsielityctoo muspeuctoemrputer 6 In some attributes UX professionals and end users agree, but in some the opinions seems to be opposite.
The results of the conducted surveys clearly present the divergence of understanding UX matters between UX professionals and end users. When the personal characteristics of end users are taken into account, the divergence of results is even more amplified. Even among UX professionals there are big differences in answers when their personal characteristics are considered.
Need for clarification of different areas of UX exists and not only from the UX professional viewpoint but from the common end user viewpoint as well. This clarification makes it possible to design and implement better end user experience for devices, software, etc.
Information collected from the conducted surveys is used to create an UX database. This database in co-operation with an UX tool under development will offer a simple way for a software developer to focus on those UX attributes that the selected target group considers as important. UX database with the UX tool will be presented in a future paper.