Web server logs have been used via techniques such as user profiling and recommendation systems to improve user experience on websites. The data contained within server logs however has generally been inaccessible to nontechnical stakeholders on website development projects due to the terminology and presentation used. We describe a process that uses visualisation to enable these stakeholders to identify questions about site usage including user profiling and behaviour. The development of this tool utilising Web 2.0 technologies is described as well as feedback from the first stage of user evaluation on a realworld multi-national web development project called e-Bug. The potential for this process to elicit user attributes and behaviour that can be incorporated into automated user profiling systems is also discussed.
Research into online user behaviour has been aided by the relative ease of collecting
feedback data using implicit methods such as web server logs [
This has had a dramatic impact on the user experience e.g. Amazon [
There have been several commercial attempts (Google Analytics2, Sawmill3,
WebTrans4) that have tried to make server logs, and therefore user behaviour, more
accessible to site owners. However, these applications analyse generic features of
sites that do not answer specific questions that certain stakeholders will have and do
not help them identify trends in user behaviour due to the sheer volume and technical
nature of the information presented [
A potential solution to this problem is to use techniques from the field of Software
Visualisation (SV) to make the data contained within server logs more accessible to
non-technical stakeholders in website development projects. Using these methods
utilises the innate pattern matching ability [
This paper describes the process of developing and using visualisation techniques to disseminate site usage information to non-technical stakeholders, in order to identify potential attributes for user profiling and recommendation systems. An ongoing multinational project in e-Health, called e-Bug (www.e-bug.eu), has been used as a test-bed and feedback from project stakeholders is detailed. The future possibilities of this technique are discussed as well as general implementation issues from using Web 2.0 technologies.
Visualisation is concerned with making large amounts of information more
comprehensible for the user by using a visual representation. Software Visualisation
has been successfully used by software engineers to “make software more visible”
[
Currently the server logs from the e-Bug project are analysed using a proprietary application called Sawmill. This produces standard reports that cover information such as visits, hits, content viewed, visitor demographics and systems and referrers. These reports are produced monthly and uploaded onto the e-Bug website7. It was found however that although the project partners expressed a high degree of interest in the website statistics during meetings, the format that the reports were currently in were not easily accessible to non-technical users. This was mainly due to the terminology used and the statistics presented not answering specific questions that the project partners had regarding the users of the site [D. Farrell 2009, pers. comm.]. It was decided therefore that the server logs from the e-Bug project website would make a suitable test-bed to use visualisation techniques to analyse and present the statistics in a way that reduced the confusion and elicited potential attributes for user profiling. 3
A User Centred Methodology (UCD) [
Sketching has been used previously to create code visualisation software [
At this stage two potential metaphors were identified: a weather map metaphor and a timeline metaphor. After discussion with members of the project team it was decided to begin by developing the weather map metaphor as this would support one of the main features that was missing from the current reports: accurate geographical distribution of the users of the site. 3.1
Having identified possible interface designs for the application, an online prototype system was developed and suitable technologies explored for creating the map metaphor. The following figure shows the first version of the prototype.
The interface incorporates two main visualisations. An area on the left hand side of the screen that shows the number of visitors and page views in a particular year and month (and their daily distribution) using simple bar charts. The area on the right contains a map with individual visitors denoted by their location with a marker (in this case the e-Bug logo). Users can select particular months from the drop down menu on the left and navigate the map using the navigation icons and the mouse pointer.
The map was created using the Google Maps API, which uses JavaScript to make asynchronous calls (AJAX) to display the map and the markers. The data for the markers is stored in an XML file that is generated by a PHP page parsing a CSV file that is created using Sawmill8. The CSV file contains paired values of a users’ hostname and the number of page views that came from that IP address. PHP is then used along with the GEOIP Lite Open Source reverse geolocation database9 to calculate a longitude and latitude for each hostname. These are then saved in an XML file in the following format: <marker lat="40.6333" lng="-7.8333"/>
The bar charts were created using the Google Charts API, which creates dynamic images based on parameters passed in the querystring, for example: <img src=”http://chart.apis.google.com/chart?cht=p3&chd=t:60 ,40&chs=250x100&chl=Hello|World” alt=”Bar chart” /> The parameters were determined using PHP pages and CSV files that contain monthly and daily totals of visits and page views.
This prototype was then uploaded to the e-Bug website and feedback was elicited from members of the e-Bug project team from seventeen European countries, as well as researchers involved in similar projects at UK Universities as part of the UCD process. The evaluation was in the form of an email with a set of open-ended questions that respondents were asked to answer regarding the interface. The main focus of this exercise was to ascertain whether the information that was being represented was clear enough, whether appropriate metaphors were being used and also whether there were any other statistics that users would be interested in. As this is an ongoing project, feedback has so far been received from nine respondents.
The majority of respondents reacted positively to the interface and the visualisation and a number of them were able to give detailed feedback, indicating that they were able to understand what the page was showing and what it did not. The main recurring points from this feedback are detailed below: Add representation that shows “magnitude of visitors” as it is difficult to gauge repeat visitors, number of pages viewed and markers that overlap. 8 The data from Sawmill was used rather than the raw server logs due to the fact that Sawmill filters out certain web crawlers as well as using custom filters that have been created to remove certain IP addresses. 9 http://www.maxmind.com/app/geolitecity Add specific place markers to the map that do not appear (unless at a higher zoom level). Add specific evaluation areas/overlays onto the map10. Show the density of visitors in each area i.e. show visitors per 100,000 population to get more meaningful comparisons. Add in a view of popular pages downloads and where they originate from. Highlight returning visitors. Add in a view that shows the times of day that various pages are being accessed e.g. if the games are being viewed outside of school hours this could indicate that students are playing them at home. Ability to compare months and countries.
One of the most interesting points noted by the stakeholders however was the fact that the data being represented itself is a potential area of confusion. For example, a number of users gave the general impression that they did not know the difference between a visitor and a hit. It became clear that the target users of this application do not posses the same knowledge that experts in the field take for granted and further investigation into this area is being conducted.
Following on from this, a second version is currently being developed to take into account the feedback and also to tackle some of the issues that have been raised with regards to the interface and the information that users would like displayed. A screenshot(s) from the second iteration of the software is shown below:
As well as markers and statistics for visitors, information regarding pack downloads (educational resources for teachers in Word and PowerPoint files) has been included and split into “Junior” and “Senior” versions.
This version of the application also uses an updated visitors’ visualisation that takes into account the number of page views from a particular users. The well-known temperature scale visualisation used on weather maps has been utilised to be able to differentiate between the levels of activity in various regions. 10 this can potentially be achieved using the Google Polylines’ API
Initial feedback has already indicated that visual representations of the data have allowed the non-technical stakeholders in the project to start to identify user types and user behaviour. One particular interest is whether pupils are accessing the games pages at home or at school and whether the tool can identify whether it is a student viewing the website or a teacher. By geographically representing visitors in relation to the location of target schools, along with the time they are accessing the site can potentially achieve this simple user profiling task.
This example and others detailed in Section 4 indicate that providing non-technical stake holders with a visual representation of the server logs has allowed them to communicate requirements for further analysis which can either be integrated into the filters used in the Sawmill application or into the visualisation tool. Without the use of visualisation techniques, it is doubtful that these questions regarding the users of the site and their behaviour would have been raised.
Further investigation of user profiles and understanding of national profiling differences is a subject of our ongoing research.
There are a number of advantages with using Web 2.0 technologies such as the various Google API’s and AJAX such as being able to create richer and more interactive online interfaces but the main advantage relates to being able to utilise users’ pre-existing skills and experience. The majority of users have prior experience with interfaces such as Google Maps and in the same way that the Desktop has become the standard metaphor used for operating systems, maps and markers and the various methods of interaction that Google has developed have become a standard in this area. Being able to “piggy-back” on to that frees the user from the interface and allows them to focus on the visualisation, even though this application is a bespoke solution.
An associated advantage is that Google is a global organisation and so is its
software. The potential users of this software are from a diverse set of countries with
a number of different languages and levels of expertise. With Google being even
more popular in Europe than the US [
The other advantage is the increased speed in development. Being able to harness pre-existing API’s allows for rapid prototyping and the ability to demonstrate a working concept to users to elicit feedback almost immediately and also allows for faster changes and incremental versions.
Finally, the fact that Web 2.0 technologies are also designed to be accessible via a number of different browsers and platforms also allows for speedier access and dissemination of the information which is vital for cross-nation projects such as eBug.
One of the main problems with the Google Maps marker metaphor is the problem of occlusion, something that is common when using 3-D visualisations. If a user visits the site numerous times or downloads numerous pack or pages it is difficult to represent that with numerous markers on the map as they will overlap with one another. This can partially solved with the colour coding of markers but the accuracy of the geolocation database and the fact that numerous visitors can originate form the same area means that the markers often overlap. To improve this a method for clustering the markers so that close “neighbours” are represented by one marker and for this information to be presented textually once a user clicks on a clustered marker are being investigated.
A related problem is the amount of data that can be represented using these tools and the limitations of the browser. During testing of the application it was found that once around five thousand markers were placed on the screen using the standard Google method, the browser would slow down and become unusable. For this prototype this problem was solved by filtering out duplicate markers and also nonEuropean hits (as this was not required at this stage in the site’s development). However once the site it is launched and publicised further this year, there will be an increase in visitors and therefore an increase in markers. Clustering methods are therefore currently being investigated.
One final problem that was highlighted from user feedback was that relying on users having had prior experience on Google Maps means that for those who have not, or those who do not realise that this is a Google Maps interface, have initial problems with the interface. Adding extra methods of navigation or instructional video/instructions are currently being piloted.
The process of identifying appropriate visualisations to allow non-technical users to start to identify site usage from server logs is important for successful web site development and evaluation. The process presented in this paper has provided a number of insights into the potential of using Web 2.0 tools and metaphors for visualisation and dissemination of information. Although at an early stage, the tool is already providing insights into a number of usage patterns on the site which are enabling non-technical stakeholders of the e-Bug project to start to identify distinct user profiles and most importantly to start to be able to utilise the data stored in server logs more readily.
Future work will include an investigation into pre-existing taxonomies that exist of
software visualisation [
Following on from this, the tool will be used in an investigation into user behaviour on the e-Bug website in order to see whether researchers can identify usage trends visually and what are the attributes of these trends e.g. time of day a user visits plus geographical location might indicate whether they are a pupil or a teacher. This will then feed directly into the development and tailoring of content for the site and the potential for incorporating this into an automated profiling system will be investigated.