Annotation of online documents has become an important practice, allowing the addition of valuable information to existing resources without corrupting them, thus introducing a novel way of user collaboration. The MADCOW annotation system provides services for annotating multimedia contents in HTML pages. Introducing groups with their related services to the MADCOW system enhances the security and privacy of annotations for group members as well as their collaboration. Usability problems arose with respect to the identi cation of signi cant matches between users' interests and group topics, to support the process of admitting users to groups. Therefore, the use of matching methods based on ontologies and URL comparison enhances the joining process with respect to time, e ort and relevance. This research illustrates our current work for introducing groups to the MADCOW system, and inspects the Groups-Users relevance problem by analyzing and implementing relevance measurements.
The MADCOW1 system is a tool enabling its users to annotate contents (texts,
images and video clips) of Web pages with textual annotations [
Annotations published to MADCOW are saved in a dedicated server and can be of three types:
Collaboration among users is enhanced by public annotations while privacy
and security are enhanced by private ones [
The MADCOW system supplies group owners with capabilities to manage their groups and to extend some of these capabilities to group moderators (group owner and moderators are collectively called authorisers ). They are able to create, update and delete their groups, and to select one of three di erent policies to allow users to join their groups, i.e. to become members :
2. Invite: users can join the groups only if invited by some group member with authoriser status. 3. Apply : users apply for joining the group, and admission is subject to approval by an authoriser.
A user invited to a group is given reader or writer privileges. A reader can only read annotations. A writer can both read and submit annotations.
Further operations allow a ner management of group composition. Atomic operations for management of group members and of their annotations have been formalized and used to describe more complex ones. In particular, group owners may execute the Extraction, Union, Intersection and Subsumption operations.
Originally, only a manual process was available to allow users to join a group: a group authoriser had to select from the list of all MADCOW users the set of users to be invited, while a user had to search for groups to send join requests to their authorisers.
Eight undergraduate classes from di erent disciplines, for a total of 152 students, have been involved in a supervised test which lasted for 15 days. Di erent Human Computer Interaction metrics (such as understandability of individual steps and interface ease of use) have been tested, while the time needed to Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
complete each operation was collected in the background. Table 1 presents information about the number of times the di erent operations on groups were used and on the average time needed to complete an operation. 1. Time-E ort : The manual process is a tedious operation that consumes a lot of time (99.25 sec. as average invitation time from Table 1) and e ort, especially with a huge number of users. 2. Irrelevance: Group authorisers have no idea about "who are the interested users to invite", and users have little indications about "which groups to join".
An automatic groups-users suggestion process has therefore been introduced. Authorisers are presented with the set of those users whose public annotations are mostly related to the group topics and could likely be interested in becoming writers. The same technique is used to suggest groups for users, so that they are presented with those groups that could interest them. Ontology -based and URLbased matching processes have been used to implement a group-user matching after de ning ontological domains in the system, so that each domain is referred to some ontology and is the possible core topic for a set of groups.
Class Match Measure (CMM) presented in [
This research represents a Ph.D. work supervised by Prof. Paolo Bottoni at the Department of Computer Science at Sapienza University of Rome, Italy. 2
Several tools allow creation of annotations and collaboration between annotators in various domains, but few of them deal with groups, mostly supplying public annotations only. Among those which do o er groups, little support is given for group management, making the transfer of annotations among groups di cult. Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
The Diigo Toolbar2 lets users create their own groups and invite other members, and provides some services for groups archiving and dissemination. A.nnotate3 takes a snapshot of a document, Web page or image, to produce a read-only copy, which can be annotated and shared with other users, with some approximation to a sort of grouping. In Bounce4 a user can produce notes for snapshots of Web pages and collaborate on them with other users.
Considering group operations, some work has been done in the context of
classifying documents according to user annotations [
Ontologies as representatives of knowledge were used in several domains.
Paralic and Kostjal [
Several works deal with measuring the relevance of an ontology to a collection
of terms. In [
MADCOW enhances the annotation process by both introducing groups and
its related services as well as solving the problem of group-user matching by
combining ontology-based [
My Ph.D. project is concerned with analyzing and implementing all MADCOW group-related services in order to enhance the group joining process, ensure security and privacy, and support collaboration for group members. Currently, our work is focused on enhancing the process of joining groups by automating the suggestion of users to groups' authorisers and of groups to users. 3.1
The work on groups in MADCOW began by implementing their preliminary services (Adding, Updating and Deleting Groups) as well as: 2 http://www.diigo.com/ 3 http://a.nnotate.com/ 4 http://www.bounceapp.com/ 5 http://jena.sourceforge.net Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
{ Group Management: handling of invitations and applications to groups, appointment/removal of moderators, blacklists of group writers, which prevent them from adding new annotations until their removal from the blacklist, de nition of restrictions on display of annotations in a group, listing all public groups in the system, and ordinary search for groups by their titles. { Atomic Groups Operations: creation of new groups could be done manually by any user, or could be the result of atomic operations (except for subsumption). Considering groups with their contained members as sets of elements, the MADCOW system provides the following collection of setbased operations6 (depicted in Figure 2): 1. Extraction: creation of a new group from an existing one, while keeping the original one unchanged. 2. Union: creation of a new group with all members from two existing ones, with the option to delete or keep the original ones. 3. Intersection: creation of a new group with members in common between two existing ones. 4. Subsumption: deleting a group whose members are all members of another one, absorbing their annotations in the latter.
A detailed description for all related groups management is given in [
Our e orts now are directed to solve the problems (time, e ort and irrelevance) of the group joining process. We propose an automatic matching process between users and groups, so that groups' authorisers are presented with a ranked list of the most relevant users, and users are presented with a ranked list of the most relevant groups. An ontology- and a URL-based matching processes are proposed to automate groups-users suggestions.
Current work (with its related terms, concepts, methodology) is described
in [
Ontology-Based Matching Before discussing the whole process, we introduce some de nitions to illustrate the role of ontologies in the process of groups-users matching: 1. Domain: a unique name designating the area of knowledge with which an ontology is associated. 6 Special considerations are implemented to manage members' privileges and annotations before and after each operation.
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
(a) Extraction (b) Union (c) Intersection (d) Subsumption 2. Concept: the name associated with a node in some ontology. 3. Terms: lexicalisations of the concepts in some ontology. 4. Tags: words (possibly terms in some ontology) provided by a user to characterise the annotation content.
Domains-Ontologies Association Simply, the idea relies on using ontologies (as a universal source of knowledge) to represent sets of related groups, and executing a match between users' annotations (public and private) and concepts in ontologies available in a repository to better match groups and users. We introduced the notion of MADCOW domains, representatives for ontologies inside the system. Each domain has to be associated with an ontology before it is used to represent groups in the system. Associating ontologies with domains minimizes the time needed in the matching algorithm since each domain represents a set of groups. Referring a group to a domain could be done on a manual or automated basis. Groups' owners could manually refer their groups to the most relevant domains (by checking their associated ontologies concepts), or, if they are not satis ed with the available domains, they can provide the system with Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
a set of terms that best describe the intention of their groups. The system performs a matching to present the most appropriate ontologies (ranked according to their group relevance) to let owners choose one of them. The selected ontology concepts become the tags to be used when submitting annotations to the group. Figure 3 depicts the Groups-Domains-Ontologies association.
The following pseudocode illustrates the process of associating a group with a domain. The code starts by executing the function selectGroup() that saves the selected group in the variable selectedGroup. The names of all the available domains in the system are loaded via loadDomains() that saves them in domains variable. A loop is entered which rst enables the owner to select one of the domains with selectDomain(domains) function; after that, the execution of checkDomain(selectedDomain.getTerms()) enables the owner to check whether the domain terms represent the intention of the selected group or not. The loop keeps going till either the owner is satis ed with the selected domain, as indicated by the function checkdomain() returning true, or all the domains have been considered. In the rst case the execution of associate(selectedGroup,selectedDomain) creates an association between the selected group and domain. In the second case, the owner can use the automatic support for group-domain association. This process starts by declaring an empty list matchedOntologies; then the function askForTerms() asks the owner to provide a set of terms which he or she deems representative of the group and saves them in the collection terms. A matching between these terms and the terms in all ontologies available in the Ontology Repository then takes place by executing the function CMM(). Each ontology that satis es the matching condition is added to matchedOntologies. Finally, a call to rank() generates a ranking of all the contents of matchedOntologies according to their relevance values. The function selectOntology() enables the group owner to select the most appropriate ontology from the list of matched ontologies. The system then checks whether or not the selected ontology is associated with a domain by exProc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
ecuting the function isAssociated(). If it is not associated, then the function associateWithDomain() is executed to associate it with a domain related to its title. In the nal step, associate() generates an association between the selected group and the domain linked with the selected ontology. selectedGroup = selectGroup ( ) ; domains = loadDomains ( ) ; r e p e a t f selectedDomain = selectDomain ( domains ) ; i f ( checkDomain ( selectedDomain . getTerms ( ) ) ) f a s s o c i a t e ( selectedGroup , selectedDomain ) ; break ; g u ngt i l ( allDomainsConsidered ( ) ) ; i f ( ! group . c h e c k A s s o c i a t i o n ( ) ) f matchedOntologies = new L i s t ( ) ; terms = askForTerms ( ) ; foreach ( ontology in MADCOW. O n t o l o g i e s ) i f (CMM( terms , ontology . getTerms ( ) ) >0)
matchedOntologies . add ( ontology ) ; matchedOntologies . rank ( ) ; s e l e c t e d O n t o l o g y = s e l e c t O n t o l o g y ( ) ; i f ( ! s e l e c t e d O n t o l o g y . i s A s s o c i a t e d ( ) )
s e l e c t e d O n t o l o g y . associateWithDomain ( s e l e c t e d O n t o l o g y . T i t l e ) ; a s s o c i a t e ( selectedGroup , s e l e c t e d O n t o l o g y . getDomain ( ) ) ; g Annotations Submission For annotations submitted as public or private, users can use any set of tags as metadata associated with the annotation. Tags attached to these annotations will be used in the matching process to propose groups to users, and users to authorisers (only for public annotations).
The following pseudocode represents the submission of public, private and grouped annotations. The code rst executes the function submitToGroup() to check if users submitted annotations to one of the groups they own or they are members in. In this case the function assignTags() is executed to make a group terms become annotation tags. If users decide to submit public or private annotations, the function attachTagsToAnnotation() is executed, in this case users are free to attach some informative tags to their annotations to better describe the intention for the annotation submission by executing the function assignTags() that executes the function askForTags() which in turn asks users to provide tags to be assigned with annotations. i f ( submitToGroup ( annotation , group ) )
assignTags ( annotation , getGroupTerms ( group ) ) ; e l s e i f ( attachTagsToAnnotation ( ) )
assignTags ( annotation , askForTags ( ) ) ; Groups-Users Matching Group owners could ask the system to suggest users to their groups. As a consequence, the system executes a textual comparison between all ontology concepts and all the tags employed by users to adorn their public annotations. The same comparison takes place when users ask the system to propose some groups to send request joins. Comparison here includes both public and private annotations of users. Figures 4(a) and 4(b) depict the Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
matching between a group and an ontology and between a domain and a user, respectively. (a) Group-Ontology Matching (b) Domain-User Matching
The CMM (exact and partial) is meant to evaluate the coverage of an ontology for a given set of search terms. By assuming that a given group is characterised by a set of terms, this measure is used to calculate to which extent an ontology covers the terms provided by the group. Hence, we measure the relevance of an ontology to a group in terms of matches between group terms and ontology lexemes. We use the same measure to calculate the relevance of groups for users depending on the terms associated with a group and the set of tags employed by users to adorn their annotations.
Domain-Annotation Matching Formalisation Given a domain D and an annotation A, let LD = fr1; r2; : : : ; rng be the set of lexicalisations of domain terms and let T = ft1; t2; : : : ; tmg be the set of tags. The matching between D and A is the process of checking the existence, within A, of a term from LD, considering exact and partial matches.
URL-Based Matching We are also discussing another way for automating the groups-users suggestion by matching URLs that are annotated by both group Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
members and non-grouped users, assuming them as indicators of common interests related to the group intent. From this point of view, group authorisers could be interested in those users sharing those interests. The same process could be applied by users who are searching for interesting groups to join. URL-based matching is illustrated in Figure 5.
The following pseudocode illustrates the URL-based matching between a group G and all users not members in G: groupURLs = G. getURLs ( ) ; u s e r s = g e t A l l U s e r s ( ) ; usersMatch = new User [ u s e r s . g e t S i z e ( ) ] ; foreach ( groupURLs as gURL) foreach ( u s e r s as u s e r ) foreach ( u s e r . getURLs ( ) as uURL)
i f (gURL == uURL) usersMatch [ u s e r ]++; ranks ( usersMatch ) ;
The code begins by executing the member function getURLs() that retrieves all annotated URLs related to the group G (for all members in G) and stores the result in the array groupURLs. The list of all users that are not members in G are stored in the array users as a result for executing the function getAllUsers(). A nested loops then are executed in which a list of all URLs annotated by a given user are retrieved by the member function getURLs() and matched against the array groupURLs. The result of the accumulative matching for every user with the user ID are stored as an element in the associative array usersMatch to be used after that in the function ranks() for ranking purposes.
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org).
Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
We built a repository from 6 di erent ontologies: Animals (with 899 concepts),
Plants (709), Finance (2037), Artificial Intelligence (2386), Vehicles
(168) and Viruses (296) gathered from [
We conducted a pilot test on the matching process. 17 participants were divided into 3 disjoint sets (6, 6, 5). Users from the rst set created 16 various MADCOW groups and assigned domains for them, then manually invited participants from the second set to join the groups (20 invitations were created). These accepted the invitations and submitted annotations to the joined groups (26 annotations). Participants in the third set submitted private and public annotations (31 annotations), all adorned by suitable tags. We asked users to annotate a set of similar websites to check URL matching (for this we used 10 di erent websites). Group owners requested the system to suggest members and invitations were sent (9 invitations). Participants of the third set required the system to suggest proper groups and sent membership requests (12 requests). By comparing the joint duration for group owners of the processes of obtaining user suggestions / selecting users / sending invitations with that of the manual invitation process (in the rst work), we note that the average invitation duration decreased from 99.25 (in Table 1) to 10.6 seconds. 4
Introducing groups to MADCOW system solved the tension between users' security and privacy and improved collaboration. Proper groups-users matching enhanced the groups-users suggestions, minimizing time and e ort for both groups authorisers and users, and solved the irrelevance problem. The reduction in average time for user invitation between the two tests encourages us to investigate Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org). Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
other relevance measures that could provide further re nements in groups-users matching as far as relevance is concerned.