<?xml version="1.0" encoding="UTF-8"?>
<TEI xml:space="preserve" xmlns="http://www.tei-c.org/ns/1.0" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.tei-c.org/ns/1.0 https://raw.githubusercontent.com/kermitt2/grobid/master/grobid-home/schemas/xsd/Grobid.xsd"
 xmlns:xlink="http://www.w3.org/1999/xlink">
	<teiHeader xml:lang="en">
		<fileDesc>
			<titleStmt>
				<title level="a" type="main">Is Your Ontology a Burden or a Gem? -Towards Xtreme Ontology Engineering</title>
			</titleStmt>
			<publicationStmt>
				<publisher/>
				<availability status="unknown"><licence/></availability>
			</publicationStmt>
			<sourceDesc>
				<biblStruct>
					<analytic>
						<author>
							<persName><forename type="first">Olga</forename><surname>Tatarintseva</surname></persName>
							<email>tatarintseva@znu.edu.ua</email>
							<affiliation key="aff0">
								<orgName type="department">Department of IT</orgName>
								<orgName type="institution">Zaporozhye National University</orgName>
								<address>
									<addrLine>66 Zhukovskogo st</addrLine>
									<postCode>69600</postCode>
									<settlement>Zaporozhye</settlement>
									<country key="UA">Ukraine</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">Vadim</forename><surname>Ermolayev</surname></persName>
							<affiliation key="aff0">
								<orgName type="department">Department of IT</orgName>
								<orgName type="institution">Zaporozhye National University</orgName>
								<address>
									<addrLine>66 Zhukovskogo st</addrLine>
									<postCode>69600</postCode>
									<settlement>Zaporozhye</settlement>
									<country key="UA">Ukraine</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">Anna</forename><surname>Fensel</surname></persName>
							<email>anna.fensel@sti2.at</email>
							<affiliation key="aff1">
								<orgName type="institution">FTW Forschungszentrum Telekommunikation Wien GmbH</orgName>
								<address>
									<addrLine>Donau-City-Straße 1</addrLine>
									<postCode>A-1220</postCode>
									<settlement>Vienna</settlement>
									<country key="AT">Austria</country>
								</address>
							</affiliation>
							<affiliation key="aff2">
								<orgName type="institution">STI Innsbruck</orgName>
								<address>
									<addrLine>Technikerstr. 21a</addrLine>
									<postCode>A-6020</postCode>
									<settlement>Innsbruck</settlement>
									<country key="AT">Austria</country>
								</address>
							</affiliation>
						</author>
						<title level="a" type="main">Is Your Ontology a Burden or a Gem? -Towards Xtreme Ontology Engineering</title>
					</analytic>
					<monogr>
						<imprint>
							<date/>
						</imprint>
					</monogr>
					<idno type="MD5">223EB5003E6C9EDC59665A54F4442685</idno>
				</biblStruct>
			</sourceDesc>
		</fileDesc>
		<encodingDesc>
			<appInfo>
				<application version="0.7.2" ident="GROBID" when="2023-03-24T13:14+0000">
					<desc>GROBID - A machine learning software for extracting information from scholarly documents</desc>
					<ref target="https://github.com/kermitt2/grobid"/>
				</application>
			</appInfo>
		</encodingDesc>
		<profileDesc>
			<textClass>
				<keywords>
					<term>ontology engineering methodology</term>
					<term>stakeholder commitment</term>
					<term>OntoDocWiki</term>
					<term>xtreme ontology engineering</term>
				</keywords>
			</textClass>
			<abstract>
<div xmlns="http://www.tei-c.org/ns/1.0"><p>One of the commonly acknowledged shortcomings of Semantic Technologies that prevents their wide adoption in industry is the lack of the commitment by the intended domain experts and users. This shortcoming becomes even more influential in the domains that change sporadically and require appropriate changes in the respective knowledge representations. This discussion paper argues that a more active involvement of the intended user community, comprising subject experts in the domain, may substantially ease gaining the required commitment of the critical mass of the domain users to the developed domain ontology. As a possible approach for building an instrumental platform for that, the paper suggests the use of the Semantic MediaWiki based collaboration infrastructure for maintaining and discussing ontology descriptions by the community of its intended users and developers. We also report how a prototypical ontology documentation wiki has been used for gaining the commitment of ontology users in the ACTIVE European project.</p></div>
			</abstract>
		</profileDesc>
	</teiHeader>
	<text xml:lang="en">
		<body>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="1">Introduction</head><p>Building and refining practically useful knowledge representations in different domains is one of the major challenges in making semantic technologies publicly accepted today. The problem is not only in creating the proper encodings of the tacit knowledge of subject experts, or user behavior observations <ref type="bibr" target="#b0">[1]</ref> but also in gaining a commitment to the developed ontologies by the users who are supposed to exploit these modules of formalized and explicit knowledge -either directly in their daily activities or by empowering their software tools. Evidently creating ontologies is not a routine task. It requires substantial intellectual effort.</p><p>Moreover, refining ontologies, making them better covering the intended requirements of the user community, is even more challenging and effort consuming. As well known from knowledge elicitation practice, five subject experts will most definitely have seven<ref type="foot" target="#foot_0">1</ref> different opinions. So, the commitment of those individuals can only be reached if a knowledge representation is aligned and harmonized alone their subjective and tacit interpretations of the domain knowledge.</p><p>Gaining a commitment to the ontology by a wider group or a community of intended users is even more complex. One reason is that the majority of users have to adapt themselves not only to the suggested knowledge representation but also to the knowledge carried by this formal representation -which could both be novel to them. In our opinion the difficulty of gaining users' commitment is the major obstacle for a broader adoption of the semantic technologies in industries and the reason <ref type="foot" target="#foot_1">2</ref> for the criticism expressed to those technologies. The problem becomes even more challenging for the knowledge about the domains that change frequently. Ontologies describing those domains have to be changed accordingly. The changes in the knowledge representations have to be accepted by the subject experts and the users.</p><p>Hence, offering support for facilitating a better and less effort consuming comprehension, alignment and harmonization of knowledge representations by a user community may become a substantial step forward in reusing domain knowledge by knowledge workers and their software systems. Contemporary ontology engineering methodologies put insufficient emphasis on offering ways to gaining such a commitment. The analysis of this shortcoming is given in Section 3.</p><p>We believe that a more active involvement of the intended ontology users in the processes of ontology development and refinement is required for lowering their comprehension barriers. A software tool facilitating this active involvement will inevitably be a collaboration platform that allows discussing knowledge representations and expressing opinions and arguments by any community member.</p><p>Developing tools for collaborative knowledge engineering and knowledge reuse is one of the mainstreams in the semantic technologies community. However, the vast majority of the tools available today are tailored to the use by knowledge engineers, but not by domain experts or users. One interesting exception is the development of ontology games and collaborative (social) semantic mark-up tools for Web 2.0. Yet, these approaches yield too lightweight models -insufficiently expressive for the majority of industrial applications. The analysis of the state of the art in collaborative platforms for ontology engineering, onto-gaming and semantic mark-up on the Social Web is given in Section 4.</p><p>One possible solution for the outlined problem is making these divergent courses meet. Tool support for ontology engineering would benefit from adopting "croudsourcing" features of collaborative knowledge representation development by Social Web users. A meeting point that will allow for the proper comprehension of knowledge structures is a collaborative platform for presenting and discussing the documentation of the ontologies by the subject experts and intended users alone the development process. There are several obstacles on this way. One is the lack of a proper incentive mechanism motivating subjects to take their active part. Another one lies in the nature of the work to be done -it is out of the scope of the core professional competence of intended audience. Yet more obstacles are caused by the lack of the tool support for: (i) the development and versioning of the ontology documentation in line with the evolution of the ontology; (ii) the discussion of ontology documentation as a representation of knowledge that is more easily comprehendible by the users than the code of the ontology. Last but not least is the need for a mechanism of reaching and spreading consensus among the participants. The requirements to the envisioned collaborative platform are presented in Section 5.</p><p>Our experience in developing and experimenting with a prototype collaboration platform for involving domain specialists in the active discussion of domain knowledge representations is presented in Section 6. The prototype platform implements some of the outlined requirements. It is based on the Semantic MediaWiki <ref type="bibr" target="#b1">[2]</ref> with an extension for moderated discussions. The prototype platform has been used in ACTIVE Project (http://active-project.eu/) for representing and discussing the PSI Suite of Ontologies (http://isrg.kit.znu.edu.ua/ontodocwiki/) describing projects and processes in microelectronic engineering design.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2">Why is an Ontology Often a Burden?</head><p>Given the effort and intellect invested into the development of ontologies as consensual descriptive formal models of domains of discourse <ref type="bibr" target="#b2">[3]</ref> in the last two decades, it could have been expected that ontologies had already become the core enablers for the ICT infrastructures and services in many industrial branches. However this is not entirely the case. "Unfortunately, the number and quality of actual, "non-toy" ontologies available on the Web today is remarkably low" <ref type="bibr" target="#b3">[4]</ref>. Several technical limitations and practical challenges preventing easy adoption remain unsolved. These barriers for technology and methodology uptake as perceived by industries<ref type="foot" target="#foot_2">3</ref> (c.f. <ref type="bibr" target="#b3">[4]</ref>) are as follows: (i) Unjustified benefits. Industrial users and policy makers tend to think that using ontologies in their industrial setting is an artificial requirement to a large extent. They (sometimes wrongly) assume that the required information could be presented and processed in a more easy way using lighter-weight mark-up languages (like XML) and corresponding parsers. (ii) Considerable effort expectations. Industries consider that even if the need for an ontology is justified, the effort required for building it is too large to be acceptable for the incurred benefits of use. (iii) Insufficient expressivity and comprehension gap. Industrial technical specialists fear that the expressivity of ontology specification languages is insufficient for fully and adequately describing their subject domain. Consequently, it can not be granted that the intended users of the ontology easily grasp the meaning of the ontology elements as intended by the knowledge engineers who created the code. (iv) Computational overhead and poor scaling. Software developers in industry estimate that ontology based software solutions are too heavy-weight. The software spends too much computation power for ontology processing and reasoning because of the complexity of the problems that are solved. As a result ontology-based applications scale quite poorly to be acceptable in industrial settings. More lightweight solutions are demanded. (v) Insufficient maturity of ontology engineering process. If it is not their core competence, industrial engineers will seek for and accept a methodology that is well defined and based on the use of standardized working patterns. However, the leading experts in the field state that the development of ontologies is still much more a craft work or a non-trivial mental exercise than a rigorous and standardized engineering process. One of the particular shortcomings is that ontology engineering cycles are too long. The result is that we can not build ontologies that adequately follow the changes in quickly evolving domains.</p><p>Last but not least, and as a consequence of the combination of the outlined barriers, people in industries hardly believe that ontologies will solve their practical problems and help effectively in the development of their applications. Therefore it is often difficult to obtain their commitment to the ontologies offered to or developed for them.</p><p>It may be also noticed that the requirements the industries implicitly put up-front in (i-v) are not properly balanced and are sometimes clashing. For example a desire to have easily comprehendible lightweight knowledge representations is contrary to the demand of more expressive power for a more adequate representation of a domain. One good approach to resolve those clashes is to offer a layered representational structure with a more coarse and easy to grasp descriptions on the top down to fully detailed and formally coded knowledge representation modules in the bottom. As suggested in <ref type="bibr" target="#b4">[5]</ref> those layers may be offered as different representation facets to different categories of specialists and at different development phases.</p><p>We do not intend to resolve all the fears of industrial experts in this paper. Our objective is to evaluate the existing ontology engineering approaches, development praxis, and methodologies by looking at how they facilitate better and broader commitment to the developed knowledge representation artifacts and, by that, relax some of the existing barriers.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3">Shortcomings of Ontology Engineering Methodologies</head><p>Knowledge Engineering as a subfield of Artificial Intelligence or broader -of Computer Science is a vibrant research discipline for already more than two decades. It involves integrating knowledge into computer systems. Knowledge has to be therefore represented in a way that a computer system is able to process. These representations are often elaborated as ontologies using the instruments provided by Ontology Engineering. This discipline develops knowledge representation frameworks to ensure adequate rigor for making the outputs tractable and processible by machines. Formal knowledge representation languages (e.g. OWL 2.0 <ref type="bibr" target="#b5">[6]</ref>) are developed and standardized for that purpose.</p><p>Ontology Engineering is also concerned about the development of the methods and methodologies for building ontologies to fulfill the requirements of the intended user audience. The results are sought to cover the user interpretations of the common sense or a target subject domain so much and completely as the expressivity of the formal representation allows. Several ontology engineering methodologies have been developed up to date. In particular METHONTOLOGY <ref type="bibr" target="#b6">[7]</ref>, DILIGENT <ref type="bibr" target="#b7">[8]</ref>, On-To-Knowledge <ref type="bibr" target="#b8">[9]</ref>, Uschold&amp;King <ref type="bibr" target="#b9">[10]</ref>, Delphi <ref type="bibr" target="#b10">[11]</ref>, Compendium <ref type="bibr" target="#b11">[12]</ref>, HCOME <ref type="bibr" target="#b12">[13]</ref>, CommonKADS <ref type="bibr" target="#b13">[14]</ref>, NeOn <ref type="bibr" target="#b14">[15]</ref> are the methodologies that are mentioned in the literature most frequently. A reader may be pointed to <ref type="bibr" target="#b15">[16]</ref> for a more comprehensive list and analytical survey. Recently the methodologies taking into account the economical aspects of ontology engineering appear -e.g. OntoCOM <ref type="bibr" target="#b16">[17]</ref>.</p><p>Among those we are particularly interested in the methodologies which explicitly support: (i) collaborative ontology engineering; and (ii) ontology refinement process with evolving requirements. These methodologies are METHONTOLOGY, DILIGENT, On-To-Knowledge, Compendium and NeOn. In terms of ontology engineering lifecycle all the five methodologies suggest a variation of a process schematically pictured in Fig. <ref type="figure" target="#fig_0">1</ref>, where the iterative parts are described in terms of either spiral or iterative waterfall process models.</p><p>The differences are: (i) METHONTOLOGY distinguishes support and development activities and focuses on knowledge elicitation and result evaluation routine (ii) DILIGENT focuses on distributed deployment and local changes and provides an argumentation framework for harmonization (iii) On-to-Knowledge distinguishes ontology refinement as a separate important stage of the development process (iv) NeOn offers a flexible scenario-based decision procedure for choosing the most appropriate lifecycle model and puts significant emphasis on the re-use of ontology patterns and available distributed ontologies (v) Compendium explicitly concentrates on the ways of organizing collaborative work at knowledge elicitation and evolving prototyping phases With respect to the relaxation of the barriers for gaining ontological commitment the contribution of all the mentioned methodologies is limited. NeOn suggests reusing good ontology engineering practices in the form of ontology patterns. Implicitly it suggests that using these good practices results in making the ontologies more correct and reliable -thus the commitment to these results is expected to be higher. Only OntoCOM elaborates the incentives for individuals and organizations for introducing ontologies. However it does not suggest mechanisms for gaining the commitment when the ontology is being developed. Compendium offers an approach to collaborative development based on moderated discussions and accounting for the evolution of the requirements. Unfortunately it does not mention the incorporation of the experts who carry the tacit knowledge about the subject domain that has to be elicited. None of the reviewed methodologies pays attention to the presentation of the developed knowledge representation in the form that is easily comprehendible by the intended users. The ontology documentation activity is considered only a support activity to the development process. None of the methodologies, except DILIGENT and NeOn to some extent provide the means for reaching consensus in ontology design decisions. DILIGENT does that by offering a harmonization framework. NeOn suggests consensual seeds in the form of reusable design patterns.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4">Social Tagging and Games with a Purpose</head><p>One of the possible ways to check if the conceptualization of the domain is correct and complete is to evaluate the model against the outcome of users' grasp of the meaning of the content of the representative set of relevant documents. From the other hand, the labels or comments the users put on these documents or resources may be effectively used to infer the conceptualization. Such labels are often denoted as tags or annotations.</p><p>If web resources of different modalities are thought of as the representative set of data we find ourselves in the exploding field of collaborative or social tagging and annotation -a substantially characteristic part of the Web 2.0 phenomenon. A good survey of the field of social tagging is offered by Gupta et al in <ref type="bibr" target="#b18">[19]</ref>. Tags created by the community of online users are exploited for different purposes. Taxonomy generation is one of the applications particularly relevant to our subject. The shortcomings of social tagging on Web 2.0 as analyzed in <ref type="bibr" target="#b18">[19]</ref> as follows: (i) tags are simple bags of words without any more expressive semantics; (ii) tags are often not correct, especially if generated by spammers; (iii) tags are often ambiguous because different users apply terms to documents in different ways; tags are often sparse and do not cover the elements of the resource uniformly. Those shortcomings effect in low quality of tags. The reason is that the taggers do not use the terms of a consensual domain model in their activity.</p><p>Semantic annotation and tagging approaches further refine social tagging techniques by offering the collections of terms that are taken from such knowledge representations in the forms of taxonomies, folksonomies, thesauri. Please refer to [20] for a comprehensive analyses of the requirements and results in the field. However, the backbone knowledge representations have to be obtained before semantic annotation may be undertaken. This remains the work for ontology engineers and is not regarded as a task for non-specialist users. Hybrid approaches for collaborative tagging and annotation aimed at enrichment of the seed knowledge representations by the activity of the user community doing tags or annotations are also reported. For example <ref type="bibr" target="#b20">[21]</ref> reports about the implementation of such a hybrid approach in Digital Libraries by tagging data through a combination of a standardized model, a harvesting protocol and a metadata mapping. It is concluded that both the custodians and users of digital repositories enabled with this collaborative annotation system benefit from the potential value of collaborative tagging without a need for a prior knowledge of the backend annotation systems. The weaknesses of traditional social tagging systems are attempted to be overcome by combining the best features of the Social and Semantic Webs. Unfortunately the problem of motivating users actively taking part in annotating resources remains open even in the reported advanced collaborative semantic tagging systems.</p><p>As already mentioned above, good ontologies should match consensual interpretations of domain knowledge by the representative set of domain users as closely and completely as possible. Therefore, ontology engineering in any form is by its very nature a process that has to involve as many domain experts and intended users as affordable. Increasing numbers of people are willing to spend their time using Web 2.0 applications, for example in adding tags and sharing their mark-ups in a group or community. Remarkably, it is totally on the contrary for ontology development -nobody reported about involving big user groups in creating knowledge representations so far. As recognized by many experts in Semantic Technologies one of the possible reasons is that traditional ontology engineering methodologies detach the effort from the benefits (c.f. <ref type="bibr" target="#b21">[22]</ref>) hence de-motivating the involvement of those people whose interpretations of the domain are critically required.</p><p>One of the promising approaches for motivating more people take part in creating or refining ontologies is offering social software or, alternatively, a game with a purpose to a group of intended users. Several results in this direction are reported in the literature. Ontology creation can be implicitly embedded in social software, namely, social networking portals where users would be creating, evolving and confirming ontology items implicitly in the background, while simply providing information to a the portal for sharing content and communication with other users <ref type="bibr" target="#b22">[23]</ref>. Games with a purpose is another approach that has been used mainly for collaborative tagging of resources having different modalities: images <ref type="bibr" target="#b23">[24]</ref>, music <ref type="bibr" target="#b24">[25]</ref> -to mention just few. Gaming approach has also been tried for inferring human intentions from their recorded actions (Common Consensus game, <ref type="bibr" target="#b25">[26]</ref>) and for evaluating how well commonsense facts fit to the interpretations of random users (FACTory Game by Cycorp, http://game.cyc.com/). For involving users in creating domain knowledge representations several game scenarios have been developed <ref type="bibr" target="#b21">[22]</ref> for ontology building and refinement, ontology matching, annotating content using lightweight ontologies. Even though ontology backed up social networking portals and games with a purpose scenarios differ in relation to the users' motivation to contribute, they are in compliance to the OntoElect approach proposed in the next Section. Both approaches offer possibilities to identify whenever users start to agree on and share certain ontological items</p><p>The positive features that are common to all these gems of related work are: (i) the pattern of user involvement adopted from Web 2.0 is used to motivate people taking part; (ii) all games with a purpose hide their purpose under the gaming scenariooffering fun in reward for providing useful results; (iii) the scenarios are designed in a way that assists in structuring the pool of players by their reputation.</p><p>There is also the shortcoming which is inherent to this approach -the knowledge representations or mark-ups that are crafted by non-specialist players can only be lightweight. Otherwise the overhead for ramping-up the players would consume all the offered incentives. Though ontology fragments are aligned with consensual user interpretations, it would be hard to ensure that the quality of those fragments is sufficient for, say, industrial use. Therefore, a joint motivated involvement of ontology engineering professionals, subject experts, and a sufficiently big group of intended users with domain knowledge and expertise is required.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5">OntoElect Approach for Xtreme Ontology Engineering</head><p>Charles Petrie in his editorial article <ref type="bibr" target="#b26">[27]</ref> argued that the correctness of "… semantics, is evident in its use …". Ontologies are often denoted as descriptive theories that specify domain semantics -so they may only be validated by the users in their daily work in that domain. Martin Hepp in <ref type="bibr" target="#b3">[4]</ref> backs up this view by stating that "… commitment can be achieved only by successful joint action -that is, successful usage of the ontology". Emphasizing the role of user commitment he observes that the perceived utility of the ontology grows with the number of users who commit to it. Hence, an effective ontology engineering methodology has to offer a mechanism for gaining the commitment by the intended group of users -the sooner -the better and as broadly as possible. We believe that a correct way to go is to involve the subject experts and the intended users in the development of the ontology at the earliest phase possible.</p><p>Our proposal of a possibly effective approach for attracting subject experts to play a more active role in the development and ownership of ontologies is inspired by the observations of social and political life -in public election campaigns. Indeed, the desired outcome of an election campaign for every candidate is to gain as much commitment of the electorate as possible. Such a commitment is measured adequately by the number of votes. The candidates compete for the votes by presenting their programmes, making coalitions, taking part in public debates -proving that his or her programme is the best match to the expectations of the majority of the electorate.</p><p>In the case of ontology engineering alternative ontology seeds for the same subject domain could be treated as election candidates. Each candidate ontology offered in a, so to say, ontology election campaign may be evaluated compared to the other candidates by the ability to answer the competency questions of the electorate. The more competence in answering the requirements of the electorate is demonstrated by the ontology, the more commitment it is potentially able to gain. The members of the electorate are the intended users in the domain. Their commitment could be measured in simple votes or using a more sophisticated scoring mechanism. A good example of such a mechanism is evaluating submissions in a peer review process. Candidate ontologies may be presented by their election committees composed of the knowledge engineers and subject experts who took part in the initial development of the artifact. The presentations could be compared by the electorate like it happens in politics to the political programs. The candidate ontologies may be invited and take part in the public debates. Their competences will thus be cross-checked by the members of the competing election committees. The results of the debates will provide more evidence to the electorate for making their informed votes. It may happen that none of the candidate ontologies receives the majority in the campaign. Such an outcome in politics may be treated as an event leading to a one more round of elections. In ontology engineering a new election round may also be used constructively for the refinement of the candidates. The development teams may make use of the election results by concentrating on answering the competency questions they failed to answer by the previous revision. Coalitions may also be fruitful if it turns out that merging some candidate ontologies will substantially improve their joint competency. The election committees of the merged ontologies of course have to reach an agreement on how their "societal influence" will be redistributed after their victory in elections. Following the outlined procedure for ontology election, if a particular ontology gets the majority then it can be expected that the electorate's commitment to this ontology is strong enough to ensure its smooth uptake. Hence, a high level research hypothesis in our proposal is:</p><p>Introducing "democracy" in ontology engineering by incorporating a competitive and transparent procedure of ontology elections makes the process effective, development cycles shorter, and enables better results transfer to industry. The effectiveness is ensured by the fact that the developed artifacts will be appropriately refined following the intended requirements of the users in iterations (election rounds) with active participation of these intended users. The incentive for the active involvement of the domain specialists and knowledge engineers is reciprocal as they share common objectives. Moreover, both parties are naturally motivated by the competitive nature of the process. The iterations become shorter and pursue better defined and more focused objectives that reflect the desires of the domain specialists adequately. Hence, a better transfer is ensured by the fact that the commitment of the intended users to the winner is the strongest among the alternative candidates.</p><p>There are several research questions that have to be answered in more detail and rigor for proving this research hypothesis: (i) Why will industrial domain specialists be willing to join election committees and vote in election rounds? This research question is very similar in its nature to the question about the proper motivation for people to join political groups and vote in elections. The answer may be sought by devising appropriate reciprocal incentive schemes motivating industrial domain specialists and knowledge engineers. For that looking at the results in several European projects may be useful. FP7 ACTIVE project derives the recommendation on possible incentive schemes, in particular for ontology development, by looking at the teams of knowledge workers as social structures. The high level objective of the FP7 INSEMTIVES project (http://www.insemtives.org/) is to bridge the gap between human and computational intelligence and providing incentives for users to contribute to the massive creation of semantic content. A general framework for organizing campaigns may be adapted from the EU infrastructure project SEALS (http://www.seals-project.eu/) which develops a reference infrastructure, the SEALS Platform, to facilitate the formal evaluation of semantic technologies. (ii) What are the proper ontology representation notation and collaboration platform for transparent election debates?</p><p>The first part of this research question is about the proper balance between the expressivity of the notation for representing ontologies and the ease of comprehension of this notation for the users that are not knowledge engineers. The expressivity has to be equal to the tractable subset of the chosen ontology specification language (for example OWL 2.0 -a de-facto standard ontology representation language to date). For ensuring the ease of comprehension we have to take into account that the representatives of the electorate are industrial knowledge workers -the engineers who develop, adapt, or adopt IT solutions in their businesses. A UML-based language is one of the commonly used notations for these professionals. Therefore, a visualized ontology specification in UML or its extension (for example OntoUML <ref type="bibr" target="#b27">[28]</ref>, or other UML variants with appropriate expressiveness <ref type="bibr" target="#b28">[29]</ref>) accompanied by the textual description of ontology elements in a natural language would be appropriate -please refer to Section 6 for more details.</p><p>The second part of the question is about the collaboration platform that enables efficient debates. We consider that Semantic MediaWiki (SMW) with extensions is very appropriate as such a platform. An argument in favor of a Wiki-based infrastructure is that it is a Web 2.0 platform that intrinsically supports the exchange of opinions and has been extensively used for collaborative content development, "crowd-sourcing", user community development, etc. (iii)</p><p>How to ensure the swift convergence of the series of election rounds to the appearance of a single good scorer ontology? The answer to this research question has to be sought by looking for the proper set of heuristic rules and policies for ontology elections. It could be rightfully argued that such a statement is a way too succinct to answer the question. However we do not have a more detailed recommendation at the moment and leave this very important issue for future research and experiments.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6">Xtreme Documenting and Ontology Discussions On-Line</head><p>One of the hypotheses we pointed out is: the ontology developed with active involvement of the intended users has to be presented in a form that is easily comprehendible by these users. A rich self explanatory notation with a user interface that is native for the target user group has to be exploited for that. We believe that a proper way of presenting the ontology to the subject experts is the documentation of the ontology as it combines the formal definitions of the ontology elements in textual and graphical representation with the informal descriptions of the semantics of those elements. Of course the documentation has to be developed in line with the ontology design to be available in proper time for discussing the design decisions. Ideally, the documentation describing a concept, a property, an ontological module has to appear at the same time with the appearance of the design of this ontology element. Apart of that, the documentation has to contain the information about the ontological context of the element and the information about its evolution.</p><p>As a proof of concept we have developed the electronic documentation site for ontologies -OntoDocWiki (http://isrg.kit.znu.edu.ua/ontodocwiki/). This resource is based on the SMW with LiquidTreads extension (www.mediawiki.org/wiki/ Extension:LiquidThreads) as a basic collaboration infrastructure. Currently the resource contains the documentation of the PSI Suite of Ontologies v.2.3. This Suite of Ontologies has been developed in the Performance Simulation Initiative (PSI) project <ref type="foot" target="#foot_3">4</ref> and further refined for the needs of the ACTIVE Project. The wiki articles represent documentation for the Suite of Ontologies (Fig. <ref type="figure" target="#fig_1">2</ref>), each individual ontology module (Fig. <ref type="figure" target="#fig_2">3</ref>), each individual concept (Fig. <ref type="figure" target="#fig_3">4</ref>) have been semi-automatically generated <ref type="bibr" target="#b29">[30]</ref> based on the reference specification <ref type="bibr" target="#b30">[31]</ref>. As pictured in Fig. <ref type="figure" target="#fig_3">2-4</ref>, the articles comply with all the outlined requirements for describing ontology elements in an easily comprehendible and non-professional user friendly, yet informative manner. They combine textual descriptions with the graphical (UML) representation of the ontological contexts. These graphical representations are implemented as image maps and allow easy navigation in the pictured ontological contexts. For example, as pictured in Fig. <ref type="figure" target="#fig_1">2</ref>, clicking the package representing the E2H ontology in the package structure diagram opens the article describing the E2H ontology (Fig. <ref type="figure" target="#fig_2">3</ref>). Similarly, the class diagram of the particular ontology allows navigation to individual concept articles (e.g. the concept of a Happening, Fig. <ref type="figure" target="#fig_3">4</ref>) or related ontological modules represented in the ontological context.  At a concept level the documentation is informative enough for allowing a nonprofessional user to evaluate the semantics of the concept and the surrounding ontological context <ref type="bibr" target="#b28">[29]</ref>. As shown in Fig. <ref type="figure" target="#fig_3">4</ref>, the article documenting a concept contains the information about:  The relationship of the concept to the higher-level or foundational ontologies. For example, the concept of a Happening is described as a subclass of an AtomicAction -the concept of the PSI Upper-Level ontology.</p><p> The explanation of the semantics of the datatype properties.</p><p> The object properties grouped by the type of relationship: subsumption (if a concept is a subclass of another concept), part-whole relationships (represented as aggregations or compositions in the UML class diagram), and associations. The descriptions of individual properties are structured in a way to present the evolution of those properties. For the properties that have been changed in the current version the information about the change is given. For those properties that have been introduced or became obsolete the rationale for this design decision is described. For the properties that may be used differently depending on the deployment of the Suite the variants of use are explained. For example, in some applications, like ProjectNavigator <ref type="bibr" target="#b31">[32]</ref> the Suite of PSI Ontologies may be used without its Upper-Level ontology. In ACTIVE project the PSI Suite of Ontologies has been used as the background knowledge for describing the subject domain of one of its case studies -knowledge processes in microelectronic engineering design. The fully functional prototype of the ACTIVE Design Project Visualizer <ref type="bibr" target="#b32">[33]</ref> has been developed and validated based on this backbone knowledge representation. Prototype development has been done by several project partners. Only one of the partners was the owner of the PSI Suite -the rest were not familiar with these ontologies. Therefore we had to develop an ontology documentation and discussion resource for ramping-up the collegues. It turned out that OntoDocWiki became very helpful in both: explaining the PSI ontologies to the software developers, some with a marginal background in knowledge engineering; and collaboratively refining the ontologies in response to the project requirements. In fact the v.2.3 release of the Suite has been developed in this collaboration based on the extensive use of the OntoDocWiki platform.</p><p>One of the features that proved to be useful and effective in ramping-up the users was discussions about ontology elements. Fig. <ref type="figure" target="#fig_4">5</ref> pictures an example of such a discussion about the semantics of the concept of a Happening. The utility of a discussion is straightforward -anyone may pose a question or present an argument about the semantics of an element in the ontological context of the related wiki article; anyone else may offer an answer or a counter-argument. The outcome of the discussion is often a better and deeper comprehension of the semantics. It also turned out some times that the discussions led to the changes in the ontology.</p><p>Apart of the more active involvement of the users in the ontology development process, the use of the Wiki as a platform allowed making the arguments or the statements in discussions more grounded by linking existing articles to the parts of the discussion statements. For example, the links to the items describing the concepts that are related for the discussion thread were inserted in the headings and the discussion statements using MediaWiki markups for hyperlinks. Those were either internal OntoDocWiki articles (e.g. Event) or the pointers to the external resources (for example <ref type="bibr" target="#b33">[34]</ref>) offered as back-up information in support of the discussion statements.</p><p>Fig. <ref type="figure" target="#fig_5">6</ref> shows some of the usage statistics for the OntoDocWiki. It is topical to notice that the peaks in access hits (one was on the 12-th of October 2010) depict the activity of users exactly in the periods of ontology discussions that occurred in different phases of the prototype development. It is also interesting that the most frequently visited OntoDocWiki pages were the ones describing the ontology concepts that were the most complex in semantics compared to the others. So, it took people more time and more visits to comprehend these ontology elements. Such information on the intensiveness of the use of different parts of ontology documentation turns out to be extremely important for the knowledge engineering team. Indeed, it objectively measures the complexity of the comprehension of the ontological concepts that could become a problem for gaining the ontological commitment by the community of the intended ontology users. These measures may be valuable in adjusting the tactics and foci in the ontology election campaigns. </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="7">Concluding Remarks</head><p>Ontology engineering as a field has been in intensive research and development for a substantial period of time. As we have hopefully shown in the paper, ontology engineering technology has passed the peak of inflated expectations on the Gartner's hype cycle curve (see e.g. http://en.wikipedia.org/wiki/Hype_cycle). Currently the position is in the proximity of the through of disillusionment, probably a little bit shifted to the slope of enlightenment. Most evidently, semantic technologies in broad and ontology engineering technologies in particular, will be better accepted by industries if and when they reach Gartner's plateau of productivity. Our intension while writing this discussion paper was to analyze the reasons for the disillusionment and, perhaps, to shed the light on the possible way up the slope to the region of industrial maturity. We believe that the right way is marked by the increase in the commitment of the intended industrial users.</p><p>We have outlined our views on a possible methodological framework for ontology engineering -OntoElect. We believe that our approach may be capable to relax several barriers on the way of gaining better and broader commitment to the use of ontologies in industrial applications. These beliefs are backed-up by our experience in implementing and using some of the elements of this framework in our research and development work in several European projects. We had positive experience in testing parts of the approach. For example, our work on PRODUKTIV+ <ref type="foot" target="#foot_4">5</ref> ontologies happened to be the informal competition of the two separate groups which views further converged to a single (merged) ontology suite -in debates. Another example that highlights the necessity of involving industrial subject experts and users as early as possible was our work on the PSI Suite of Ontologies together with the experts and users from Cadence Design Systems GmbH. The use case for testing the user friendly way for representing ontologies on a collaborative platform was the one in the ACTIVE project -presented in Section 6.</p></div><figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_0"><head>Fig. 1 .</head><label>1</label><figDesc>Fig.1. Ontology engineering lifecycle specified in ISO/IEC 24744 graphical notation for describing methodologies<ref type="bibr" target="#b17">[18]</ref>.</figDesc></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_1"><head>Fig. 2 .</head><label>2</label><figDesc>Fig. 2. OntoDocWiki article describing the PSI Suite of Ontologies v.2.3.</figDesc><graphic coords="11,125.34,331.26,344.64,203.46" type="bitmap" /></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_2"><head>Fig. 3 .</head><label>3</label><figDesc>Fig. 3. OntoDocWiki article describing the PSI Environment, Event, and Happening (E2H) ontology v.2.3.</figDesc><graphic coords="12,124.86,170.40,345.60,180.00" type="bitmap" /></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_3"><head>Fig. 4 .</head><label>4</label><figDesc>Fig. 4. OntoDocWiki article describing the concept of a Happening of the PSI E2H ontology v.2.3.</figDesc><graphic coords="12,125.16,389.04,344.94,183.42" type="bitmap" /></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_4"><head>Fig. 5 .</head><label>5</label><figDesc>Fig. 5. OntoDocWiki discussion page for the concept of a Happening.</figDesc><graphic coords="13,124.74,301.20,343.58,197.10" type="vector_box" /></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_5"><head>Fig. 6 .</head><label>6</label><figDesc>Fig. 6. The statistics of use of the OntoDocWiki resource provided by Google analytics.</figDesc><graphic coords="14,126.36,148.86,342.22,167.58" type="bitmap" /></figure>
			<note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="1" xml:id="foot_0">The numbers are indicative.</note>
			<note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="2" xml:id="foot_1">… apart of the incurred computational overhead.</note>
			<note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="3" xml:id="foot_2">This information has been acquired from a several year experience of industrial partnerships in knowledge intensive research and development projects.</note>
			<note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="4" xml:id="foot_3">PSI is the accomplished R&amp;D project of Cadence Design Systems GmbH.</note>
			<note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="5" xml:id="foot_4">PRODUKTIV+ is the accomplished R&amp;D project funded by the German Bundesministerium für Bildung und Forschung (BMBF).</note>
		</body>
		<back>

			<div type="acknowledgement">
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Acknowledgements</head><p>PSI Suite of Ontologies has been developed in the PSI project funded by Cadence Design Systems GmbH. OntoDocWiki development has been partially supported by the ACTIVE project funded in part by the European Commission Framework Programme 7. The authors would also like to thank the anonymous reviewers for their thorough comments that helped substantially to improve the paper.</p></div>
			</div>

			<div type="references">

				<listBibl>

<biblStruct xml:id="b0">
	<analytic>
		<title level="a" type="main">Toward Principles for the Design of Ontologies Used for Knowledge Sharing</title>
		<author>
			<persName><forename type="first">T</forename><forename type="middle">R</forename><surname>Gruber</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Int. J Human-Computer Studies</title>
		<imprint>
			<biblScope unit="volume">43</biblScope>
			<biblScope unit="page" from="907" to="928" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b1">
	<analytic>
		<title level="a" type="main">Semantic Wikipedia</title>
		<author>
			<persName><forename type="first">M</forename><surname>Krötzsch</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Vrandecic</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Völkel</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><surname>Haller</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Studer</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">J of Web Semantics</title>
		<imprint>
			<biblScope unit="volume">5</biblScope>
			<biblScope unit="issue">4</biblScope>
			<biblScope unit="page" from="251" to="261" />
			<date type="published" when="2007">2007</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b2">
	<analytic>
		<title level="a" type="main">Ontologies: Principles, Methods, and Applications</title>
		<author>
			<persName><forename type="first">M</forename><surname>Uschold</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Grüninger</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Knowledge Eng. Rev</title>
		<imprint>
			<biblScope unit="volume">11</biblScope>
			<biblScope unit="issue">2</biblScope>
			<biblScope unit="page" from="93" to="155" />
			<date type="published" when="1996">1996</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b3">
	<analytic>
		<title level="a" type="main">Possible Ontologies: How Reality Constrains the Development of Relevant Ontologies</title>
		<author>
			<persName><forename type="first">M</forename><surname>Hepp</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">IEEE Internet Computing</title>
		<imprint>
			<biblScope unit="volume">11</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="90" to="96" />
			<date type="published" when="2007">2007</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b4">
	<analytic>
		<title level="a" type="main">Theoretical foundations and engineering tools for building ontologies as reference conceptual models</title>
		<author>
			<persName><forename type="first">G</forename><surname>Guizzardi</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Semantic Web</title>
		<imprint>
			<biblScope unit="volume">1</biblScope>
			<biblScope unit="issue">2</biblScope>
			<biblScope unit="page" from="3" to="10" />
			<date type="published" when="2010">2010</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b5">
	<monogr>
		<ptr target="http://www.w3.org/TR/owl2-primer" />
		<title level="m">OWL 2 Web Ontology Language Primer</title>
				<imprint/>
	</monogr>
</biblStruct>

<biblStruct xml:id="b6">
	<monogr>
		<author>
			<persName><forename type="first">A</forename><surname>Gómez-Pérez</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Fernández-López</surname></persName>
		</author>
		<author>
			<persName><forename type="first">O</forename><surname>Corcho</surname></persName>
		</author>
		<title level="m">Ontological Engineering</title>
				<meeting><address><addrLine>London</addrLine></address></meeting>
		<imprint>
			<publisher>Springer</publisher>
			<date type="published" when="2004">2004</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b7">
	<analytic>
		<title level="a" type="main">DILIGENT: Towards a fine-grained methodology for distributed, loosely-controlled and evolving engineering of ontologies</title>
		<author>
			<persName><forename type="first">H</forename><forename type="middle">S</forename><surname>Pinto</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><surname>Tempich</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Staab</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Sure</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">16th European Conf. on Artificial Intelligence. ECAI</title>
				<editor>
			<persName><forename type="first">R</forename><forename type="middle">L</forename><surname>De Mántaras</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">L</forename><surname>Saitta</surname></persName>
		</editor>
		<imprint>
			<publisher>IOS Press</publisher>
			<date type="published" when="2004">2004</date>
			<biblScope unit="page" from="393" to="397" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b8">
	<monogr>
		<title level="m" type="main">On-To-Knowledge methodology: On-To-Knowledge: Semantic Web enabled Knowledge Management</title>
		<author>
			<persName><forename type="first">Y</forename><surname>Sure</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Studer</surname></persName>
		</author>
		<editor>Davies, J.</editor>
		<imprint>
			<date type="published" when="2002">2002</date>
			<publisher>J. Wiley and Sons</publisher>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b9">
	<analytic>
		<title level="a" type="main">Towards a Methodology for Building Ontologies</title>
		<author>
			<persName><forename type="first">M</forename><surname>Uschold</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>King</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">IJCAI&apos;95 Workshop on Basic Ontological Issues in Knowledge Sharing</title>
				<editor>
			<persName><forename type="first">D</forename><surname>Skuce</surname></persName>
		</editor>
		<meeting><address><addrLine>Montreal, Canada</addrLine></address></meeting>
		<imprint>
			<date type="published" when="1995">1995</date>
			<biblScope unit="page">10</biblScope>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b10">
	<analytic>
		<title level="a" type="main">A collaborative approach to ontology design</title>
		<author>
			<persName><forename type="first">C</forename><forename type="middle">W</forename><surname>Holsapple</surname></persName>
		</author>
		<author>
			<persName><forename type="first">K</forename><forename type="middle">D</forename><surname>Joshi</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Comm. ACM</title>
		<imprint>
			<biblScope unit="volume">45</biblScope>
			<biblScope unit="page" from="42" to="47" />
			<date type="published" when="2002">2002</date>
			<publisher>ACM Press</publisher>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b11">
	<analytic>
		<title level="a" type="main">Augmenting design deliberation with compendium: The case of collaborative ontology design</title>
		<author>
			<persName><forename type="first">S</forename><surname>Buckingham-Shum</surname></persName>
		</author>
		<author>
			<persName><forename type="first">E</forename><surname>Motta</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Domingue</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">HypACoM 2002: Facilitating Hypertext-Augmented Collaborative Modeling</title>
				<imprint>
			<date type="published" when="2002">2002</date>
		</imprint>
		<respStmt>
			<orgName>Maryland University</orgName>
		</respStmt>
	</monogr>
	<note>ACM Hypertext&apos;02 Workshop</note>
</biblStruct>

<biblStruct xml:id="b12">
	<analytic>
		<title level="a" type="main">HCOME: tool-supported methodology for collaboratively devising living ontologies</title>
		<author>
			<persName><forename type="first">K</forename><surname>Kotis</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><forename type="middle">A</forename><surname>Vouros</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><forename type="middle">P</forename><surname>Alonso</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">SWDB&apos;04: Second International Workshop on Semantic Web and Databases</title>
				<imprint>
			<publisher>Springer-Verlag</publisher>
			<date type="published" when="2004">2004</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b13">
	<monogr>
		<title level="m" type="main">Knowledge Engineering and Management. The CommonKADS Methodology</title>
		<author>
			<persName><forename type="first">G</forename><surname>Schreiber</surname></persName>
		</author>
		<imprint>
			<date type="published" when="1999">1999</date>
			<publisher>MIT Press</publisher>
			<pubPlace>Cambridge, USA</pubPlace>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b14">
	<monogr>
		<title level="m" type="main">1: NeOn Methodology for Building Contextualized Ontology Networks</title>
		<author>
			<persName><forename type="first">M</forename><forename type="middle">C</forename><surname>Suárez-Figueroa</surname></persName>
		</author>
		<imprint>
			<date type="published" when="2008">D5. 2008</date>
			<biblScope unit="volume">4</biblScope>
		</imprint>
		<respStmt>
			<orgName>The NeOn Project</orgName>
		</respStmt>
	</monogr>
	<note type="report_type">Technical report</note>
</biblStruct>

<biblStruct xml:id="b15">
	<monogr>
		<title level="m" type="main">A Contextual Approach to Ontology Reuse: Methodology, Methods and Tools for the Semantic Web</title>
		<author>
			<persName><forename type="first">E</forename><surname>Pâslaru-Bontaş</surname></persName>
		</author>
		<imprint>
			<date type="published" when="2007">2007</date>
			<pubPlace>Berlin</pubPlace>
		</imprint>
		<respStmt>
			<orgName>Freie Universität Berlin</orgName>
		</respStmt>
	</monogr>
	<note type="report_type">PhD Thesis</note>
</biblStruct>

<biblStruct xml:id="b16">
	<analytic>
		<title level="a" type="main">A Cost Estimation Model for Ontology Engineering</title>
		<author>
			<persName><forename type="first">E</forename><surname>Simperl</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><surname>Tempich</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Sure</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">ISWC 2006. LNCS</title>
				<editor>
			<persName><forename type="first">I</forename><forename type="middle">F</forename><surname>Cruz</surname></persName>
		</editor>
		<meeting><address><addrLine>Berlin, Heidelberg</addrLine></address></meeting>
		<imprint>
			<publisher>Springer</publisher>
			<date type="published" when="2006">2006</date>
			<biblScope unit="volume">4273</biblScope>
			<biblScope unit="page" from="625" to="639" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b17">
	<analytic>
		<title level="a" type="main">Standardizing Methodology Metamodelling and Notation: An ISO Exemplar</title>
		<author>
			<persName><forename type="first">B</forename><surname>Henderson-Sellers</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><surname>Gonzalez-Perez</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">UNISCON 2008. LNBIP</title>
				<editor>
			<persName><forename type="first">R</forename><surname>Kaschek</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">C</forename><surname>Kop</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">C</forename><surname>Steinberger</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">G</forename><surname>Fliedl</surname></persName>
		</editor>
		<meeting><address><addrLine>Berlin, Heidelberg</addrLine></address></meeting>
		<imprint>
			<publisher>Springer</publisher>
			<date type="published" when="2008">2008</date>
			<biblScope unit="volume">5</biblScope>
			<biblScope unit="page" from="1" to="12" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b18">
	<analytic>
		<title level="a" type="main">Survey on Social Tagging Techniques</title>
		<author>
			<persName><forename type="first">M</forename><surname>Gupta</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Yin</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Han</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">SIGKDD Explorations</title>
		<imprint>
			<biblScope unit="volume">12</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="58" to="72" />
			<date type="published" when="2010">2010</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b19">
	<analytic>
		<title level="a" type="main">Semantic annotation for knowledge management: Requirements and a survey of the state of the art</title>
		<author>
			<persName><forename type="first">V</forename><surname>Uren</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Cimiano</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Iria</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Handschuh</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Vargas-Vera</surname></persName>
		</author>
		<author>
			<persName><forename type="first">E</forename><surname>Motta</surname></persName>
		</author>
		<author>
			<persName><forename type="first">F</forename><surname>Ciravegna</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Science. Services and Agents on the World Wide Web</title>
		<imprint>
			<biblScope unit="volume">4</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="14" to="28" />
			<date type="published" when="2006">2006</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b20">
	<analytic>
		<title level="a" type="main">HarvANA -Harvesting Community Tags to Enrich Collection Metadata</title>
		<author>
			<persName><forename type="first">J</forename><surname>Hunter</surname></persName>
		</author>
		<author>
			<persName><forename type="first">I</forename><surname>Khan</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Gewrber</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">8th ACM/IEEE-CS Joint Conference on Digital Libraries</title>
				<editor>
			<persName><forename type="first">A</forename><surname>Paepcke</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">J</forename><surname>Borbiha</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">M</forename><surname>Naaman</surname></persName>
		</editor>
		<meeting><address><addrLine>New York; New York</addrLine></address></meeting>
		<imprint>
			<publisher>ACM</publisher>
			<date type="published" when="2008">2008</date>
			<biblScope unit="page" from="147" to="156" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b21">
	<analytic>
		<title level="a" type="main">Games with a Purpose for the Semantic Web</title>
		<author>
			<persName><forename type="first">K</forename><surname>Siorpaes</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Hepp</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">IEEE Intelligent Systems</title>
		<imprint>
			<biblScope unit="volume">23</biblScope>
			<biblScope unit="issue">3</biblScope>
			<biblScope unit="page" from="50" to="60" />
			<date type="published" when="2008">2008</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b22">
	<analytic>
		<title level="a" type="main">Community-driven Ontology Construction in Social Networking Portals</title>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">V</forename><surname>Zhdanova</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Web Intelli. and Agent Sys</title>
		<imprint>
			<biblScope unit="volume">6</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="93" to="121" />
			<date type="published" when="2008">2008</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b23">
	<analytic>
		<title level="a" type="main">Games with a Purpose</title>
		<author>
			<persName><forename type="first">L</forename><surname>Von Ahn</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Computer</title>
		<imprint>
			<biblScope unit="volume">29</biblScope>
			<biblScope unit="issue">6</biblScope>
			<biblScope unit="page" from="92" to="94" />
			<date type="published" when="2006">2006</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b24">
	<analytic>
		<title level="a" type="main">Tagatune</title>
		<author>
			<persName><forename type="first">E</forename><surname>Law</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proc. Int&apos;l Conf. Music Information Retrieval</title>
				<meeting>Int&apos;l Conf. Music Information Retrieval<address><addrLine>Ismir</addrLine></address></meeting>
		<imprint>
			<publisher>Austrian Computer Soc</publisher>
			<date type="published" when="2007">2007</date>
			<biblScope unit="page" from="361" to="364" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b25">
	<analytic>
		<title level="a" type="main">Common Consensus: A Web-Based Game or Collecting Commonsense Goals</title>
		<author>
			<persName><forename type="first">H</forename><surname>Lieberman</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Smith</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Teeters</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proc. Workshop Common Sense for Intelligent Interfaces, ACM Conf. Intelligent User Interfaces</title>
				<meeting>Workshop Common Sense for Intelligent Interfaces, ACM Conf. Intelligent User Interfaces</meeting>
		<imprint>
			<publisher>ACM Press</publisher>
			<date type="published" when="2007">2007</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b26">
	<analytic>
		<title level="a" type="main">Pragmatic Semantic Unification</title>
		<author>
			<persName><forename type="first">C</forename><surname>Petrie</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">IEEE Internet Computing</title>
		<imprint>
			<biblScope unit="volume">9</biblScope>
			<biblScope unit="issue">5</biblScope>
			<biblScope unit="page" from="95" to="96" />
			<date type="published" when="2005">2005</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b27">
	<analytic>
		<title level="a" type="main">Ontological Foundations for Structural Conceptual Models</title>
		<author>
			<persName><forename type="first">G</forename><surname>Guizzardi</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Telematica Instituut Fundamental Research Series</title>
		<imprint>
			<biblScope unit="volume">15</biblScope>
			<date type="published" when="2005">2005</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b28">
	<analytic>
		<title level="a" type="main">Using Contexts in Ontology Structural Change Analysis</title>
		<author>
			<persName><forename type="first">V</forename><surname>Ermolayev</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Copylov</surname></persName>
		</author>
		<author>
			<persName><forename type="first">N</forename><surname>Keberle</surname></persName>
		</author>
		<author>
			<persName><forename type="first">E</forename><surname>Jentzsch</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W-E</forename><surname>Matzke</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="s">CIAO, CEUR-WS</title>
		<editor>Ermolayev, V., Gomez-Perez, J.-M., Haase, P., Warren, P.</editor>
		<imprint>
			<biblScope unit="volume">626</biblScope>
			<date type="published" when="2010">2010</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b29">
	<analytic>
		<title level="a" type="main">Applied Research and Development in Cooperation with Industry</title>
		<author>
			<persName><forename type="first">V</forename><surname>Ermolayev</surname></persName>
		</author>
		<author>
			<persName><forename type="first">O</forename><surname>Tatarintseva</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Information Technologies in Education</title>
		<imprint>
			<biblScope unit="volume">5</biblScope>
			<biblScope unit="page" from="16" to="26" />
			<date type="published" when="2010">2010</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b30">
	<analytic>
		<title level="a" type="main">Performance Simulation Initiative</title>
		<author>
			<persName><forename type="first">V</forename><surname>Ermolayev</surname></persName>
		</author>
		<author>
			<persName><forename type="first">E</forename><surname>Jentzsch</surname></persName>
		</author>
		<author>
			<persName><forename type="first">N</forename><surname>Keberle</surname></persName>
		</author>
		<idno>PSI-ONTO-TR-2010-1</idno>
	</analytic>
	<monogr>
		<title level="m">The Suite of Ontologies v.2.3. Reference Specification</title>
				<imprint>
			<date type="published" when="2010">2010</date>
		</imprint>
		<respStmt>
			<orgName>VCAD EMEA, Cadence Design Systems, GmbH</orgName>
		</respStmt>
	</monogr>
	<note type="report_type">Technical Report</note>
</biblStruct>

<biblStruct xml:id="b31">
	<analytic>
		<title level="a" type="main">Holonic simulation of a design system for performance analysis</title>
		<author>
			<persName><forename type="first">R</forename><surname>Sohnius</surname></persName>
		</author>
		<author>
			<persName><forename type="first">E</forename><surname>Jentzsch</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W</forename><surname>Matzke</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proc. of the 3rd international conference on Industrial Applications of Holonic and Multi-Agent Systems</title>
				<meeting>of the 3rd international conference on Industrial Applications of Holonic and Multi-Agent Systems<address><addrLine>Berlin, Heidelberg</addrLine></address></meeting>
		<imprint>
			<publisher>Springer-Verlag</publisher>
			<date type="published" when="2007">2007</date>
			<biblScope unit="page" from="447" to="454" />
		</imprint>
	</monogr>
	<note>HoloMAS &apos;07</note>
</biblStruct>

<biblStruct xml:id="b32">
	<analytic>
		<title level="a" type="main">Increasing predictability and sharing tacit knowledge in electronic design</title>
		<author>
			<persName><forename type="first">V</forename><surname>Ermolayev</surname></persName>
		</author>
		<author>
			<persName><forename type="first">F</forename><surname>Dengler</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><surname>Fortuna</surname></persName>
		</author>
		<author>
			<persName><forename type="first">T</forename><surname>Stainer</surname></persName>
		</author>
		<author>
			<persName><forename type="first">T</forename><surname>Bosser</surname></persName>
		</author>
		<author>
			<persName><forename type="first">E-M</forename><surname>Melchior</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Context and semantics in knowledge management</title>
				<editor>
			<persName><forename type="first">P</forename><surname>Warren</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">E</forename><surname>Simperl</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">J</forename><surname>Davies</surname></persName>
		</editor>
		<meeting><address><addrLine>Berlin, Heidelberg</addrLine></address></meeting>
		<imprint>
			<publisher>Springer Verlag</publisher>
			<date type="published" when="2011">2011</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b33">
	<analytic>
		<title level="a" type="main">An Ontology of Environments, Events, and Happenings</title>
		<author>
			<persName><forename type="first">V</forename><surname>Ermolayev</surname></persName>
		</author>
		<author>
			<persName><forename type="first">N</forename><surname>Keberle</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W.-E</forename><surname>Matzke</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Proc 31st IEEE Annual International Computer Software and Applications Conference</title>
				<meeting>31st IEEE Annual International Computer Software and Applications Conference<address><addrLine>Turku, Finland</addrLine></address></meeting>
		<imprint>
			<date type="published" when="2008">2008</date>
			<biblScope unit="page" from="539" to="546" />
		</imprint>
	</monogr>
</biblStruct>

				</listBibl>
			</div>
		</back>
	</text>
</TEI>
