This project supports application for a practice-based [4] PhD by Peter Goodall. The project will produce a system architecture and proof-of-concept laboratory implementation to model, instrument, prototype, and evaluate the e ectiveness of several alternate system designs which are intended to enable small end-user communities to evolve specialized ontologies and annotations for entities important to them. Depending on available resources, the laboratory may also be used to study bridging-subset ontologies for interchange, federation and seeding of community ontologies. There will be a discussion of the strengths and weaknesses of various approaches tried by others, and a re ection on the usefulness of the system resulting from innovative work of this project.
Each living human community, small or large, cultural, scienti c or commercial has its characteristic evolving shared nomenclature which enables discussion and action involving entities and activities important to that community. Further, communities that interact or exchange information with each-other require intersecting terminologies.
Ontology is the concept which encompasses terminology with its meanings and mechanisms. Ontology has philosophical and computational streams of interest to this project: Philosophical Ontology has been described as `that branch of metaphysics concerned with the nature or essence of being or existence' [17]. Its modern child - Computational Ontology can be described as `a conceptualization of some universe of discourse, embodied as a declarative formal abstraction'[14].
This project is motivated by observation of a contradiction between the
curatorial and cultural perspectives of an Australian Research Council Linkage Grant
(now inactive) which I project-managed - known as `The Virtual Museum of the
Paci c'[
The Cultural Collections section of our project partner - The Australian Museum, has over many decades iterated through developing or adopting controlled vocabularies for cataloging their Paci c Collection. They currently use a concise in-house taxonomy for very practical reasons.
While preparing for the launch of the project web-site, we spent time working with representatives of some Paci c communities. They were concerned that the collection taxonomy had little relationship with their own way of describing their artifacts within the collection.
Both the cultural and curatorial ontologies struggled to be e ective in the context of the collection. Both the curatorial sta and cultural owners were experts in the conceptualization of their particular perspectives of the collection, and both need aid in ongoing development of terms to suit their both separate and overlapping needs.
My project goal is to research and develop systems for exploring ontologies computationally emergent from technically-informal community annotation. 2
Social - many communities of commitment or interest [
Commercial - because nearly every modern large-scale electronic market or business, has a shortage of technical experts to formally classify objects and data for cataloging and recommendation.
Both of these application areas could greatly bene t from further development of categorization generated from community or customer based annotation. 3
Eleanor Rosch and others, in a number of seminal papers [18,19] developed the notions of Basic Categories and Prototype Theory, prosecuting the idea that there are a number of fundamental levels of category that have a degree of coherence amongst end-users. This experimental work inspires some con dence in community labeling of objects being used to produce derivable category structure. More recently work on the Rational Model of Categorization and the Feature Induction Model examined by Sanborn [20] show promise for inferring categories from user tags.
The work of Cimiano and colleagues [
Mika [15] provides an extended ontology tripartite hypergraph model, splitting the hypergraph into three bipartite graphs associating actors with concepts, concepts with instances and actors with instances. These bipartite graphs are then folded using matrix operations to create various useful a liation networks, whose properties are used to develop lightweight ontologies demonstrated via a number of case-studies.
A data-model for capture, archiving and processing of user tagging events
requires some thought. A user labels an object while engaged in some cognitive
context, the person chooses a symbol relevant to that context as a reminder
of the concept from that context. Some investigation of processes and schools
of Semiotics [
The time-trace of these tags bears an interesting a nity with Dynamic Topic
Modeling and Topic Hierarchies [
As noted in the Problem Description of this overview, development of
category systems by domain experts who are not ontologists is discouraged by the
complexity of traditional approaches. This complexity is re ected in the
software tools readily available. Protege [13] is an example of a quality open-source
ontology development system, many others are available in various states of
development usage and repair [
Research Questions 1. How to model simple hierarchies of community generated categories? 2. What is the relationship between community and classi cation? 3. Are time-based data-sets available which demonstrate category drift within a community over time? 4. How much information external to the tagging discourse is required for an ontology to emerge? 5. Is emergence as a computational model valid? 6. How to represent collections of tagging events in a way that is simple and e ective, yet consistent with relevant semiotic theory? 7. How to model requirements for selecting suitable experimental datasets and feeds 8. How to access and transform datasets/feeds for ingestion into the system, and select appropriate tool-sets for this? 9. How to represent and derive ontologies from tagging event streams? 10. Which formal models of tags and ontologies are suitable for incremental and bulk computation? 11. De ne a system architecture for processing, interaction, and visualization. 5
These hypotheses are still very much under development. 1. Basic-level ontologies can be computationally emergent from a suitably annotated tagging event stream. The emergent ontologies will be judged useful by the annotating community. 2. Ontologists or curators working in the domain of a community's emergent ontologies (as derived above) will judge those ontologies to be useful to their work. 3. A proof of concept technical work-bench/tool-chain can be constructed from Free and Open Source Software (FOSS) and reasonably generic cloud computing resources, which can be used to e ectively experiment with extraction of latent categories in tagging discourses. 4. Cohesiveness of a subject domain's community of interest corresponds to the coherence of the vocabulary used when tagging objects from that domain. 5. A cohesive community of interest tagging their domain of interest, will produce a stream of tagging events whose latent categorizations will converge into a set of basic and superordinate categories [19], which are recognized as useful by that community. 6. A tagging event data-set with a long-enough time-base will demonstrate temporal drift of its latent categories. 7. Scalable algorithms can be found to derive basic and superordinate categories incrementally from suitable annotation event streams. 6
{ the fundamental relationships between semiotics, classi cation, community, crowd-sourcing, folksonomy. { computational topics - such as Formal Concept Analysis, ontology extraction from text, functional programming, analysis packages such as R, Incanter and Pandas. 2. Characterize and discover suitable data-sets. It is important to nd, where possible, suitable existing data-sets for testing hypotheses before committing to the expense and di culty of user-testing.
{ There are a number of ad-hoc web-sites indicating the existence of possibly suitable data-sets, many of these sites are out-of-date or informally curated. { Formal experimental data-set registries are being actively implemented.
These also need characterization and selection to nd examples suitable for this project. { Stream based data-feeds of tagging events are particularly desirable, especially if the same sources have archives of their data-feeds. 3. Identify end-user communities that have an interest in curation of their subject areas. Although very disappointing, strong advice that the di culty of progressing an ethnographic project through ethics committee process has caused me to design experiments using contemporary western user domains. I am still expecting to use members of some communities of interest - where groups have overlapping domains but divergent interests. 4. Perform preliminary processing of data-sets to re ne technical approach and use that to inform further development of my thesis. I expect to iterate through this cycle several times. 5. Once algorithms for deriving categorizations latent in the test data-streams are performing adequately it will be feasible to begin on user-interface development and visualization. I expect to base the display and tagging interface for end-users on the The Virtual Museum of the Paci c, and iteratively evolve it from that base. 6. Once an end-user user-interface is working under informal testing, nalization of user-testing strategy should be developed. Resources such as Amazon Mechanical Turk will probably be used for scalable user-testing. 7. Finalize the system architecture for the experimental work-bench. At this stage of the project I should have su cient knowledge of the characteristics of data-sets, user-iteration and analysis tools to perform this task with some con dence. Initially I am strongly motivated to use FOSS where practicable, and to implement the system so that it can be scaled by moving from a Linux-based workstation onto a cloud platform. 8. Complete the thesis and package the system for archiving. 7
I don't regard my possible success as a sign of failure of others, rather I hope to answer a variant requirement with my own synthesis. Most of the work done with Ontology development has been very formal, which excludes most busy domain-experts from their development.
I hope to produce some interesting results which are a step on the way to generating useful, evolvable user-ontologies, and that the 'workbench' of tools I compose will be useful to others in the eld, or a good starting point for further development. 8
There are three facets of evaluation for this project: 1. Subject domain usefulness - If a community of end-users tags objects from their domain of interest, they should recognize the extracted categories and rate them as useful 2. Ontologist usefulness - A working ontologist or curator should nd the extracted categories useful in themselves, and in a standard format that they can use of in their normal working environment. They should also recognize features in the project's developed workbench that they would like to see as an improvement in their professional working environment. 3. Workbench deployment e ectiveness - The running and deployment of the work-bench should be accessible to a researcher in ontology extraction who has a reasonable ability to maintain and con gure a Linux workstation. 4. Demonstrate that relationships between annotators can be convincingly deduced by agent a liation analysis from the tripartite graph of agent, object, and annotation. Agents would need some veri able, relevant relationship whose description is independent of the tagging dataset. 5. Given a reasonably large tagging event stream from a particular community, determine how well computationally emergent ontologies from that dataset are received by that community. 6. Use temporal sliding windows to generate subsets of a large tagging event stream as input to observe if temporal category drift is observable. 7. Demonstrate computation that generates emergent ontologies incrementally in an environment where the whole data-set cannot be held in system RAM. 13. Gennari, J.H., Musen, M.A., Fergerson, R.W., Grosso, W.E., Crubzy, M., Eriksson, H., Noy, N.F., Tu, S.W.: The evolution of protg: an environment for knowledge-based systems development. International Journal of Human-computer studies 58(1), 89123 (2003), http://www.sciencedirect.com/science/article/pii/S1071581902001271 14. Gruber, T.R., et al.: A translation approach to portable ontology speci cations.
Knowledge acquisition 5, 199199 (1993) 15. Mika, P.: Ontologies are us: A uni ed model of social networks and semantics.
Web Semantics: Science, Services and Agents on the World Wide Web 5(1), 5{15 (Mar 2007), http://www.sciencedirect.com/science/article/
B758F-4MYF67P-1/2/56984a3ddf4632bb98b722551cdb1151 16. Nth, W.: Handbook of semiotics. Indiana University Press (1995), http: //books.google.com/books?hl=en&lr=&id=rHA4KQcPeNgC&oi=fnd&pg=PR9&dq= Handbook+of+Semiotics&ots=ddo1tWkS4f&sig=9NNalr 6O7bREpvCPhwQXB9lBn4 17. Oxford English Dictionary: "ontology, n.". (2004),
http://www.oed.com/view/Entry/131551?redirectedFrom=Ontology 18. Rosch, E., Mervis, C., Gray, W., Johnson, D., Boyes-Braem, P.: Basic objects in natural categories. Cognitive psychology 8(3), 382439 (1976) 19. Rosch, E.: Principles of categorization. Concepts: core readings p. 189206 (1999), http://books.google.com/books?hl=en&lr=&id=sj1gczQ-7K8C&oi=fnd&pg= PA189&dq=%0D%0A1%0D%0APrinciples+of+Categorization%0D%0AEleanor+ Rosch,+1978+&ots=NoqbGizR2u&sig=0WEAZVboo9yC0P8tnryxMC7DT3M 20. Sanborn, A.N., Gri ths, T.L., Navarro, D.J.: A more rational model of categorization. In: Proceedings of the 28th annual conference of the cognitive science society. p. 726731 (2006), http://citeseerx.ist.psu.edu/viewdoc/ download?doi=10.1.1.163.3800&rep=rep1&type=pdf 21. Wenger, E.: Communities of Practice: Learning, Meaning, and Identity.
Cambridge University Press (Sep 1999)