In this paper the classicist and the connectionist debate is integrated in a multimedia data mining framework. Due to the ready availability of visual information digitally, the problem to store, retrieve and organize images is gaining tremendous importance and poses a serious challenge for practitioners of AI and databases alike. The central problem for image database management is the idea of an annotation associated with every image. This idea of an annotation as a representation of an image is explored in this paper. The claim is that arriving at a suitable annotation using data mining techniques however does not solve the problem of representation and reference. Most image annotation techniques are classicist, while connectionist approaches which have the advantage of contextuality have not been explored much because of the problem of systematicity. Consequently the image annotation problem can be used as a critique of both.
Knowledge Representation (KR henceforth) deals with the problem of
representing and storing knowledge. Hence, KR has a strong a nity with Epistemology.
KR has been one of the foremost problems in AI since its inception. A
representation is a surrogate for the thing itself [
McCarthy et. al [
Scalability is perhaps the most severe problem for any data mining
technique [
The two dominant paradigms in Knowledge Representation are the connectionist and the classicist paradigms. Most data mining algorithms are classicist in the sense that knowledge is almost always represented as symbols or in computational terms as bits of strings. This is especially pronounced in scenarios where the data is well structured, although the "solutions" are sometimes marred by scalability problems. The problem recurs in connectionist systems also. In selecting an architecture for solving a particular mining problem, the scalability of its implementation in the particular context is generally taken into consideration. It would not be too much of an exaggeration to state that the classicist vs. the connectionist debate is really between local vs. distributed representations.
Connectionism primarily draws its inspiration from the human brain,
although associationism which goes as far back as David Hume subsumes
connectionism. Connectionism claims that knowledge is represented in a distributed
non-symbolic manner in the connection strengths between neurons.
Connectionist architectures like neural networks have been employed for data mining since
its beginning, although the structured nature of many mining problem makes it
appear as if they are not susceptible to connectionist solutions. There is a middle
ground between the two paradigms, the so called implementational
connectionism for whom the the aim of connectionism is to nd connectionist architectures
that implement classicist architectures at a higher more abstract level. This line
of research has not been pursued much in data mining, where it can be applied
to problems like auto annotation of images. Perhaps the greatest drawback that
connectionism has is its lack of systematicity [
The task of image annotation is trivial for a human being but when the task is
transferred to the machine learning domain it becomes challenging [
We have reframed the problem of image auto-annotation to that of multirelational association rule mining where relations exist between image-based features and textual annotations. We then combined low-level image features such as color, orientation, intensity, etc. and corresponding text annotations to generate multi-relational association rules that were used to generate annotations. The relations between these media are not rigid. The idea is that annotations expressed as multiple relations stores more information as compared to when these are considered in isolation. 3
In a previous work [
The MRFP-Growth algorithm that we have used in CoMMA, builds upon the
FP-Growth algorithm and consists of two phases. The rst phase of the algorithm
involves running the MR FP-Tree algorithm separately on all the tables. This
phase is similar to the FP Tree algorithm but with one major di erence. Each
node in the tree not only keeps track of its support but also keeps track of the
indices in the dataset where it occurs. In the second phase the table that was
populated with rules mined from MR FP-Trees in the previous phase is used to
make an MR FP-tree. Once the tree is made, it is mined for rules as speci ed
earlier. The task of MR FP-Tree can thus be stated as that of nding relations
from the primary table to each of the secondary tables and then between the
secondary tables. Refer to our CoMMA paper [
Original Keywords: small
bush owers ground Auto-Annotations: small bush grass leaves radiant
owers berries ground weeds earth dry desert
Original Keywords: bush
owers ground Auto-Annotations: small
bush over forest river ground tree leaves bushes grass sky owers trees rocks
Original Keywords: lake
summit in Alaska Auto-Annotations: still lake summit Alaska near rock rocks desert melting ice walls The relations between images, their annotations and the image features are more than just statistical correlations and the relations are not rigidly de ned. The system overall ts into the traditional classicist scheme. The results of some of the CoMMA generated annotations are given in Figure 1. From these annotations it is evident that the system picks up major objects present in the picture, though the annotations are not really perfect. The results vary as the support and con dence is varied. Hence the task is to get an optimal value for these two parameters for a particular database. 4
Ever since Alan Turing formulated his test for intelligence other versions of the test have been proposed like the Total Turing Test, the Reverse Turing Test, the modi ed Turing Test, the Limited or constrained Turing Test. One can reframe the problem of image annotation to that of a constrained Turing Test where the task is to nd appropriate words to describe an image. Lets call it the "Annotation Turing Test." Although a system that can annotate images in an open ended domain is currently impossible, the Annotation Turing Test can be conducted in a limited domain, say dealing with only scenic images or sports images i.e.,a Constrained Annotation Turing Test (CATT henceforth). Section 3 described a system for automatically annotating images and although the system is far from perfect in auto-annotation of images but the fact that it and other similar systems can annotate images with reasonable accuracy gives one reasons to believe that a domain speci c auto-annotation system might not be that far o .
Now consider the question, is the system said to have the ability to understand images if it can properly annotate them? Admittedly the test has some limitations but since CATT is a constrained test to begin with, this is a nonissue. If one takes a purely functionalist stance then one is forced to say that it does, but there is clearly something wrong with this assertion. The problem arises because of the confusion between cognition and representation. The Problem with CATT is that seemingly intelligent behavior exhibited by the system is not generative. The system is brittle if moved from one domain to the next. The Turing Test would be vindicated if a general purpose non-domain speci c system could be built but the very notion of such a system has many problems. The reason that humans have a need to name objects (another way of saying annotating) is because they are situated in an environment where they have to interact with the world and that these objects have some use for them, in short a human is, as Heidegger puts it a being-in-the-world or dasein. This is perhaps the reason why the Holy Grail of AI i.e., general purpose reasoning system has been so elusive. 5
Data mining seeks to extract hidden knowledge from large amounts of data.
But what constitutes hidden knowledge really? Most traditional data mining
algorithms are either search algorithms or their derivatives. Hence data
mining is confronted with the same problems that earlier plagued many classicist
architectures i.e., the problem of determining what is relevant. In data mining
tweaking relevant parameters in an algorithm usually ensures that one gets the
desired results e.g., consider the case of Multi-relational Association Rule
Mining described in section [
Classicists architectures also fail to account for the role of context in real
world problems. Connectionism o ers the hope of doing away with the problem
of holistic representations [
What AI can learn from data mining is that although domain speci c human
capabilities can be mimicked by search methods, these are just search mechanisms
and nothing more. This is of course the lesson which should have been learnt
from the Expert System epoch in AI. It can be interpreted as either a short
coming of functionalism or a failure to distinguish between cognition and
understanding. We have stated the shortcomings of the classicist and the connectionist
paradigms with respect to the problem of image annotation and what
generalizations can we make from the lessons learned here. The symbol manipulation
paradigm of Classicism has also been attacked vigorously by Dreyfus [
Dreyfus has pointed out that classicism is in part motivated by the
rationalist tradition in Philosophy. Hence an image annotation system can annotate
images with a reasonable accuracy but such a system refers objects and
situations in the image in the same sense that the images refer to these objects in
the physical world. Thus the problem is that of intentionality. Dreyfus however
thinks [
As argued in section 4 a human is a being-in-the-world. Her intelligence is in part due to her interaction with the world. This is why an open-ended, non domain speci c annotation system is a too di cult and complex problem to be solved by any of the currently available paradigms. Of course it would not be practical to build a system that learns annotations from the world by physically interacting with it. Such a project would of course the defeat the purpose of auto annotation in the rst place. current techniques in image annotation like association rule mining, object identi cation, viewpoints etc. perform reasonably in this task. Algorithmic improvements on these approaches will certainly yield better results, even though the domain restrictions will imply nonetheless. 7
Many of the problems that are traditionally associated with AI were considered from a data mining perspective by speci cally considering the image mining problem. The strengths and weaknesses of the classicist and the connectionist paradigms were considered. The contextualization problem in classicism is a major impedent to generalization. This is one of the main reasons why image annotation systems have to restricted to certain image domains. Connectionism works best in situations where the KR is distributed. One has to consider the trade o s when considering the problem to be solved.
An image annotation can also be thought of as a representation of the image itself. We also argued that system that annotates images cannot be said to refer to anything in the image, where reference is an intentional construct. CATT also demonstrates that a genuinely referring system has to understand what it is referring. An open ended image annotation system can thus on be part of a larger general problem solving system. It does not have to be similar to Heidegger's dasein or being-in-the-world. However the establishment of Merleau Ponty's intentional arc might be a necessary condition. While considering the problem of image annotation we consider our own newly devised system CoMMA. CoMMA uses multi-relational association rules to come up with annotations for images. The system performs reasonably well within a restricted domain. However the performance deteoriates as more domains are added. Although CoMMA is classicist, the relations between annotations can be thought of as associationalist if not connectionist.
What does the Turing test really tell us about the intelligence of a system or even about intelligence at all? A debate on CATT revealed that particular intelligent capabilities are not isolated but are always part of the whole. Since a human, a being-in-the-world according to Heidegger, is always situated in an environment. It is due to her interaction with the world that meaning and reference arises, including naming objects and situations. Hence situatedness is be a necessary condition for any general level intelligent capability like annotating in an open ended domain.
Possible arenas of research for the future include extending CoMMA to give descriptions of the images, test the performance of the system in multiple disparate domains and improve upon the MR FP Tree algorithm. Perhaps the long term goal of such a project could be to test an implementational connectionist architecture or some other hybrid connectionist-classicist system that can give us a better idea about what constitutes a reference. The relation between reference and the establishment of an intentional arc should be explored.