Automated Story Illustrator Anshita Talsania Sandip Modha Hardik Joshi Charotar University of Science and L.D.R.P. College, Gujarat University, Technology (Charusat), Gandhinagar, Ahmedabad, Anand, India. India. India. sjmodha@gmail.com joshee@acm.org anshita.talsania@gmail.com Amit Ganatra Charotar University of Science and Technology (Charusat), Anand, India. amitganatra.ce@charusat.ac.in 2. DATA ABSTRACT The task uses two different data components: one is an image The Automated Story Illustration is a task under FIRE 2015 to dataset containing all possible images available for illustration be organized in DAIICT, Gandhinagar. The participants are and another one is the children's short stories that need to be required illustrate stories automatically by retrieving a set of illustrated. In this task, ImageCLEF Wikipedia Image images from an image dataset and if it is the case, identifying Retrieval 2010 is used as the image dataset. This dataset which concepts and events in the text should be illustrated. consists of 237,434 images along with their captions metadata. This paper overviews the task, the approach- the model and the Captions are available in English, French and/or German. tool used to carry out the task. Secondly, the stories that need to be illustrated are all children’s short stories. Additionally annotations are provided Keywords for each story to indicate what portion of the story needs to be Automated Story Illustrator, Illustrated story. illustrated. Annotations are in form important entities (nouns or noun phrases) and events (a combination of entities and verbs or verb phrases) that need illustration in a story. 1. INTRODUCTION The task “Automated Story Illustration” requires to illustrate stories automatically by retrieving set of images from the 3. APPROACH image dataset. The key tasks involved to are: For creating the story illustration, primarily we need to a. Identify concepts and events in the text that should be perform information retrieval – query important passages of illustrated (annotations). the story and retrieve the corresponding image representation available from the image dataset provided i.e perform two b. Selecting best illustration from the image dataset for that main tasks: particular concept/event. a. Indexing -Mapping of terms (basic indexed units) to In the FIRE task, participants are provided with multiple documents in a corpus children’s short stories which need to be illustrated using the ImageCLEF Wikipedia Image Retrieval dataset. The story text b. Retrieval - Generation of results due to a query (information as well as the important entities and events that need need) illustration [1] in it are provided. The objective is to provide To perform these tasks, several open source tools are available. one ranked list of images corresponding to each important These tools differ on the grounds of their indexing and entity and event in a story. retrieval models used. We choose the Terriertool [3] as it is The need of this research stems from the fact that we often ideal for performing information retrieval experiments. Terrier forget what we read few sentences before. Our reading can index large corpus of documents with multiple indexing memory is affected due to boredom, lack of attention or strategies. Additionally it is highly effective providing support distraction. Studies [2] suggest creating visual illustrations can of state of art retrieval approaches like DFR and BM25. improve reading memory therefore going a long way in helping children and elder people who often face reading memory problems. 3.1 Indexing Indexing using the Terrier tool is performed on the ImageCLEF dataset provided. The tool utilizes entire image caption metadata represented in form of XML and then performs indexing using configuration set within the tool. 71 each story is used as query. Figure-3 provides an outline of the retrieval process in Terrier: Figure 1: Indexing Architecture Figure 1: outlines the indexing process in Terrier The corpus data (ImageCLEF) is parsed in TREC format and Figure 3: Retrieval Architecture that data forms the collection. A Collection object extracts the raw content of each individual document and hands it in to a Document object. The Document object then removes any The input query is parsed initially post which it enters in Pre- unwanted content (e.g., from a particular document tag) and processing – entering it into same configured TermPipeline. gives the resulting text to a Tokeniser object. Unwanted The query is then handed to matching component. Weighting content is removed through the TermPipeline, which Model is instantiated (DFR model is used) and document transforms the terms removing stopwords (high frequency scores for the query are then computed. To improve the scores, terms) and stemming (prefix, suffix removal) [4]. Finally, the query moves to post processing e.g query expansion [3] - tokeniser object converts the text into a stream of tokens that taking the top most informative terms from the top-ranked represent the content of the document. The entire iterations of documents of the query, and adding these new related terms terms and is building of index is performed by into the query. The processed query is assigned scores by BasicIndexer(default indexer). matching component. Post filtering is the final step in Terrier’s We get output in form 10177882 tokens from the corpus data retrieval process, where a series of filters can remove already of 237434 images. Result of indexing is depicted below: retrieved documents, which do not satisfy a given condition. Figure-4: depicts retrieval results of the query. In the screenshot we observe that Image ID 232878matches query with a highest score of 10.7788. Figure-2: Screenshot of result of indexing performed in terrier tool 3.2 Retrieval After performing indexing, we can now initiate retrieval process using different queries. These queries are in natural Figure-4: Screenshot of retrieval results of the query in language and denote the text from the story that needs to be terrier tool illustrated. We use the event description of a story as a search query. These event descriptions are provided as a XML data in After searching every query, run files (output) are compiled the task with respective manual annotations. Each event from using the scores of search query and retrieved image id. 72 4. EVALUATION AND RESULTS 6. ACKNOWLEDGEMENTS Evaluation is conducted on the run files using standard We are grateful to Dr. Debasis Ganguly and Mr. Iacer Calixto trec_eval tools. Precision-at-K (P@K) and mean average for his guidance throughout of this task. Additionally we precision (MAP) scores are evaluated. Each important entity or would also like to thank FIRE2015 for opportunity to work event in a story will have a relevance list associated with it. under this task and facilitating the process. P@K and MAP for each annotation are computed against these relevance scores. There were a total of two groups participating and four system 7. REFERENCES [1] Diogo Delgado , Joao Magalhaes and Nuno Correia. submissions with the result shown below in Table-1. 2010. Assisted News Reading with Automated Illustrations. ACM. DOI= Table-1 RESULT http://dl.acm.org/citation.cfm?id=1874311. [2] Delgado, D. 2010. Automated illustration of news stories. IEEE. [3] Michel Beigbeder, Wray Buntine and Wai Gen Yee. 2006. Terrier: A High Performance and Scalable Information Retrieval Platform. OSIR [4] William B. Frakes, Ricardo Baeza-Yates, 1992. Book: Stemming Algorithms. PEARSON Education. A highly effective information retrieval is one with high recall and precision i.e. retrieve as many relevant documents as possible and as few non-relevant documents as possible. The results of cguj-run-2 file had 0.1545 precision. 5. CONCLUSION Automated Story Illustrator task is first time released in FIRE. The research can go a long way in illustrating short stories especially for children as well help improve reading memory. Lots of further work needs to be carried out in the task to improve the effectiveness of the result like modifying scores using different algorithms, improve the manual annotations and modify the queries or improving the indexing. 73