Large number of researchers consider essay as a tool to judge learning outcomes and intellectual capabilities and to assess the organized and integrated thoughts. Due to increase in the number of universities students and distance and ubiquitous e-learning approaches, the intention of using Computer-based Assessment Systems has rise rapidly in a decade. Manual grading of students' essays requires significant amount of time and hard work and also an expensive activity for educational institutions and need a practical solution to this task. The automated essay grading or evaluation system is solution to such need. So now-adays, most of the online competitive and universities exams are trying to evaluate the human written essays by examiners / teachers as well as by machines like automated essay grading system. Such system has to significantly focus on vocabulary and text syntax, and text semantics. The research paper focus on the existing automated essay grading systems, their functional technologies and proposes a methodology to overcome the issues related to them while evaluating such as grammatical and semantic error as well as influence of local and regional languages in Hindi essays.
ववश्वास साथ लोग अपने घरो के आँगन मे रंगोली या आलपोना से सजाते है” and “राखी के पण्ु य पवव पर घर मे ककसी के ननधन
से यह त्योहा र खोटा हो जाता है”, the influence of regional or local language can be clearly identified. Presently Computer-based Assessment Systems (CbAS) are Copyright © 2017 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. rare whose primary focus is to provide automatic grading and evaluation of Hindi essays. Automated Essay Grading (AEG) Systems, also known as Automated Writing Evaluation Systems or Automated Essay Assessors. The aim of such system is to score student’s essays on a specific topic and give feedback to the student on deficiencies in his/her essay. These systems lower down the burden of the evaluator as unable to give personalized attention to the student’s needs. Such systems provide human capability of reading and writing and also time-to-time feedback to the writers/students which help them to improve their writing skill.
PEG is one of the initial implementations of automated
essay grading. PEG is a statistical approach based on the
assumption that the quality of essays is reflected by the
measurable proxes [
IEA is a domain-independent tool based on the Latent
Semantic Analysis (LSA) technique that was originally
designed for indexing documents and text retrieval [
Hindi Dependency Treebank (henceforth HDT) uses karaka - a syntactico-semantic relation as an intermediary step to express the semantic relations through vibhaktis [10]. Each karaka has a default vibhakti. In linguistics, grammatical relations (also called grammatical functions or grammatical roles, or syntactic functions) refer to functional relationships between constituents in a clause [11]. The role of grammatical relations in theories of grammar is greatest in dependency grammars, which tend to posit dozens of distinct grammatical relations. Every head-dependent dependency bears a grammatical function. Semantic analysis can be done using HDT as it includes Part-ofspeech, Chunk Information, and Dependency Information. For each sentence, the output of HDT has four columns which are mentioned below, 1st Column represents Token or chunk id such as 1, 1.1, 2, 2.2 etc. 2nd Column indicates the actual word or word groups in the sentence having the attribute 'name' for naming. 3rd Column specifies part of speech
A precise research in this decade has helped us to understand the AI & Machine Learning techniques based existing AEG systems, going through some limitations to propose a methodology which could work under Indian context. The methodology is based on the series of semantic evaluations.
For checking grammatical or semantic error, HDT of each sentence is captured. In this methodology, the features obtained from treebank are used to develop machine learning techniques to identify the errors. The machine learning procedure analyzes each noun, pronoun and verb and postposition associated with it. It also analyzes the number and gender agreement between noun/pronoun and verb.
Hindi is a free-word-order language but its default word order of sentences is Subject-Object-Verb (SOV). The object may be direct or indirect or both. In Hindi, postposition or vibhaktis/case marker is used instead of preposition in English language and is combined with noun or pronoun or more generally a noun phrase. Vibhaktis like ने ,को ,से,का के ,को ,में, etc. are attached as suffix with noun or pronoun. Sentences in Hindi may follow default word order conventions for coding the information of grammatical relations. Hindi language has rich morphological case in which the subject and object and other verb arguments are identified in terms of the case markers that they bear (e.g. nominative, accusative, dative, genitive, ergative, etc.). The subject in a sentence must agree with the finite verb in person, number, and gender to be grammatical correct. A sentence is considered to be ungrammatical if it contains syntactic error. Let us consider the following sentences, Eg1. राम ने रावण मारा .
Eg2. लड़की स् क ू ल में जाती हैं./ लड़की स् क ू ल को जाती हैं. Eg3. गोपाल अपने भाई से लंबी हैं । Although Eg1. is ungrammatically as it is missing “को ” after रावण
in the sentence, HDT considers it to be grammatically correct as shown by the dependency structures of the sentences “राम ने रावण मारा ”, “राम ने रावण को मारा ” and “राम रावण को मारा ” in the figure 1, 2 and 3 respectively where k1 indicates Karta karaka /Nominative Case (having ‘ने’ case marker) and k2 represent Karma karaka /Accusative case (having ‘को’ case marker).
This indicate that absence of case marker is not treated as grammatical error by HDT. Sentences in eg2. are grammatical but it could be more proper by eliminating the “में” and “को ” as “लड़की ्स क ू ल जाती हैं”. In eg3: गोपाल अपने भाई से लंबी हैं, the verb “लंबी” does not agree with the subject “गोपाल” as it possesses masculine gender whereas the verb here has feminine gender. Now consider the sentence “चला जाऐंगा अपने आप सुन ील” which is ungrammatical too. Hence, the gender and number agreement helps in lexical analysis.
Semantic knowledge provided information such as animacy, named entity categories and verb selectional restrictions. Named entity tag information is used to match the category of pronoun and their referent. The semantic class information (noun category) is used for the finding facts and fact evaluation in essays. The pairs which do not have semantic feature match are filtered out. Using the semantic knowledge for each word, semantic analysis is performed. "Semantic Analysis" refers to a formal analysis of meaning, and "computational" refer to approaches that in principle support effective implementation [12]. Semantic analysis involves the identification of the intended meaning at the word level i.e. word-sense disambiguation, as word has multiple meanings in different contexts. Semantic analysis also helps to understand that how different sentence and textual elements fit together. The analysis began with the identification of word senses computationally, exploring the interrelationships between the elements of a sentence, and relations between sentences (e.g., coreference), and examine the semantic relations and sentiment analysis. The dependency structures shown in figure 1,2, and 3 indicates that HDT shows the meaning of these sentences to be correct although they are grammatically incorrect. The dependency structure shows the relation of noun phrases and verb phrases which are semantically interrelated. Semantic knowledge analyzes multiple words and identifies their relations between as hypernymy & hyponymy and meronymy & holonymy too. Hindi WordNet is a system for bringing together different lexical and semantic relations between the Hindi words [13]. For each word (lexical item) there is a synonym set, or synset, in the Hindi WordNet, representing one lexical concept. Further, each synset is mapped to a concept ontology which defines the semantic properties of lexical items of a given synset.
[fruit],भारत [India], खेल [game], मोटर [automobile] Hence, the verbs and adverbs can be matched against the attributes related to various senses and shall manage the correlation between the segments of the sentences or clauses.
The proposed methodology improves automated assessment by incorporating vast semantic attributes and grammar checking to overcome the issues related to automated essay evaluation systems. The system has to be evaluated on the basis of dependency and the supporting information from WordNet about sense and correctness of the sentences. In future, the size and variety of the corpus has to be increased. The factors of grammar checking other than number and gender agreements are considered as future research directions.
This work was supported by Research and Development Laboratory, Department of Computer Science and Engineering at Bhilai Institute of Technology, Durg, Chhattisgarh, India, awaiting sponsorship from suitable funding agencies.