To support foreign language learners, we propose a system that automatically completes the learner's writing in a foreign language. The proposed system completes the continuation of the learner's input using a language model. The learner writes a part of a sentence using our system. Our system provides automatic completion by showing the candidate expressions, each of which completes the sentence. Among the candidates, another machine-learning classifier predicts the candidates that the learner is unlikely to know in a learner-adaptive manner. The system scaffolds each learner by showing the candidate expressions that the learner is unlikely to come up with during writing. In experiments, we show qualitative results.
The complexity and intricacy of language learning necessitate a carefully orchestrated approach, particularly concerning the acquisition and application of appropriate vocabulary in a second-language context. Vocabulary knowledge plays a crucial role in expressing ideas effectively and communicating proficiently. However, due to the diverse abilities and levels of language learners, as well as the range of unfamiliar words they encounter, developing a universally effective method of support poses unique challenges.
This study addresses the aforementioned challenges by proposing an innovative approach that leverages artificial intelligence (AI) to facilitate second- language vocabulary acquisition. While previous studies and traditional teaching methods have offered some support in clarifying grammatical constructs, such as prepositions, there is a substantial gap in providing targeted assistance for vocabulary building. The primary difficulty lies in accurately predicting the specific words that individual learners struggle with when writing compositions, which necessitates extensive data collection and analysis.
In the field of second language acquisition, two key concepts are recognized: receptive vocabulary
and productive vocabulary [
Building on this premise, this study hypothesizes that augmenting a learner’s receptive vocabulary would subsequently enhance their productive vocabulary. To achieve this, we propose an AI-based support system that emphasizes targeted vocabulary building through a reading vocabulary test. This AI system employs machine learning methods to evaluate a learner’s receptive vocabulary, identify gaps, and predict potential areas of difficulty in their productive vocabulary.
The proposed system, which involves a classifier and a masked language model, operates as follows: First, the learner is asked to take a vocabulary test that requires them to determine the meaning of words in a sentence. This test typically takes approximately 30 minutes.
Next, the data from this test are used to train or fine-tune a machine-learning classifier that allows for learner-adaptive prediction. This model determines whether a given expression is difficult for the learner in a learner-adaptive manner. It is important to note that productive vocabulary is nearly always a subset of receptive vocabulary. Therefore, if the classifier determines that an expression is difficult (i.e., not part of the learner’s receptive vocabulary), it is highly likely that the expression is also absent from the learner’s productive vocabulary.
Our system functions as follows: Using the system, the learner writes part of a sentence, and the system presents candidate expressions, each of which can complete the sentence. This automatic completion is conducted by a typical masked language model, which simply predicts the next word based on the learner’s input. This masked language model enables the learner to explore potential candidates for the rest of the sentence. Then, the previously mentioned classifier for determining difficult words for the learner identifies candidates that are particularly difficult but important for the learner. The learner can discover expressions that complete the sentence, especially those with which they are un- familiar. Moreover, the system scaffolds the learner by continuously displaying important candidates predicted to be unfamiliar to them.
To evaluate this study, ideally, we require large datasets consisting of learners’ partial sentences to be autocompleted. However, to the best of our knowledge, such a learner corpus does not exist, making rigorous evaluation difficult. In this presentation, as a preliminary study, we will showcase the system’s construction, provide a demonstration of how it works, and present qualitative results from several examples. Finally, we present a qualitative evaluation of the proposed system.
1.1.
Several studies have been proposed to support second language writing using neural language
models [
An intelligent reading support system with the personalized prediction of each second language
learner’s vocabulary was previously proposed [
Next, in step (b), each user submits a half-written text to the system. The system forwards it to the LM module for auto-completion of the input. The LM module generates a ranked list of the top-k candidate words to complete the half-written input. Then, each word in the ranking of the completed text is input to the personalized classifier. The personalized classifier predicts the probability that the user knows the word. By examining the words that complete the input, the system can identify which expressions are likely to be unfamiliar to the user. In the provided example, each of the words “appointed” and “hereditary” is a suitable continuation for the phrase “Mayors are,”. However, the learner is predicted to be unfamiliar with “hereditary”.
Figure 2 is the actual screenshot of the proposed system shown in Figure 1. Two text boxes are shown. The upper one displays the half-written input, while the bottom one displays the auto-completed results. In the bottom box, each line consists of three components. The first value is a score that is returned by the “topk” function of the transformers library and indicates how well the completed word suits the input. The second value indicates how likely the user is to know the completed word. Finally, the third is the completed input.
In Figure 2, we can see that the word “appointed” is predicted to be most likely to complete the input. The word “hereditary” is also considered appropriate as an expression that completes the input. However, it is not likely to be known to the learner as shown in Figure 2.
In fact, although the majority of the data in [
We built the personalized classifier in Figure 1 using the vocabulary-test result dataset [
For the LM module, most language models can be used. We used the BERT (Bidirectional Encoder
Representation From Transformer) model [
While, in this paper, we used BERT, which is a masked language model, we can also use other types of language models, such as large language models (LLMs), including ChatGPT (https://chat.openai.com/). Most language models, including masked language models and causal language models, can be used as “LM” in Figure 1 because the model simply has to predict the next word given a context. Hence, also, no fine-tuning is required in our system. Hence, a simple call of ChatGPT can also be used as “LM” in Figure 1.
Here, we show some examples to qualitatively analyze the proposed system. Figure 3 shows an example to show that the proposed system can understand basic English grammar and can handle the idiom “have to”. Various nouns could be used as objects for the verb “have”, such as in “do I have time to do that?”. However, in the example of Figure 3, the noun for the object is already filled in due to the presence of “what”, so nouns such as “time” cannot be the object of the verb “have”. Therefore, in Figure 3, a preposition or adverb correctly comes after the word “have”. Especially, due to the phrase “To do so”, the most likely word to complete the input is correctly predicted as “have to”. This result implies that the language model can scaffold the learner by correctly adding the word “to” when the learner does not remember that the preposition “to” is necessary to complete the idiom.
Figure 4 shows an example in which the system is used to find a difficult expression. Here, the definition of the word is specified in the phrase “A set of many islands are called”. Then, the language model returns a list of the word that is suitable for this definition. Here, the word “archipelago” comes the second of the list. According to the Weblio dictionary, “archipelago” is very difficult word. This result shows that the system can scaffold the learner by correctly showing the word suitable for the definition even if the learner does not know the word that the learner wants to express and used the other plain expression to indicate the word.
In this paper, we showed that a writing support system that scaffolds language learners. By a simple combination of two machine-learning models, namely a language model and a personalized classifier, the proposed system can auto- complete the second language learner’s input. The system can also output how likely the expression is known to the learner. Through the qualitative analysis, we showed that our system can effectively scaffold the learner by showing the word that the learner may not be able to think of.
Future work includes more thorough user studies of the system. Another important future work includes the retraining/re-fine-tuning of the model for learning difficult expressions based on the learner’s choice among the candidates.
This work was supported by JST ACT-X, Grant Number JPMJAX2006, Japan.
The references should be formatted according to the following gudelines: A paginated journal article
[