Word embedding techniques are techniques that have been extensively studied in natural language processing (NLP) to obtain vector representations of words typically using neural networks. The word2vec is a seminal paper in these lines of studies (Mikolov et al. 2013) . The following papers report improvement of their accurateness to represent words as vectors, typically by comparing the distances between word vectors with human judgments on semantic proximity between words (Pennington, Socher, and Manning 2014) . Early studies on word embeddings address how to make one vector for each word. As one vector representation is modeled to point one meaning, this limitation is obviously problematic to deal with polysemous words. Several previous studies tackled this problem and proposed methods to estimate the number of a word’s meanings and to estimate an embedding for each meaning of the word (Athiwaratkun, Wilson, and Anandkumar 2018) . However, recently, contextualized word embeddings (Peters et al. 2018; Devlin et al. 2019) became quickly popular. With these methods, we can obtain an embedding for each usage, or occurrence, of a word, considering the context of the occurrence of the word in a running sentence. These methods can also be seen as a method to estimate word embeddings for polysemous words, with an extreme assumption that each occurrence of a word has different meanings. As contextualized word embeddings are shown to be successful in many tasks, in current NLP, the former strategy to estimate both the number of meanings of a word and an embedding for each meaning is employed only when it is necessary.

Following the rise of word embedding techniques, visualization studies were propose to visualize word embeddings. The study by (Smilkov et al. 2016) simply reported that their development of a tool to visualize embeddings for different words. The study by (Liu et al. 2017) introduces applying visualization of word embeddings to analyze semantic relationships between words. Both paper deals with principal component analysis (PCA) and t-SNE (Maaten and Hinton 2008) for visualization. To our knowledge, we are the first to visualize contextualized word embeddings, in which each occurrence of a word, rather than a word, is visualized, with a practical purpose on language education.

In addition to the visualization, our method can also predict the usages that each learner is familiar/unfamiliar with, in a personalized manner, when vocabulary test result data of dozens of learners are provided, such as the data in (Ehara 2018) . While there exist previous studies (Ehara 2018; Lee and Yeung 2018; Yeung and Lee 2018) for predicting the words that each learner is familiar/unfamiliar with using such data by using simple machine-learning classification, our method tackles a more difficult problem that deals with predicting which usages of a word is known/unknown to the learner.

While our proposed method is novel as a visualization, software tools that search the usages of an inputted word for educational purposes and display them itself are not novel: such software is known as concordancers. Concordancers target learners, educators, and linguists as primary users. They are interactive software tools that retrieve all usages of the inputted word in a large corpus and display the list of the usages, each of which comes with the surrounding word patterns (Hockey and Martin 1987) . Concordancers were also studied to support translators, who are second language learners in many cases (Wu et al. 2004; Jian, Chang, and Chang 2004; Lux-Pogodalla, Besagni, and Fort 2010) .

Figure 2 shows a screenshot from a current concordancer 1. In this screenshot, the word “book” is searched. Then, the list of word usages is shown. Each word usage comes with surrounding words so that language learners can see how the word is used. While the list is sorted in alphabetical order of the previous word, we can see that the list shows “a book” and “the book” in totally different positions and are not helpful for language learners. While some concordancers support listing the usage of “book” as nouns by attaching texts with part-of-speeches in advance, this is not helpful to see the different usages of the word when the part-of-speeches of the usages are identical. For example, the word “bank” have polysemous meanings sharing the same part-of-speech: one as financial organizations, and another as embankments.

In this study, a part of our goals is to identify complex usages of a word in a running text. In other words, for one word, one usage of the word in running text is complex for a learner, and another usage of the word is not. There are previous studies that identify complex words in a personalized manner in the NLP literature (Ehara et al. 2012; Lee and Yeung 2018) . These studies predict the words that each learner knows based on each learner’s result of a short vocabulary test, which a learner typically takes 30 minutes to solve. Also, there are also many studies that identify complex usages in a non-personalized manner, as summarized in (Paetzold and Specia 2016; Yimam et al. 2018) .

However, to our knowledge, the task of identifying complex usages in a personalized manner is novel. Our method is also novel in that it trains how to visualize the usages so that learners can visually understand the usage differences by using the learners’ vocabulary test data.

Principal component analysis (PCA) and t-SNE (Maaten and Hinton 2008) are famous dimension reduction methods, and t-SNE is notable for its intuitiveness and well clustered

We built a preliminary system and conducted some experiments to see how contextualized word embedding vectors are plotted in the system. Figure 4 depicts such an example of searching for the word book. Users can directly type the word in the textbox shown at the top of Figure 4. Below is the visualization of the usages found and their list. Each dark-colored point is linked to each usage. Two dark colors are used to color each usage point according to the results of a Gaussian mixture model (GMM) clustering with 2 components, as this value was reported to work well (Athiwaratkun, Wilson, and Anandkumar 2018) . The light-red colored point is the probe point: the usages are listed in the nearest order of the probe point. No usage is linked to the probe point. Users can freely and interactively drag and move the probe point to change the list of usages below the visualization. Each line of the list shows the usage identification number and the surrounding words of the usage, followed by a checkbox to record the usage so that learners can refer to it later. In Figure 4, the probe point is on the left part of the visualized figure. In the first several lines of the list, the system successfully shows the usages of the word book as a publication. In contrast, Figure 5 depicts the case in which the users drag the probe point from the left to the right of the visualization. The first several lines of the list show the usages of the word book, which means to reserve. We can see that the words surrounding the word book vary: merely focusing on the surrounding words, such as “to” before book, cannot distinguish the usages of book, which means to reserve, from the usages of book for reading.

The GMM clustering was accurate but not perfect: 0 errors in the 42 usages of “book”, 1 error in the 22 usages of “bank”, when manually checked in the excerpt. Hence, learners can choose not to use this, as in the video. Figure 6 shows the variance of the usage vectors of each word against its log frequency in the excerpt. It showed a statistically significant moderate correlation (r = 0:56, p < 0:01 by F-test), implying that frequent words tend to have complex usages.

From the example of “book” in the previous sections, we can easily see that the usages of “book” about reading are more frequent than those of “book” about a reservation. Hence, when counting the number of usages, it is intuitive to assume that learners are not familiar with all usages but are familiar with the usages within a certain radius in the vector space. This is the motivation of our method descried in the next section.

Before entering the technical details of our visualization method in the next section, we show some usage prediction result examples of our method in a manner similar to the previous examples of “book” so that readers can intuitively understand our motivation, as shown in Figure 7 and Figure 8. The markers are changed to triangular to denote that the colors reflect prediction results, rather than the GMM-based clustering results explained above. The coloring and darkness of the points in the visualization follow those of the previous examples; the red light-colored point is the probe point, and the other dark points denote usages. Figure 7 shows an example of the familiar usage prediction in case of searching the word “haunt”. The right-hand side of the cross-marked circle is the area in which usages are predicted to be familiar to this learner. The probe point is located within the circle. We can see that the usages of “haunt” about chasing are listed below. Figure 8 shows another example of “haunt”. As the probe point is located outside of the circle, in the left side of the visualization, the list below shows the list of the usages predicted to be unfamiliar to this learner. We can see that“haunt” about “to curse” are mainly listed. dvi freq(vi) = =

log(freq(vi) + 1) N (~ci; ; Xi) ni X tanh (M k=1 (2) (3) ReLU( de(G~ci; G~xk;i)))(4) As stated in the Related Work section, some previous studies address methods to predict the words that a learner knows based on his/her short vocabulary test result. However, since our application requires personalized prediction of the usages of the word that the learner does not know. Hence, we propose a novel model that does this.

Let us write the set of words as fv1; : : : ; vI g, where I is the number of words (in type), and write the set of learners as fl1; : : : ; lJ g, where J is the number of learners. Then, in previous studies, based on the Rasch model (Rasch 1960; Baker 2004) , the following logistic regression model Equation 1 is used to predict whether learner lj knows word vi or 1 not. Here, (x) := 1+exp( x) and yi;j is the response of the learner in the vocabulary test; yi;j = 1 if learner lj answered correctly to the question of word vi, and yi;j = 0 otherwise. We have two types of parameters to tune: alj is the ability of learner lj and dvi is the difficulty of word vi.

P (yi;j = 1jlj ; vi) = (alj dvi ) (1)

Here, how to model dvi , or the difficulty parameter of word vi, is the key to our purpose. Previous studies report that the negative logarithm of the word frequency correlates well with the perceived difficulty of words (Tamayo 1987; Beglar 2010) . As in Figure 1, our key idea is to count the frequency of word usages only within a certain distance from the typical usage of the word. Hence, we propose the following model to implement this idea.

For each vi, we have ni vectors that are vector representation of each of the ni occurrences of word vi. We write these vectors as Xi = f~x1;i; : : : ; ~xni;ig. Each vector ~xk;i is T1 dimensional. Among Xi, let ~ci be the one closest to their center n1i Pkn=i1 ~xk. Let freq(vi) be the frequency of the vectors in Xi within distance measured from the central vector ~ci. We write this frequency simply as freq(vi) = N (~ci; ; Xi). Here, n is the number of usages of word vi and let each ~xk be each usage vector obtained from contextualized word embedding methods. Let ReLU(z) = max(0; z) be the rectiThe tricky part is that Equation 3 can be approximately written as Equation 4, whose parameter can be easily tuned and optimized by using neural machine learning framework such as PyTorch. In Equation 4, due to the ReLU function, negative values within the function is simply ignored. Hence, as de is the Euclidean distance, if = 0, i.e., the size of the circle is 0, the terms inside ReLU is negative, and freq(vi) = 0. If de(G~ci; G~xk;i) > 0, due to M and tanh, the resulting value is almost 1. This means that we are counting only the cases that surpasses de, i.e., counting the usages within measured from ~ci.

Notably, the following characteristics are important to understand our model.

Notably, the proposed model is not merely a logistic regression. Our model has more parameters such as ; ~ci; alj ; G. Because of having different extra parameters compared to the logistic regression, to train our model, we typically need to use a neural network machine learning framework to model and optimize, such as PyTorch. To optimize using such models, as it is difficult to differentiate the loss functions of such models by hand, the model loss function is desirable to be mostly continuous and smooth so that its parameters can be tuned using auto-gradient. We specifically designed Equation 4 to meet these conditions. In the experiments, we used the Adam optimization method (Kingma and Ba 2015) to optimize the loss function.

As Equation 4 is mostly continuous and smooth, matrix G can also be trained by using deep-learning framework software. As G is a projection matrix from T1 to T2, if we set T2 = 2 to consider a projection to a two-dimensional space, training G via supervisions means training visualization via supervisions. Here, in our task setting, the supervisions are vocabulary test dataset of second language learners, i.e., a matrix in which the (j; i)-th element denotes whether learner lj correctly answered the question of word vi. j : Personalized In Equation 4, for easier understanding, we write to be a constant that does not depend on learner index j. In reality, we can personalize by making dependent to learner index j as j ; in this case, each learner lj has his/her own region that he/she can understand, and the radius of this region is j . This personalized version is the one that we used in the experiments.