The question of whether a computer is capable of writing text with creative features such as poetry (and if so, how it will differ from human creation) has become a popular one among researchers in the fields of natural language generation (NLG), computational creativity, and, broadly, artificial intelligence (AI). While a significant number of studies have long attempted to achieve automatic responses to a series of questions, automatic poetry generation has been receiving increasing attention in recent times. This is a challenging research area, because it requires a very high level of skill to satisfy the formal conditions and content of the poem.
Automated poetry creation not only indicates technological progress, but it represents a new creative approach altogether that is entirely different from the existing concept and principles of poetry creation. The shift in the perception of poetry creation methods dates back to 1920, even before computers were accessible. It can be observed in a poem by Tristan Tzara, a poet who participated in a new European art movement called Dadaism in the early 20th century. His poem [5] is as follows:
To make a Dadaist poem: Take a newspaper. Take a pair of scissors. Choose an article as long as you are planning to make your poem. Cut out the article. Then cut out each of the words that make up this article and put them in a bag. Shake it gently. Then take out the scraps one after the other in the order in which they left the bag.
Copy conscientiously. The poem will be like you. And here are you a writer, infinitely original and endowed with a sensibility that is charming though beyond the understanding of the vulgar. Zara's poem, published in 1920, proposed a complete departure from traditional poetry. In it, the poet finds adequate poetic words and combines them according to the rules; their intentions, feelings, or causality cannot be found. This recalls the definition of an algorithm, meaning a procedure or method for solving a problem or a step for performing a task.
With the advent of computers, experimental attempts were made to generate poems. Bailey [1] suggested semi-automatic poem generation, emphasizing the potential of computer use in poem creation. The French Atelier of Literature Assisted by Maths and Computers (ALAMO) group [2] proposed 'rimbaudelaires,' a method to combine existing poems in order to create new ones, in which the structure of a poem by Rimbaud was filled with the vocabulary of Baudelaire's poems.
The use of deep learning algorithms such as recurrent neural networks (RNNs) and LSTM in computational creativity has evolved the concept of automatic poetry generation. In this context, the present study proposes a Korean poetry generation model based on deep learning which aims to imitate the writing style of a particular poet.
First, four poets, Kim So Wol, Yoon Dong-Joo, Baek Seok, and Jeong Ji-yong, who represent Korea's modern poetry and remain popular amongst Koreans, were selected. These poets wrote noteworthy poems within the forms of free poetry and lyric poetry during the Japanese occupation.
The rests of the paper are organized as follows. We start by reviewing previous works in the Section 2. Section 3 describes the approach adapted in our experiments. And we illustrate the evaluation in Section 4. The conclusion of this paper and future work are demonstrated in Section 5.
If Bailey [1] demonstrated the possibility of computer-generated poetry, Gervás [4] marked the beginning of the automatic poetry generation, and various studies have been since. Wu and Tosa [10] proposed a poem generation system based on Haiku phrase corpus. When the user enters a word or phrase, the system finds expressions containing it in the corpus and creates a poem by combining them. Manurung and Thompson [6] developed a system using genetic algorithms. The poetry generation system, called McGonagall, finds one of several candidate poems with no grammatical error and clear meaning transmission according to stochastic search. Das and Gambäck [3] presented a syllable-based poetry generator. When a user enters a sentence, the syllabification engine understands its rhythm and generates the appropriate sentence that follows it.
Recently, along with the advance of machine learning, poetry generation using deep learning have emerged. Wang et al. [9] propose the machine poetry generator based on LSTM for imitating Chinese poet Du Fu's writing styles. Given the first character, this model produces a poem reflecting the tone and rhythm of Du Fu's poem. Zugarini and Maggini [11] also suggested a system using LSTM to generate terrests, a feature of Dante Alighieri.
Poetic generation researches in various languages such as English, Japanese, Chinese, and Italian are being actively conducted. Regarding Korean poem generation, Park et al. [7] introduced a model for generating poems using Sequence Generative Adversarial Networks (SeqGAN). Korean poem generation is just beginning. In this context, we present a Korean poetry generation model based on deep learning aiming to imitate the style of a particular poet.
The whole experiment workflow is shown in Figure 1. At the beginning of the experiment, we first collected 400 poems written by four poets. The poetic work of the target authors is usually not enough to successfully train deep neural networks, so we collected a total of 1000 pieces by adding other poems written during the Japanese colonial period. Before training the data using LSTM, we removed old language, Chinese character, etc. from the poem in the pre-processing process and then numbered each line of the poem. When a user enters a keyword and clicks a poet, a poem is generated.
We attempted a quantitative analysis of poetry text to define the style of each poet. To do this, we extracted high frequency vocabularies using part-of-speech (POS) tagging From the results of the stylistic analysis, certain characteristics could be determined. First, vocabulary related to nature appeared frequently in the poems by the specified poets; second, the primary emotion influencing their poetry is sorrow; and third, sensory expressions representing nature are often used. Finally, the use of first-person pronouns 'I' and determiner such as 'this' was high. Table 1 shows the results of the stylistic analysis.
Nature 윤동주 (Yun Dong-Joo) 밤 (night), 하늘 (sky), 별 (star), etc. 정지용 (Jeong Ji-yong) 바다 (sea), 물 (water), etc. 김소월 (Kim So Wol) 산 (mountain), 나무 (tree), etc. 백석 (Baek Suk) 새 (bird), 개구리 (frog), etc. Emotion 슬프 (sad), 외롭 (alone), 괴롭 (painful), 서럽 (sorrow), etc. Sense 붉은 (red), 푸른 (blue), 밝은 (bright), 검은 (black), 높은 (high), 뜨거 운 (hot), etc. Pronoun 나 (I) Determiner 이(this), 그(that), etc.
Table 1: The results of the stylistic analysis.
Fig. 2: The word cloud of top frequent word.
We present an approach based on LSTM to generate Korean poems with a specific style. As shown in Figure 3 and 4, we constructed a word-level encoder-decoder LSTM network, entered the author's name, poem title, the line number of the poem, and trained to minimize the difference between the target sentence and the sentence generated by the network. When the sequence (author name, poem title, poem's line number) enters the encoder network, this network performs word embedding and inputs it to the LSTM network to extract the feature values of the input sequence. The decoder network applies Attention Mechanism to train the relationship between input sequences and output sentences. Hidden size of each network is 256 Dimension and depth of layer is 3. The maximum length that can be trained is given by 30-word sequence length. All LSTM networks were initialized to zero before training.
Some researchers have suggested criteria for evaluating automatically composed texts. For example, Manurung et al. [6] introduced grammaticality, meaningfulness, and poeticness, while Sten et al. [8] proposed adequacy, fluency, readability, and variation.
In the present study, readability, meaningfulness, and grammaticality were adopted, and five evaluators chose 60 poems out of 1000 based on the three criteria. However, some errors were found in the chosen poems, the most prominent of which were syntactic and semantic errors. In Korean, adjectives are generally placed before nouns. However, sentences that violate such syntactic rules were found. Additionally, in some cases, Fig. 3: The training process for Korean poetry generation. sentence meanings could not be gathered despite the absence of syntactic errors. In order to improve on these aspects, a significant amount of further study is required.
When a user clicks on one of the four poets and enters the desired keyword, a new poem reflecting the poet's style is created. Even if the same keyword is entered repeatedly, new results will be generated each time. The reproductions of the poets' vocabulary and emotions were analyzed in 60 poems. Words related to nature, such as rivers, skies, mountains, and the sea appeared in the title and content of the poem, and emotional words representing sorrow and loneliness were used. However, some meaninglessly repeated words affected the readability of the poems.
Automatic poetry composition is a highly challenging problem because poetry is a genre of literature that expresses human imagination and creativity. Several studies have been conducted to generate poems in various languages such as English, Chinese, and Japanese. In this paper, an LSTM-based approach to produce a Korean poem with a specific style is presented. When a user enters a keyword and clicks on a poet, the model creates a poem that reflects the poet's writing style. Korean poem generation research is just beginning; this work is expected to contribute to Korean text generation research.
This research was supported by the Korea Creative Content Agency, under the Ministry of Culture, Sports and Tourism.