The recent improvement of powerful Large Language Models is the key to automatically produce satisfactory written and spoken language, in contrast to the constraints of conventional template-based solutions. However, the most advanced models can be costly and complex to integrate into practical applications, especially in business contexts where the output quality significantly matters. This study presents a tailored pipeline for data-to-text and text-to-speech generation, primarily harnessing the availability of open source pre-trained language models and leveraging established Natural Language Processing tasks. As a use case, we worked on the automatic generation of both textual and video product descriptions from the structured information about the product features. The pipeline involves all the required steps, providing the final trained and customized model. The obtained descriptions showed the capability of replicating the overall semantic, lexical and linguistic style of the corresponding human counterpart, despite being based on a cost-efective model.
As the number of applications for Artificial Intelligence (AI) has grown, several technologies
have gained particular interest, especially with regard to automation techniques. This aspect
increased the importance of Natural Language Processing (NLP) and Natural Language
Generation (NLG), allowing computers to understand human language and respond accordingly,
showing significant potential and a wide array of applications based on solid deep learning
solutions [
Data-to-text architectures serve as the foundation for enhancing services by transforming
structured data into useful information. They enable the generation of text to compensate for
the shortage of human resources. Over the years, diferent approaches have been proposed
for diferent domains [
This paper represent an extended abstract of a recent proposal [
Our main contributions include: (1) the development and evaluation of a Named Entity Recognition (NER) model to detect product features and product title; (2) an integrated pipeline using the open GPT-2 model; (3) the generation of a complete textual product description and the corresponding video based on the list of product features.
Fig. 1 shows the general overview of the proposed pipeline for producing complete descriptions starting from the list of attributes. We worked on two main steps: (i) the training phase, for ifne-tuning the pre-trained model (Section 2.1); and (ii) the generation phase, for creating a new textual description and the corresponding video according to the given input (Section 2.2).
The core element of our pipeline is the usage of a Neural Network (NN)-based model. In the current instance of the proposed pipeline, we selected GPT-2 as reference solution for the text generation model. In fact, GPT-2 is open source, free of charge and gives access to the generated model. The overall process is designed to enhance the ability of the language model to produce coherent and relevant product descriptions. Therefore, it involves specific data pre-processing steps. In particular, we developed a customized Named Entity Recognition (NER) model to transform the given dataset and map the descriptions with the corresponding list of features.
We collected several product descriptions from diferent sources (see Section 3). Irrelevant features have been eliminated and the stored information has been reduced to product title and product description only. Since short descriptions (less than 100 characters) are unhelpful for training, they have been removed, whereas long descriptions (i.e. more than 750 words for GPT-2 small) have been split multiple times to comply with the length-related constraints of the model. 2.1.2. NER model A primary objective was to automatically establish a link between the structured data and the corresponding textual description. For detecting product features and titles within the textual descriptions, we proposed a customised NER model. It was based on the introduction of two specific tags to denote the product title ( <prod>) and each feature/attribute (<attr>). Then, the ifnal dataset for training was built, obtaining the desired semi-structured data linking the list of features (extracted using NER) and the corresponding text (i.e., the original description).
For each element of the training dataset, we merged the input data extracted using the NER model and the corresponding description into one single object to feed the GPT-2 trainer. As shown in Table 1, specific tokens were included to enrich the tokenizer of GPT-2: (1) product name, (2) product features list, (3) product description. The sequence tokens (<OVERV_START> and <OVERV_END>) replaced the original GTP-2 ones (i.e. GPT-2 bos and GPT-2 eos, respectively).
Indicate beginning and end of sequence Delimit start and end of product title Indicate beginning and end of features list Separate one feature from another Indicate beginning and end of the description Token <OVERV_START><OVERV_END> <NAME_START><NAME_END> <FEAT_START><FEAT_END> <NEXT_FEAT> <DESCR_START><DESCR_END>
Once obtained the new trained model, it was possible to proceed with the automatic generation of the product description. However, post-processing was still required to improve the overall quality of the results. It returns a complete HTML-based textual description which is used to generate the video presentation accordingly.
Considering our real-world use case, the proposed approach automatically retrieved the required information from a MongoDB instance, manipulating the collected data to be a suitable input for our model (i.e., including the introduced special tokens). Filtering is supported by the pipeline to define the desired list of features (and categories of features) employed for the description generation and/or a block-list to remove unessential information (e.g. packaging, suppliers). Each product feature is defined by the union of the corresponding key-value pair (or the key alone for boolean values) to better support GPT-2 in understanding the received input.
For the actual text generation, there are diferent decoding solutions [
Finally, the temperature is a crucial parameter to be set carefully. In fact, it defines the creativity of the model in generating text, thus higher values may lead to incorrect, misleading or out of topic text.
Minor post-processing is then required to ensure better quality and usability. Based on the special tokens <DESCR_START> and <DESCR_END>, the description is automatically retrieved and processed. Common mistakes are removed, as well as redundant sentences according to a similarity score. Lastly, subtitles and paragraphs are highlighted to create the final comprehensive and organized textual description.
The Video Generation module returns the video description given the obtained text. We divided this block in two main actors: a distance module and a video and audio generation module.
Distance module. The video is designed as a slideshow about the given product, with a text-to-speech model for reading the description. The synchronization of audio and video ensures that whenever a sentence featuring a particular feature is spoken, it promptly displays on the screen. It is based on sentence embedding, a technique mapping sentences to real-valued vectors, considering as input both the textual description and the list of features. As model, we used all-MiniLM-L6-v2. For improving the accuracy of the sentence embedding, we created simple sentences from the original features (e.g., "this product has a 4K resolution"). (a) Display and Network section. (b) Audio section.
Audio and video generation. Slides consider the title and the image of the product for providing suitable graphics. Each paragraph triggers the creation of a new slide, with the paragraph title and features from the sentences displayed in sync with the audio narration. The Google Text-to-Speech (gTTS) library orchestrates the synchronized audio generation. To elevate consumer engagement, supplementary elements like background music and visually appealing slide templates are introduced. Fig. 2 illustrates two sample slides for reference.
In this section, we report the obtained results, both for the NER model and the generated
text evaluation. For training and testing the pipeline, we merged diferent free datasets about
technological products of popular e-commerce [
The 10-fold Cross Validation method has been used over 58 product descriptions and considering two distinct tags (i.e., ATTR and PROD tags). For describing the overall performance of the NER model, we defined some specific metrics. The Perfect match rate, which is the ratio of exactly retrieved attributes and product names with respect to the whole set of detected elements; Partial match rate, indicating the detected entities with partial overlapping, thus not the complete original element; False alarm, counting the portion of incorrectly detected entities with no counterpart in the reference set (complementary of the Total match rate); Miss rate, providing the rate of undetected entities, neither partial; Misclassification rate , the ratio of both perfect and partial matches assigned to the wrong tag. The complete assessment is detailed in Table 2.
The Total match rate, comprising Perfect match and Partial match, serves as a key indicator of system performance. Notably, it stands at 82.55% for attributes, while 96.90% for product names, indicating highly satisfactory result. Misinterpreting a feature as a product name occurs less frequently (6.69% only), whereas it is more prevalent for product attributes (14.68%). This discrepancy arises from the limited number of product names relative to the diverse array of attributes within textual descriptions.
The statistical assessments was performed by using the common metrics BLEU-4 and GLEU
(Google-BLEU) [
A grid search has been performed to identify the model response to diferent configurations, according to the values of (i.e., 0.5, 0.8, 1.2), (i.e., 35, 50) and (i.e., 0.9, 0.95). By averaging over 3 runs, we obtained better results for = 1.2, = 50, = 0.9. For our test dataset, we had 0.450 ± 0.007 for BLEU and 0.510 ± 0.005 for GLEU. This indicates that the generated text generally exhibits strong correlation with the reference text.
We have launched an online survey to gather public opinions on the obtained product descriptions to have a human evaluation. In particular, we proposed two text comparisons including both the original and the generated description for a specific product. Users were asked to select the most fitting description, according to their individual perception.
Perfect match Partial match Total match False alarm Miss Misclassification
ATTR Mean 56.35% 26.19% 82.55% 17.45% 25.78% 6.69%
Std 8.70% 4.85% 6.15% 6.15% 8.04% 4.07%
PROD Mean Std 68.66% 28.17% 28.25% 27.86% 96.90 % 6.55% 3.10% 6.55% 21.33% 17.94% 14.68% 13.34%
(a) Generated description (b) Reference description
We collected feedback from 87 volunteers with diverse ages and backgrounds. The survey revealed that in 56.3% of cases, our generated text was rated as either better or equal to the original text. Therefore, the product descriptions produced by our system closely resemble the original ones, making it evident that the proposed model possesses the capability to replicate the linguistic elements of the training dataset, including text structure and paragraph delineation, relying on the provided list of product features as structured input only.
Finally, we used structured information collected from a MongoDB database of a real website2 to generate product descriptions, assessing the efectiveness of our pipeline in a practical scenario. For online e-commerce and websites, the length of the description is a key aspect to provide a useful service: therefore, longer descriptions are required. However, the implemented version of GPT-2 (768 tokens) is a strong restriction. Then, our solution allows features selection by list of categories and the final text is obtained by iteration, organizing the description in the corresponding paragraphs. As a reference, Fig. 4 shows a complete example about a Smart TV: the text is consistent with common online product descriptions, describing all the diferent aspects of the product in each paragraph.
The advent of significantly powerful NLG models has made possible the integration of systems capable of automating time-consuming tasks. However, the resources required for more advanced models are not negligible. In our work, we studied the feasibility and efectiveness of automatically transforming structured data into engaging textual and visual narratives, by leveraging on the capabilities of the open-source GPT-2 model. Our proposed pipeline deals with all the necessary steps to fully integrate an NLG solution in a real-world scenario, starting from data manipulation towards the generation of both text and video descriptions. Using structured data as entry point (i.e., list of product features) implies more constraints, as the model must remain consistent with the given input to avoid reporting incorrect information. By comparing the generated text with the reference one, our analysis showed that the system was actually able to generate a coherent text and replicate the linguistic style of the given context, transforming raw data into more compelling and narrative information.
Currently, our solution is confined to the small version of GPT-2 which provided interesting results and human-like structures and contents. However, moving to the medium or large versions would be optimal for fully harnessing the model’s potential, ofering better performance. Our future work will focus on analyzing diferent models and configurations, supported by the acquisition of a more suitable training dataset, both in terms of quantity and linguistic quality.