Artificial Intelligence and Natural Language Processing Applied to Design María Villalba Language Processing and Information Systems Group (GPLSI), University of Alicante, Spain Abstract Artificial Intelligence (AI) and Natural Language Processing (NLP) are transforming how designers approach and execute their tasks, fundamentally altering the landscape of various design disciplines, including industrial, fashion, interior, interface and visual communication design. This research explores how generative AI and NLP can significantly enhance the creativity and functionality of design processes. The goal is to investigate and develop applications of AI and NLP in design, evaluate the integration of these technologies into existing workflows, and address ethical considerations essential for sustainable innovation. By examining the intersection of AI and NLP in design, this research aims to develop a system that integrates these technologies to facilitate the creative process of designers and to establish practical guidelines. Keywords Natural Language Processing, Large Language Models, Design 1. Introduction Design is a discipline focused on the interaction between users and the human-created envi- ronment, considering aesthetics, functionality, context, culture, and society. It includes a wide range of applications, from physical objects and digital environments to business strategies and service systems. Over the last 20 years, the integration of Artificial Intelligence (AI) in design has led to an era of unprecedented transformation. AI is transforming how designers work, providing new working models across various design sectors, including a specific focus on fashion trend prediction, thereby opening new frontiers of innovation, efficiency and creativity. This technological evolution has sparked increased interest among researchers, aiding in the development of new methods and AI-based applications for tasks in different design sectors. These advancements have enabled computers to automatically perform some tasks that were traditionally done by humans. While it is true that AI will radically alter design tasks and who performs them, many researchers agree that its greatest impact will lie in complementing and enhancing human creativity, rather than replacing it [1, 2].The field of design is populated by designers from all specializations, but regardless of the type of design, the process of designing is at the core of their practice. According to the International Council of Design (ICoD) [3], the most common types of designs are: • Visual communication design: specializes in creating visual elements ranging from graphic design work such as books and magazines, websites, apps, to the conceptualization of Doctoral Symposium on Natural Language Processing, 26 September 2024, Valladolid, Spain. Envelope-Open maria.villalba@ua.es (M. Villalba) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings more abstract systems related to brand identity and strategic planning. • Industrial or product design: focuses on creating products intended for industrial manufac- turing, applying their understanding of materials, manufacturing methods, and ergonomic principles to design furniture, electronic devices and more. • Fashion Design: focuses on creating clothing, accessories, and other items, utilizing designers’ expertise in materials, sewing techniques, pattern design and manufacturing. • Interior design: focuses on creating interior environments, using designers’ knowledge of materials, construction elements, decorative coatings and space distribution to design interiors that are functional and aesthetically pleasing. • Interaction design: encompasses both service design and UI/UX design, where designers work to design websites, web applications, and other digital tools. The previously mentioned categorization helps to better understand the diversity of the design world, and it is important to note that these categories are not rigid, and some disciplines may overlap or not fit perfectly, reflecting the multifaceted and constantly evolving nature of design. Each of these types of design is uniquely influenced by AI, resulting in a diverse and exciting landscape of applications. In particular, Natural Language Processing (NLP) has emerged as a powerful tool for understanding and generating creative content that responds to complex human contexts and needs, thereby amplifying the capacity for innovation in design. Within this field, Large Language Models (LLMs) play a crucial role, facilitating the automation of complex tasks and expanding creative capabilities in the interaction between designers and computational systems. However, it is critical to consider the ethical implications of their use, especially in terms of potential biases that may arise. The relevance of these technologies in design lies in its innate ability to optimize processes and unlock new horizons of innovation. Over time, AI has evolved from a tool for automating simple tasks to becoming a creative collaborator, capable of generating ideas, predicting trends and understanding complex human interactions. AI has revolutionized the field of design, ushering in a new era where technology and creativity merge. This article represents the research plan that will be carried out, focusing on the analysis of the role of generative AI and NLP in the field of design. The increasing integration of these advanced technologies represents a significant impact that warrants detailed exploration. This research aims to deeply analyze the convergence between different AI and NLP tools and design across multiple disciplines, identifying their impact and relevance and analyzing the integration of ethical considerations to ensure that technological advances are applied responsibly and fairly. By virtue of this, the present research has the potential to contribute to the understanding and application of AI and NLP in the field of design, providing an updated and comprehensive perspective of this exciting and constantly evolving field. This exploration will allow us to identify current and future trends, and establish a framework that can guide designers and researchers in the effective integration of AI-based tools into their creative processes. It aims to foster a closer collaboration between artificial intelligence technology and traditional design practices, maximizing the benefits and minimizing the risks associated with its implementation. 2. Background and related work Numerous studies have been conducted on the use of AI in design, analyzing its concrete applications across various domains such as engineering and industrial design [4, 5, 6], interior design [7, 8],fashion design [9], interaction design [10, 11] and visual communication design [12]. In particular, special attention has been paid to generative artificial intelligence and the use of NLP within these domains. 2.1. Generative AI applied to design Generative Artificial Intelligence is revolutionizing various design fields by introducing auto- matic and efficient methods for creating and optimizing prototypes. In fashion design, AI tools are transforming how designers experiment with patterns, colors, and textures, enabling the creation of collections that dynamically respond to emerging trends and consumer preferences, for example, the study by Jeon et al. (2021) [13]. Furthermore, the application of AI in interior and environmental design is beginning to show promising results, where generative models assist in creating sustainable and ergonomically viable spaces. As seen in the study by Guoxing Chen (2022) [14], by utilizing data-driven insights and simulation techniques, these tools can propose layouts and material selections that optimize for energy efficiency and user comfort, significantly enhancing the livability and sustainability of built environments. In industrial design, AI algorithms enable rapid prototype development by predicting and optimizing the functionality and aesthetics of products before manufacturing, which reduces costs and de- velopment times, such as a study by Vasantha et al. (2022) [15]. As illustrated by the study conducted by Villalba and Palomar (2024), which is currently under review, significant trends have been observed in the application of AI to industrial design. These trends include: • The use of Natural Language Processing (NLP), language models, and generative AI in the definition and analysis of customer requirements. • The use of advanced Deep Learning (DL) techniques for the automated evaluation and optimization of prototypes. These trends highlight the increasing applications of AI in design and demonstrate how tech- nological advances are reshaping the field. Due to this, it is crucial to explore how NLP can further amplify these advancements. The potential of NLP will be analyzed in the next section, where its specific applications in the design field will be examined. 2.2. NLP applied to design In the following subsections, we analyze the most notable applications of NLP within the field of design. These applications highlight the diverse ways in which NLP enhances design processes and outcomes. 2.2.1. NLP for visual content generation NLP has become a key tool for facilitating and enhancing communication between designers and computational systems, especially in image generation and visual content creation. In design communication, NLP helps to translate verbal requirements into detailed technical or visual specifications, improving precision and efficiency in project planning, as demonstrated in the study by West et al. (2018) [16]. In NLP-based image generation, this technology is used to convert textual descriptions into detailed visual images. This approach has found significant applications in advertising and marketing, where product descriptions are transformed into attractive and accurate visual representations, enhancing consumer interaction and product presentation. For example, a study by Sawant et al. (2021) [17] demonstrates how NLP can be utilized to interpret textual descriptions and convert them into detailed visual representations, facilitating the creation of more accurate and contextually relevant design elements. A study conducted by Saharia et al. [18] proposed the Imagen model, a text-to-image diffusion model with an unprecedented degree of realism and depth of language understanding. Imagen builds on a LLM for understanding text and utilizes a diffusion model’s strengths for generating high-fidelity images. In their study, Avrahami et al. (2022) [19] introduce a new method for editing natural images using text descriptions. This approach integrates a language-image model (CLIP) with a difusion model (DDPM) to ensure that text-driven edits blend seamlessly into the original images, enhancing both realism and contextual accuracy. 2.2.2. NLP for requirement extraction and analysis NLP is used to extract and analyze requirements from textual data, which is vital in the early stages of the design process. NLP techniques can sift through large volumes of user feedback, product reviews, and other text sources to identify functional requirements and user preferences. For instance, in the fashion industry, NLP can analyze social media posts, fashion blogs, and online reviews to predict emerging trends and consumer preferences, enabling designers to create collections that align with market demands. Text classification and sentiment analysis are commonly applied to user reviews to highlight features that need improvement or are highly valued by customers. These insights are crucial for designers to prioritize development areas and align product specifications with user expectations. A notable implementation of this is described in the research by Wu et al. (2019) [22], where NLP was leveraged to automate the extraction of user requirements from online product reviews, significantly speeding up the iterative design cycles. Additionally, Akay et al. (2021) developed a system that utilizes deep learning-based NLP to automate the extraction of structured functional requirements from textual design documents [20]. 2.2.3. NLP for user interaction NLP technologies play a crucial role in enhancing user interaction through the development of intelligent user interfaces. These interfaces employ NLP to understand and respond to user inputs in a natural manner, making technology more accessible and intuitive. Chatbots and virtual assistants are prime examples of NLP applications that simulate human-like interactions, allowing for a more personalized user experience. For example, in the field of e-commerce, NLP-driven chatbots guide users through complex catalogs to find products that meet their needs, as shown in the deployment by Sasidharan Pillai (2023) [21]. Additionally, Zukerman and Litman (2001) discuss how NLP and user modeling can synergistically enhance user interactions by adapting system responses to user needs and preferences, thereby improving communication effectiveness and system usability in user interfaces [22]. 2.2.4. NLP for design automation Natural Language Processing (NLP) is increasingly being integrated into design tools to automate routine tasks, such as generating documentation and assisting in the creation of design models from textual descriptions. This automation not only speeds up the design process but also helps maintain consistency and compliance with standards. For instance, automated documentation tools powered by NLP can interpret design changes and automatically update design documents accordingly. This reduces the manual effort required and minimizes human error, as explored by Akay and Kim (2021) [23]. Their research demonstrates the potential of NLP to significantly streamline the extraction of functional requirements from design documentation, enhancing the efficiency of the design process. Furthermore, generative design systems use NLP to interpret user inputs and generate design alternatives, which are then refined through iterative processes. These systems enable designers to explore a broader array of design options more efficiently, enhancing creativity and innovation in design practices. 3. Description of the proposed research and hypotheses The primary objective of this research will be to investigate and develop the AI and NLP appli- cations on design and understand how AI is transforming this field in terms of methodologies, tools, and outcomes. 3.1. Objectives In this research the following objectives will be pursued: 1. Identify and classify the current applications of AI and NLP in design, in order to establish a clear overview of its uses, advantages, and limitations across different types of design. 2. Analyze use cases where AI and NLP has been implemented in successful design projects, to understand best practices and the challenges encountered during its integration. 3. Investigate the role of AI and NLP, as well as the perception and acceptance of these technologies among design professionals, through surveys and interviews conducted by researchers, to assess current attitudes and barriers to its adoption. 4. Develop language and image generation models that utilize NLP and LLMs for design applications, detailing the specific NLP tasks and techniques that will be investigated, such as trend prediction, requirement extraction, and bias detection. a) Generate image databases from the fashion domain. b) Develop a fashion trend observation system based on NLP using digital fashion entities. c) Develop the image model. 5. Ensure the quality of the corpus used for language models by implementing image bias detection and mitigation techniques, and address ethical issues and biases explicitly with clear methodologies. 3.2. Hypotheses For the study of the application of artificial intelligence and natural language processing on design, the following hypotheses are pursued: 1. The implementation of AI and NLP techniques in design process leads to significant improvements in optimization, innovation, and customization of final products, compared to traditional methods. 2. In fashion design, the use of AI and NLP to analyze trends and consumer preferences re- sults in collections that are more closely aligned with current market demands, increasing customer satisfaction and business profitability. 3. The application of AI in industrial design facilitates faster and more accurate creation of prototypes, improving the efficiency of the product development process and reducing operational costs. 4. The use of AI technologies in interior design allows for deeper and more accessible cus- tomization, enhancing user experience and expanding design options without significantly increasing costs. 5. The integration of AI and NLP in interface design improves the usability and accessibility of digital applications, contributing to a better user experience. 6. In the field of visual communication, artificial intelligence facilitates the generation of innovative and adaptive graphic content, enhancing visual communication in terms of relevance and visual appeal for different audiences. 4. Methodology A comprehensive methodology will be developed to validate the research hypotheses. This sec- tion will outline the experimental design, data collection, preprocessing, and analysis methods. 4.1. Experimental design The experimental design will include a series of controlled experiments to evaluate the effec- tiveness of AI and NLP applications in various design fields. • Data Collection: Data will be collected from multiple sources, including design documents, user feedback, and online reviews. For the fashion trend prediction, data from social media, fashion blogs, and retail websites will be used. • Data Preprocessing: Text data will be cleaned and preprocessed using standard NLP techniques. Images will be processed using computer vision techniques to ensure consis- tency and quality. Specifically, for fashion design, NLP techniques will be employed to analyze text data from fashion-related sources to extract relevant trends and consumer preferences. • Analysis Methods: Advanced machine learning and deep learning models will be used to analyze the data. For NLP tasks, techniques such as text classification, sentiment analysis, and trend prediction will be applied. 4.2. Evaluation metrics Clear evaluation metrics will be defined to measure the success of the proposed approaches. • Accuracy: The accuracy of trend predictions and requirement extraction will be measured using standard metrics such as precision, recall, and F1-score. • User Satisfaction: Surveys and user studies will be analyzed to measure user satisfaction and the perceived impact of AI and NLP tools on their design process. • Efficiency: The time and resources saved by using AI and NLP tools compared to tradi- tional methods will be evaluated. 4.3. Validation techniques The results will be validated using multiple techniques to ensure robustness and reliability. • Cross-validation: To prevent overfitting, cross-validation techniques will be employed. • User Studies: Real-world user studies will be conducted to validate the practical effective- ness of the developed models and tools. • Comparative Analysis: The performance of the proposed methods will be compared against existing state-of-the-art approaches to demonstrate their quality. 5. Research issues to discuss Several research issues are suggested and briefly discussed to advance the application of AI and NLP in design effectively and efficiently. • Use of LLMs for design: Integrating Large Language Models (LLMs) into the design process is revolutionizing how designers interact with digital tools. These models facilitate the automation of manual tasks and the generation of design elements from natural language descriptions, enhancing their creativity. • Image Bias Detection and Mitigation Analyzing current techniques and developing new methodologies to identify and correct biases during the training and application phases of models is essential. • Trend Prediction in Fashion The research will explore the application of AI and NLP in predicting fashion trends. By analyzing text data from fashion blogs, social media, and retail websites, the study will develop models to forecast upcoming trends in both textual and visual content. This will help designers and brands stay ahead of market demands and enhance their decision-making processes. Acknowledgments This research has been funded by the Generalitat Valenciana (Conselleria d’Educació, Investi- gació, Cultura i Esport), through the project: NL4DISMIS: Natural Language Technologies for dealing with dis- and misinformation (CIPROM/2021/021) and through the project: CORTEX ”Conscious natural text generation” (PID2021-123956OB-I00). References [1] N. Cila, Designing human-agent collaborations: Commitment, responsiveness, and support, in: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, Association for Computing Machinery, New York, NY, USA, 2022, pp. 1–18. doi:10.1145/3491102.3517500 . [2] J. Heer, Agency plus automation: Designing artificial intelligence into interactive systems, Proceedings of the National Academy of Sciences 116 (2019) 1844–1850. doi:10.1073/ pnas.1807184115 . [3] W. d. M. Peter Kneebone, International council of design, https://www.theicod.org/en, 2024. Last accessed: April 29, 2024. [4] I. K. Nti, A. F. Adekoya, B. A. Weyori, O. Nyarko-Boateng, Applications of artificial intelligence in engineering and manufacturing: A systematic review, Journal of Intelligent Manufacturing 33 (2022) 1581–1601. [5] N. Yüksel, H. R. Börklü, H. K. Sezer, O. E. Canyurt, Review of artificial intelligence applica- tions in engineering design perspective, Engineering Applications of Artificial Intelligence 118 (2023) 105697. doi:https://doi.org/10.1016/j.engappai.2022.105697 . [6] Y. Tsang, C. Lee, Artificial intelligence in industrial design: A semi-automated literature survey, Engineering Applications of Artificial Intelligence 112 (2022) 104884. doi:https: //doi.org/10.1016/j.engappai.2022.104884 . [7] M. L. Castro Pena, A. Carballal, N. Rodríguez-Fernández, I. Santos, J. Romero, Artificial intel- ligence applied to conceptual design. a review of its use in architecture, Automation in Con- struction 124 (2021) 103550. doi:https://doi.org/10.1016/j.autcon.2021.103550 . [8] A. Samuel, N. R. Mahanta, A. C. Vitug, Computational technology and artificial intelligence (ai) revolutionizing interior design graphics and modelling, in: 2022 13th International Conference on Computing Communication and Networking Technologies (ICCCNT), IEEE, 2022, pp. 1–6. [9] Y. K. Lee, How complex systems get engaged in fashion design creation: Using artificial intelligence, Thinking Skills and Creativity 46 (2022) 101137. doi:https://doi.org/10. 1016/j.tsc.2022.101137 . [10] P. Chanchamnan, C. Ho, S. San, Design in the age of artificial intelligence: A literature review on the enhancement of user experience design with ai, IEEE Access (2022). [11] A. M. Abbas, K. I. Ghauth, C.-Y. Ting, User experience design using machine learning: a systematic review, IEEE Access 10 (2022) 51501–51514. [12] S. Mayahi, M. Vidrih, The impact of generative ai on the future of visual content marketing, arXiv preprint arXiv:2211.12660 (2022). [13] Y. Jeon, S. Jin, P. C. Shih, K. Han, Fashionq: An ai-driven creativity support tool for facilitating ideation in fashion design, in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, Association for Computing Machinery, New York, NY, USA, 2021, pp. 7–18. URL: https://doi.org/10.1145/3411764.3445093. doi:10. 1145/3411764.3445093 . [14] G. Chen, A data-driven intelligent system for assistive design of interior environments, Computational Intelligence and Neuroscience 2022 (2022) Article ID 8409495. doi:10.1155/ 2022/8409495 . [15] G. Vasantha, D. Purves, J. Quigley, J. Corney, A. Sherlock, G. Randika, Assessment of predictive probability models for effective mechanical design feature reuse, Artificial Intelligence for Engineering Design, Analysis and Manufacturing 36 (2022) e17. doi:10. 1017/S0890060422000014 . [16] A. West, J. Clifford, D. Atkinson, ” alexa, build me a brand” an investigation into the impact of artificial intelligence on branding, The Business & Management Review 9 (2018) 321–330. [17] R. Sawant, A. Shaikh, S. Sabat, V. Bhole, Text to image generation using gan, in: Proceedings of the International Conference on IoT Based Control Networks & Intelligent Systems - ICICNIS 2021, 2021. Available at SSRN: https://ssrn.com/abstract=3882570 or http://dx.doi. org/10.2139/ssrn.3882570. [18] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, J. Ho, D. J. Fleet, M. Norouzi, Pho- torealistic text-to-image diffusion models with deep language understanding, in: S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh (Eds.), Advances in Neural Information Processing Systems, volume 35, Curran Associates, Inc., 2022, pp. 36479–36494. URL: https://proceedings.neurips.cc/paper_files/paper/2022/file/ ec795aeadae0b7d230fa35cbaf04c041-Paper-Conference.pdf. [19] O. Avrahami, D. Lischinski, O. Fried, Blended diffusion for text-driven editing of natural images, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 18208–18218. [20] H. Akay, M. Yang, S.-G. Kim, Automating Design Requirement Extraction From Text With Deep Learning, in: Volume 3B: 47th Design Automation Conference (DAC), International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, 2021, p. V03BT03A035. doi:10.1115/DETC2021- 66898 . [21] A. Sasidharan Pillai, Advancements in natural language processing for automotive virtual assistants enhancing user experience and safety, Journal of Computational Intelligence and Robotics 3 (2023) 27–36. URL: https://thesciencebrigade.com/jcir/article/view/161. [22] I. Zukerman, D. Litman, Natural language processing and user modeling: Synergies and limitations, User Modeling and User-Adapted Interaction 11 (2001) 129–158. URL: https://doi.org/10.1023/A:1011174108613. doi:10.1023/A:1011174108613 . [23] H. Akay, S.-G. Kim, Extracting functional requirements from design documentation using machine learning, Procedia CIRP 100 (2021) 31–36. URL: https://www.sciencedirect.com/ science/article/pii/S2212827121004637. doi:https://doi.org/10.1016/j.procir.2021. 05.005 , 31st CIRP Design Conference 2021 (CIRP Design 2021).