The emergence of large language models (LLMs), particularly ChatGPT, has fundamentally transformed user expectations regarding interaction with software, including Decision Support Systems (DSS). While LLMs have introduced intuitive dialogue-based interfaces and the ability to process and generate natural language text, they have also raised concerns over reliability, trust, and stability due to phenomena like "hallucinations." This paper analyzes how these developments have reshaped the requirements for modern DSS and explores strategies for adapting existing systems to meet new expectations. Key requirements now include support for natural language queries and responses, contextual query processing, integration of unstructured data, utilization of open-source information, and adaptive response generation based on user profiles. The paper advocates for a service-oriented architecture (SOA) approach, enabling modular upgrades-such as adding an API layer and interaction services-without altering the DSS's core mathematical or data models. This modular design supports quick adaptation to evolving standards while preserving the DSS's integrity, thus ofering a practical pathway to enhance usability, trustworthiness, and relevance in an AI-driven era.
The emergence of modern large language models (LLM) [
Using AI in DSS has always been quite natural [
As a result, ordinary users, on the one hand, appreciated the LLM capabilities and expressed a desire to see similar capabilities in modern DSS, on the other hand, users have accumulated AI distrust, which must also be taken into account when developing modern DSS.
However, since there are a large number of high-quality and useful DSS, in addition to the new DSS development, the question arises about the adapting existing DSS possibility to new requirements and the such adaptation feasibility.
This work is devoted to the analysis of how the requirements for modern DSSs have changed and the technology development that will allow adapting the existing DSSs to these requirements.
Any DSS that does not support natural language queries is perceived by the user as something outdated. Of course, a DSS should have a graphical interface and an intuitive menu, but users are not only accustomed to the ability to communicate with programs in natural language, but also understand that it is not very dificult.
Everyone understands that it’s no longer necessary to independently develop your own converter
word2vec [
It’s even possible to do without direct sentence encoding, and ask questions to LLM, working in the dialogue mode, in which the user’s request is contained in a package with additional information. By asking a number of such questions, it’s possible to clearly decompose the user’s request into simpler formalized instructions for the DSS.
As we can see, in this issue both user expectations and developer capabilities coincide. Therefore, a natural language query becomes the norm for DSS.
It’s also necessary to pay attention that most natural language processing systems are designed to work with English. If the DSS must process queries in Ukrainian, it’s advisable to use the corresponding LLMs, since when translating text from Ukrainian to English and vice versa, the content may be distorted.
From technical standpoint, consistent response text generating is no more dificult than request analyzing. The process doesn’t look very complicated, but there are a few issues to consider.
The modern user is used to formulating relatively short queries and at the same time wants the system
to understand them. Of course, without the query context, it’s impossible to give a correct answer.
LLMs have taught users that they can clarify their queries, i.e. add new requirements formulated as
short queries that do not make sense without analyzing the preliminary conversation. Of course, you
can force the user to work with the DSS correctly, but for users accustomed to modern technologies, this
causes irritation and, accordingly, reduces the motivation to use such a DSS. Therefore, the accumulation
of information that creates the query context is a necessary condition for modern competitive DSS
development. Technologies such as Retrieval-Augmented Generation (RAG) [
Another context source is the user’s personal data. On the one hand, no one wants to share their personal data, no one wants to identify themselves, everyone is about their privacy concerned. But the modern Internet, social networks, and Chat GPT use user information. When an online translator translates a text, it knows the user’s preliminary requests and the sites list visited. Accordingly, it knows the user’s interest area. Using this knowledge, it makes the translation more accurate. And although people do not want to share information about themselves, they have already gotten used to the fact that all modern services adapt to them. We can say that this is already a modern era requirement, since users expect such behavior from software. Accordingly, the DSS should, if possible, meet these requirements.
Of course, the DSS should not be a full-fledged chatbot that can support a conversation on any topic, but it’s still desirable to take into account the context and previous user’s requests. In addition, diferent user groups require diferent levels of information generalization. For example, users with little experience need more detailed information. Specialists are more focused on facts, and managers tend to receive general information with conclusions. And the modern user expects the DSS to understand over time in what format the answers are needed.
Any DSS is based on a models set, and these models don’t necessarily have to use AI. The value of DSS models is in their quality and reliability. However, in recent years, users have become accustomed to being able to ask any questions. And it often happens that user questions go beyond the mathematical apparatus, embedded in the DSS. Even 10 years ago, the opinion was established that requests that go beyond the DSS models definition zone should be answered with a refusal. However, at that time, the LLM didn’t exist. If we already use the LLM to analyze requests and formulate responses, why can’t we use other capabilities? The responses’ reliability will usually be much lower, but with the right combination of the DSS mathematical apparatus and the LLM capabilities, additional capabilities can be obtained.
In response to a user query such as "how will a 10% increase in the oil cost afect the rockets cost?" the DSS may refuse to answer, since the rockets cost formula included in its mathematical model doesn’t include the oil cost. Or it can use the LLM to determine the impact of oil cost on each of the factors included in the formula for calculating rocket cost, and thus calculate a new cost. Of course, this calculation will be approximate, but it’s unknown what exactly the user wants and what accuracy suits him. The DSS task is to warn about the quality of the result obtained. For example, add the phrase "A model of analogies and data from open sources was used in the calculations" to the answer. Thus, the DSS doesn’t limit the user, but warns about a decrease in the answer authenticity.
As is known, "hallucinations" appearance is a disadvantage not only of Chat GPT, but of all LLMs in general. Even if the developers define this as temporary dificulties, it’s clear that the "hallucinations" possibility will decrease sharply in the near future, but will not disappear completely. Therefore, the DSS task is to prevent the "hallucinations" appearance in responses. To reduce the "hallucinations" number in the answers, you can reduce the load on LLM. It’s necessary to explain in detail what is meant by load.
In fact, the LLM is a very powerful tool with many capabilities. Sometimes DSS developers are tempted to transfer part of their work to the LLM, for example, ask it to calculate a value quality or make a conclusion or description on its own. It isn’t very dificult to do - just to form a response request and get a quick result. Exactly such of LLM using leads to the appearance of "hallucinations". Moreover, during DSS testing, there is a high probability of not detecting these "hallucinations". But if the "hallucination" is detected by the user, then the trust in the DSS will be completely violated. If the DSS has a probability of a "hallucination" appearance at the output, then for the user it’s no better than Chat GPT. The user will not figure out where there are more or fewer "hallucinations". For him, the very fact of their possible appearance is a sentence for DSS. Therefore, the issue of LLM correct use is very important.
However, in order to completely prevent the "hallucinations" occurrence, it’s necessary to do
postprocessing and check the responses generated by the LLM. For this, trigrams can be used [
Another problem that occurs when updating the LLM during response generation is that the LLM may give diferent responses to same requests. It’s especially true for LLMs with online access.
Accordingly, if a user received one answer to a request yesterday, and today received a diferent answer to the same request, this will lead to irritation at the very least, and may also lead to the emergence of entire DSS mistrust. Any user expects that, in the absence of changes, the DSS answers to the same questions will be the same. Otherwise, the user has a feeling of misunderstanding how the DSS works, and accordingly, he stops trusting it.
Unfortunately, when using dynamically updating systems like Chat GPT, additional efort is required to achieve stability.
To solve this problem, you can save user queries, and you can work with software fixed versions. Whatever the case, this problem cannot be avoided.
Users of Google, Chat GPT and other quality systems are accustomed to the fact that they can correct an incorrect answer and the same error will not happen again next time. It’s clear that large companies have powerful support departments and improve their systems due to such corrections. It’s their development strategy part. But users have already gotten used to such capabilities, and therefore the DSS should also have them. Unfortunately, it’s necessary to spend human resources and time to analyze the error and correct the mathematical model. It can’t be done quickly. If the DSS uses, for example, neural networks for modeling, then time will be needed for training and testing. The user, in turn, wants his correction to be immediately taken into account by the DSS. To resolve this conflict, the following approach is used - a post-processing module is added to the DSS output, which accumulates and takes into account user feedback. It allows, on the one hand, the user to immediately see the corrected results, and on the other hand, it gives the developers the opportunity to accumulate and analyze user corrections for a certain period, and then make a new DSS modification. From a scientific point of view, this approach can’t be called self-training or self-improvement. But for the user, what is important isn’t what happens behind the scenes of the user interface, but that the system has taken into account his comments.
When developing such post-processing, there is a need to search for new requests to match added exceptions. For this, it’s also possible to use LLM according to the schemes described above. But a large number accumulation of exceptions significantly increases the time and computing resources necessary for processing requests. That’s why the SSP should be updated regularly.
It can be said that "unfortunately, the average level of DSS professional knowledge user has dropped significantly." Or it can be said that "the DSS powerful development the has allowed users to quickly master new areas of activity." Both statements are true and complement each other. Indeed, a modern user, on the one hand, very easily masters new areas of activity, on the other hand, he lacks basic knowledge and his professional education level isn’t very high. Should this be taken into account when developing a DSS? The answer is "definitely yes." The task of modern DSS developers is to do everything possible so that such users can master the work on the DSS as quickly as possible. The use of correct reference, visualization of information, calculations of auxiliary quantities, a user-friendly interface - all this makes mastering the DSS easier and faster. If earlier the presence of such capabilities was an additional plus of the DSS, now it has become a necessary condition for its use. Of course, the development of such capabilities takes a lot of time and resources, while not adding scientific novelty. But a modern DSS can’t be targeted at a small group of specialists, which will make it uncompetitive.
All of the above indicates that modern DSS should have additional properties, but, as we can see, these
additional properties don’t require changes in either the mathematical apparatus of the DSS or the
data structure. Moreover, most of the necessary improvements are the same for certain classes of DSS.
Accordingly, it’s possible to propose a technology that will accelerate the adaptation of existing DSS to
modern requirements. In our opinion, the use of service-oriented architecture (hereinafter SOA, from
Service-oriented Architecture) [
In fact, most of the new properties concern the sphere of interaction between the DSS and the user. Accordingly, if an application programming interface (hereinafter API, from Application Programming Interface) is added to the DSS, then the interaction services with the user and the outside world can use DSS as a service that performs mathematical calculations, modeling and other specialized operations. The service of interaction with users is relatively universal and can be used to work with a whole class of similar DSS.
Accordingly, the DSS modification comes down to adding a module to the DSS that will provide API support.
The services set may vary depending on the tasks. In our opinion, based on the above arguments, the basic additional services set should include: • Natural Language Query Service; • Natural Language Postprocessing and Response Generation Service; • Query context generation service; • Interaction service with external information sources; • Unstructured information processing service.
In fact, the DSS also becomes one of the services that can be replaced without changing the structure of service interaction (Figure 1).
The user still has the ability to interact with the DSS through the DSS Graphical User Interface, which was developed specifically for this DSS, and the ability to interact with the DSS through response services appears.
This technology allows adapting the DSS to modern requirements with minimal efort.
It doesn’t require changing the programming language in which the DSS is written, nor changing the DSS itself architecture. Adding an API module should not take much time, but it requires efort to understand the DSS itself structure. If a set of additional services has already been created and interaction interfaces have been agreed upon, then in fact, creating and connecting an API module will allow the user to interact with the DSS through the corresponding services.
Compared to modifying the entire DSS, this approach is more expedient and faster.
The paper analyzes the impact of LLM emergence on the requirements for modern DSS.
The following requirements for DSS are described, which should be taken into account when developing new DSS and improving existing ones: 1. Requirements afected by new capabilities: • Natural language style queries; • Responses generation in natural language style; • Unstructured information processing; • Data from open source usage; • Taking into account knowledge about the user (request context).
• DSS protection against the "hallucinations" appearance; • Stability.
• DSS self-improvement; • Taking into account user qualifications.
The paper proposes a technology for DSS adapting to modern requirements using a service architecture that ensures the DSS improvement with minimal efort.
By using API, minimal changes are made to the DSS structure. And using of service architecture allows to apply of modules that use LLM to work with diferent DSSs.
As a result, the user has the opportunity to interact with DSS both through the basic user interface and through additional services that provide support for new functionality that meets modern requirements.