The article is devoted to the development of Markov models for assessing the readiness of artificial intelligence (AI) systems in critical areas, taking into account retraining procedures. A conceptual model of an information and control system with AI (AI-ICS) is proposed, which includes a state space, failures, and maintenance procedures, such as online verification and retraining. A feature of the developed singleand multi-fragment Markov models is that they allow for the assessment of AI-ICS readiness, taking into account various parameters, both traditional for software and hardware systems (failure and recovery rates), and parameters of planned and reactive retraining processes and the resulting change in the corresponding system indicators. It is shown that the multi-fragment model surpasses the single-fragment one in accuracy, demonstrating the ability to account for adaptation through retraining. Prospects for future research are discussed, including two-version structures that increase safety by reducing Common Cause Failures Risks, and the development of diversification technologies in the creation of AI.
1.1.
Systems of artificial intelligence (AI) play a key role in critical areas, where they provide
automation of complex processes, real-time analysis of large volumes of data, and decision-making
under uncertainty. In medicine, AI is used for diagnosing diseases based on medical images and
predicting pandemics; in transport, for controlling autonomous vehicles; in the energy sector, for
optimizing resource distribution and conducting predictive maintenance; and in the defense and
security sectors, for threat analysis, risk prediction, decision-making support, and humanitarian
demining [
Errors in such systems can have catastrophic consequences, including loss of human lives,
power grid blackouts with significant economic losses, or security breaches in defense systems due
to erroneous decisions or cyberattacks [
Modern research is focused on increasing the protection of AI from attacks and model
anomalies. In [
In general, a certain deficit of mathematical models for AI systems can be concluded, which allow the calculation of complex availability indicators that take into account various parameters of the AI itself and training processes, as well as software and hardware platforms, enabling the investigation of the dependencies of these indicators on parameter changes and the formation of recommendations for ensuring compliance with the requirements for such systems. 1.2.
The goal of the study is the development of Markov models for assessing the readiness of nonredundant AI systems, taking into account changes in parameters due to retraining and the formation of recommendations for increasing readiness and dependability.
Objectives are the following: • the development of a conceptual model of an information and control system with AI (AIICS) (section 2), which takes into account the features of the failure and recovery processes of its components and the system as a whole; • the development and study of Markov models of the AI-ICS (section 3); • the analysis of the modeling results of the respective advantages and limitations in the use of models, as well as the formulation of recommendations for the selection of system parameters to increase readiness (section 4); • the determination of the main contribution, results, and directions for further research (section 5).
The research methodology is based on: • the consideration of the AI-ICS as a set of the model part of artificial intelligence, its software and hardware implementation, and the environment with cyber-physical effects on different system components; • the detailing of the AI-ICS failure and recovery model, taking into account the main factors and types of faults; • the use of single- and multi-fragment Markov models that allow for the consideration of parameter changes during training and possible improvement of the model part of the AI system due to online verification processes.
A conceptual model of an information and control system with AI (AI-ICS) is based on a software and hardware platform and an AI model that performs control functions (Figure 1). It describes the system's state space, which includes working states Sw, failure states Sf, and maintenance states Sm: S=Sw∪ Sf ∪ Sm , (1) where Sw={Sq , S'1 }, Sf ={S HW , SSW , S AI }, Sm={SOV , SRL }.
• SHW, SSW, SAI – are the failure states of the hardware, software, and AI model; • SOV – are the online verification states; •
Online verification (OV) checks the system's compliance with requirements in real-world conditions, while Retraining (Re-Learning) adapts the AI model to new conditions or eliminates errors. Transitions between states are modeled as a mapping T: S→S, where the failure rates (λHW, λSW, λAI) and recovery rates (μHW, μSW, μAI) determine the system's reliability dynamics.
For the study of AI systems, the functioning of the so-called single-version architectures was considered, which allows to simplify the models at the initial stage of the study and obtain adequate values of the input parameters of the model from today's available sources. Markov models have been developed to analyze the readiness of a single-version AI system, which take into account planned and reactive additional training. 3.1.
The state space of a single-fragment model (Figure 2) includes the following states: • S0: full operability; • S1: partial performance (eg reduced accuracy); • •
with the conditions that P0 (t = 0) = 1 and for any moment t: (2) Availability function can be presented by following formula: P0( t )+ P1( t )+ P2( t )+ P3( t )=1 ,
A ( t )= P0( t )+ P1( t ) .
The multi-fragment model (Figure 3) expands the system's state space by adding a new postretraining fragment (S5-S9) and the probability Dp=0.8 of successful scheduled retraining. Parameters λ56′, λ57′, λ67′ in the second fragment are reduced due to model updates (Table 2).
Figures should be centered, and their captions should be placed below them.
Thus, the multi-fragment model takes into account the separation of pre-learning states into for scheduled pre-learning, S4 for reactive pre-learning in Fragment 1 (analogously, S8 and S9 in Fragment 2). After reactive retraining (S4→S5), the system switches to Fragment 2, where the parameters λ56′, λ57′, λ67′ differ from λ01, λ02, λ12 due to model updates. After routine retraining With S3, the system returns to S0 with probability Dp or moves to S5 with probability 1−Dp. Fragment 2 follows the structure of Fragment 1, but with new parameter values reflecting the effect of prelearning.
The single-fragment Markov model was implemented in MATLAB using the function fM1.m, which constructs vertex matrix V (defining states S₀ to S₃ with coordinates and colors for visualization) and edge matrix E (defining transitions with intensitiesλ01, λ02, λ12, μ03, μ13, μ23, μ30). The script m_01.m sets global parameters (Table 1), builds the transition matrix A via matrixA.m, solves the Kolmogorov differential equations using ode15s with the stiffness handler stiff.m over a 100hour interval, computes availability A(t) as (4), and plots A(t) along with individual state probabilities.
As a result of the investigating the first model by Matlab, it was found that (Figure 4): • the availability function A(t) drops rapidly from 1 to 0.919 in the first 5 hours due to high frequencies of transitions to the pre-learning state (S3) and system degradation. In the future, A(t) stabilizes at the level of 0.893, with the system spending 82.5% of the time in the state of full working capacity (S0), 6.9% – in partial working capacity (S1), 2.7% – in incapacity (S2) and 7.9% – in additional training (S3); • the relatively high probability of entering the S3 state is due to planned additional training (μ03=0.033), which significantly reduces availability. Reactive retraining (μ13=0.1, μ23=0.2) effectively reduces time in S1 and S2, but does not compensate for losses from S3; • to improve performance, it is recommended to reduce the frequency of scheduled retraining or speed up the recovery process.
Thus, the Markov model for a retraining AI system provides a methodological basis for analyzing its behavior, taking into account both planned and reactive adaptation strategies, but at the same time does not allow to assess the reliability of the AI system under the conditions of making changes to the system during the retraining process.
The multi-fragment model was implemented in MATLAB using the function fM2.m, which extends the single-fragment approach to n fragments (determined by array sizes of λ01, λ02, λ12), constructing expanded V and E matrices with 5 states per intermediate fragment (S0-S4 or equivalents) and 4 for the last, incorporating probability Dp for branching in retraining transitions (μ30, μ35, μ45). The script m_02.m sets array-based parameters for 3 fragments, builds A via matrixA.m, solves the ODE system using ode15s with stiff.m over a 500-hour interval, computes aggregated availability Ag(t) as the sum of probabilities for full/partial operability states, groups probabilities into P1-P5 for state categories, and plots Ag(t) with these groups. For comparison, m_02_1.m computes and plots the difference ΔA(t) between multi- and single-fragment availabilities. As a result of modeling the multi-fragment model (Figure 4), it was found that: • the availability factor gradually decreases to 0.916 for the first 5.56 hours of operation. The rate of decrease of A(t) slows down with time, and then, at t = 500, it stabilizes at the level of 0.899, which indicates the achievement of a stationary state of the system; • the sum of the probabilities of being in the states of full operability (S1, S6, S11) decreases to 0.855 at t = 500, which is 85.5% of the initial value. This indicates that the system retains its initial operability for a significant part of the time, although it gradually loses it due to transitions to other states; • the sum of the probabilities of being in the states of partial operability (S2, S7, S12) increases to 0.045 (4.5% of the maximum value) at t = 500, indicating a limited time of the system in partially operable states. The system is completely inoperable for a small fraction of the time (0.025, or 2.5%); • the sum of the probabilities of being in the states of planned retraining (S4, S9, S14) demonstrates a significant fraction of the time spent on retraining or adaptation processes (0.075, or 7.5%). We note that the additional states that model the processes of reactive retraining show an extremely low probability of the system being in these states (2.61×10⁻⁵), associated with rare events.
a) b) c)
In addition, we conclude that the high proportion of (S4, S9, S14) (7.53%) can be due to intensive planned retraining processes with a frequency of μ03 = 0.033. This significantly affects the overall availability of the modeled system, reducing A(t). The states that reflect partial operability and inoperability remain at relatively low levels due to fast reactive processes (μ13 = 0.1 and μ23 = 0.2), which effectively reduce the time the system spends in these states.
Thus, retraining processes are the dominant factor limiting the system readiness. It is clear that the single-fragment model is simpler and predicts the system behavior faster, but underestimates the readiness in the long term due to the generalized approach.
The multi-fragment model, taking into account the dynamics of the parameters, provides higher accuracy and better adaptation to real conditions, which is confirmed by the higher value of A(t) in the steady state. For AI systems, where detail and long-term stability are important, the multifragment model is a better choice, although it requires more complex tuning.
The main contribution of the research is suggested Markov’s models and results of their investigation. These models allow describing processes of re-learning and changing AI-ICS parameters that impact on system availability. Due to theses models can be improved accuracy of availability assessment.
The proposed Markov models allow assessing the readiness of AI-ICS, taking into account various parameters, both traditional for software-hardware systems, and parameters of re-learning processes and the resulting change in the corresponding indicators.
The multi-fragment model exceeds the single-fragment model in accuracy (A(t)=0.89999 versus 0.893348), demonstrating the possibility of taking into account adaptation through re-learning.
Future research can be aimed at: • first, detailing Markov models by taking into account more complex failure scenarios due to cyberattacks or hardware degradation, which will increase the accuracy of predicting the behavior of AI systems; • second, developing requirements and substantiating quantitative values for AI characteristics such as ethics and legality. These steps will contribute to the development of AI Safe and Secure Systems Engineering as a separate discipline that meets the current needs of critical industries; • third, researching two-version structures that increase safety by reducing Common Cause Failures Risks, but their implementation is complicated by the need to develop diversification technologies when creating AI.
Acknowledgements This research was partly supported by project «Dependability and resilience of intelligent UAVUSV complexes with combined application strategies», 0124U000945, MES of Ukraine. Declaration on Generative AI
[13] A. Yanko, V. Krasnobayev, A. Hlushko, and S. Goncharenko, "Neurocomputer operating in the Residue Class System," Advanced Information Systems, vol. 9, no. 2, pp. 84–92, Apr. 2025. doi:10.20998/2522-9052.2025.2.11 [14] R. Naidu, "Bio-inspired AI Architectures: Lessons from nature for building next-Generation
Learning Systems," IJARIIE, URL: https://surl.li/daolxs [15] M. Joshi, "Adaptive Learning through Artificial Intelligence," SSRN Electronic Journal, 2023.
doi:10.2139/ssrn.4514887 [16] J. Mendonça, F. Machida, and M. Völp, "Enhancing the reliability of perception systems using N-version programming and rejuvenation," 2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), pp. 149–156, Jun. 2023. doi:10.1109/dsn-w58399.2023.00044 [17] B. K. Konidena, J. N. Malaiyappan, and A. Tadimarri, "Ethical considerations in the development and deployment of AI Systems," European Journal of Technology, vol. 8, no. 2, pp. 41–53, Mar. 2024. doi:10.47672/ejt.1890 [18] S. T. Mortaji and M. E. Sadeghi, "Assessing the reliability of Artificial Intelligence Systems: Challenges, metrics, and future directions," International Journal of Innovation in Management, Economics and Social Sciences, vol. 4, no. 2, pp. 1–13, Jun. 2024. doi:10.59615/ijimes.4.2.1 [19] J. Perez-Cerrolaza et al., "Artificial Intelligence for safety-critical systems in industrial and Transportation Domains: A survey," ACM Computing Surveys, vol. 56, no. 7, pp. 1–40, Apr. 2024. doi:10.1145/3626314 [20] M. Al-Hawawreh, Z. Baig, and S. Zeadally, "AI for Critical Infrastructure Security: Concepts, challenges, and future directions," IEEE Internet of Things Magazine, vol. 7, no. 4, pp. 136–142, Jul. 2024. doi:10.1109/iotm.001.2300181 [21] F. Wotawa, On the use of available testing methods for verification. In Proceedings of the Workshop on Artificial Intelligence Safety 2021 (SafeAI 2021). February 8, 2021. pp. 1-6. URL: https://ceur-ws.org/Vol-2808/Paper_29.pdf [22] G. G. Shayea, M. H. Zabil, M. A. Habeeb, Y. L. Khaleel, and A. S. Albahri, "Strategies for protection against adversarial attacks in AI models: An in-depth review," Journal of Intelligent Systems, vol. 34, no. 1, Jan. 2025. doi:10.1515/jisys-2024-0277