<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Leveraging Trustworthy AI for Automotive Security in Multi-Domain Operations: Towards a Responsive Human-AI Multi-Domain Task Force for Cyber Social Security</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Vita Santa Barletta</string-name>
          <email>vita.barletta@uniba.it</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Danilo Caivano</string-name>
          <email>danilo.caivano@uniba.it</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Gabriel Cellammare</string-name>
          <email>g.cellammare1@studenti.uniba.it</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Samuele del Vescovo</string-name>
          <email>samuele.delvescovo@imtlucca.it</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Massimiliano Morga</string-name>
          <email>m.morga@serandp.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Annita Larissa Sciacovelli</string-name>
          <email>annitalarissa.sciacovelli@uniba.it</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>SER&amp;P, Spin-of of University of Bari Aldo Moro</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Scuola IMT Alti Studi Lucca</institution>
          ,
          <addr-line>Piazza S.Francesco, 19, 55100 Lucca</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Università degli studi di Bari Aldo Moro</institution>
          ,
          <addr-line>Piazza Umberto I, 70121 Bari, Apulia</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Multi-Domain Operations (MDOs) emphasize cross-domain defense against complex and synergistic threats, with civilian infrastructures like smart cities and Connected Autonomous Vehicles (CAVs) emerging as primary targets. As dual-use assets, CAVs are vulnerable to Multi-Surface Threats (MSTs), particularly from Adversarial Machine Learning (AML) which can simultaneously compromise multiple in-vehicle ML systems (e.g., Intrusion Detection Systems, Trafic Sign Recognition Systems). Therefore, this study investigates how key hyperparameters in Decision Tree-based ensemble models-Random Forest (RF), Gradient Boosting (GB), and Extreme Gradient Boosting (XGB)-afect the time required for a Black-Box AML attack i.e. Zeroth Order Optimization (ZOO). Findings show that parameters like the number of trees or boosting rounds significantly influence attack execution time, with RF and GB being more sensitive than XGB. Adversarial Training (AT) time is also analyzed to assess the attacker's window of opportunity. By optimizing hyperparameters, this research supports Defensive Trustworthy AI (D-TAI) practices within MST scenarios and contributes to the development of resilient ML systems for civilian and military domains, aligned with Cyber Social Security framework in MDOs and Human-AI Multi-Domain Task Forces.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Multi-Domain Operations</kwd>
        <kwd>Trustworthy AI</kwd>
        <kwd>Cyber Social Threat Intelligence</kwd>
        <kwd>Automotive Security</kwd>
        <kwd>Human-AI Responsive Collaboration</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Multi-Domain Operations (MDOs) constitute the central paradigm of modern military strategy,
emphasizing the integrated coordination of capabilities across Space, Air, Land, Sea, and Cyber domains to
produce synergistic malicious efects [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. These operations generate complex, multi-surfaced challenges
aimed at complicate the adversary’s Military Decision-Making Process (MDMP) [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ]. Within this
framework, Multi-Domain Task Forces (MDTFs) play a key role by aligning defence resources across
both physical and informational environments [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. A core challenge in MDOs is the lack of defensive
synchronization between kinetic and cyber capabilities across domains [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. To address this, the Cyber
Social Security (CSS) framework has been introduced to integrate cyber defence within traditional
domain-based strategies, forming the CSS-MDO framework [
        <xref ref-type="bibr" rid="ref6 ref7">6, 7</xref>
        ].
      </p>
      <p>
        The CSS-MDO framework adopts a multidimensional approach, where the five active warfare domains
are represented along the horizontal axis, each equipped with specialized tools, methods, and procedures
[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. These are aligned with the vertical axis representing the Detection-Response-Prevention processes.
In this framework, unlike other military domains, the "cyber" domain is not only a "link" between the
various "traditional" domains but it has its own ofensive/defensive identity [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. This bi-dimensional
structure identify operational tiers (i.e. horizontally and vertically) in which MDTFs may operate [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>
        Smart cities constitute vulnerable assets within future MDOs, especially involving the Cyber and
Land domains. Among the critical components of a smart city’s Internet of Things (IoT) infrastructure,
Connected and Autonomous Vehicles (CAVs) represent a prominent point of vulnerability [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. As
foundational enablers of shared and electric mobility, CAVs are central to optimizing urban transportation
and improving sustainable travel models [
        <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
        ]. However, in this rapidly evolving and innovation
context, this high-value assets’ attack surfaces are expanding and complicating in ways challenging the
capacity of any "cyber-social" blue team to efectively detect and mitigate threats. Specifically, attackers
(more or less skilled) may exploit known vulnerabilities in In-Vehicle Networks (IVNs) based on the
Controller Area Network (CAN) protocol [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. These attacks may be part of broader, coordinated
military/civilian activities conducted across multiple domains to create complex and orchestrated dilemmas
for defenders [13].
      </p>
      <p>Within the context of MDOs and the realted Multi-Domain Threats (MDTs), one of the attackers goals
may be to compromise Electronic Control Units (ECUs), which function as critical communicational
nodes within IVNs [14]. This threat can also afect the autonomous vehicle’s driving system by exploiting
various IVNs levels. Such intrusions can lead to anomalous or unsafe behavior in the afected vehicle,
thereby posing direct risks to its functionality and safety [15]. Furthermore, an attacker can compromise
physically road signs to compromise autonomous driving’s features of the vehicle. In response to these
threats, the integration of Artificial Intelligence (AI) and Machine Learning (ML) technologies has
emerged as a promising avenue for reinforcing vehicular cybersecurity. Among these approaches, there
are ML-based Intrusion Detection Systems (IDSs) as well as Trafic Sign Recognition Systems (TSRS),
which are specifically designed to detect abnormal communication patterns and unauthorized access
attempts within IVNs [16].</p>
      <p>So, an adversary can manipulate input data, such as images or CAN bus frames, at the testing or
deployment phase [17]. These perturbations are imperceptible to human observers yet are suficient
to deceive the targeted ML model into producing incorrect classifications [ 18, 19]. AML attacks are
generally categorized into three scenarios. These are discriminated by the level of knowledge the
attacker possesses about the internal architecture and parameters of the victim model. Among these,
the Black-Box setting is considered the most plausible and accessible from an adversarial perspective,
as it assumes no prior access to or knowledge of the model’s internal workings [20, 21]. Moreover,
aligning with the conceptual framework of MDTs, in which adversaries operate across multiple domains
simultaneously, it is plausible to hypothesize scenarios in which attacks are exploited on diferent
surfaces of a single asset. This concept can be referred to as Multi-Surface Threat (MST). Current
literature on the application of Black-Box attacks within the CAN bus frame detection task remains
limited and in an early stage of development (even in MDTs and MSTs scenarios).</p>
      <p>
        Therefore, the primary goal of this paper is to investigate about the role of specific hyperparameters
associated with Decision Tree (DT)-based ensemble Technology Transfer (TT) models. These are
Random Forest (RF), Gradient Boosting (GB), and Extreme Gradient Boosting (XGB). These are the core
of the supervised ML-based victim IDS for the CAN bus frame detection task. It is assumed that the IDS
(installed onboard the vehicle) is subjected to a Black-Box AML attack i.e. the Zeroth Order Optimization
(ZOO) in a pure evasive Black-Box setting. This type of attack is conceptualized as a part of a MST,
so a Single-Surface Threat (SST), falling into the Cyber component of a complex MDT. This threat is
framed into a MDO. So, this work tries to address how variations in selected hyperparameters afect the
time required to generate adversarial examples for each targeted ML model. Results indicate that the
number of bagging trees in RF and the number of boosting rounds in GB have a significant impact on
the attack time. Thus, the same does not hold for the boosting rounds in XGB. These hyperparameters,
in the cases of RF and GB, can be interpreted as intrinsic (or deterrence) defense against the ZOO attack.
Appropriately values for these hyperparameters may lead to a trustworthy AI-by-design approach
for In-Vehicles (I-V) ML systems’ robustness [22], [17]. All of that can contribute to the concept of
Defensive Trustworthy AI (D-TAI) i.e. AI for defence purposes. In particular, the work underscores the
relevance of robustness [23] and security [24] properties in the design and deployment of ML models
within adversarial environments [25]. Additionally, Adversarial Training (AT) time is analyzed to better
understand the attacker’s "window of opportunity", proving the combo between hyperparameters’
correct tuning and AT can slow the attacker’s steps, fostering the activities of the blue team. Finally,
this work aims to contribute to the proper education regarding responsible development of ML systems
in both civilian and military (public/private) contexts for industries and academies, promoting their
integration into the CSS-MDO [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] framework as part of a defence strategy coordinated by
HumanAI Multi-Domain Task Forces (H-AI MDTF). The proof of concept is the qualitative identification of
the positive impact of this trustworthy AI-by-design practice on the vertical axes of the CSS-MDO
framework.
      </p>
      <p>In summary, the research questions (RQs) are:
• RQ1: Can the hyperparameters related to the number of bagging trees in RF, the number of
boosting rounds in GB, and the number of boosting rounds in XGB afect the time needed to
generate ZOO adversarial examples, when applied to supervised ML-based IDS for the CAN bus
frame detection task (in a Black-Box attack scenario)?
• RQ2: Can the combination of the timing for the AT and the correct setting of the previous
parameters slow down the attacker’s actions by reducing the attacker’s "window of opportunity"?
• RQ3: Is it possible to qualitatively quantify the (positive) impact of these values on the
Detection</p>
      <p>Response-Prevention axes of the CSS-MDO framework (for H-AI MDTFs educational purpose)?</p>
      <sec id="sec-1-1">
        <title>So, the main contributions are:</title>
        <p>• Empirically evaluate the influence of hyperparameters as the number of bagging trees in RF,
the number of boosting rounds in GB, and the number of boosting rounds in XGB on the time
needed to generate adversarial examples exploiting the ZOO attack, when applied to supervised
ML-based IDS in the CAN bus frame detection task (considering a Black-Box threat model).
• Understand whether the combination of the correct setting of the previously mentioned
parameters and the time needed to execute the AT can slow down or reduce the attacker’s "window of
opportunity".
• Qualitatively quantify the (positive) impact of these values on the "Detection", "Response" and
"Prevention" axes of the CSS-MDO framework (for H-AI MDTFs educational purpose).</p>
        <p>The paper is organized as follows: section 2 describes the related works; section 3 describes the
experimentation setup; section 4 shows the results; section 5 conclude the work and explain the future
developments.</p>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>2. Related Work &amp; Research Gap</title>
      <sec id="sec-2-1">
        <title>2.1. Evasion Black-Box AML for CAN Bus Frame Detection</title>
        <p>To the best of our knowledge, the scientific literature addressing Black-Box AML attacks against
ML-based IDS in the context of CAN bus frame detection remains relatively underdeveloped actually.</p>
        <p>Zenden et al. [26] examined the Fast Gradient Sign Method (FGSM) attack’s impact on various ML and
DL models’ performance within a surrogate Black-Box attack scenario. Their study further demonstrated
that adversarial training serves as an efective mitigation strategy, yielding notably positive outcomes.
The evaluated models included BL-DNN, BL-Ensemble, SOTA-CNN, and SOTA-LSTM architectures
[26]. The experiments were conducted using a subset of the Survival dataset [27]. The results indicated
that ML models were particularly susceptible to the FGSM attack, exhibiting an accuracy degradation
of approximately 40%.</p>
        <p>Longari et al. [19] proposed a novel methodology for executing Black-Box evasion attacks (in a pure
scenario) against ML-based IDS within the context of online CAN bus frame detection. Their method
targets the entire transmission flow by analyzing segments of CAN payloads. The attack strategy
employs a sliding window technique over the payload data, rather than processing entire examples
in isolation [19]. The experimentation utilizes the “ReCAN” dataset, specifically the “ID C-1” subset
collected from a real Alfa Romeo Giulia Veloce vehicle [19]. A range of ML algorithms were evaluated
as the IDS core, including Small-LSTM, Small-GRU, Large-GRU, CANnolo, Neural Network, Vector
Auto-Regressive (VAR), and Hamming distance-based models [19].</p>
        <p>Instead, Aloraini et al. [18] have conducted an adversarial attack using a substitute victim IDS, trained
on data extracted from the OBD-II interface. This dataset is diferent from the one used to train the
real victim IDS [18]. This scenario constitutes a non-pure Black-Box due to the transferability of the
adversarial examples exploited [18]. The victim IDS models were: a baseline proprietary IDS based on
Deep Neural Network (DNN) and one state-of-the-art model, i.e. MTH-IDS. The surrogated models
were a DNN and a DT. The dataset exploited for the surrogated model is the Car Hacking Dataset [28].
Several White-Box AML attacks were considered like FGSM, Basic Iterative Method (BIM), Projected
Gradient Descent (PGD) and Jacobian-based Saliency Map Attack (JSMA) [18]. The experimental results
have shown the decrease of the F1 scores from 95% to 38% and from 97% to 79% respectively for the real
victim models [18].</p>
        <p>These works do not consider attacks conducted directly against the target IDS in a pure Black-Box
scenario in a TT context. Moreover, they do not explore the application of state-of-the-art Black-Box
AML techniques that are not explicitly tailored to this specific task. In contrast, the study by Barletta et
al. [29] investigates the application of the ZOO attack within a pure Black-Box setting for the same
task, focusing on supervised ML algorithms. Originally developed for image recognition tasks, the
ZOO attack was exploited using the OTIDS dataset [30]. The victim models included DT, RF, GB, and
XGB (contestualized in TT scenarios). Experimental findings revealed a reduction in weighted accuracy
of approximately 70%. Furthermore, adversarial training was once again validated as an efective
countermeasure against such attacks.
2.2. Evasion Black-Box AML Attacks &amp; Defensive Trustworthy AI in CAN Bus Frame</p>
        <p>Detection for MDOs
This paper aims to explore the concept of trustworthy AI for defensive purposes (specifically the
robustness dimension) as a proactive defense mechanism (i.e. D-TAI) for ML-based systems in the
automotive domain. Among existing countermeasures, AT is the most widely adopted technique to
enhance the security posture of ML-based systems and mitigate the risks posed by Black-Box AML
attacks [31, 32, 29]. However, the current literature reveals a significant gap concerning the role of
robustness-by-design oriented programming practices in the development of ML-based systems for
CAN bus frame detection under Black-Box AML attack scenarios.
2.3. Defensive Trustworthy AI Impact Evaluation on CSS-MDO Framework for H-AI</p>
        <p>MDTFs
It is necessary to consider the impact of best ML-based systems programming practices (for D-TAI
and especially for robustness of ML models) on Single Surface threats (considered as a part os MSTs
targetting various I-V Systems) mapped within the CSS-MDO framework’s policies. In other words,
it is necessary to consider the positive impact of properly educating MDTFs about these practices
emphasizing the virtuous collaboration between human and (well-setted) artificial agents in H-AI
MDTFs. All of that can be a piece of the MDTs big puzzle in MDOs scenario. Accordingly, this paper
seeks to underscore this critical need for future research.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Methodology</title>
      <p>In this section (useful for answering all RQs), details about the Black-Box attack scenario, the ZOO attack
pipeline, the empirical estimation of attack and AT time and the qualitative analysis of the parameters’
impact on the CSS-MDO framework’s axes are discussed. All this is about the CAN-based SST. This
work is based on Python 3.9. The implementation of the ML models is provided by the Scikit-learn
library. The XGBoost model implementation is based on the xgboost library. The Pandas framework is
used to handle the dataset. The ZOO attack implementation is provided by the Adversarial Robustness
Toolkit (ART) [33]. The working machine is supported by an AMD Ryzen 5 2600 Six-Core Processor
and 16 GB of RAM.</p>
      <sec id="sec-3-1">
        <title>3.1. CAN-based SST Attack Scenario</title>
        <p>This phase addresses the RQ1. Considering the possibility of MSTs (in the examined case impacting
several I-V systems simultaneously), an example of that could be a simultaneous threat on the
CANbased IVN protected by an IDS for the CAN bus frame detection task and on the network enabling the
Trafic Sign Recognition System (TSRS). Figure 1 describe the attack scenario.</p>
        <p>The CAN-based SST attack begins with a Vulnerability Assessment and Penetration Testing (VAPT)
phase targeting the CAN based IVN, aiming to compromise a single ECU. This initial step facilitates
both the exfiltration and injection of CAN frames, enabling the attacker to infer behavioral patterns of
the target IDS. The ultimate goal is to penetrate the IDS module itself [34], thereby obtaining the true
label assigned to each preprocessed frame for the generation of corresponding adversarial examples.
Additionally, the attacker may observe the IDS’s output by compromising a module that interfaces with
the IDS. Importantly, the attacker operates under a pure Black-Box scenario, with no prior knowledge
of the victim system’s internal architecture or parameters [29].</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Attack Pipeline for the CAN-based SST</title>
        <p>This work is based on the Barletta et al. [29] attack pipeline, useful for training the victim ML models.
The OTIDS dataset [30] is prelaborated following the Bari et al. [31] pipeline. The final dataset version
is splitted into three parts:  (i.e. the 60% part),  and  (i.e. the other 20% parts). ML models useful for
empirical estimation are: RF bagging-based, GB and XGB (in their default configurations). The attack
pipeline extracted is composed by these steps (for each victim ML model):
1. IDS training on the  dataset (obtaining  );
2. Adversarial examples sets’ generation i.e. ′ and ′ (starting from the  and  sets) on  ;
3. Training on  +  + ′ dataset, obtaining  ++′ (Adversarial Training);
A K-Fold Stratified Cross Validation (K-FSCV) (with  = 5) is performed before running the ZOO
attack on the  subdataset. The ZOO attack follows the default configuration except for:
• the learning_rate setted to 0.1 (default is 0.01). The attacker probably want to converge very fast
(during the gradient descent);
• the max_iter setted to 50 (default is 10). The attacker probably want to get examples very close to
the normal ones (by increasing the number of trials);
• the variable_h setted to 0.2 (default is 0.0001). The attacker probably wants the adversarial
examples quickly (enlarging the extremes of the search range);</p>
        <p>This research work is not interested in quantifying the impact of the attack on the victim ML
performances’ since Barletta et al. [29] have already explored that. For this reason, phase three of the
attack pipeline directly considers the execution of the adversarial training.
3.3. Empirical Estimation of the Hyperparameters’ Influence &amp; AT time on the</p>
        <p>CAN-based SST
This subsection is useful for answering RQ1 and RQ2. Ideally, a Vehicle-Security Operations Center
(Vehicle-SOC) involved in a MDTF would prefer to exploit a ML-based IDS that maximizes the time
required for generating adversarial examples while minimizes the coutermeasure’s time, thereby
increasing the attack’s operational cost. Considering that, certain hyperparameters of ensemble-based
ML models specifically RF, GB, and XGB can be conceptualized as intrinsic defensive mechanisms,
potentially influencing the computational efort needed to generate input adversarial examples. The
approach adopted in this study involves measuring the time (seconds) required to generate 92270
adversarial examples for each model (i.e. RF, GB, XGB), as a function of incrementally varied hyperparameter
values. Time is detected after about five minutes of computation. These include the number of bagging
trees in RF and the number of boosting rounds in both GB and XGB. The empirical assessment is
conducted on  , focusing on the second phase of the previously described attack pipeline, and it
is limited to the ′ dataset, assuming its consistent with ′. The goal is to determine whether a direct
proportional relationship exists between these hyperparameters and the adversarial example generation
time. This analysis seeks to provide actionable insights into optimal hyperparameter configurations and
the selection of the most defensive ensemble model. Finally, only for models demonstrating a "defensive"
tendency of the mentioned hyperparameters, the time required to perform an AT (considering the
training set  +  + ′ i.e. 461350 examples) is evaluated as the values of the hyperparameters vary.
This is useful to better understand the extent of the attacker’s "window of opportunity." It is assumed
that once introduced into the CAN network, the attacker proceeds with the compromise of the IDS
system.
3.4. Hyperparameters’ Impact Qualitative Analysis on CSS-MDO Framework
This subsection is useful for answering RQ3. Considering the critical nature of the scenario that
surrounds this research work, an impact (positive) analysis realted to the MDTFs education about
the best programming practices (improving the resilience of ML models to Black-Box attacks) is an
important milestone to underline the right importance of these. All this is intended to emphasize the
fruitful collaboration between human and artificial agents in future Human-AI MDTFs. The actual
qualitative analysis is based on the Land and Cyber domains. This analysis comes from an high-level
)







(























2 · 105
1</p>
        <p>Attack Time
1,243.65 ·  + 8,876.2 Lin. Reg.
qualitative risk assessment related to this SST. This second analysis is adopted considering multiple
hypothetical negative consequences: the potential to incite a climate of terror through anomalous
vehicle behavior and the cognitive disruption of civilian and military operators, the reputational damage
to the national infrastructure and institutions [29], and the inherently risk-averse perspective guiding
the human evaluator point-of-view [35].</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Result &amp; Discussion</title>
      <sec id="sec-4-1">
        <title>4.1. Empirical Evalution of Attack &amp; AT Time</title>
        <p>Figure 2 presents an estimation of the needed time to generate 92270 adversarial examples
(corresponding to subset ′) as a function of the bagging trees number (i.e. the examinated hyperparameter)
in the RF model. The results demonstrate a linear relationship between the number of estimators
and the generation time, as evidenced by the regression line. For instance, with 81 estimators, the
expected generation time approaches approximately 31 hours. This finding indicates that the selected
hyperparameter influences not only the model’s predictive performance but also its robustness against
ZOO attack. Therefore, the response to RQ1, concerning the RF model, is afirmative.</p>
        <p>Figure 3 presents the same type of the previous estimation but applied to the GB model. In this
case, the examined independent variable is the number of boosting rounds. The results clearly reveal a
directly proportional relationship between the number of boosting rounds and the time required to
generate the adversarial examples, as indicated by the regression line. For instance, approximately 2
hours are needed for 80 estimators. These findings support the same conclusions drawn for the RF
model, thereby afirmatively answering RQ1 in the context of GB.</p>
        <p>Figure 4 depicts the estimation for the XGB model, again using the number of boosting rounds as
the independent variable. Unlike the RF and GB cases, the results do not exhibit a consistent linear
relationship between the number of estimators and the adversarial generation time, as confirmed
by the regression analysis. For example, around 9 hours are required when 80 estimators are used.
Consequently, the answer to RQ1 in the case of XGB is negative.</p>
        <p>Figure 5 illustrates the evolution of AT time as a function of the number of bagging trees in the
RF model. A clear direct proportionality is observed between the two variables, with a maximum of
approximately 150 seconds for 105 trees. Notably, some configurations show local minima in training
time, which can be attributed to tree configurations that limit the depth of the learned patterns. A
similar phenomenon is observable in Figure 6, which depicts the impact of the number of boosting
rounds on training time for the GB model. In this case, the maximum time reaches approximately 670
seconds for 105 boosting rounds. When comparing the estimated attack and AT times for both models,
the RF model demonstrates a more favorable trade-of.</p>
        <p>Furthermore, in both scenarios, this combination of training and attack times efectively reduces
the attacker’s "window of opportunity" (thus strengthening the defence) during an attack on the IVN,
particularly when considering the potential additional attack delay introduced by the execution of
AT. The needed AT time is significantly less than the time provided to the H-AI MDTF to detect the</p>
        <p>Training Time
6.01 ·  + 57.23 Lin. Reg.
intrusion in the CAN network, making this countermeasure almost necessary in online systems. So, the
answer to RQ2 is afirmative.
4.2. Hyperparameters’ Impact on Educating MDTF along CSS-MDO Framework</p>
        <p>Axes for MDOs
Table 1 shows the impact of the previous analysis on the CSS-MDO framework axes for MDOs. This is
the answer to RQ3. Generally, the education of MDTFs (even H-AI) about this analysis has a "Very High"
impact due to the complementary motivations [29]. An "High" impact is observed on the "Prevention"
axis since deterrence does not categorically prevent the execution of the attack. By properly educating
MDTFs on this, it is also possible to gain valuable time in critical defensive operations. Indeed, exploiting
an IDS that is highly robust to AML attacks helps ensure the sustained operational integrity of afected
vehicles. Such robustness can be particularly critical during high-stakes MDOs, where maintaining
functionality for as long as possible can be decisive.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusion &amp; Future Work</title>
      <p>MDOs aim to integrate capabilities across all active domains of warfare to achieve coordinated,
crossdomain efects against targeted systems. Within this context, Smart Cities and in particular CAVs
emerge as critical and highly vulnerable assets, especially to cyberattacks leveraging Black-Box AML
techniques. A perfect targets in such attacks can be ML-based IDSs employed for securing CAN-based
IVNs. These threats are particularly concerning in the broader context of MDTs and, more specifically,
within MSTs. Despite increasing attention, current research on Black-Box AML attacks targeting CAN
frame detection remains sparse and at an early stage of development.</p>
      <p>This paper evaluates (RQ1) the influence of hyperparameters related to DT-based state-of-the-art TT
ensemble models (i.e. RF, GB, XGB), underlying an IDS targeted by the ZOO attack (seen as a SST) in a
pure Black-Box scenario, on adversarial example generation time. Additionally, it understand (RQ2)
whether the combo of the correct setting of the hyperparameters (under exam) and the AT needed time
can slow down or reduce the attacker’s "window of opportunity". Finally, it qualitatively assess (RQ3) the
educational impact on MDTFs of this analysis mapped into the CSS-MDO framework contributing to the
integration of artificial agents within MDTFs leading to H-AI MDTFs. The experimental results indicate
a direct proportional relationship between the number of bagging trees in RF and boosting rounds in
GB with the time required to exploit the ZOO attack, a trend not observed in XGB. Thus, only RF and
GB exhibit hyperparameters that may serve as intrinsic defense mechanisms contributing to D-TAI. RF
is recommended for its superior robustness considering the reduced AT time needed. Generally, the
educational impact on MDTFs of such evidence is rated very high considering the important possibility
of controlling the attack timing. All this, leads to labeling the collaboration between human and AI in
H-AI MDTFs as functional to increase the resilience of social cyber attacks.</p>
      <p>Some future directions of this work are to perform the empirical analysis by considering diferent
values related to the attack hyperparameters (even the default ones); to base the analysis on additional
Black-Box (and White-Box) attacks as well as additional state-of-the-art datasets in the automotive
context. In addition, it is also considered to extend the analysis on datasets and IDSs exploited in
additional systems of national interest (i.e. IoT networks, aircraft, underwater vehicles), taking into
account the possibility to integrate this intelligence to educate on risk management by exploiting a
cyber social wargame also. Accompanying the analysis of AT times, it is useful to understand the attack
times following the application of the countermeasure. Regarding the educational impact analysis, a
clear development is to assess the attack impact also considering a victim TSRS in a MST.</p>
    </sec>
    <sec id="sec-6">
      <title>6. Acknowledgments</title>
      <p>This work was partially supported by the following projects: SERICS - ”Security and Rights In the
CyberSpace - SERICS” (PE00000014) under the MUR National Recovery and Resilience Plan funded
by the European Union - NextGenerationEU; Patto territoriale "Sistema universitario pugliese" – CUP
F61B23000370006; CYBER-PREDICT: Cyber vulnerability ranking prediction by prescription, Avviso
“Reti - Sostegno alla ricerca collaborativa”, - Codice Progetto DUQVKW0; SETH: Security Education
and Training Hybrid-Wargame, Avviso “Reti - Sostegno alla ricerca collaborativa”</p>
    </sec>
    <sec id="sec-7">
      <title>Declaration on Generative AI</title>
      <sec id="sec-7-1">
        <title>The author(s) have not employed any Generative AI tools.</title>
        <p>[13] F. Tommasi, C. Catalano, M. Fornaro, I. Taurino, Mobile session fixation attack in micropayment
systems, IEEE Access 7 (2019) 41576–41583. doi:10.1109/ACCESS.2019.2905219.
[14] S. Rajapaksha, H. Kalutarage, M. O. Al-Kadri, A. Petrovski, G. Madzudzo, M. Cheah, Ai-based
intrusion detection systems for in-vehicle networks: A survey, ACM Comput. Surv. 55 (2023). URL:
https://doi.org/10.1145/3570954. doi:10.1145/3570954.
[15] F. Sommer, J. Dürrwang, R. Kriesten, Survey and classification of automotive security attacks,
Information 10 (2019). URL: https://www.mdpi.com/2078-2489/10/4/148. doi:10.3390/info10040148.
[16] A. Alfardus, D. B. Rawat, Intrusion detection system for can bus in-vehicle network based on
machine learning algorithms, in: 2021 IEEE 12th Annual Ubiquitous Computing, Electronics &amp;
Mobile Communication Conference (UEMCON), 2021, pp. 0944–0949. doi:10.1109/UEMCON53757.
2021.9666745.
[17] and European Union Agency for Cybersecurity, A. Malatras, I. Agrafiotis, M. Adamczyk, Securing
machine learning algorithms, 2021. URL: https://op.europa.eu/publication-detail/-/publication/
c7c844fd-7f1e-11ec-8c40-01aa75ed71a1. doi:doi/10.2824/874249.
[18] F. Aloraini, A. Javed, O. Rana, Adversarial attacks on intrusion detection systems in in-vehicle
networks of connected and autonomous vehicles, Sensors 24 (2024). URL: https://www.mdpi.com/
1424-8220/24/12/3848. doi:10.3390/s24123848.
[19] S. Longari, F. Noseda, M. Carminati, S. Zanero, Evaluating the robustness of automotive intrusion
detection systems against evasion attacks, in: Cyber Security, Cryptology, and Machine Learning:
7th International Symposium, CSCML 2023, Be’er Sheva, Israel, June 29–30, 2023, Proceedings,
Springer-Verlag, 2023, p. 337–352. URL: https://doi.org/10.1007/978-3-031-34671-2_24. doi:10.
1007/978-3-031-34671-2_24.
[20] B. Wu, Z. Zhu, L. Liu, Q. Liu, Z. He, S. Lyu, Attacks in adversarial machine learning: A systematic
survey from the life-cycle perspective, 2024. arXiv:2302.09457.
[21] S. Kotyan, A reading survey on adversarial machine learning: Adversarial attacks and their
understanding, 2023. arXiv:2308.03363.
[22] E. U. A. for Cybersecurity (ENISA), Artificial intelligence and cybersecurity research, 2023.</p>
        <p>URL: https://www.enisa.europa.eu/publications/artificial-intelligence-and-cybersecurity-research.
doi:10.2824/808362.
[23] H.-L. E. G. on AI European Commission, Ethics guidelines for trustworthy ai, 2024. URL: https:
//digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai.
[24] N. I. of Standards, Technolgy, Ai fundamental research - security, 2023. URL: https://www.nist.</p>
        <p>gov/artificial-intelligence/ai-fundamental-research-security.
[25] S. Goellner, M. Tropmann-Frick, B. Brumen, Responsible artificial intelligence: A structured
literature review, 2024. URL: https://arxiv.org/abs/2403.06910. arXiv:2403.06910.
[26] I. Zenden, H. Wang, A. Iacovazzi, A. Vahidi, R. Blom, S. Raza, On the resilience of machine
learning-based ids for automotive networks, in: 2023 IEEE Vehicular Networking Conference
(VNC), IEEE, 2023. doi:10.1109/vnc57357.2023.10136285.
[27] M. L. Han, B. I. Kwak, H. K. Kim, Anomaly intrusion detection method for vehicular networks
based on survival analysis, Vehicular Communications 14 (2018) 52–63. URL: https://www.
sciencedirect.com/science/article/pii/S2214209618301189. doi:https://doi.org/10.1016/j.
vehcom.2018.09.004.
[28] H. M. Song, J. Woo, H. K. Kim, In-vehicle network intrusion detection using deep convolutional
neural network, Vehicular Communications 21 (2020) 100198.
[29] V. S. Barletta, D. Caivano, C. Catalano, S. D. Vescovo, Black-box adversarial ml attacks on ids
and multi-domain impact analysis for threat intelligence in automotive scenarios, in: 2024 IEEE
International Workshop on Technologies for Defense and Security (TechDefense), 2024, pp. 132–
137. doi:10.1109/TechDefense63521.2024.10863442.
[30] H. Lee, S. H. Jeong, H. K. Kim, Otids: A novel intrusion detection system for in-vehicle network
by using remote frame, in: 2017 15th Annual Conference on Privacy, Security and Trust (PST),
2017, pp. 57–5709. doi:10.1109/PST.2017.00017.
[31] B. S. Bari, K. Yelamarthi, S. Ghafoor, Intrusion detection in vehicle controller area network (can)
bus using machine learning: A comparative performance study, Sensors 23 (2023). doi:10.3390/
s23073610.
[32] B. Badjie, J. Cecílio, A. Casimiro, Adversarial attacks and countermeasures on image
classificationbased deep learning models in autonomous driving systems: A systematic review, ACM Computing
Surveys 57 (2024) 1–52.
[33] M. Nicolae, M. Sinn, T. N. Minh, A. Rawat, M. Wistuba, V. Zantedeschi, I. M. Molloy, B. Edwards,</p>
        <p>Adversarial robustness toolbox v0.2.2, CoRR abs/1807.01069 (2018). arXiv:1807.01069.
[34] V. S. Barletta, D. Caivano, C. Catalano, M. De Vincentiis, A. Pal, Machine learning for automotive
security in technology transfer, in: A. Rocha, H. Adeli, G. Dzemyda, F. Moreira, V. Colla (Eds.),
Information Systems and Technologies, Springer Nature Switzerland, Cham, 2024, pp. 341–350.
[35] M. T. Baldassarre, V. S. Barletta, D. Caivano, D. Raguseo, M. Scalera, Teaching cyber security: The
hack-space integrated model, in: Italian Conference on Cybersecurity, volume 2315, 2019. URL:
https://ceur-ws.org/Vol-2315/paper06.pdf.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>F. T.</given-names>
            and,
            <surname>Nato</surname>
          </string-name>
          <article-title>'s approach to multi-domain operations: From the perspective of the economics of alliances</article-title>
          ,
          <source>Defence and Peace Economics</source>
          <volume>35</volume>
          (
          <year>2024</year>
          )
          <fpage>281</fpage>
          -
          <lpage>294</lpage>
          . URL: https://doi.org/10.1080/10242694.
          <year>2023</year>
          .
          <volume>2235502</volume>
          . doi:
          <volume>10</volume>
          .1080/10242694.
          <year>2023</year>
          .
          <volume>2235502</volume>
          . arXiv:https://doi.org/10.1080/10242694.
          <year>2023</year>
          .
          <volume>2235502</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>A.</given-names>
            <surname>Gilli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gilli</surname>
          </string-name>
          ,
          <string-name>
            <surname>G. G.</surname>
          </string-name>
          <article-title>and, Nato, multi-domain operations and the future of the atlantic alliance</article-title>
          ,
          <source>Comparative Strategy</source>
          <volume>44</volume>
          (
          <year>2025</year>
          )
          <fpage>73</fpage>
          -
          <lpage>91</lpage>
          . URL: https: //doi.org/10.1080/01495933.
          <year>2024</year>
          .
          <volume>2445491</volume>
          . doi:
          <volume>10</volume>
          .1080/01495933.
          <year>2024</year>
          .
          <volume>2445491</volume>
          . arXiv:https://doi.org/10.1080/01495933.
          <year>2024</year>
          .
          <volume>2445491</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>F.</given-names>
            <surname>Thrope</surname>
          </string-name>
          , E. Heinz,
          <article-title>Improving cyberspace intelligence preparations for us army multi-domain operations (</article-title>
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>T.</given-names>
            <surname>Wójtowicz</surname>
          </string-name>
          ,
          <string-name>
            <surname>D.</surname>
          </string-name>
          <article-title>Król, Multi-domain battle: new doctrine of the united states armed forces</article-title>
          ,
          <source>Zeszyty Naukowe Akademii Sztuki Wojennej</source>
          (
          <year>2018</year>
          )
          <fpage>64</fpage>
          -
          <lpage>78</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>F.-S.</given-names>
            <surname>Gady</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Stronell</surname>
          </string-name>
          ,
          <article-title>Cyber capabilities and multi-domain operations in future high-intensity warfare in 2030, Cyber Threats</article-title>
          and NATO 2030:
          <article-title>Horizon Scanning and Analysis (</article-title>
          <year>2020</year>
          )
          <fpage>151</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>V. S.</given-names>
            <surname>Barletta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Caivano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Catalano</surname>
          </string-name>
          ,
          <string-name>
            <surname>M. de Gemmis</surname>
          </string-name>
          , D. Impedovo,
          <article-title>Cyber social security education</article-title>
          , in: Extended Reality: International Conference,
          <source>XR Salento</source>
          <year>2024</year>
          , Lecce, Italy, September 4-
          <issue>7</issue>
          ,
          <year>2024</year>
          , Proceedings,
          <string-name>
            <surname>Part</surname>
            <given-names>IV</given-names>
          </string-name>
          , Springer-Verlag, Berlin, Heidelberg,
          <year>2024</year>
          , p.
          <fpage>240</fpage>
          -
          <lpage>248</lpage>
          . URL: https: //doi.org/10.1007/978-3-
          <fpage>031</fpage>
          -71713-0_
          <fpage>16</fpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>031</fpage>
          -71713-0_
          <fpage>16</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>V. S.</given-names>
            <surname>Barletta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Calvano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Sciacovelli</surname>
          </string-name>
          ,
          <article-title>Cyber social security in multi-domain operations</article-title>
          ,
          <source>in: 2024 IEEE International Workshop on Technologies for Defense and Security (TechDefense)</source>
          ,
          <year>2024</year>
          , pp.
          <fpage>41</fpage>
          -
          <lpage>46</lpage>
          . doi:
          <volume>10</volume>
          .1109/TechDefense63521.
          <year>2024</year>
          .
          <volume>10863352</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>S. G.</surname>
          </string-name>
          <article-title>della Difesa Italiana, The italian defence approach to multi-domain operations (approccio della difesa alle operazioni multidominio</article-title>
          ), https://www.difesa.it/assets/allegati/31787/2.1defence_ approach_to_mdos.pdf,
          <year>2022</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>T.</given-names>
            <surname>Campisi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Severino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Al-Rashid</surname>
          </string-name>
          ,
          <string-name>
            <surname>G. Pau,</surname>
          </string-name>
          <article-title>The development of the smart cities in the connected and autonomous vehicles (cavs) era: From mobility patterns to scaling in cities</article-title>
          ,
          <source>Infrastructures</source>
          <volume>6</volume>
          (
          <year>2021</year>
          ). URL: https://www.mdpi.com/2412-3811/6/7/100. doi:
          <volume>10</volume>
          .3390/ infrastructures6070100.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>H.</given-names>
            <surname>Olufowobi</surname>
          </string-name>
          , G. Bloom, Chapter 16 - connected cars:
          <article-title>Automotive cybersecurity and privacy for smart cities</article-title>
          , in: D. B.
          <string-name>
            <surname>Rawat</surname>
            ,
            <given-names>K. Z.</given-names>
          </string-name>
          <string-name>
            <surname>Ghafoor</surname>
          </string-name>
          (Eds.),
          <source>Smart Cities Cybersecurity and Privacy</source>
          , Elsevier,
          <year>2019</year>
          , pp.
          <fpage>227</fpage>
          -
          <lpage>240</lpage>
          . doi:https://doi.org/10.1016/B978-0
          <source>-12-815032-0</source>
          .
          <fpage>00016</fpage>
          -
          <lpage>0</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>D.</given-names>
            <surname>Morris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Madzudzo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Garcia-Perez</surname>
          </string-name>
          ,
          <article-title>Cybersecurity threats in the auto industry: Tensions in the knowledge environment</article-title>
          ,
          <source>Technological Forecasting and Social Change</source>
          <volume>157</volume>
          (
          <year>2020</year>
          )
          <article-title>120102</article-title>
          . URL: https://www.sciencedirect.com/science/article/pii/S0040162520309288. doi:https://doi. org/10.1016/j.techfore.
          <year>2020</year>
          .
          <volume>120102</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>H.</given-names>
            <surname>Qin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Yan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Ji</surname>
          </string-name>
          ,
          <article-title>Application of controller area network (can) bus anomaly detection based on time series prediction</article-title>
          ,
          <source>Vehicular Communications</source>
          <volume>27</volume>
          (
          <year>2021</year>
          )
          <article-title>100291</article-title>
          . URL: https://www. sciencedirect.com/science/article/pii/S2214209620300620. doi:https://doi.org/10.1016/j. vehcom.
          <year>2020</year>
          .
          <volume>100291</volume>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>