=Paper=
{{Paper
|id=Vol-1144/paper8
|storemode=property
|title=Comparison of Fuzzy Membership Functions for Value of Information Determination
|pdfUrl=https://ceur-ws.org/Vol-1144/paper8.pdf
|volume=Vol-1144
|dblpUrl=https://dblp.org/rec/conf/maics/MiaoIHT14
}}
==Comparison of Fuzzy Membership Functions for Value of Information Determination==
https://ceur-ws.org/Vol-1144/paper8.pdf
Comparison of Fuzzy Membership Functions
for Value of Information Determination
Sheng Miao and Robert J. Hammell II
Department of Computer and Information Sciences
Towson University, Towson, MD USA
smiao1@students.towson.edu; rhammell@towson.edu
Timothy Hanratty
Computational Information Science Directorate
US Army Research Laboratory, Aberdeen Proving Ground, MD USA
timothy.p.hanratty.civ@mail.mil
Ziying Tang
Department of Computer and Information Sciences
Towson University, Towson, MD USA
ztang@towson.edu
Abstract Recently, a fuzzy associative memory architecture was
Network-centric military operations are redefining used to develop a system to calculate VoI in complex
information overload as military commanders and staffs are military environments based on the information’s content,
inundated with vast amounts of information. Recent source reliability, latency, and the specific mission context
research has developed a fuzzy-based system to assign a
under consideration (Hanratty, Hammell, and Heilman
Value of Information (VoI) determination for individual
pieces of information. This paper presents an investigation 2011; Hammell, Hanaratty, and Heilman 2012). Military
on the effect of using triangular and trapezoidal fuzzy intelligence analysts were used as subject matter experts to
membership functions within the system. provide the fuzzy association rules from which the system
was constructed, and preliminary results from the system
have been demonstrated and “validated” in principal and
Introduction context (Hanratty et al. 2012; Hanratty et al. 2013). Efforts
Today’s military operations utilize information from a are continuing towards a more formal validation of the
myriad of sources that provide overwhelming amounts of system and to empirically evaluate the effects of the
data. A primary challenge of decision makers at all levels system on intelligence analyst performance (Newcomb and
is to identify the most important information with respect Hammell 2012; Newcomb and Hammell 2013).
to the mission at hand, and often do so within a limited This paper presents an investigation on the effect of
amount of time. The process of assigning a Value of using two different membership functions within the
Information (VoI) determination to a piece of information fuzzy-based system and a comparative analysis of the
has historically been a multi-step, human-intensive differences between them. The paper is organized as
exercise requiring intelligence collectors and analysts to follows: the next section presents background information
make judgments within differing operational situations. on VoI as well as the design of the original fuzzy system.
This is followed by a section that discusses the
Research was sponsored by the Army Research Laboratory and was accomplished under
experimental framework used for this work, and then a
Cooperative Agreement Number W911NF-11-2-0092. The views and conclusions contained in this section describing the experiments and results. The paper
document are those of the authors and should not be interpreted as representing the official
policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. concludes with a section that provides conclusions and
The U.S. Government is authorized to reproduce and distribute reprints for Government purposes
notwithstanding any copyright notation herein. future work.
Value of Information (VoI) VoI System
While it is likely that numerous characteristics could be
In order to turn large amounts of disparate information into
applicable to determining VoI, the aspects of source
useful knowledge, it is vital to have some way to judge the
reliability, information content, timeliness, and mission
importance of individual pieces of information; the Value
context were used as the starting point to develop an
of Information (VoI) metric is used to do this. Ranking the
automated VoI system.
“value” of information is a formidable task involving not
A Fuzzy Associative Memory (FAM) model was chosen
only the sheer amount and diversity of information, but
to construct the prototype fuzzy system. A FAM is a k-
also the idea that the value of a piece of information will
dimensional table where each dimension corresponds to
likely be influenced by the specific mission context to
one of the input universes of the rules. The ith dimension
which it will be applied.
of the table is indexed by the fuzzy sets that compromise
Before going further, it is useful to briefly address what
the decomposition of the ith input domain. Fuzzy if-then
is meant by information “value”, and differentiate it from
rules are represented within the FAM. For the prototype
what could be meant by information “quality”. One
system, three inputs are used to make the VoI decision:
viewpoint is that “quality” refers to the fitness of data with
source reliability, information content, and timeliness (how
respect to the inherent attributes of the data (accuracy,
mission context contributes to the determination will be
precision, timeliness, freshness, resolution, etc.) while
explained shortly).
“value” addresses the utility of the data within a specific
The overall architecture of the fuzzy system is shown in
application context (Bisdikian et al. 2009). The definition
Fig. 1. Instead of using one 3-dimensional FAM, two 2-
used in this paper comes from that provided by (Wilkins,
dimensional FAMs were used. The reasoning behind this
Lee, and Berry 2003). Wilkins considers the practical
decision was presented in detail in (Hammell, Hanratty,
importance of the information to the receiver, suggesting
and Heilman 2012) but essentially it provided a simpler
that information with value supports the receiver’s ability
knowledge elicitation process, decreased the total number
to make informed decisions.
of fuzzy rules, and provided a potential for the output of
the first FAM to be useful on its own.
VoI Determination
As seen in Fig. 1, two inputs feed into the Applicability
U.S. military doctrinal guidance for determining VoI is FAM: source reliability (SR) and information content (IC);
vague at best (US Army 2006; NATO 1997) and does not the output of this FAM is termed the information
address integrating mission context into the decision. The applicability decision. Likewise, two inputs feed into the
guidance provides two tables for judging the “reliability” VoI FAM: one of these (information applicability) is the
and “content” of a piece of data, with each characteristic output of the first FAM; the other input is the information
broken into six categories. Reliability relates to the timeliness rating. The output of the second FAM, and the
information source, and is ranked from A to F (reliable, overall system output, is the VoI metric.
usually reliable, fairly reliable, not usually reliable, The fuzzy rules represented in the FAMs capture the
unreliable, and cannot judge). Information content is relationships between the input and output domains. Since
ranked from 1 to 6 (confirmed, probably true, possibly both FAMs have two inputs and one output, all the fuzzy
true, doubtfully true, improbable, and cannot judge). rules in the system will be of the form "If X is A and Y is B,
Doctrinal guidance does not provide any process for then Z is C", where A and B are fuzzy sets over the input
combining these determinations into a VoI metric. domains and C is a fuzzy set over the output domain. For
Additionally, it is obvious that combining only these two example, an actual rule in the Applicability FAM might be:
assessments of a piece of information would fall far short "if Source Reliability is Usually Reliable and Information
of representing all the critical aspects for a useful VoI Content is Probably True, then Information Applicability is
determination.
Two other potential data characteristics include mission
context and timeliness. Timeliness relates to how long ago
the piece of information was collected, while mission
context is set by the operational tempo of the military
operation underway. The operational tempo relates to the
decision cycle for the mission; that is, the time that can or
will be used to plan, prepare, and execute the mission. Fast
tempo operations may have a decision cycle measured in
minutes to hours, while slower tempo operations may be
measured in months or longer. Figure 1. VoI System Architecture
Highly Applicable." Knowledge elicitation from military midpoints has been referred to as a TPE system (Sudkamp
intelligence Subject Matter Experts (SMEs) was used to and Hammell 1994). Fig. 3(a) shows the TPE
construct the fuzzy rules (Hanratty et al. 2012). decomposition of a domain ranging from 1 to 5; Fig. 3(b),
Within the Applicability FAM, the two input domains 3(c), and 3(d) illustrate isosceles triangular decompositions
(source reliability and information content) are divided into of the same range that do not adhere to the restriction of
five fuzzy sets following the guidance provided in (US having bases of the same width. Triangular
Army 2006). The omission of the “cannot judge” category decompositions, with and without bases of the same width,
from both of the input domains is explained in (Hammell, are included in our experimental framework.
Hanratty, and Heilman 2012). The “information In addition to using triangular membership functions,
applicability” output domain was decomposed into nine trapezoidal decompositions are another approach we would
fuzzy sets (ranging from not applicable to extremely like to explore. Similar to the triangles, isosceles
applicable) while the VoI output domain utilized eleven trapezoids both with and without bases of the same width
fuzzy sets (ranging from not valuable to extremely are considered. Fig. 5(a) shows the decomposition of a
valuable). domain ranging from 1 to 5 using isosceles trapezoids with
Up to this point, the contribution of mission context has bases of the same width; Fig. 5(b), 5(c), and 5(d) depict
not been apparent. To account for differing mission similar decompositions using isosceles trapezoids without
tempos, three separate VoI FAMs were derived to represent the requirement for equally sized bases.
three different tempos. Missions were characterized as While we mentioned several forms of membership
either 'tactical' (high-tempo), 'operational' (moderate- functions from which to choose, we selected trapezoidal
tempo), or 'strategic' (slow-tempo). The system selects the and triangular fuzzy sets for two primary reasons. First,
correct VoI FAM based on the indicated mission context, the membership degree calculations for both are linear,
thereby utilizing the appropriate fuzzy rule base to produce thereby facilitating high computational efficiency. This is
the VoI determination. significant since the purpose of the fuzzy VoI system is to
More detailed descriptions of the FAMs, the fuzzy rules help intelligence specialists find the most important
bases, the domain decompositions, and other information within a potentially large amount of data while
implementation aspects of the prototype system can be frequently adhering to restrictive time constraints.
found in (Hanratty et al. 2013). The series of surveys and The second reason is that these two forms can help in
interviews with SMEs that were used to integrate cognitive the data acquisition process. As implied earlier, significant
requirements, collect functional requirements, and elicit the knowledge elicitation efforts using intelligence specialists
fuzzy rules is presented in (Hanratty et al. 2012). as Subject Matter Experts (SMEs) were required to
The VoI system has been demonstrated to the SMEs and construct the initial fuzzy rules; likewise, any membership
its output has met SME expectations (Newcomb and function optimization will be determined by the SMEs.
Hammell 2012). Note that there is no current system The triangular and trapezoidal functions are more visually
against which the results can be compared. As such, the understandable and provide an environment more
system has not been tested comprehensively due to the conductive to human-in-the-loop knowledge acquisition.
human-centric, context-based nature of the problem and Based on these two reasons, trapezoidal and triangular
usage of the system. Formal validation of the VoI system membership functions are often used (Zimmerman 1996).
requires a comprehensive experiment which is currently To facilitate the analysis of various domain
under development separately. decompositions using the triangular and trapezoidal fuzzy
sets, we compare them from different aspects and display
the results visually. Three categories of experiments are
Experimental Framework presented in the next section. First, results from using
A major factor in the design of any fuzzy system relates to “standard” triangular and trapezoidal decompositions are
the decomposition of the input and output domains into compared, where “standard” means the use of isosceles
fuzzy sets. The “shape” of the fuzzy sets defines the shapes with bases of the same width (Fig. 3(a) and 5(a)).
membership functions for the system. While there are Next, “standard” triangular fuzzy membership functions
numerous shapes for fuzzy sets (triangular, trapezoidal, are compared with “customized” triangular fuzzy
Gaussian, bell, and the like), triangular membership membership functions, where “customized” means that the
functions were used in the initial VoI system. To further restriction for bases of the same width is removed (Fig.
facilitate computational efficiency, it was also required that 3(b), 3(c), and 3(d)). Finally, “standard” trapezoidal fuzzy
the triangles were isosceles with bases of the same width; sets are compared with “customized” trapezoidal fuzzy
this triangular decomposition with evenly spaced decompositions (Fig. 5(b), 5(c), and 5(d)).
Results
This section provides the experimental results from
comparing triangular and trapezoidal fuzzy set membership
functions. Three subsections will be used to present the
results. First, a comparison of the “standard” triangular
and trapezoidal sets will be shown. Next, several
(a)
“customized” triangular decompositions will be compared
with the initial TPE fuzzy sets. Finally, several
“customized” trapezoidal decompositions will be
compared with the standard trapezoidal fuzzy sets.
Standard Triangular vs Standard Trapezoidal
Fig. 2 compares the FAM outputs for the standard (TPE)
triangular fuzzy sets (a, c) and the standard trapezoidal
fuzzy sets (b, d). Fig. 2a and 2b show the applicability
FAM output for the two models; that is, the relationship
between source reliability (x-axis) and information content
(y-axis). The values of two inputs are from one to five,
(b)
with the smaller value of one being “better” (better
reliability/content) and five meaning “worse” (less
reliability/content). The applicability output values vary
from one to nine where the larger values represent better
applicability; the colors vary from blue to red where the
higher value is in blue (high applicability meaning reliable,
probable information) and the lower value is in red
(unreliable, improbable information).
Fig. 2c and 2d show the value of information (VoI)
FAM output based on the two inputs of applicability and
timeliness. Applicability is as mentioned above.
Timeliness reflects the temporal age of the information,
with values ranging from one to three: one means “recent” (c)
while three means “old”. As with the applicability graphs,
the VoI values are represented in the color shades within
the graph. The numerical values for VoI range from zero
to ten (blue meaning ten; red meaning zero) and the higher
values represent higher VoI (more valuable information).
The mission context is assumed to be “tactical”.
Comparing results for the models, the output landscape
of the triangular fuzzy models (a, c) looks smoother while
the trapezoidal fuzzy models (b, d) produce some fairly
well defined rectangles. To see why, note that when an
input (in these standard models) has a membership value
equal to 1 in a fuzzy set, the input belongs only to that (d)
fuzzy set (see Fig. 3(a) and 5(a)). For example, in the
triangular fuzzy model, only the integer input values (1, 2,
etc.) belong to just one fuzzy set; that is, there is only one
input value in each triangular fuzzy set that will have a
membership equal to one. For the trapezoidal fuzzy sets,
however, there are several values in each set that have a
membership equal to one and, thus, belong to only that
fuzzy set. This creates areas within the color graphs that Figure 2. Applicability and VoI: Standard
have the same calculated output values for applicability or Triangular and Trapezoidal Fuzzy Sets
VoI, thereby producing the more pronounced rectangles.
Note these rectangles are seen within the color graph and at
the four corners of the graph.
Standard Triangular vs Customized Triangular
In the experiments shown below, due to space limitations,
comparisons between the different models will be (a)
illustrated using the results of the applicability FAM only.
Fig. 3 and 4 are used to compare the applicability values
for the standard and customized triangular fuzzy models;
Fig. 3 shows the fuzzy set shapes for all domains in the
standard model (3(a)) and the customized models (3(b),
3(c), 3(d)), and Fig. 4 provides the associated color graphs.
Case 1
In the standard model, both domains (source reliability and
information content) are decomposed following the TPE
restrictions as illustrated in Fig. 3(a). The resulting color
graph for the standard (TPE) model is shown in Fig. 4(a).
In the customized model, both inputs shrink the third fuzzy (b)
set from the standard 2 to 4 width (on the x-axis) to the
customized width of 2.5 to 3.5 as shown in Fig. 3(b); the
corresponding color graph is Fig. 4(b).
Compared with the applicability distribution of the
standard model, the color graph for the customized model
is much less smooth. It is also clear that two very similar
color belts cross in the middle of the graph (as outlined);
the edges are 2.5 and 3.5 (both vertically and horizontally).
The middle of the graph for the customized model has
similar color values; however, the outer edge of the color
(c)
(a)
(b)
(d)
(c)
(d)
Figure 3. Standard and Customized Figure 4. Applicability: Standard and
Triangular Fuzzy Membership Functions Customized Triangular Fuzzy Sets
belt has smaller changes than that in standard model and extreme high or low values corresponding to dark blue and
the inner edge has larger changes which cause the visible dark red.
boundaries. Also, four rectangles in a solid color around
the center are observable (and outlined) in Fig. 4(b). The Standard Trapezoidal vs Customized Trapezoidal
reason for the observed differences is that in the Fig. 5 and 6 are used to compare the applicability values
customized model, inputs between 2 to 2.5 and 3.5 to 4 for the standard and customized trapezoidal fuzzy models;
only belong to one fuzzy set. This leads to smooth Fig. 5 shows the fuzzy set shapes for all domains in the
visualization and solid squares of the same color. On the standard model (5(a)) and the customized models (5(b),
other hand, input values between 2.5 to 3.5 belong to two 5(c), 5(d)), and Fig. 6 provides the associated color graphs.
fuzzy sets. The third fuzzy membership degree is changed
Case 1
faster (narrower triangle; slope is larger) than that in the
In the standard model, both domains (SR and IC) are
standard model. As a result, this enhances the
decomposed as illustrated in Fig. 5(a). The resulting color
representation of boundaries.
graph for the standard model is shown in Fig. 6(a). In the
Case 2 customized case, the middle fuzzy set is still an isosceles
In this case, the third fuzzy set is assigned a wider range, trapezoid but the width is smaller than the other sets, as
encompassing the entire input domain, as shown in Fig. depicted in Fig. 5(b). The left and right bottom points are
3(c). Again, in the standard model both domains are 2.5 and 3.5; note the upper base is the same 2.75 to 3.25 as
decomposed following the TPE restrictions (3(a)). in the standard trapezoidal model. The corresponding color
The values in the customized color graph in Fig. 4(c) graph is shown in Fig. 6(b).
look smoother in the center with sharp variations occurring As in Case 1 of the triangular fuzzy model, the color
in red and blue at the corners; the red and blue corner graph for this customized trapezoidal model illustrates two
values have a much smaller area than in the standard fuzzy similar color belts crossing in the middle of Fig. 6(b).
triangular model. The reason is that the third fuzzy set in Because the middle fuzzy set is narrower and more inputs
the customized model affects all the fuzzy membership belong to only the second or fourth fuzzy set, the edges
degree calculations since it spans the entire input domain. corresponding to the middle SR and IC input values are
For high value inputs, the third fuzzy set causes lower smaller and more pronounced than those of the standard
FAM values to join the calculation, resulting in a lower trapezoidal model in Fig. 6a. Also, the neighboring
output value than that in the standard model. rectangles of solid color are larger than that in the standard
The reverse occurs for the low value inputs; the middle model. Note that the areas associated with the four corners
fuzzy set contributes higher FAM values to the are similar in both the standard and customized color
applicability result. Thus, the red and blue boundaries graphs.
contract to the corners of the customized graph as
Case 2
compared to the standard triangular fuzzy model results.
Considering the opposite setup with the middle fuzzy set as
Case 3 shown in Fig. 5(c), this case sets the middle fuzzy set to
Considering that some users maybe prefer a wide range in cover a wider input range, from 1 to 5. However, the upper
the middle fuzzy sets (most IC and SR inputs would fall in base is still fixed from 2.75 to 3.25 and all other sets are
the “middle”) but smaller ranges at the edges (only the same as in the standard trapezoidal model.
extreme IC and SR inputs are considered “best” or Fig. 6(c) illustrates the associated color graph. The result
“worst”), Fig. 3(d) shows a fuzzy set pattern to provide of the customized trapezoidal model reveals a similar trend
such a system. In this model, the two ends are made as that of the corresponding triangular model; more areas
narrower (range from 1 to 1.5 and 4.5 to 5), which means in the middle values can be observed and sharp variation
only a small range of inputs belong to these sets. The happens in the corners as compared to the standard
middle set has a wide input scope, which is from 1.5 to 4.5. trapezoidal model in Fig. 6(a). Nevertheless, the graph still
Meanwhile, the input ranges of other two fuzzy sets are presents the basic features of the trapezoidal fuzzy model -
reduced appropriately. some rectangles in similar colors exist in the color graph
Fig. 4(d) shows the applicability distribution based on which are not as obvious in the triangular fuzzy model.
this customized model which is much more “blocky” than
Case 3
that of the standard TPE model shown in Fig. 4(a).
Based on the same scenario as with Case 3 for the
Because the middle fuzzy set is extended and covers
triangular fuzzy sets, this customized trapezoidal model
numerous inputs, the resulting output has a number of
sets up a wide middle fuzzy set and narrower side sets as
areas in the middle values. The contraction of the other
shown in Fig. 5(d). In this setup, only very high or low
fuzzy sets causes much of the graph area to show up in the
value inputs are regarded as extreme conditions.
orange and cyan colors, while only the corners have
(a)
(a)
(b)
(c)
(b)
(d)
Figure 5. Standard and Customized
Trapezoidal Fuzzy Membership Functions
Fig. 6(d) shows the applicability distribution based on
the customized model. Compared with the result of the
standard trapezoidal fuzzy model in Fig. 6(a), similar
results occur as with Case 3 for the triangular model. The
areas in the middle values are larger than those in the
standard trapezoidal model (Fig. 6(a)) and only small (c)
sections of dark red and blue in the corners represent the
extreme high and low applicability values. Moreover, the
result of this customized model retains the features of a
trapezoidal fuzzy model which produces larger areas in the
graph of solid colors. However, one difference is that
color boundaries between the rectangles are much
narrower in the customized model. This makes the
boundaries more pronounced and provides well-defined
solid color rectangles.
Conclusion and Future Work (d)
This paper presents two approaches for codifying the
contextual underpinnings (framework) and cognitive
interpretation for capturing VoI utilizing source reliability,
information content and latency based on triangular and
trapezoidal fuzzy membership functions. While both
approaches for capturing VoI are intuitively simple to
comprehend and computationally easy to calculate,
differences are observed.
The first major difference observed is that when using
Figure 6. Applicability: Standard and
the triangular approach the results of the color graphs were
Customized Trapezoidal Fuzzy Sets
strikingly different than those of the trapezoidal approach. performance of data fusion systems. In Handbook of Multisensor
Using the triangular fuzzy model produced graphs that Data Fusion, D. Hall and J. Llinas (Eds.), CRC Press, Boca
Raton, FL, 2001.
were infinitely smoother in their transition between
Hammell II, R.J.; Hanratty, T.; and Heilman, E. 2012. Capturing
calculated values. The trapezoid models, on the other
the Value of Information in Complex Military Environments: A
hand, produced plots that appeared “blockier”, lending to Fuzzy-based Approach. In Proceedings of the 2012 IEEE
larger areas of continual homogeneous values. International Conference on Fuzzy Systems (FUZZ-IEEE 2012)
A second major difference observed is the increased as part of the 2012 World Conference on Computational
flexibility for representing membership functions afforded Intelligence (WCCI 2012), 142-148. Brisbane, Australia.
by the trapezoidal representation. Using the trapezoids Hanratty, T.; Hammell II, R.J.; Bodt, B.; Heilman, E.; and
allowed an ‘interval of values’ that maximized the Dumer, J. 2013. Enhancing Battlefield Situational Awareness
Through Fuzzy-based Value of Information. In Proceedings of
individual membership functions (top of the trapezoids) as the 46th Annual Hawaii International Conference on Systems
compared to the triangle representation that permitted only Sciences (HICSS 2013), 1402-1411. Maui, Hawaii.
one. The introduction of the trapezoid dramatically Hanratty, T; Heilman, E; Dumer, J.; and Hammell II, R.J. 2012.
increases the ability of the user to capture representations Knowledge Elicitation to Prototype the Value of Information. In
over the more simplistic triangular shape. Proceedings of the 23rd Midwest Artificial Intelligence and
With this understanding, one might mistakenly chose Cognitive Sciences Conference (MAICS 2012), 173-179.
Cincinnati, OH.
one approach over the other, thinking on one hand the
trapezoidal approach is inferior because of the “blocky Hanratty, T.; Hammell II, R.J.; and Heilman, E. 2011. A
Fuzzy-Based Approach to the Value of Information in Complex
effect” or on the other hand superior because of the added Military Environments. In Proceedings of the Fifth International
flexibility. The fact is both approaches have their own Conference on Scalable Uncertainty Management (SUM 2011).
strengths and weaknesses. For example, depending on the Dayton, OH.
context of the situation, the blocky effect might provide a North Atlantic Treaty Organization (NATO). 1997. Standard
better representation of the military function being Agreement (Edition 8) Annex.
modeled. An example of this effect can be seen when Newcomb, A.; and Hammell II, R.J. 2013. Validating a Fuzzy-
comparing a logistic battle function against that of a based Mechanism for Improved Decision Support. In
tactical combat battle function. For the logistics operations Proceedings of the 14th IEEE/ACIS International Conference on
Software Engineering, Artificial Intelligence, Networking and
the fidelity of the information required for moving Parallel/Distributed Computing (SNPD 2013), 143-148.
equipment can be significantly less critical than that Honolulu, Hawaii.
required when conducting a combat cordon and search Newcomb, A.; and Hammell II, R.J. 2012. Examining the
operation; as such, the logistical representation of VoI may Effects of the Value of Information on Intelligence Analyst
very well be represented with larger areas of homogeneous Performance. In Proceedings of the 5th Annual Conference on
values (blocking effect). Information Systems Applied Research (CONISAR 2012). New
Orleans, LA, http://proc.conisar.org/2012/pdf/2227.pdf.
Ultimately the goal of this research is targeted to
improve the higher-level information fusion process (Hall, Sudkamp, T. and Hammell II, R.J. 1994. Interpolation,
Completion, and Learning Fuzzy Rules. IEEE Transactions on
Hall, and Tate 2001) - effectively interleaving the human Systems, Man, and Cybernetics, 24-2:332-342.
computer interaction (HCI) with the lower-level fusion
US Army 2006. US Army Field Manual (FM) 2-22.3, Human
process. To accomplish this goal further refinement of the Intelligence Collector Operations.
VoI approach is necessary and includes the following Wilkens, D.E., Lee, T.J., and Berry, P. 2003. Interactive
activities: 1) vetting the VoI approaches with subject execution monitoring of agent teams. Journal of Artificial
matter experts to provide direct feedback on applicability, Intelligence Research (JAIR), 18:217-261.
2) exercising the VoI construct within a task network Zimmermann, H.J. 1996. Fuzzy Set Theory and its Applications
model to assess the potential impact, and 3) conducting (Third edition). Boston/Dordrecht/London: Kluwer Academic
human-in-the-loop experiments to measure how Publishers (1996) ISBN 0-7923-9624-3.
cognitively aligned interfaces improve task performance.
References
Bisdikian, C., Kaplan, L. M., Srivastava, M. B., Thornley, D. J.,
Verma, D., & Young, R. I. 2009. Building principles for a
quality of information specification for sensor information. In 12th
International Conference on Information Fusion. Seattle, WA.
Hall, M. J., Hall, S.A., and Tate, T. 2001. Removing the HCI
bottleneck: How the human-computer interface (HCI) affects the