The potential applications of Wireless Sensor Networks (WSNs) span a very wide range. One of the application domains is in healthcare industry. The diverse WSN application requirements signify the need of application specific methodology for system design and performance evaluation. Moreover the performance of typical wireless network is stochastic in nature, Probability is an essential instrument needed to assess the performance characteristics. Before WSNs widespread involvement in life or death critical applications, an urgent need is a generic systematic evaluation methodology for decision makers to evaluate performance among alternative solutions taking into account the cohesion characteristic. This paper offers a quantative decision making procedure to incorporate performance deviation as a target performance metric. Decision making is guided by goals and objectives for the particular application specified by application domain experts.
We have witnessed in recent years the emergence of WSNs in healthcare. These
applications aim to improve and expand the quality of care across wide variety settings
and for different segments in the healthcare system. They range from real-time
patient monitoring in hospitals, emergency care in large disasters through automatic
electronic triage, improving the life quality of the elderly through smart environments,
to large-scale field studies of human behavior and chronic diseases [
Essentially, multiple-criteria evaluation is a well studied realm. There exist plenty of
methodologies in multi criteria decision making domain [
Source of Uncertainty of WSN performance Several factors contribute to the fact that wireless sensor networks often do not work as expected when deployed in a real-world setting. Firstly, there is possibility of wrong expectation from system designer side: analytical model does not fit into the problem in hands. That is often a problem for inexperienced designers. For all simulation or other experiments methods, first step is to eliminate the possibility of this kind of profound design problem in preliminary stage. Secondly, there is possible wrong expectation from simulation results: Simulation modeling can not faithfully reflect the System Under Test (SUT).
Except the designer’s preliminary problem of analytical model mismatching design
target, we can further identify fault point of performance disagreement between
expectation and real world implementation into components of WSNs hierarchy [
All these factors contribute to the uncertainty of the sensor network behavior and
function. These elements increase the probability of network functionality deviation
from its normal operation and affects its' collected data accuracy.
In order to effectively develop parameters, we in [
P = F (spatial, temporal, traffic load pattern, SUT) (1)
The parameters in equation (1) as show in Figure 1 represent system level
performance dynamics involving all four factors. As the input elements display statistical
behaviors, output performance definitely will have a statistical distribution pattern
with certain norm and deviations for specific scenarios. Since wireless performance is
inherently statistical in nature, accurate performance testing must account for these
random components [
WSNs sense the targeted phenomenon, collect data and make decision to store, aggregate or send the data according to distributed local algorithm. The modulated electromagnetic waves propagate in free-space; interact with the environment through physical phenomenon such as reflection, refraction, fast fading, slow fading, attenuation and human activities. Even with the best wireless protocols, the best chipsets, the best RF design, the best software, wireless performance is going to vary. Wireless performance is inherently statistical in nature, and accurate performance evaluation must reflect this nature.
We observed that currently most ad hoc evaluations in wireless network field,
especially in WSNs research, no matter in the form of test bed experiment or simulation,
only focus on mean value of performance metrics, and do not pay much attention on
performance deviation. For some applications, average performance is sufficient for
data gathering and collective data analysis. However, average ‘throughput’, ‘lifetime’,
‘reliability’ or ‘delay response’ are not sufficient enough to predict performance on
certain application scenarios. Any dip in performance, no matter how short, can result
in dropped packets that cause visual artifacts or pixilation of the image of wireless
video monitoring application. In extreme cases like in healthcare emergency
monitoring application, any dropped packet may cause life or death difference. Consequently,
the viewer/user experience is completely dependent on the wireless system
‟worstcase‟ performance [
Figure 2 represents Probability Density Function‟ (PDF) of a sample performance metric with four different Systems Under Test (SUT), each with different average and standard deviation (variability) parameters. The graph illustrates „normal‟ probability distribution revealing statistical characteristics of metric X, representing at least approximately, any variable that tends to cluster around the mean as norm , shown as, but not necessary, the familiar „bell curve‟. It shows the relative probabilities of getting different values. It answers the question: What is the chance I will see a certain result? What is the mean value or norm of the respective SUT at this specific performance metric? How cohesive and stable is the performance for each SUT?
Examining the random process represented by the red curve in figure 3, we would expect outcomes with a value around „0‟ to be twice as prevalent as outcomes of around 1.25 (40% versus 20%). However, in some cases, we are more interested in a threshold performance value as benchmark value than individual probability point, what is the probability of having performance being less or greater than a threshold value? A transformed PDF ,Cumulative Distribution Function (CDF) (Figure 3.) helps answer this question.
When you use probability to express your uncertainty, the deterministic side has a
probability of one (or zero), while the other end has a flat (all equally probable)
probability. For example, if you are certain of the occurrence (or non-occurrence) of an
event, you use the probability of one (or zero). If you are uncertain, and would use the
expression “I really don't know,” the event may or may not occur with a probability of
50 percent. This is the Bayesian notion that probability assessment is always
subjective. That is, the probability always depends upon how much the decision maker
knows. Due to statistics science the quality of information and variation are inversely
related. That is, larger variation in data implies lower quality data (i.e., information)
[
To characterize wireless performance, CDF graphs can provide immensely useful
information for system designers to compare performance of alternative solutions.
Ideally, we prefer better mean value and smaller deviation, but if the ideal choice is
unavailable, we have different optional solution to choose. All the choices should be
put in application specific context to choose the right protocol for right application.
The principle is:
─ Drastic different mean value, if mean value represent positive metrics, like
throughput or lifetime, bigger is better, ideally we prefer SUT with bigger mean
value and less deviation.
─ Same mean value, different deviation: long tail means performance not stable, we
prefer smaller deviation.
─ Slightly different mean value, but one with long tail, we prefer stable over slightly
improved peak performance.
─ But if the optimal solution is not available, we have choice over performance
stability and higher norm performance according to different application scenario.
(option 1) Higher performance potential but less predictable performance
(option 2) Less performance potential but higher stable performance
Reference [
WSNs energy-oriented research originates from conflict between application performance requirements and limited energy supply by battery. In foreseeable future, it will remain as a bottleneck for its widespread development unless a breakthrough at relevant material science field occurs. However, we can not overemphasize energy conservation while ignoring application specific requirements. To what extent we should emphasize the importance of energy aspect comparing other QoS objectives depends on application scenarios. Tradeoffs have to be made on per application basis.
Typically WSN lifetime (energy efficiency), response time (delay), reliability, and throughput are among the main concerns in the design and evaluation process. Under the constraint of wireless sensor node size, the energy budget and computing resources are unfeasible to afford any luxury algorithms. Under such constraint, there does not exist a perfect optimal solution satisfying all performance metrics in the problem (NP hard problem), rather, the question we sought is how to tradeoff multiple criteria explicitly leading to more informed and better decisions. The methodology should be general enough to contain different application scenario according to decision maker’s preferences.
There have been few works on application-driven protocol stack evaluation for
WSNs. Our evaluation methodology, similar to analytic hierarchy process (AHP) [
The end-user configures the relative importance of the individual design metrics in a function that maps the individual metric values to a single SPI value. Firstly we define the default overall objective function as a weighted sum of the individual design metric normalized values as other AHP methodologies normally do: separation of defining design metric, and weighing those functions’ importance in an overall objective function.
SPI norm = a * m (L) + b * m (R) + c * m (T); (2)
Here (a, b, c) represent corresponding weight of performance metrics such as lifetime (L), reliability(R) and timeliness (T). (m (L), m (R), m (T)) represents the mean value of multiple measurement of corresponding metric. A key feature of our approach is that, we introduce the statistical analysis of the resulting experiment data, not only using measurement mean value as supposed normalized value (which is not realistic representation of the dynamic truth of the wireless network nature), we introduce deviation of performance measurement PDF as a critical secondary performance metric to emphasis the importance of performance stability and cohesion. Even a higher mean performance metrics, if the performance spreads over a wide spectrum of measurement, not cohesive to so called norm performance (mean) value, it will be problematic for certain application scenarios which require consistent performance , such as health monitoring application and multimedia application. We introduce stability performance index, as:
SPI stability = a ' * (1/δ 2 (L)) + b ' *(1/ δ 2 (R)) + c ' * (1/δ 2 (T)) (3)
Here (a’, b’, c’) indicates the relative importance of the metrics cohesive characteristic of metrics (L, R, T) represented by deviation (1/δ 2 ). So overall we have: n SPI= SPI norm + SPI stability = (Wi * Metrici (mean) Wi' * (1/ i2 )) (4) i1
Here ‘n’ represent the number of metrics considered, Wi = (a, b, c…) represents respectively the user specified relative importance of the performance metrics. And W '
i = ( a’, b’, c’ …) indicates the relative importance of the metrics cohesive characteristic. The relative importance of each design metric as weight is assigned by considered application specific scenarios, how important in your application is certain metric (network lifetime, reliability, throughput, delay, etc ) respectively? How important is cohesive characteristic of performance to your application? Which metric is utmost important for you? 4
System evaluation process starts with the end users as application experts who know very well what kind of performance they needs; they specify the most concerned QoS performance metrics, and the weights of each metric.
The WSNs designers decide the initial parameters according to the literature studies and previous experiment experience. Then for each performance metric start the iterative experiment process as such:
Repeat l experiment measurements, record each experiment the state of each parameter xi as f ji f (1<i<n, 1< j<l) and corresponding performance measurement j (1<j<l). Then use linear aggregation and P-value to decide significant parameters to metrici . 2. Design space reduces from n parameters to m parameters for metrici .
Tune the parameters based on the reduced parameters set, Repeat l experiment measurements, record the state of each parameter xi as f ji (1<i<n, 1< j<l) and corresponding performance measurement j (1<j<l). Then use the Choquet nonlinear aggregation model as described in later chapter to decide the most effective parameters set including interaction effect of individual parameter. 4. Now we have finally approach the effective parameters set. Tune the effective parameters set, repeat measurement and get the performance curve, get the (metrici( 2)) and metrici (mean) for metrici . 7. For competing solution for pair wise comparison, repeat the above process and get
SPI value and compare:
Notice that we can setup threshold value as prerequisite filter for minimum requirement, any time if Metrici mean or deviation is less than the threshold value, the candidate solution is not qualified for further comparison due to unsatisfactory for minimum user specification.
If { every metrici (mean)> threshold(mean) And metrici( 2)< threshold( 2) }
Single Performance Index: SPI= n (Wi *Metrici(mean) Wi' *(1/i2)) i1 )
SPI=0 (not satisfy minimum requirement,
c ' *(1/ δ 2 (R)) …….
and corresponding stability indicator:
Prerequisite Filter: er 1. Repeat above procedure for any oth
SUT alternatives, 2. Pairwise Comparison: If SPI system1 > SPI system2 3. Conclusion System1 perform better than system 2 Under user specified requirement.
Initial parameters Development Parameter significance analysis for metrici
Design space reduction for metrici
Parameters analysis for metrici
interaction
Repeat experiment and measurement in effective design space for metrici
PDF /CDF for metrici :
acquisition
metrici ( 2 ) ;
metrici (mean)
In this paper, we introduce a procedure to evaluate WSNs application performance
according to application scenarios, in our approach, uncertainty attributes contribute
to final performance index. Our future research direction is how to capitalize data
mining technique to further dig deep experiment data and distil invaluable collective
information from randomness. “Large-scale random phenomena in their collective
action create strict, nonrandom regularity” [