=Paper=
{{Paper
|id=Vol-2457/paper2
|storemode=property
|title=A Self-Powered Wireless Sensor Network
|pdfUrl=https://ceur-ws.org/Vol-2457/2.pdf
|volume=Vol-2457
|authors=Roger N. Alegret,Raul Aragones,Carles Ferrer
|dblpUrl=https://dblp.org/rec/conf/cpsschool/AlegretA019
}}
==A Self-Powered Wireless Sensor Network==
https://ceur-ws.org/Vol-2457/2.pdf
A Self-Powered Wireless Sensor Network
Roger N. Alegret1,2[0000−0002−0906−8138] , Raul Aragones1,2[0000−0002−3960−6312] ,
and Carles Ferrer1[0000−0002−1475−8790]
1
Autonomous University of Barcelona, 08193 Bellaterra, Barcelona, Spain
roger.nicolas@e-campus.uab.cat, {raul, carles}@uab.cat
http://www.uab.cat
2
Alternative Energy Innovations SL, 08221 Terrassa, Barcelona, Spain
{roger,raul}@aeinnova.com
http://www.aeinnova.com
Abstract. SARSA is one of the most advanced algorithms in reinforce-
ment learning and artificial intelligence, this algorithm lets the actor to
learn about its environment and to act accordingly. Batteries are the
main stopper for adopting wireless communications in the industry. In-
dustrial processes are a source of waste energy. This paper explores the
different components of the IIoT devices, which changes are needed in
order to be powered by the energy on its surroundings, and finally, how
those points above can be tackled using artificial intelligence.
Keywords: Energy Harvesting · Waste Heat · WSN · IIoT · Routing
Protocols · Mesh Networks · Reinforcement Learning.
1 Introduction
Nowadays, Wireless Sensor Networks (WSN) are a recurrent topic of investi-
gation, technology is towards to create smaller devices with greater computer
capacity, while trying to extend the life of the device. Another important issue
that is evolving a lot in recent times is the harvesting of energy from where it was
not believed possible to scavenge it. This evolution has been from an experiment
in a laboratory [9] unto it has become a technology to keep in mind while the
process of building new devices is being done [10]. If these two topics are mixed,
we find ourselves in a new paradigm called Energy Harvesting Wireless Sensor
Networks (otherwise called EH-WSN) [5]. In this new paradigm, we can see the
research done so far in WSN is relatively obsolete, since the main objective that
was being sought so far was to extend the battery life by following different
configurations and procedures: from reducing the duty cycle [8] of the device to
routing protocols of packages such as LEACH and all its variants [14].But if these
two concepts are combined, we can see the WSN has virtually unlimited power,
since there is a generating source that can charge the battery in a relatively less
time than the node will reach its depletion which, that means, it could even be
possible to remove the battery of the node [9]. This opens up a new range of
possibilities for operating the node that has not yet been contemplated by the
date, and the main objective shifts from extending the battery life to maximize
the network usage.
This harvested energy can come from many sources; technologies such as so-
lar panels, piezoelectric components, thermoelectric generators, radio-frequency
transceivers and wind turbines, to cite some of them, are able to harvest energy
from the environment and power devices attached to them [4]. Thereafter, the
energy harvested is stored into an energy buffer, normally a battery or a super-
capacitator, and it is consumed by the node when it needs to capture, process,
send or manage data to the network. Thus, the system is able to harvest en-
ergy continuously and burst it when the node requires more energy than the
harvested on that concrete moment of time. This field of research, searching a
way to balance the energy harvested during a period of time and the energy
consumed on the same period is known as Energy Neutral Operation Nodes.
The main contribution of this paper is a novel architecture for EH-WSN
adopting reinforcement learning algorithms to control the node consumption as
well as selecting the channels to transmit information.
This paper presents a brief state of the art of energy harvesting wireless
sensor networks and which technologies are used right now. Specially, in section
2 it goes deeper on the algorithms for prediction-free energy managers. In section
3, it dissects the different parts of an energy harvesting wireless sensor node and
which responsibility takes each part in order to understand how they are related
with the other parts. Following this point, in section 4, a discussion of which
issues and challenges needs to be faced in order to evolve in the field of energy
harvesting wireless sensor networks. Finally, in section 5 a network architecture
is proposed in order fill the gap between the nodes and the network when the
nodes are powered using energy harvesting techniques.
2 Related Work
In order to minimize the consumption of the node, two different approaches
are used, the first one is modifying the transmission policy; the other one is
modifying the duty cycle [3].
On the transmission policy side, three types of policies can be done: fixed
transmission policy, variable transmission policy and using probability distribu-
tion in order to transmit data. On this field, the investigation is mainly done
on the probability distribution, given as a result the LEACH protocol an all its
variants, as mentioned in 1.
LEACH protocol assumes each node has enough transmitting power to reach
the base station, but using it all the time would be a waste of energy; the solution
goes through creating clusters inside the WSN, the nodes communicate to the
cluster heads and the cluster heads compress and re-transmit the data to the
sink station. Doing this, the nodes that send data to the cluster head do not need
to use their radio transmitter at maximum power, allowing them o save energy.
Then, in the next round a new cluster head is selected, being not possible to
select the same cluster head for P rounds, where P are the number of desired
cluster heads. Therefore, the probability to be the cluster head again is 1/P.
On the duty cycle policy side, two types of policies can be done: sleep/wake-
up duty cycle policies and maximum/minimum duty cycle policies. Both policies
try to minimize the consumption by playing around with the duration of the
duty cycles. The difference here is while sleep/wake-up duty cycle policies keep
the nodes awaken while there is a connection in progress and put the nodes to
sleep when there is no connection, and a scheduling is done in order to wake
up the nodes; maximum/minimum duty cycle policies keep the nodes working
on maximum capacities when the harvested energy is more than the energy
consumed and the duty cycle is changed to a minimum performance when the
energy harvested is less than the typical energy consumption on active mode,
trying to prevent an energy-negative state; changing again to maximum perfor-
mance when the energy harvested is more than the typical energy consumption
in active mode.
The duty cycle policies are implemented in the energy managers. The energy
manager is an element inside the system in charge of controlling the energy
available in the buffers or collectors and to tell to the system the amount of
energy able to use in a concrete moment of time. These energy managers can
use previous data in order to decide the next action, this paradigm is known as
predictive energy managers. The main problem of this concept is the collected
information is only valid for the same region and cyclical events. On the other
hand, we can find prediction-free managers, these managers have two strong
points, the first one is that they have the ability to learn from their environment
without any previous data, the second one is that they can be deployed anywhere,
because they are not using previous captured information.
The first prediction-free energy manager was LQ-Tracker [13], proposed in
2007 by Vigorito et al. The main objective of this energy manager is to adapt the
duty cycle using the state of charge of the energy buffer using a linear quadratic
tracking. In 2014, P-FREEN [11] was proposed by Peng and Low, P-FREEN is
an energy manager that maximizes the duty-cycle of the sensor node. In order to
work, P-FREEN needs to know the state of charge of the energy buffer and the
harvested energy in that moment. If one of these values are below a threshold,
the node works using the minimum energy needed to be kept alive; otherwise, the
node will be adapted to work using the combination of the energy buffer and the
harvested energy. Two years later appeared Fuzzyman [2], proposed by Aoudia
et al. This energy manager uses a set of IF-THEN rules in order to work. The
input of these rules is extracted from the combination of the energy harvested in
the previous cycle and the residual energy on the energy buffer. The result gives
the total amount of available energy for the node on that cycle. Finally, in 2018,
Aoudia et al. proposed RLMan [1]. This energy manager uses reinforcement
learning in order to maximize the quality of the service while trying to avoid
power failures. In order to work, it only needs the state of charge of the energy
buffer.
The research done until this point always consider a node sending directly to
the base station, without the possibility of conforming a mesh network. Also, it
always consider the modification of the duty-cycle in order to work, leaving the
signal power of the node or the channel used during the transmission out of the
game.
3 Schema of a EH-WSN
Basically, an EH-WSN node is divided on four main pieces: the energy harvester,
the energy storage, the energy manager and the node itself. Figure 1 shows a
schema of a generic EH-WSN node.
Fig. 1. Generic Energy Harvesting WSN Node
The energy harvester is the component of an EH-WSN node that scavenges
energy from the environment and converts that energy harvested to electricity,
commercial products can be found on the market in many forms, such as photo-
voltaic panels, wind and water turbines, vibrational scavengers or thermoelectric
power generators.
Then, this energy is transferred to the energy manager. This component is
in charge to control what to do with the energy harvested. That means it must
make a decision whether charge the batteries, transfer the energy directly to the
processor or power the processor using the batteries previously charged.
Finally, there is the node itself, the node is in charge of measure a physical
process through a sensing interface and use the radio module in order to send
the data to the network. It is also in charge of controlling the different cycles
and the power consumption. The radio module can be connected to a gateway
directly, creating a point to point communication, or it can be connected with
other radio modules creating a mesh. When a mesh is created, the node also
needs to control the time slots when the data is sent and when the radio needs
to be switched as a receiver in order to retransmit data sent by the other nodes
conforming the network.
4 Issues and Challenges in Energy Harvesting Wireless
Sensor Networks
One of the main problems that are faced when talking about energy harvesting
wireless sensor networks is the mismatch between the harvester and the node.
Martinez et al. [7] created a tool to help hardware developers and designers in
order to solve this problem. But, the problem of that tool is that it is needed
to be fed with information previously captured or theoretical information if no
practical data is available. So, it is only useful when the energy generated is a
known value. Another big problem is that the most of the research done until
now tries to solve a very specific problem, without taking in account they are
part of a more complex system; and then, when two specific advances are tried to
be glued in order to produce something better, they are not always compatible.
Finally, in industrial sectors, when a deployment is done, it is expected to be
operative for ages and this problem cannot be easily tackled. New technologies
does not need to match with the previous ones, creating a barrier for accepting
these advances.
On the other hand, industrial processes are a source of wasted energy. Com-
panies are developing devices in order to recover this energy, and using it as a
power source for monitoring devices is a must. The deployment of wired sensors
costs around 50e /meter just for the wiring. Industrial battery-based sensors
have an average live of 6-24 months, that means, every two years, batteries need
to be replaced.
5 Proposed Architecture
In order to tackle all the problems mentioned above, we present a new system
that tries to solve the challenges and issues exposed above. The architecture
proposed consists in an undetermined number of nodes, one or more gateways,
a security manager and a network manager. This structure follows the struc-
ture a of the mesh network proposed in WirelessHART standard [15]; with two
differences explained below.
The main proposal is a shift on the energy manager from monitoring the
level of the battery to monitoring the amount of energy that is produced in the
energy harvester. This change is done because, as it is said in the introduction,
if the battery is removed, the node can continue working due to the energy
that comes from the energy harvester. But the problem here is the node does
not know how much energy will be produced in a concrete period because of
the uncertainty of the process from where the energy is extracted. In order to
solve that, the node needs to learn its capabilities. This learning is done using
artificial intelligence algorithms, concretely, a reinforcement learning algorithm
called SARSA (State-Action-Reward-State-Action) [12]. In equation 1 we can
see the SARSA prediction method
Q(st , at ) ← Q(st , at ) + α[rt+1 + γQ(st+1 , at+1 ) − Q(st , at )] (1)
where Q(st , at ) are the initial conditions, α is the learning rate and γ is the
discount factor.
The reinforcement learning process can be defined as a list of states where
for each state there are some actions to be done. Depending on the action, the
reward will be different and will affect the probability of doing the same action in
the next round of the state. This process can be split in two parts. The first one
uses the amount of generated energy in one concrete period of time as a state;
and the actions for each state will be related to adapt the node to consume
that energy generated. The second one is related to the communication, it scans
the channels that are used before doing any transmission, and selects a channel
in that moment. This selection is done taking in account the energetic cost of
sending a message. In both cases, an exploration or learning period is needed in
order to know the environment and act accordingly.
The novelty proposed in this paper makes a change on how the network is
formed. In WirelessHART there is a field that informs to the network manager
the amount of remaining energy in the battery. This field is no longer used,
and instead of this, the information sent is the state of the energy manager.
With this information, the network manager needs to create and distribute the
network paths to the nodes. This task must be repeated each time a threshold
of minimum network paths is exceeded. Also, the selection of the transmission
channel is new compared to the WirelessHART protocol, which is based on Time
Slot Channel Hopping (TSCH), where the node knows which channel is needed
to be used in advance [6]. In our proposal, just the paths are needed, but not
the channel to be used.
6 Conclusions
In this paper a new architecture is presented to tackle the issues and challenges
exposed in the section 4. Concretely, a new energy manager has been proposed
and explained the advantages against the existing ones. In future research, this
energy manager must be tested and compared with other methods focused to
extend the network lifetime. Besides, this energy manager is in charge of the
resilience of each node, handling power output and channel used in order to
avoid interferences between nodes thanks to applying the reinforcement learning.
Finally, thanks to this new energy manager, a modification of WirelessHART
network has been suggested. This suggestion is necessary for two reasons: there
is no existing industrial wireless protocol that handles well the energy harvesting
wireless sensor nodes and because of the energy constraints of the network, one
cannot delegate to the nodes the self-organizing capacity. Instead of this, it is
needed an element in the network able to calculate and distribute the different
paths between the nodes.
7 Acknowledgments
This research is funded in part by the Industrial Doctorates Plan of the Gen-
eralitat de Catalunya, under Grant No. DI-2016/50 and Innoenergy Industrial
Doctorates Plan under Grant No. PHDS1701.
References
1. Ait Aoudia, F., Gautier, M., Berder, O.: RLMan: An Energy Manager
Based on Reinforcement Learning for Energy Harvesting Wireless Sen-
sor Networks. IEEE Transactions on Green Communications and Network-
ing 2(2), 408–417 (Jun 2018). https://doi.org/10.1109/TGCN.2018.2801725,
https://ieeexplore.ieee.org/document/8279506/
2. Aoudia, F.A., Gautier, M., Berder, O.: Fuzzy power management
for energy harvesting Wireless Sensor Nodes. In: 2016 IEEE Interna-
tional Conference on Communications (ICC). pp. 1–6. IEEE, Kuala
Lumpur, Malaysia (May 2016). https://doi.org/10.1109/ICC.2016.7510767,
http://ieeexplore.ieee.org/document/7510767/
3. Babayo, A.A., Anisi, M.H., Ali, I.: A Review on energy management schemes in
energy harvesting wireless sensor networks. Renewable and Sustainable Energy
Reviews 76, 1176–1184 (Sep 2017). https://doi.org/10.1016/j.rser.2017.03.124,
https://linkinghub.elsevier.com/retrieve/pii/S1364032117304598
4. Basagni, S., Naderi, M.Y., Petrioli, C., Spenza, D.: Wireless Sensor Networks
with Energy Harvesting. In: Basagni, S., Conti, M., Giordano, S., Stojmen-
ovic, I. (eds.) Mobile Ad Hoc Networking, pp. 701–736. John Wiley & Sons,
Inc., Hoboken, NJ, USA (Mar 2013). https://doi.org/10.1002/9781118511305.ch20,
http://doi.wiley.com/10.1002/9781118511305.ch20
5. Chandrakasan, A., Amirtharajah, R., Seonghwan Cho, Goodman, J., Kon-
duri, G., Kulik, J., Rabiner, W., Wang, A.: Design considerations for dis-
tributed microsensor systems. In: Proceedings of the IEEE 1999 Custom In-
tegrated Circuits Conference (Cat. No.99CH36327). pp. 279–286 (May 1999).
https://doi.org/10.1109/CICC.1999.777291
6. Fabien Joseph Chraim: Wireless sensing applications for critical industrial envi-
ronments. Ph.D. thesis, University of California, Berkeley (Oct 2014)
7. Huerta, B.M.: Exploiting Spatio-Temporal Correlations for Energy Management
Policies. Ph.D. thesis, Universitat Autnoma de Barcelona, Bellaterra, Cerdanyola
del Valls, Barcelona, Spain (2015)
8. Kamalinejad, P., Mahapatra, C., Sheng, Z., Mirabbasi, S., M. Le-
ung, V.C., Guan, Y.L.: Wireless energy harvesting for the In-
ternet of Things. IEEE Communications Magazine 53(6), 102–
108 (Jun 2015). https://doi.org/10.1109/MCOM.2015.7120024,
http://ieeexplore.ieee.org/document/7120024/
9. Kansal, A., Hsu, J., Zahedi, S., Srivastava, M.B.: Power management in en-
ergy harvesting sensor networks. ACM Transactions on Embedded Comput-
ing Systems 6(4), 32–es (Sep 2007). https://doi.org/10.1145/1274858.1274870,
http://portal.acm.org/citation.cfm?doid=1274858.1274870
10. Martinez, B., Vilajosana, X., Chraim, F., Vilajosana, I., Pister, K.S.J.: When
Scavengers Meet Industrial Wireless. IEEE Transactions on Industrial Elec-
tronics 62(5), 2994–3003 (May 2015). https://doi.org/10.1109/TIE.2014.2362891,
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6922531
11. Peng, S., Low, C.: Prediction free energy neutral power manage-
ment for energy harvesting wireless sensor nodes. Ad Hoc Networks
13, 351–367 (Feb 2014). https://doi.org/10.1016/j.adhoc.2013.08.015,
https://linkinghub.elsevier.com/retrieve/pii/S1570870513001789
12. Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. Adaptive com-
putation and machine learning series, The MIT Press, Cambridge, MA, second
edition edn. (2018)
13. Vigorito, C.M., Ganesan, D., Barto, A.G.: Adaptive Control of
Duty Cycling in Energy-Harvesting Wireless Sensor Networks. In:
2007 4th Annual IEEE Communications Society Conference on Sen-
sor, Mesh and Ad Hoc Communications and Networks. pp. 21–
30. IEEE (Jun 2007). https://doi.org/10.1109/SAHCN.2007.4292814,
http://ieeexplore.ieee.org/document/4292814/
14. Yuvaraj P., Vikram K., Venkata Lakshmi Narayana K.: A Review on State of Art
Variants of LEACH Protocol for Wireless Sensor Networks. Sensors and Transduc-
ers 186, 25–32 (Mar 2015)
15. Zand, P., Mathews, E., Havinga, P., Stojanovski, S., Sisinni, E.,
Ferrari, P.: Implementation of WirelessHART in the NS-2 Simula-
tor and Validation of Its Correctness. Sensors (Basel, Switzerland)
14(5), 8633–8668 (May 2014). https://doi.org/10.3390/s140508633,
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4063040/