=Paper=
{{Paper
|id=None
|storemode=property
|title=Decentralized Multiagent Planning for Balance Control in Smart Grids
|pdfUrl=https://ceur-ws.org/Vol-923/paper03.pdf
|volume=Vol-923
}}
==Decentralized Multiagent Planning for Balance Control in Smart Grids==
https://ceur-ws.org/Vol-923/paper03.pdf
Decentralized Multiagent Planning for Balance
Control in Smart Grids
Francisco S. Melo1 , Alberto Sardinha1 , Stefan Witwicki1 ,
Laura M. Ramirez-Elizondo2 , and Matthijs T.J. Spaan2
1
INESC-ID/Instituto Superior Técnico
2780-990 Porto Salvo, Portugal
{fmelo,witwicki}@inesc-id.pt, jose.alberto.sardinha@ist.utl.pt
2
Delft University of Technology
2628 CD, Delft, The Netherlands
{l.m.ramirezelizondo,m.t.j.spaan}@tudelft.nl
Abstract. Integrating large-scale micro-generation in distribution grids
is challenging for distribution grid operators, particularly when renew-
able energy sources (RES) and micro-cogeneration are involved. In this
paper we contend that recent developments in multiagent decision mak-
ing under uncertainty can positively contribute to safe, efficient and cost-
effective operation of future distribution grids.
Keywords: Smart Distribution Grids, Decentralized Planning, Agents
1 Introduction
Electric power systems have been undergoing momentous changes over the last
decade. In the past, power was supplied predominantly by a limited number
of large power plants, mainly nuclear powered or fossil fueled, and then trans-
mitted to the consumers [5]. In the near future, production will increasingly
rely on a greater number of decentralized, mostly small-scale production sites
[5] based on renewable energy sources (RES), such as solar or wind power, and
micro-cogeneration units, such as stirling engines and fuel cells. These will be
located closer to the final consumers than traditional power plants, even at the
households themselves.
The inclusion of micro-generation enhances the overall system in terms of sus-
tainability. However, we advocate that additional improvement may be achieved
through intelligent agent-based decision making. Customers, or agents acting on
their behalf, should play an active role in managing the energy produced by
the controllable micro-generation units. Additionally, the balance between con-
sumption and supply is required for a proper and stable operation. Thus, agents
that control energy consumption can also support the distribution network by
matching the timing of their demand to the dynamic availability of the energy
supply. As a result, a more efficient operation can be obtained by reducing the
peak load while maintaining the power balance.
The massive introduction of small-scale RES-based production and active
consumption management introduces significant uncertainty in the normal op-
eration of the distribution grid. Some of the sources behind such uncertainty
are [4]: (i) Operational uncertainty, usually associated with the demand and the
supply of energy (e.g., load pattern predictions, future energy supply of solar
cells and windmills); (ii) Structural uncertainty, associated with changes in the
physical infrastructure (e.g., switches in the power grid may have to be closed
or opened to keep voltage and frequency within normal operational limits; a
distribution line breaking due to bad weather).
With the introduction of decentralized generation, several important changes
regarding the planning, operation and control of power systems have taken place,
particularly because of the following differences: decentralized generation units
are connected to the distribution network and not to the transmission network;
several types of decentralized generation units are connected to the grid by means
of power electronic interfaces (whereas large generation plants are coupled to
the electricity grid directly); the power generated by micro-generation units is
considerably less than power generated from traditional power plants (several
orders of magnitude); and renewable energy generators depend on natural and
uncontrollable sources, which adds a high level of uncertainty to the system.
Given that distribution generation will play an important role at distribution
level, power systems are forced to adapt in order to perform control actions at
this operating level as well.
In the literature, a large number of the planning and control architectures
designed for distribution network and micro-grid applications have two-level hi-
erarchical configurations and only take into account electrical parameters and
electrical interactions, even though heat outputs from micro-cogeneration units
are also available. For example, in [11] a droop control method is applied on
a system that contains renewable energy generators and storage. The control
unit optimizes the power output of the generators by communicating new droop
settings based on the information collected from the inverters, micro-generation
units and battery banks. Another example can be found in [6], where control
and power management strategies based on locally measured signals without
communication were proposed under various micro-grid operating conditions.
The real power of each decentralized generation unit is controlled based on a
frequency droop characteristic and a frequency restoration strategy [6].
In contrast to these prior works, we propose addressing the electrical flows,
but also the usable heat produced by the micro-generation units, as considered
in [8]. Moreover, we defend the incorporation of three aspects of control using an
integrated agent-based planning methodology, namely the active power control,
voltage control and control with respect to economic considerations. An economic
optimization based on forecasts will provide the set-points to the controllable
components of the system in which the active power control and voltage control
will be applied.
Hence, we envision a distribution grid that is able to self-regulate with little
human supervision. We defend the use of new decentralized planning and control
techniques for the distribution grid that take into account the dynamics and the
topology of the grid and also handle the uncertainty inherent in the production
and consumption of electricity. These techniques should allow the grid to preserve
its properties as it scales in size and should also accommodate the possibility
of massive micro-generation from renewable energy sources and from micro-
cogeneration units. Finally, it should facilitate the inclusion of new technologies
such as smart heating, ventilation, and air conditioning equipment.
2 Decentralized Planning and Optimization
Consider the distribution grid as a complex system composed of interconnected
components, many of which need to be controlled in order to optimize system
objectives. Decentralized planning accomplishes this optimization by distribut-
ing the control among a team of intelligent agents, each of which operates an
individual component. For instance, an agent controlling a particular power sub-
station decides where and when to route power. In planning its decisions, each
agent should account for uncertainty in the consequences of its actions, reasoning
over, for instance, the likelihoods of different volumes of future energy consump-
tion. Agents may only be able to base their decisions on incomplete and local
information, depending on sensory capabilities and on infrastructure supporting
information exchange throughout the system. Nevertheless, because the actua-
tion of one component may affect the state of another, the agents should work
together to formulate coordinated plans that fulfill quantifiable global objectives.
In the literature, these characteristics serve as the basis for a formal model of
multiagent decision-making called a Decentralized Partially Observable Markov
Decision Process (Dec-POMDP) [1]. The Dec-POMDP model has been hailed as
a rich, principled mathematical framework for optimization under uncertainty,
and has spawned an increasingly active area of research referred to as multiagent
sequential decision making (MSDM) under uncertainty. Power systems research
has considered the effects of uncertainty in load predictions [2], the inherent
uncertainty in wind forecasts [3] or uncertainty in unit commitment [9]. How-
ever, the decentralized optimization techniques for tackling uncertainty that we
propose have not yet been exploited in Smart Grids.
3 Discussion
Framing the control problem as one of decentralized planning, one can address
the problems of keeping the network under stable operation, performing balance
control, and economically optimizing the system, in a single integrated solution,
all while accounting for uncertainty. An appealing aspect of this application is
the structure in the distribution grid control problem that we expect can be
leveraged to improve the efficiency and scalability of decentralized planning [7].
Recent theoretical developments have established that multiagent systems in
which the interactions between agents are weakly coupled allow for significant
computational savings that can result from exploiting such weakly coupled in-
teraction structure [10]. This has resulted in increasing research efforts in the
development of better representations of the structure of multiagent systems and
better techniques for exploiting it.
Although there is yet little application of MSDM techniques to real problems,
control of smart distribution grids provides a well-motivated application domain,
and with it a golden opportunity to break free of the status quo and to develop
and validate MSDM research on realistic problems. It not only allows the testing
of conventional assumptions of existing models and algorithms that have long
been taken for granted, but it can also inspire the development of more useful
models and methods whose assumptions are more realistic. This would constitute
an important step forward in grounding recent MSDM work, and one that is
essential for maturing the field.
Acknowledgements
This work was partially supported by national funds through Fundação para a Ciência
e a Tecnologia under project PEst-OE/EEI/LA0021/2011. M.S. is funded by the FP7
Marie Curie Actions Individual Fellowship #275217 (FP7-PEOPLE-2010- IEF).
References
1. D. S. Bernstein, R. Givan, N. Immerman, and S. Zilberstein. The complexity of
decentralized control of Markov decision processes. Mathematics of Operations
Research, 27(4):819–840, 2002.
2. R. Billinton and D. Huang. Effects of load forecast uncertainty on bulk electric
system reliability evaluation. IEEE Trans. Power Systems, 23(2):418–425, 2008.
3. E. Constantinescu, V. Zavala, M. Rocklin, S. Lee, and M. Anitescu. A computa-
tional framework for uncertainty quantification and stochastic optimization in unit
commitment with wind power generation. IEEE Trans. Power Systems, 26:431–
441, 2011.
4. A. Dominguez-Garcia and P. Grainger. A framework for multi-level reliability eval-
uation of electrical energy systems. In Energy 2030 Conference, 2008. ENERGY
2008. IEEE, pages 1 –6, nov. 2008.
5. J. Kassakian and R. Schmalensee. The future of the electric grid: An interdisci-
plinary MIT study. Technical report, Massachusetts Institute of Technology, 2011.
6. F. Katiraei and M. R. Iravani. Power management strategies for a microgrid with
multiple distributed generation units. IEEE Trans. Power Systems, 21(4):1821–
1831–, 2006.
7. F. A. Oliehoek, M. T. J. Spaan, S. Whiteson, and N. Vlassis. Exploiting locality of
interaction in factored Dec-POMDPs. In Proc. of Int. Conference on Autonomous
Agents and Multi Agent Systems, pages 517–524, 2008.
8. L. M. Ramirez-Elizondo and G. C. Paap. Unit commitment in multiple energy
carrier systems. In Proceedings of the North American Power Symposium (NAPS),
2009, pages 1–6, 4-6 2009.
9. P. Ruiz, C. Philbrick, E. Zak, K. Cheung, and P. Sauer. Uncertainty management
in the unit commitment problem. IEEE Trans. Power Systems, 24:642–651, 2009.
10. S. J. Witwicki and E. H. Durfee. Towards a unifying characterization for quanti-
fying weak coupling in Dec-POMDPs. In Proc. of Int. Conference on Autonomous
Agents and Multi Agent Systems, 2011.
11. Y. Zilong, W. Chunsheng, L. Hua, and X. Honghua. Design of energy management
system in distributed power station. In Sustainable Power Generation and Supply,
2009. SUPERGEN ’09. International Conference on, pages 1–5–, 2009.