=Paper=
{{Paper
|id=Vol-3190/invited1
|storemode=property
|title=Equivariant Graph Neural Network for Crystalline Materials
|pdfUrl=https://ceur-ws.org/Vol-3190/invited1.pdf
|volume=Vol-3190
|authors=Astrid Klipfel,Zied Bouraoui,Yaël Frégier,Adlane Sayede
|dblpUrl=https://dblp.org/rec/conf/ijcai/KlipfelBFS22
}}
==Equivariant Graph Neural Network for Crystalline Materials==
https://ceur-ws.org/Vol-3190/invited1.pdf
Equivariant graph neural network for crystalline materials
Astrid Klipfel1,2,3 , Zied Bouraoui2 , Yaël Frégier1 and Adlane Sayede3
1
Laboratoire de Mathématiques de Lens (LML)-UR 2462, Artois University, France
2
Centre de Recherche en Informatique de Lens (CRIL)-CNRS UMR 8188, Artois University, France
3
Unité de Catalyse et Chimie du Solide (UCCS)-CNRS UMR 8181, Artois University, France
Abstract
Materials generation is an essential task in material science that aims to discover new materials. While most of the existing
models have shown interesting results in simulation, they struggle to produce new original and stable materials. This
paper discusses the salient properties required for material generation and studies the difficulties related to material pattern
repetition, which impacts the stability of the generated structures.
1. Introduction
Energy
Crystalline materials are involved everywhere in our
modern society. From metal alloy to semi-conductor, sev-
eral technological objects contain crystalline materials.
Discovering new materials remains a difficult task in ma-
Materials space
terial science. While existing algorithms can search for
new structures in the materials space [1], searching for Figure 1: The energy of a structure is given by its geometry
a new material with a given set of desirable properties and chemistry. Stable structures are the local minima of the
is not a trivial task. The set of potential candidate ma- energy (green dots). Relaxing a structure is the process of
terials is not countable by a computer, and the portion modifying the geometry of a structure to minimize its energy
(an unstable structure represented as a red dot converge to
of stable materials (i.e. materials that can exist with-
the nearest stable minima)
out self-destructing) is small. Moreover, estimating the
properties of a single material with chemical simulation
as Density Functional Theory (DFT) is computationally
expensive. To this end, methods based on evolution- terial. It permits the discovery of new materials since
ary algorithms (e.g. [2]) are introduced for generating minimizing the total energy of the structure allows it to
new materials from existing datasets composed of stable be more stable. Consequently, the local minima of the
materials. However, most of these approaches work by energy corresponding to stable materials and relaxation
hybridization and mutation of existing materials. As a lead to producing stable crystalline structures from an
consequence, these methods are not able to find complex initially unstable crystal around the stable structures in
materials. the materials space. This is illustrated in Figure 1.
Discovering new materials is a challenging problem. Several methods were recently introduced for chem-
But contrary to most generation problems, theoretical ical simulation either by enhancing current algorithms
chemistry provides a powerful set of tools to analyse with machine learning techniques (e.g. [3]) or by using
synthetic and real data. As a matter of fact, simulation end-to-end models (e.g. [4]). Most of these approaches
techniques like hartree-fock or DFT are able to estimate focus on accelerating DFT by approaching relaxation
the properties of a given structure by applying physics as a supervised task. End-to-end models generally re-
laws. Consequently, the stability of the materials can be quire expensive labels to be trained as interaction forces.
estimated through simulation. These methods can also However, these labels are generally not available and
be used to perform the relaxation of crystalline materials. need to be produced through DFT calculation. Thus, the
Relaxation is the process of minimizing the energy of benefits of end-to-end models is limited because such
a crystalline structure by deforming it. This process is models required data from atomistic simulation. Finally,
very common in material science to study a given ma- even if the relaxation may lead to a crystalline struc-
ture, there is no guaranty on its stability. In fact, most of
STRL’22: First International Workshop on Spatio-Temporal Reasoning the random structures do not result in stable structures.
and Learning, July 24, 2022, Vienna, Austria
Consequently, approaches based on simulation for new
$ klipfel@cril.fr (A. Klipfel); zied.bouraoui@cril.fr (Z. Bouraoui);
yael.fregier@univ-artois.fr (Y. Frégier); materials discovery are limited.
adlane.sayede@univ-artois.fr (A. Sayede) Another direction consists in directly generating stable
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0). crystalline structures using machine learning techniques.
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
In this case, the generative process can be performed have multiple edges between a pair of nodes and between
with successive actions applied to the structure [5]. This a node with itself.
process doesn’t require strictly following physics laws as
long as the stable points (i.e. local minima of the energy)
learned by the machine learning models remain the same
as the real stable points. This fact can be advantageous
in some cases because the realistic relaxation of a ran-
dom structure is not guaranteed to converge to a stable
material. As a result, learning realistic stable points with-
out realistic physics can help machine learning models Figure 2: The periodic structure of a material is represented
produce stable structures. as a lattice (in dotted lines). The multi-graph associated with
Indeed, graph neural networks (GNN) have already a material (blue arrow) can overlap on the adjacent repetition
been used for the representation and generation of or- of the lattice and a pair of nodes can be connected multiple
ganic molecules [6]. However, crystalline materials are time.
known to be more difficult to generate since they gener-
ally have more complicated chemistry. Also, they contain
repetitive patterns defined by the lattice of the crystals, Material deformation To tackle the crystalline sys-
which make them harder to process. In this paper, we tem generation problem, we should define action on the
present how graphs representation of materials are de- geometry of the material. This action can be seen as an
fined. We also discuss some important properties re- action on the lattice of the material 𝜌 resulting in the
quired for generative models. updated lattice 𝜌′ and action on the atomic positions 𝑥𝑖
resulting in the updated atomic position 𝑥′𝑖 .
2. Problem statement {︃
𝜌′ = 𝑔𝜌
. (2)
Crystalline systems As molecules, crystalline sys- 𝑥′𝑖 = [𝑥𝑖 + 𝑔𝑖 ]
tems can be defined as coloured point clouds. However,
as crystals are periodic structures, additional information In this case, the goal is to predict the action 𝑔 ∈
about how the point could is repeated in the space is re- GL3 (R) on the lattice and the actions 𝑔𝑖 ∈ R on the
3
quired to represent it. The periodicity of the material can atomic position. The atomic positions are brought back
be then represented as a network where a group of points into the lattice of the crystal by truncation.
is repeated by discrete translation, which is equivalent
to tiling space with a parallelepiped containing a cloud 3. Method
of atoms as illustrated in Figure 2.
As a result, a periodic system can be describe as atomic Equivariance To obtain meaningful actions applied to
positions 𝑥𝑖 ∈ [0, 1[3 with an associated feature space a material, we should satisfy the equivariance property.
representing the chemical information of each atom 𝑧𝑖 ∈ Indeed, translations and rotations have no impact on the
ℱ and a lattice 𝜌 ∈ GL3 (R) representing the periodicity material’s properties. Consequently, a machine learning
of the material. The infinite point cloud generated by model acting on the material should not be dependent
this representation can be defined as on the orientation and position of the structure but only
be dependent on its geometry as shown in Figure 3.
𝜌·(𝑥𝑖 +𝜏 ), 𝑧𝑖 |𝜏 ∈ Z , 1 ≤ 𝑖 ≤ 𝑛 ⊆ R ×𝐹 (1)
3 3
{︀(︀ )︀ }︀
ℎ
𝑀 ℎ·𝑀
Where 𝜏 act like a Z3 vector that translate the point 𝑔 𝑔
cloud.
𝑀′ ℎ · 𝑀′
ℎ
Material graph There are multiple ways to define Figure 3: Equivariance between the action applied on the
the graph of a material. Chemical bonds can be used material 𝑔 and the actions of translation and rotation group
to build the set of edges, but generally, edges are built ℎ ∈ R3 ∪ SO(3). 𝑀 and 𝑀 ′ denoting the original material
from the atoms under a given threshold distance of from and the material after having applied the action 𝑔 .
the 𝑘 nearest neighbourhood [7]. The resulting graph is
a multi-graph because the local environment of an atom
can be on a translated point cloud near the border of the
lattice as depicted in Figure 2. As a result, an edge can
Actions on the lattice When 𝑔 acts on 𝑟ℎ𝑜 the lattice
of the crystal, we would restrain 𝑔 to the group of actions
deforming the structure. However, any matrix in R3×3
can be decomposed as a sum of a symmetric and an anti-
symmetric matrix. As we define the action of 𝑔 in GL(3),
the anti-symmetric part of the transformation can be
seen as a rotation. But as we discussed earlier, applying
rotations on a crystal doesn’t act on a material as the
rotated material is equivalent to the original material.
This is illustrated in Figure 4. Consequently, it is better Figure 5: Actions defined on the edges of the graph can be
to restrain 𝑔 to GL(3) ∖ SO(3). In other words, to restrain decomposed into action on the atomic positions and action
𝑔 to the subset of matrices of GL(3) that are symmetric. on the lattice of the crystalline material.
References
[1] C. J. Pickard, R. J. Needs, Ab initio random struc-
Figure 4: The SO(2) group doesn’t act on the materials but ture searching, Journal of Physics: Condensed Mat-
only rotates them without deformation. ter 23 (2011) 053201. URL: https://doi.org/10.1088/
0953-8984/23/5/053201. doi:10.1088/0953-8984/
23/5/053201.
[2] A. R. Oganov, A. O. Lyakhov, M. Valle, How
Acting on the crystalline structure To act on both evolutionary crystal structure prediction
the lattice and the atomic position, we can define actions works—and why, Accounts of Chemical Re-
on the edge of the graph. These actions can be then search 44 (2011) 227–237. URL: https://doi.
decomposed into a global action on the lattice of the org/10.1021/ar1001318. doi:10.1021/ar1001318.
crystal and local action on the atomic position. In order arXiv:https://doi.org/10.1021/ar1001318,
to define these actions, the contributions of all edges are pMID: 21361336.
aggregated to compute the action on the lattice. To act [3] Y. Yang, O. A. Jiménez-Negrón, J. R. Kitchin,
on the atomic positions, only the actions of the edges Machine-learning accelerated geometry optimiza-
connected to the node are taken into account. tion in molecular simulation, The Journal of Chem-
ical Physics 154 (2021). URL: https://par.nsf.gov/
4. Conclusion biblio/10250472. doi:10.1063/5.0049665.
[4] F. Ekström Kelvinius, R. Armiento, F. Lindsten,
Crystalline materials are difficult to process because of Graph-based machine learning beyond stable ma-
the complexity and the variety of their chemistry, but also terials and relaxed crystal structures, Phys. Rev. Ma-
because of their repetitive structure. However, materials terials 6 (2022) 033801. URL: https://link.aps.org/doi/
can be represented as graphs containing both chemical 10.1103/PhysRevMaterials.6.033801. doi:10.1103/
and geometrical information about the structures. Con- PhysRevMaterials.6.033801.
sequently, geometric machine learning techniques such [5] T. Xie, X. Fu, O.-E. Ganea, R. Barzilay, T. S. Jaakkola,
as graph neural networks can be used in a wide variety Crystal diffusion variational autoencoder for peri-
of tasks including supervised and unsupervised learning. odic material generation, in: International Con-
But in order to enhance the generalization capability of ference on Learning Representations, 2022. URL:
machine learning models, some properties such as the https://openreview.net/forum?id=03RLpj-tc_.
symmetry of the actions on the lattice or the equivariance [6] V. G. Satorras, E. Hoogeboom, M. Welling, E(n) equiv-
can be beneficial. ariant graph neural networks, in: M. Meila, T. Zhang
(Eds.), Proceedings of the 38th International Confer-
ence on Machine Learning, volume 139 of Proceed-
Acknowledgments ings of Machine Learning Research, PMLR, 2021, pp.
9323–9332. URL: https://proceedings.mlr.press/v139/
This work has been supported by HPC resources from satorras21a.html.
GENCI-IDRIS (Grant 2021-[AD011013288]) and FEI INS2I [7] T. Xie, J. C. Grossman, Crystal graph convolu-
2022-EMILIE. tional neural networks for an accurate and inter-
pretable prediction of material properties, Phys. Rev.
Lett. 120 (2018) 145301. URL: https://link.aps.org/
doi/10.1103/PhysRevLett.120.145301. doi:10.1103/
PhysRevLett.120.145301.