<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Equivariant graph neural network for crystalline materials</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Astrid Klipfel</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Zied Bouraoui</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Yaël Frégier</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Adlane Sayede</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Centre de Recherche en Informatique de Lens (CRIL)-CNRS UMR 8188, Artois University</institution>
          ,
          <country country="FR">France</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Laboratoire de Mathématiques de Lens (LML)-UR 2462, Artois University</institution>
          ,
          <country country="FR">France</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Unité de Catalyse et Chimie du Solide (UCCS)-CNRS UMR 8181, Artois University</institution>
          ,
          <country country="FR">France</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Materials generation is an essential task in material science that aims to discover new materials. While most of the existing models have shown interesting results in simulation, they struggle to produce new original and stable materials. This paper discusses the salient properties required for material generation and studies the dificulties related to material pattern repetition, which impacts the stability of the generated structures.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>y
Crystalline materials are involved everywhere in our reg
modern society. From metal alloy to semi-conductor, sev- En
eral technological objects contain crystalline materials.</p>
      <p>
        Discovering new materials remains a dificult task in
material science. While existing algorithms can search for Materials space
new structures in the materials space [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], searching for Figure 1: The energy of a structure is given by its geometry
a new material with a given set of desirable properties and chemistry. Stable structures are the local minima of the
is not a trivial task. The set of potential candidate ma- energy (green dots). Relaxing a structure is the process of
terials is not countable by a computer, and the portion modifying the geometry of a structure to minimize its energy
of stable materials (i.e. materials that can exist with- (an unstable structure represented as a red dot converge to
out self-destructing) is small. Moreover, estimating the the nearest stable minima)
properties of a single material with chemical simulation
as Density Functional Theory (DFT) is computationally
expensive. To this end, methods based on evolution- terial. It permits the discovery of new materials since
ary algorithms (e.g. [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]) are introduced for generating minimizing the total energy of the structure allows it to
new materials from existing datasets composed of stable be more stable. Consequently, the local minima of the
materials. However, most of these approaches work by energy corresponding to stable materials and relaxation
hybridization and mutation of existing materials. As a lead to producing stable crystalline structures from an
consequence, these methods are not able to find complex initially unstable crystal around the stable structures in
materials. the materials space. This is illustrated in Figure 1.
      </p>
      <p>
        Discovering new materials is a challenging problem. Several methods were recently introduced for
chemBut contrary to most generation problems, theoretical ical simulation either by enhancing current algorithms
chemistry provides a powerful set of tools to analyse with machine learning techniques (e.g. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]) or by using
synthetic and real data. As a matter of fact, simulation end-to-end models (e.g. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]). Most of these approaches
techniques like hartree-fock or DFT are able to estimate focus on accelerating DFT by approaching relaxation
the properties of a given structure by applying physics as a supervised task. End-to-end models generally
relaws. Consequently, the stability of the materials can be quire expensive labels to be trained as interaction forces.
estimated through simulation. These methods can also However, these labels are generally not available and
be used to perform the relaxation of crystalline materials. need to be produced through DFT calculation. Thus, the
Relaxation is the process of minimizing the energy of benefits of end-to-end models is limited because such
a crystalline structure by deforming it. This process is models required data from atomistic simulation. Finally,
very common in material science to study a given ma- even if the relaxation may lead to a crystalline
structure, there is no guaranty on its stability. In fact, most of
STRL’22: First International Workshop on Spatio-Temporal Reasoning the random structures do not result in stable structures.
and Learning, July 24, 2022, Vienna, Austria Consequently, approaches based on simulation for new
y$aekl.lfirpefgeile@r @cruiln.firv(-Aa.rtKoliisp.ffrel()Y;.zFierdég.bioeru)r;aoui@cril.fr (Z. Bouraoui); materials discovery are limited.
adlane.sayede@univ-artois.fr (A. Sayede) Another direction consists in directly generating stable
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License crystalline structures using machine learning techniques.
CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org)
In this case, the generative process can be performed
with successive actions applied to the structure [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. This
process doesn’t require strictly following physics laws as
long as the stable points (i.e. local minima of the energy)
learned by the machine learning models remain the same
as the real stable points. This fact can be advantageous
in some cases because the realistic relaxation of a
random structure is not guaranteed to converge to a stable
material. As a result, learning realistic stable points
without realistic physics can help machine learning models
produce stable structures.
      </p>
      <p>
        Indeed, graph neural networks (GNN) have already
been used for the representation and generation of
organic molecules [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. However, crystalline materials are
known to be more dificult to generate since they
generally have more complicated chemistry. Also, they contain
repetitive patterns defined by the lattice of the crystals,
which make them harder to process. In this paper, we
present how graphs representation of materials are
deifned. We also discuss some important properties
required for generative models.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Problem statement</title>
      <p>Crystalline systems As molecules, crystalline
systems can be defined as coloured point clouds. However,
as crystals are periodic structures, additional information
about how the point could is repeated in the space is
required to represent it. The periodicity of the material can
be then represented as a network where a group of points
is repeated by discrete translation, which is equivalent
to tiling space with a parallelepiped containing a cloud
of atoms as illustrated in Figure 2.</p>
      <p>As a result, a periodic system can be describe as atomic
positions  ∈ [0, 1[3 with an associated feature space
representing the chemical information of each atom  ∈
ℱ and a lattice  ∈ GL3(R) representing the periodicity
of the material. The infinite point cloud generated by
this representation can be defined as
︀{(  · ( + ), )︀ | ∈ Z3, 1 ≤  ≤ }︀ ⊆
R3 ×  (1)</p>
      <sec id="sec-2-1">
        <title>Where  act like a Z3 vector that translate the point</title>
        <p>cloud.</p>
        <p>
          Material graph There are multiple ways to define
the graph of a material. Chemical bonds can be used
to build the set of edges, but generally, edges are built
from the atoms under a given threshold distance of from
the  nearest neighbourhood [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]. The resulting graph is
a multi-graph because the local environment of an atom
can be on a translated point cloud near the border of the
lattice as depicted in Figure 2. As a result, an edge can
have multiple edges between a pair of nodes and between
a node with itself.
        </p>
        <p>Material deformation To tackle the crystalline
system generation problem, we should define action on the
geometry of the material. This action can be seen as an
action on the lattice of the material  resulting in the
updated lattice  ′ and action on the atomic positions 
resulting in the updated atomic position ′.</p>
        <p>{︃ ′ = 
′ = [ + ]
.</p>
        <p>(2)</p>
      </sec>
      <sec id="sec-2-2">
        <title>In this case, the goal is to predict the action  ∈</title>
        <p>GL3(R) on the lattice and the actions  ∈ R3 on the
atomic position. The atomic positions are brought back
into the lattice of the crystal by truncation.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Method</title>
      <sec id="sec-3-1">
        <title>Equivariance To obtain meaningful actions applied to</title>
        <p>a material, we should satisfy the equivariance property.</p>
        <p>Indeed, translations and rotations have no impact on the
material’s properties. Consequently, a machine learning
model acting on the material should not be dependent
on the orientation and position of the structure but only
be dependent on its geometry as shown in Figure 3.</p>
        <p>′
ℎ
ℎ
ℎ ·</p>
        <p>ℎ ·  ′</p>
        <p>Actions on the lattice When  acts on ℎ the lattice
of the crystal, we would restrain  to the group of actions
deforming the structure. However, any matrix in R3× 3
can be decomposed as a sum of a symmetric and an
antisymmetric matrix. As we define the action of  in GL(3),
the anti-symmetric part of the transformation can be
seen as a rotation. But as we discussed earlier, applying
rotations on a crystal doesn’t act on a material as the
rotated material is equivalent to the original material.</p>
        <p>This is illustrated in Figure 4. Consequently, it is better Figure 5: Actions defined on the edges of the graph can be
to restrain  to GL(3) ∖ SO(3). In other words, to restrain decomposed into action on the atomic positions and action
 to the subset of matrices of GL(3) that are symmetric. on the lattice of the crystalline material.</p>
        <p>Acting on the crystalline structure To act on both
the lattice and the atomic position, we can define actions
on the edge of the graph. These actions can be then
decomposed into a global action on the lattice of the
crystal and local action on the atomic position. In order
to define these actions, the contributions of all edges are
aggregated to compute the action on the lattice. To act
on the atomic positions, only the actions of the edges
connected to the node are taken into account.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Conclusion</title>
      <p>Crystalline materials are dificult to process because of
the complexity and the variety of their chemistry, but also
because of their repetitive structure. However, materials
can be represented as graphs containing both chemical
and geometrical information about the structures.
Consequently, geometric machine learning techniques such
as graph neural networks can be used in a wide variety
of tasks including supervised and unsupervised learning.
But in order to enhance the generalization capability of
machine learning models, some properties such as the
symmetry of the actions on the lattice or the equivariance
can be beneficial.</p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <sec id="sec-5-1">
        <title>This work has been supported by HPC resources from GENCI-IDRIS (Grant 2021-[AD011013288]) and FEI INS2I 2022-EMILIE.</title>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>C. J.</given-names>
            <surname>Pickard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. J.</given-names>
            <surname>Needs</surname>
          </string-name>
          ,
          <article-title>Ab initio random structure searching</article-title>
          ,
          <source>Journal of Physics: Condensed Matter</source>
          <volume>23</volume>
          (
          <year>2011</year>
          )
          <article-title>053201</article-title>
          . URL: https://doi.org/10.1088/
          <fpage>0953</fpage>
          -8984/23/5/053201. doi:
          <volume>10</volume>
          .1088/
          <fpage>0953</fpage>
          -8984/ 23/5/053201.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>A. R.</given-names>
            <surname>Oganov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. O.</given-names>
            <surname>Lyakhov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Valle</surname>
          </string-name>
          ,
          <article-title>How evolutionary crystal structure prediction works-and why</article-title>
          ,
          <source>Accounts of Chemical Research</source>
          <volume>44</volume>
          (
          <year>2011</year>
          )
          <fpage>227</fpage>
          -
          <lpage>237</lpage>
          . URL: https://doi. org/10.1021/ar1001318. doi:
          <volume>10</volume>
          .1021/ar1001318. arXiv:https://doi.org/10.1021/ar1001318, pMID:
          <fpage>21361336</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O. A.</given-names>
            <surname>Jiménez-Negrón</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. R.</given-names>
            <surname>Kitchin</surname>
          </string-name>
          ,
          <article-title>Machine-learning accelerated geometry optimization in molecular simulation</article-title>
          ,
          <source>The Journal of Chemical Physics</source>
          <volume>154</volume>
          (
          <year>2021</year>
          ). URL: https://par.nsf.gov/ biblio/10250472. doi:
          <volume>10</volume>
          .1063/5.0049665.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>F.</given-names>
            <surname>Ekström</surname>
          </string-name>
          <string-name>
            <surname>Kelvinius</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Armiento</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Lindsten</surname>
          </string-name>
          ,
          <article-title>Graph-based machine learning beyond stable materials and relaxed crystal structures</article-title>
          ,
          <source>Phys. Rev. Materials</source>
          <volume>6</volume>
          (
          <year>2022</year>
          )
          <article-title>033801</article-title>
          . URL: https://link.aps.org/doi/ 10.1103/PhysRevMaterials.6.033801. doi:
          <volume>10</volume>
          .1103/ PhysRevMaterials.6.033801.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>T.</given-names>
            <surname>Xie</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Fu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.-E.</given-names>
            <surname>Ganea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Barzilay</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. S.</given-names>
            <surname>Jaakkola</surname>
          </string-name>
          ,
          <article-title>Crystal difusion variational autoencoder for periodic material generation</article-title>
          ,
          <source>in: International Conference on Learning Representations</source>
          ,
          <year>2022</year>
          . URL: https://openreview.net/forum?id=03RLpj-tc_.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>V. G.</given-names>
            <surname>Satorras</surname>
          </string-name>
          , E. Hoogeboom,
          <string-name>
            <given-names>M.</given-names>
            <surname>Welling</surname>
          </string-name>
          ,
          <string-name>
            <surname>E(</surname>
          </string-name>
          <article-title>n) equivariant graph neural networks</article-title>
          , in: M.
          <string-name>
            <surname>Meila</surname>
          </string-name>
          , T. Zhang (Eds.),
          <source>Proceedings of the 38th International Conference on Machine Learning</source>
          , volume
          <volume>139</volume>
          <source>of Proceedings of Machine Learning Research, PMLR</source>
          ,
          <year>2021</year>
          , pp.
          <fpage>9323</fpage>
          -
          <lpage>9332</lpage>
          . URL: https://proceedings.mlr.press/v139/ satorras21a.html.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>T.</given-names>
            <surname>Xie</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. C.</given-names>
            <surname>Grossman</surname>
          </string-name>
          ,
          <article-title>Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties</article-title>
          ,
          <source>Phys. Rev. Lett</source>
          .
          <volume>120</volume>
          (
          <year>2018</year>
          )
          <article-title>145301</article-title>
          . URL: https://link.aps.org/
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>