<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Quality Assessment of Generated Hardware Designs Using Statistical Analysis and Machine Learning</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Lorenzo Servadei</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Elena Zennaro</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Keerthikumara Devarajegowda</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Wolfgang Ecker</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Robert Wille</string-name>
          <email>robert.wille@jku.at</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>In neon Technologies AG</institution>
          ,
          <addr-line>Am Campeon 1-15, 85579 Munich</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Johannes Kepler University Linz</institution>
          ,
          <addr-line>Altenbergerstra e 69, 4040 Linz</addr-line>
          ,
          <country country="AT">Austria</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Technical University of Munich</institution>
          ,
          <addr-line>Arcisstra e 21, 80333 Munich</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Univ. of Kaiserslautern</institution>
          ,
          <addr-line>Erwin-Schrodinger-Str. 1, 67663 Kaiserslautern</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>In order to continuously increase design productivity, engineers and researchers rely on automation frameworks for hardware design purposes. This does not only guarantee an easier implementation of components, but creates a larger margin for improvement by generating design variants. Within this framework, a major problem for optimizing the generated design is retrieving data from which a prediction function (e.g. area, speed, power consumption) could be learned correctly (since a complete generation, i.e. synthesis of the hardware design, is too computationally expensive to be performed for a wide set of variants). In particular, the data used for learning the prediction function should be representative of valid design possibilities and be generated in an e cient way. As one contribution, this paper describes how Statistical Analysis (SA) and Machine Learning (ML) are used to guarantee the quality of the data. At the same time, its retrieval should avoid time consumption and manual e ort. Therefore, this paper also proposes an automatic approach to generate representative and valid con guration samples both to improve the e ciency and to avoid manual e ort during the retrieval. To point out this concept, we implement the generation of data for the estimation of the area of a Register Interface (RI) component. The proposed methods, implemented through SA and ML, allow to supervise the correctness of the generated data and the learning process itself. As a consequence, given the correctly generated data, the process of learning the RI area through a data-driven ML algorithm guarantees a still accurate (R2 = 0:98) but 600x faster estimation.</p>
      </abstract>
      <kwd-group>
        <kwd>Machine Learning</kwd>
        <kwd>Statistical Analysis</kwd>
        <kwd>Data Generation</kwd>
        <kwd>Design Automation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        In order to cope with the increasing complexity in hardware and system design,
a common approach is to rely on higher levels of abstractions. For this reason,
model-based design ows utilizing modeling languages such as UML [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] or, more
speci cally, SysML [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] become of interest. Through multi-layer abstractions and
structure modeling, they allow to de ne dependencies, relations and constraints
for the components. This allows e.g. for veri cation and validation at early stages
of the design ow (using e.g. methods such as [
        <xref ref-type="bibr" rid="ref4 ref5 ref9">4, 5, 9</xref>
        ]), but also to realize the
desired system from well de ned high-level models. In particular, this approach
is very useful for generating hardware designs that di er in some implementation
details but rely on a common source (represented in terms of an abstract model).
      </p>
      <p>
        In the following, we focus on MetaRTL [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], an approach followed at
Inneon for design centric modeling of digital hardware. However, the methods
proposed in this paper are independent of any speci c design ow and shall be
applicable to other design ows as well. The MetaRTL design ow allows the
automatic generation of hardware designs from a given abstract model (in the
following called meta-model ) and, by this, provides an agile environment which
aids designers in the generation of the hardware components and systems.
      </p>
      <p>However, the generation mechanism itself is a challenge, since the generation
framework must be resilient, robust and easily maintainable. Indeed, while the
meta-model provides a basis for the designs generation, the designer still has to
provide a corresponding con guration, i.e. a set of parameters guiding the design
generation process and de ning how the meta-model is instantiated/realized. For
example, from an abstract description (meta-model) of the CPU structure, the
designer has to select a speci c con guration for a CPU instance (e.g. No. of
cores, etc.). The complexity of today's hardware components makes it hard to
determine a proper set of con gurations which indeed yields a design satisfying
given objectives e.g. with respect to area, speed, etc. This raises the question how
to e ciently determine instances, from a given meta-model, which eventually
trigger the desired design generation.</p>
      <p>In order to retrieve an optimal design con guration for a given objective, the
designer needs to be equipped with a set of di erent con gurations as well as
estimates about the resulting area, speed, etc. of the design obtained from each of
these con gurations. While the same could be achieved manually (implementing
a set of di erent con gurations and, through a synthesis tool, obtaining the
corresponding area, speed, etc.), this would result in a time consuming process.
Moreover, the estimates would be prone to errors since, as long as not a \real"
implementation would be created for each con guration, the designer would
need to infer those characteristics from the design itself. Hence, an automated
and reliable way of exploring the space of design con gurations is needed.</p>
      <p>
        In this work, we address these problems. First, we automatically generate
values for the design con gurations from a given meta-model of a RI, a
common hardware component which regulates the data-transfer between CPU and
peripheral devices. Then, we utilize methods of Statistical Analysis (SA) and
Machine Learning (ML) for evaluating the structure of the generated data. This
is done through an analysis of the features correlation, features clustering and
Mutual Information (MI): that captures statistical relatedness and
commonalities of correctly as well as incorrectly generated data. By training a ML algorithm
on correctly generated data, we are able to maximize the prediction accuracy
(R2 = 0:98) [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] and improve 600x the speed for obtaining the RI area. Overall,
this yields a set of con gurations, including corresponding estimates, for the
generation of hardware designs { addressing the shortcomings summarized above.
As a nal remark, this whole process is obtained in an automatic manner.
      </p>
      <p>Experimental evaluations con rm the bene ts of the proposed method. While,
thus far, designers mainly determined the needed con gurations by experience,
the proposed approach provides them with a powerful tool which automatically
suggests proper con gurations, and learn successively a correct objective
prediction (e.g. area, speed). Moreover, the SA and ML allows to hint to designers
characteristics of the hardware components which were not obvious in the rst
place just by manually checking out the single design instances.</p>
      <p>The rest of this paper is structured as follows. Section 2 reviews the applied
hardware designs generation ow based on MetaRTL and motivates the
problem considered in this work. Section 3 then presents the proposed method and
outlines how datasets of hardware design con gurations are generated, and how
methods of SA and ML are applied for analyzing their quality and correctness.
Finally, Section 4 summarizes the obtained results and Section 5 concludes the
paper.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Background and Motivation</title>
      <p>With the purpose of keeping this work self-contained, this section brie y
reviews the automation framework MetaRTL, which is considered in this work
as a representative of a model-based approach for generating hardware designs
starting from meta-models. Afterwards, we outline the main challenge of the
corresponding hardware designs generation ow, namely how to e ciently and
correctly obtain hardware con gurations compliant to the requirements of the
design (w.r.t. the complementary metrics such as area, speed, etc.). Determining
which con guration indeed satis es imposed constraints for the desired
objectives is a non-trivial task.
2.1</p>
      <p>
        The MetaRTL Design Flow
The MetaRTL framework (originally proposed in [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]) provides an environment
for hardware design generation. MetaRTL allows to de ne abstractions and
properties for each hardware component (e.g. the Register Interface, the Timer etc.),
and generates the design based on a chosen con guration. The automation and
abstraction of the design process leads to an increase in productivity and to a
rapid work- ow.
      </p>
      <p>The MetaRTL generation- ow follows the three level Model Driven
Architecture (MDA) abstractions, and is optimized for supporting the hardware
generation. The rst layer, called Model-of-Things (MoT), captures the abstract
formalized con gurations, which corresponds to the Computation Independent
Model (CIM) in the original MDA description. The second layer, called
Modelof-Design (MoD) corresponds to the Platform Independent Model (PIM) and
de nes the micro-architecture for the given hardware speci cation. This is the
core model for MetaRTL, as it de nes the hardware architecture. The third layer,
namely the Model-of-View (MoV) corresponds to the Platform Speci c Model
(PSM) and is the least abstract model with a mapping to the target view. It can
be conceptualized as a Abstract Syntax Tree (AST), de ned by an abstraction
of the target low-level design implementation.
2.2</p>
      <p>Open Problem: Determining the Desired Con guration
Using the MetaRTL ow described in Section 2.1, di erent hardware designs can
be generated from the same meta-model. To this end, a meta-model is de ned
rst in the MoT layer which provides a generic description of the system and/or
application to be realized. With this respect, the problem we address in this
paper is how to correctly and automatically map con gurations of hardware
designs (i.e. instances of the meta-model) to output predictions on a learned
function. In order to achieve this, we develop methods for generating potential
design con gurations and inspect their statistical properties. In the remainder
of this work we describe this process, as well as our contributions for improving
it, by means of the following running example.</p>
      <p>Example 1. We consider a Register Interface (RI) application as example. To
this end, a meta-model (as shown in Figure 1) is created. This de nes
dependencies and relations between the RI sub-components. The meta-model is
composed of four main classes, namely the Interface, the Unit, the Bit eld and the
Contained. The Interface class contains all the properties of the same,
including most importantly the DataWidth and the AddressWidth of each Interface.
Each Interface can contain one or more Units, i.e. logic entities which group the
Bit elds. Each Unit is indeed related to one or more Bit elds by means of the
Contained component, which are pointers to the Bit elds and allows to decouple
them to a single Unit usage. The Bit elds are the minimal information set that
we can read and write at once in the RI, and real center of the meta-model.</p>
      <p>With the meta-model available, the designer can instantiate di erent
hardware realizations by setting parameters, which we denote as input values (or
feature values ) for the prediction. With features we indicate instead each
attribute to which the input values are referred. The respective parameters are
thereby usually restricted through explicit constraints for avoiding completely
invalid results. Furthermore, the respectively chosen values will eventually a ect
the quality of the generated hardware designs e.g. with respect to area, speed,
etc.
Outputs</p>
      <p>Bit elds Size (X2)
No. Bit elds (X3)</p>
      <p>No. Units (X4)
No. HwRd (X5) 0
No. HwWr (X6) 0
No. SwRd (X7) 0
No. SwWr (X8) 0
No. Virtual (X9)</p>
      <p>No. DC (X10) 0
No. LUTs (Y1)</p>
      <p>No. SRs (Y2)
0
0
0
x5
x6
x7
x8
0
x10
x1
x2
x3
0
x4 u
x4 u
x4 u</p>
      <p>m
Example 2. Consider again the meta-model of the running example shown in
Figure 1. Instances of this meta-model are restricted by constraints as shown in
Table 1, e.g. the total number of instances of the component Unit is limited by
the value of the maximum number of Units, denoted with m, while the size of
the Unit is identi ed by u. This allows to reduce the single feature space and
adapt the generator to realistic use cases. Each feature may eventually a ect
the generate designs with respect to area, speed, etc. For the area prediction
of the RI component, we enlisted as outputs of the prediction Look Up Tables
(No. LUTs ) and the number of the Slice Registers (No. SRs) generated in the
synthesis process. These values in fact determine the surface of the implemented
RI component.</p>
      <p>Now, the challenge is how to properly determine the values for these features.
Although, as stated above, the con gurations values de ne the quality of the
generated design e.g. with respect to area, speed, etc., their relation is often not
obvious. More precisely, the designer is faced with the question, how to determine
a set of possible con gurations which eventually allow to optimize the desired
objectives (e.g. instantiating a realization with a certain area, power, etc.). This
problem has often a non-trivial solution: by increasing the number of features,
a higher number of combinations has to be considered. Some of these will be
considered as valid (e.g. No. SwRd + No. HwRd &gt; 0 etc.), some of them are
falling out of the valid con gurations subset (e.g. No. SwRd + No. HwRd = 0,
etc.). Finally, enlarging the features space may introduce non-obvious e ects,
complicating the interdependence in the data.</p>
      <p>In the following, we propose a solution to this problem. In order to ease the
corresponding descriptions, we thereby focus on determining proper con
gurations which generate designs satisfying an area prediction task (further objectives
such as speed can then be addressed in a similar fashion). As an output for the
prediction problem, we forecast the amount of LUTs and SRs generated in the
synthesis run. More speci cally, in order to optimize objectives of the design
con gurations (e.g. chip area, cost, speed, etc.), we need a set of generated data,
from which the prediction function can be determined. For these data to be a
valid input, they should be compliant to de ned constraints and present speci c
statistical relations among their features. In fact, as we generate data sharing
the same constraints for each dataset, these properties could be observed and
o er a glimpse on the correctness of the con gurations. Furthermore, these
conditions allow to learn, through a data driven approach, an accurate, robust and
representative approximation of the true design values. This is of high interest
for preventing the designer from selecting erroneous settings and carry them to
a further implementation process. This could indeed escalate in wrong designs
and poorly evaluated manufacturing products, which would lead to high costs
and expensive re-design solutions.
3</p>
    </sec>
    <sec id="sec-3">
      <title>Proposed Solution</title>
      <p>Hardware Synthesis tools are able to take as input the hardware design
generated from MoT les, and output the number of Con gurable Logic Blocks
(CLBs) and the implementation of the intended hardware function. From the
synthesis report, the area of the hardware design is identi ed by number of LUTs
and SRs that constitute the CLBs. This approach has two main drawbacks for
optimization purposes. First of all, it does not explicitly return the function to
be optimized. Second, it requires manual e ort and computation power, as the
process implies the synthesis of the hardware implementation of the whole
platform. ML algorithms can overcome these issues, by learning to map the design
con gurations to the desired prediction. This approach needs nevertheless
correctly generated data to learn from. Without those, it is impossible to have a
reliable data-driven algorithm. To this end, in this section we propose ML and
SA as viable methods for supervising the automated generation of data,
structuring a robust and rapid rst step for an optimization work- ow. This approach
overcomes the synthesis run for retrieving the area of the design and outlines a
prediction function for performing this task.
3.1</p>
      <p>
        The Problem of Area Forecast
In this paper, we generate con gurations as input data for a supervised
learning problem, in the form of a multiple regression. The output variables are the
No. LUTs and No. SRs, indicated as Y1 and Y2. The predictors, usually named
features, are denoted with X1; : : : ; X10 and are retrieved from the MoT con
gurations. The dataset is composed of n = 319 MoTs data samples and can be
represented as f(x(1); y(1)); : : : ; (x(n); y(n))g, where each x(i) = (x(1i); x(2i); : : : ; x(pi))T
is the vector of feature measurements for the i-th case. The amount of
generated data is dependent on the attening of the learning curve of the algorithm
for the area forecast: this shows that additional samples would not enhance the
prediction score of the ML model. For an in-depth description of the regression
problem and ML algorithms applied, we refer to this paper [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
3.2
      </p>
      <p>Data Generation and Features Space Exploration
In order to create a robust generator, we establish a range of possible feature
values given by the design required implementations. This means that the
congurations have to match with the design experience on application use cases.
As a consequence, the computational time for generating a representative
sampling is reduced. The con guration boundaries used for this paper, referred to
the RI design, are shown in the column Constraints of the Table 1. Even if the
constraints restrict the space of search for the optimal RI area, the number of
possible features combinations leads to a non-trivial problem. Our approach for
randomizing the data generator is to sample, within the constraints of the RI,
the Units and Bit elds properties. In addition to that, in each generation we
apply one out of four mapping functions, inspired by possible design use cases,
for setting the Bit elds inside the Units. The algorithms used to map the Units
are the following:
{ Compact mapping algorithm: dense Bit elds mapping inside each Unit;
{ Random mapping algorithm: random number of Bit elds mapped inside each</p>
      <p>Unit;
{ Basic mapping algorithm: one Bit eld mapped inside each Unit;
{ Combined mapping algorithm: jointly use of Basic and Compact mapping
algorithms.
3.3</p>
      <p>Analysis of the Generated Data
The automated data generation is a fast and exible way to retrieve input data
for any data-driven algorithms. This process nonetheless has some caveats. In
fact, the generation of wrongly constrained data is pernicious for an accurate area
forecasting. Further than that, the dataset generated for the learning model, in
case of non-trivial problems, should represent extensively the values of the
features provided. After building a data generator, in this paper we address thus
this problem: we present several approaches for testing the data-generator and
evaluating the correctness of the generated dataset. In our case the data
generator is implemented through a python script which samples, as aforementioned,
from a constrained features space.</p>
      <p>The rst step of the proposed solutions consists in measuring the correlation
among features for the generated datasets. This value indicates how much
features are dependent on each other, giving an intuition on how a feature value
changes according to other ones. In order to achieve this, we iteratively plot the
features values of the dataset, two at times: it results that the correlation among
features is mainly monotonic, i.e. either the variables increase in value together,
or as one variable value increases, the other variable value decreases. According
to these considerations, we compute the Spearman's Rank Correlation Coe
cient (SRCC) on the predictors, two at a time. The SRCC, denoted with ,
expresses the relation between the distance d between the feature components
xi and yi and the number of samples n. This coe cient provides an overview
of the features correlation inside the RI, and can be computed, if the ranks are
distinct integers, as
= 1
6 P d2</p>
      <p>i
n(n2
1)
(1)
- d: pairwise distance of the ranks of the observations xi and yi of the variables</p>
      <p>X and Y ;
- n: number of samples.</p>
      <p>As a second support for the identi cation of wrong constraints in the data
generator, we apply an agglomerative clustering algorithm, a subset of the
family of hierarchical clustering. This approach, from a data-driven perspective,
reconstructs the context of the given features, by grouping them together. This
procedure allows to match the designer's prior information about the features
used with the clusters retrieved in the data. As a rst step, we exclude the
presence of outliers after performing an analysis on the generated design features.
After an evaluation of di erent linkage methods on our data, we decide to adopt
the average linkage clustering. In fact, we aim to nd averaged statistics that
match the given design knowledge on the meta-model structure and this linkage
performs a meaningful clustering on the data. In the average linkage clustering,
the distance between clusters is computed as the mean distance among elements
(i.e. the mean of the distance d(x; y), corresponding to the modulus of the
difference of the two elements x and y) of each pair of clusters in the dataset, as
shown in the following equation</p>
      <p>D(H; G) =
1</p>
      <p>X
kG kH x2G;y2H
d(x; y)
where d(x; y) represents the distance between x and y, which belong to clusters
G and H, respectively.</p>
      <p>- d(x; y): distance between x 2 G and y 2 H;
- G, H: clusters;
- kG, kH : No. of elements, respectively in clusters G, H.
(2)
(3)</p>
      <p>Starting from a single cluster per feature, the algorithm iteratively merges
them until the desired groups number Q, set previously by a speci c
hyperparameter, is reached. As a criterion for the agglomeration, the algorithm merges
clusters where the mean distance between pair of elements is the least among all
clusters at each cycle. The nal outcome of the algorithm is a number of clusters
Q which include T features, with Q &lt; T.</p>
      <p>
        As a further approach to the analysis of the generated data, we introduce
a features selection algorithm for measuring how each feature in uences the
nal prediction of number of LUTs and SRs. We use ranking algorithms to
highlight the di erent importance of each feature for predicting the two output
responses. The corresponding importance is computed by means of the MI. This
metric, indeed, does not assume any monotonic relationship among features,
but considers only the degree of their relatedness. The ML algorithm applied
computes the MI from the K-Nearest-Neighbors (KNNs) statistics. This value
comes from an iterative process applied to the independent variable X (e.g. No.
Units, X4) and the dependent output variable Y (e.g. No. LUTs, Y1) in order
to quantify the impact of X on the uctuation of Y . In the paper [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], the KNNs
distance for computing the MI among two continuous variables is calculated
as the maximum distance between the projected datapoint xi and yi and the
corresponding next point x0i and yi0 on the subspaces X and Y . By means of
the KNNs distance, and by introducing the hyperparameter k as a result of a
grid search which considers the consistency of the model over permutations on
the features, we are able to compute the MI I(X; Y ), as shown in the following
equation
      </p>
      <p>I(X; Y ) = (k)</p>
      <p>1=k+
(sx) + (sy) + (n)
- I(X; Y ): MI between the variable X and the variable Y;
- : digamma function, de ned as (x) 1d (x)=dx;
- ::: : average over all i 2 [1; : : : ; n];
- sx: number of datapoints with jjxi xj jj x(i)=2, where x(i)=2 is the
distance from zi = (xi; yi) to the k-th datapoint projected in the X subspace;
- sy: number of datapoints with jjyi yj jj y(i)=2, where y(i)=2 is the
distance from zi = (xi; yi) to the k-th datapoint projected in the Y subspace;
- n: number of samples.
3.4</p>
      <p>
        Machine Learning for Area Forecast
We proceed, after the data generation and validation, into the area forecast
problem. ML provides a set of algorithms for functions approximation: this has been
often exploited in the hardware design literature for the forecast and
optimization of power consumption [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], SoC performance [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] and area of chip
components [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. To this end, parametric ML algorithms have been often preferred, as
they guarantee a fast and inexpensive computation of the predicted value
(inference), once the approximated function has been learned [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Since the restrained
dimension of the dataset, after comparing di erent ML algorithms (e.g. Random
Forest, Gradient Boosting, Linear Regression) for the area forecast, we select a
Multilayer Perceptron (MLP) as the best performing algorithm, as described in
[
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. The MLP corresponds to the simplest case of Feedforward Arti cial Neural
Network (FFNN) in which, each node is a neuron that uses a nonlinear
activation function [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. FFNNs provide a general framework for representing nonlinear
functional mappings between a set of input variables and a set of output ones.
Thanks to the non-high number of data samples, the monotonic variables
correlation and the restrained amount of features, we are able to perform a grid search
by considering as hyperparameter the No. Neurons of the unique Hidden Layer,
in a interval from 1 to 12. We perform a nested 4-folds Cross-Validation (CV),
which provides a ne tuning of the hidden layer size hyperparameter together
with a satisfactory and robust model accuracy.
      </p>
      <p>
        In the algorithm, we average out the prediction and hidden layer size for 30
di erent models (emodels = 30). Each model performs a nested 4-folds CV for
nding the best test scores (outer 4-folds CV), best parameters (inner 4-folds
CV) and grid search over the hidden layers size (inner 4-folds CV). As a solver,
after a comparative search, we apply a Quasi-Newton method, called L-BFGS.
The algorithm is based on the BFGS recursion for the inverse Hessian matrix
H. The L-BFGS method approximates the Hessian with a rst order matrix
with sparse vectors, in order to limit the usage of memory of the algorithm,
as shown in [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. As a regression score measurement, we adopt the coe cient
of determination R2, which provides a value for the explained variance of the
output variable.
4
      </p>
    </sec>
    <sec id="sec-4">
      <title>Results Evaluation</title>
      <p>With the scope of evaluating the SA and ML pipeline, we generate a
Baseline Dataset (BD) by implementing the aforementioned mapping algorithms
and other three anomalous datasets of 200 samples each. These latter cases
contravene the constraints imposed in the original random generator, and they
gradually enhance the severity level of the anomalies. We consider the following
anomalous datasets:</p>
      <p>{ ADI: dataset with an additional number of empty Units;
{ ADII: dataset where all Bit elds created are not readable by hardware
devices nor software;
{ ADIII: dataset where one or more Bit elds are included in the same Unit
several times.</p>
      <p>The anomalous datasets bypass in an incremental way the set of constraints of
the BD dataset: it is important to remark that the anomalies are present on the
dataset level (population), as we create each dataset with prede ned mapping
and constraints. Thus, we focus on dataset statistics, and not on single design
(individual) analysis. As a st approach to the analysis of the generated data,
we compute and plot the Spearman's between features. The outcome of the
analysis is a symmetric matrix. The SRCC is bounded in the interval 1 1,
where = 1 means a full-inverse correlation, = 0 indicates an absence of
correlation, and = 1 corresponds to a full-direct correlation. In Figure 2 we
plot the correlation values for the BD dataset. For the anomalous datasets,
we only provide some comments on the changed coe cients, by observing their
correlation matrix.</p>
      <p>As expected, we can observe here the intrinsic structure of the RI, in a
statistical fashion. Indeed, the min( BD) = 0:68 corresponds to the correlation
between Containeds Size and No. Bit elds. This shows the decoupling in the
RI between the Contained and the Bit eld component, so that a reference to a
same Bit eld can be added in di erent Units several times. For the anomalous
datasets, we point out the following principal changes. In ADI, all the Bit elds
properties have a low correlation w.r.t. the No. Units, as in the dataset there
exists an additional random number of empty Units. In the ADII, we notice that
there are no correlation values between the No. HwRd and No. SwRd properties,
whereas the No. HwWr and No. SwWr have tighter bound to the No. Bit elds
( = 0:98), as either the HwWr or the SwWr property needs to be True, for
the Bit eld to be valid. Last, we observe a correlation interval decrease in ADIII
(0 ADIII 1). In particular, the Bit elds Size and Containeds Size are fully
uncorrelated with the No. Units, since Bit elds and Containeds are redundant
in the Units ( ADIII = 0 and ADIII = 0). The correlation between features
(X1;X4) (X2;X4)
clearly varies, according to the entity of the anomaly in the data.</p>
      <p>As a second step for the features analysis, in order to reconstruct meaningful
groups of features in the dataset, we use an agglomerative clustering ML
algorithm with average linkage. The table shows the nal clustering results of the
algorithm. As number of nal clusters to be outlined, we select Q = 5. The value
of Q is chosen based on a features analysis of the RI meta-model since, from
design experience, it is possible to retrieve 5 features groups in this component.
As show in Table 2, for the BD, all features related to the Bit eld properties
compose the Cluster1BD. In the same way, other related entities concerning
the Units (e.g. Containeds Size (X1) and No. Units (X4)) are kept in the same
cluster (Cluster2BD), whereas independent properties are set in a separate
onefeature cluster. In the ADI, the No. Units (X4) are separated from other clusters
(Cluster2ADI ), whereas most of the Bit eld properties are kept together, as a
result of the basic mapping algorithm and the presence of empty Units. With
respect to the ADII, it is possible to observe the same cluster including the No.
HwRd (X5) and No. SwRd (X7) features, as the corresponding properties HwRd
and SwRd are both constantly set to False (Cluster3ADII ). In a similar fashion
the No. HwWr (X6) and No. SwWr (X8) contribute to the same Cluster1ADII ,
as the properties HwWr and SwWr are likely to be set to True. Finally, we
observe how in the ADIII the Containeds Size (X1) and Bit elds Size (X2) are
positioned by the algorithm in a same cluster (Cluster4ADIII ), whereas the No.
Units (X4) does not have any other agglomerated features (Cluster3ADIII ).
This shows how the repetition of Bit eld references in the Containeds lets their
relatedness to the Units diminish. The obtained results outline proximity and
distances among features for the BD, ADI, ADII and ADIII, and show the
different internal structure of the datasets. In particular we observe that, for the
BD, the algorithm can retrieve correctly all the assumed clusters of features in
the design, di erently than for the anomalous datasets case.</p>
      <p>As a nal step of the features analysis, we evaluate the importance and
ranking of each single feature w.r.t. the outputs of our regression by means
of the MI score. In Figure 3 it is shown how each single feature is related to
No. LUTs. The MI between predictors and the response variable No. LUTs is
very high for the BD, with a 0:75 M IBD 1:77. In particular, No. Units
and Containeds Size have the highest MI with the No. LUTs value. From a
design experience point of view, this seems feasible. Observing the MI of the</p>
      <p>ADI, the range is 0:60 M IADI 1:17. In particular, features with high MI
for the BD are not ranked similarly for the ADI (e.g. No. Units have the last
ranking position for the ADI dataset). Concerning the ADII, it is noticeable,
as by constraints, the absence of MI between No. SwRd, No. HwRd and the
No. LUTs. Furthermore, the features ranking appears di erent and the total MI
(M Itot) decreases (M ItBotD = 10.85, M ItAoDtII = 4.22). Finally in ADIII, because
of the di erent constraints of the dataset and anomalies produced, the total
MI equals M ItAoDtIII = 1.2, with about 9x reduction from M ItBotD w.r.t. the No.
LUTs.</p>
      <p>Fig. 3: MI on the No. LUTs</p>
      <p>We evaluate as well the MI between each one of the features and the response
variable No. SRs, as shown in Figure 4. Con rming the hardware design
knowledge, the most important feature in the M IBD ranking w.r.t. the No. SRs is
the Bit elds Size, that is the aggregated size of all the bit elds present in the
RI. The MI shared between the two variables is in fact M I(BXD2;Y2) = 1.76. As
a second feature for importance, the No. Bit elds has a M I(BXD3;Y2) = 1.43; this
ranking position matches as well with the designer knowledge. The M ItBotD =
12.57, whereas it results lower for the ADI, where M ItAoDtI = 10.52. The M ItAoDtI
is furthermore distributed in a more uniform way, showing that the di erent
degree of relatedness between M I(AXD2I;Y2) and between M I(AXD3I;Y2) is diminishing
the relative importance of X2 and X3. The reason behind that is the additive
number of empty units and the consequently lower bit eld density in the ADI.
The two remaining datasets have a much lower M Itot, respectively M ItAoDtII =
0.54 and M ItAoDtIII = 0.17. The lower values of MI w.r.t. the No. SRs are due
to the very little information between the single feature and the output. This is
particularly evident in the M ItAoDtIII where, adding repetitively the same bit eld
to a unit, the relatedness of the aggregated bit eld properties towards the units
and the nal value of No. SRs clearly decreases.</p>
      <p>The results obtained show that features ranking and M ItBotD diverge deeply
from the anomalous to the representative dataset. This serves as a valuable
metric for assessing the quality and representativeness of the generated designs.</p>
      <p>
        After ensuring the quality of the generated dataset, we implement the ML
algorithm for the area prediction, as described in detail in [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. As a rst step, we
compute a grid search for the MLPs network size over the emodels. The results
      </p>
      <p>Fig. 4: MI on the No. SRs
of the search show 9 No. Neurons 11. The R2 averaged score of the nested
CV (inner k-folds with k = 4, outer k-folds with k = 4) is R2 = 0:98 in the test
phase, which guarantees a satisfactory prediction score, as well as convergence
with the training score and robustness. The Root of the Mean Squared Error
(RMSE) in terms of LUTs, is 57, where 432 No. LUTs 3246 and the mean
No:LUT s = 1456.7. The RMSE in terms of SRs is 33, where 73 No. SRs
857 and the mean No:SRs = 432.3. In the inference phase, which consists in
the area prediction once the model is trained, the estimation is 600x faster than
the synthesis run for obtaining the RI area value.
5</p>
    </sec>
    <sec id="sec-5">
      <title>Conclusion and Future Works</title>
      <p>In this paper, we reviewed and analyzed ML and SA algorithms for supporting
the process of automated data generation in the design con guration. Through
the example of the RI, we could show how SA and ML can help in the correct
learning of a mapping function to the RI area in a certain constraints boundary,
and express useful metrics for pinpointing the validity and quality of the design
settings. Indeed, the algorithms used are able to detect incorrectly generated
datasets, which could lead to a skewed or invalid area prediction. This leads to a
fundamental contribution to the ML area estimation algorithm which, through
representative data, can forecast the RI area with R2 = 0:98 and 600x faster
than the design generation - synthesis cycle. As a future work, we plan to increase
the number of hardware components considered and approach further objectives
through ML algorithms. This would determine additional dependencies, but also
increase the potential of the proposed solutions. As the problem will grow in
dimensionality, we think that ML and SA may be even more valuable methods
for supporting hardware design con gurations, objectives optimization and for
understanding complex relations among design features.
6</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgements</title>
      <p>This project was partially funded by BMBF and supported by ITEA under the
umbrella of COMPACT.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>W.</given-names>
            <surname>Ecker</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Schreiner</surname>
          </string-name>
          .
          <article-title>Introducing model-of-things (mot) and model-of-design (mod) for simpler and more e cient hardware generators</article-title>
          .
          <source>In 2016 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC)</source>
          , pages
          <fpage>1</fpage>
          <lpage>{</lpage>
          6,
          <string-name>
            <surname>Sept</surname>
          </string-name>
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>Sanford</given-names>
            <surname>Friedenthal</surname>
          </string-name>
          , Alan Moore, and
          <string-name>
            <given-names>Rick</given-names>
            <surname>Steiner</surname>
          </string-name>
          .
          <article-title>Omg systems modeling language (omg sysml™) tutorial</article-title>
          .
          <source>In INCOSE international symposium</source>
          , volume
          <volume>18</volume>
          , pages
          <fpage>1731</fpage>
          {
          <year>1862</year>
          . Wiley Online Library,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>Seymour</given-names>
            <surname>Geisser</surname>
          </string-name>
          and Wesley O Johnson.
          <article-title>Modes of parametric statistical inference</article-title>
          , volume
          <volume>529</volume>
          . John Wiley &amp; Sons,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>Martin</given-names>
            <surname>Gogolla</surname>
          </string-name>
          , Fabian Buttner, and
          <article-title>Mark Richters</article-title>
          .
          <article-title>USE: A UML-based speci - cation environment for validating UML and OCL</article-title>
          .
          <source>Science of Computer Programming</source>
          ,
          <volume>69</volume>
          (
          <issue>1-3</issue>
          ):
          <volume>27</volume>
          {
          <fpage>34</fpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>Frank</given-names>
            <surname>Hilken</surname>
          </string-name>
          , Philipp Niemann,
          <string-name>
            <given-names>Martin</given-names>
            <surname>Gogolla</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Robert</given-names>
            <surname>Wille</surname>
          </string-name>
          .
          <article-title>Filmstripping and unrolling: A comparison of veri cation approaches for UML and OCL behavioral models</article-title>
          .
          <source>In International Conference on Tests and Proofs</source>
          , pages
          <volume>99</volume>
          {
          <fpage>116</fpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>Alexander</given-names>
            <surname>Kraskov</surname>
          </string-name>
          , Harald Stogbauer, and
          <string-name>
            <given-names>Peter</given-names>
            <surname>Grassberger</surname>
          </string-name>
          .
          <article-title>Estimating mutual information</article-title>
          .
          <source>Physical review E</source>
          ,
          <volume>69</volume>
          (
          <issue>6</issue>
          ):
          <fpage>066138</fpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Sankar</surname>
            <given-names>K</given-names>
          </string-name>
          <string-name>
            <surname>Pal and Sushmita Mitra</surname>
          </string-name>
          .
          <article-title>Multilayer perceptron, fuzzy sets, and classi - cation</article-title>
          .
          <source>IEEE Transactions on neural networks</source>
          ,
          <year>1992</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>Adam</given-names>
            <surname>Powell</surname>
          </string-name>
          , Christos Savvas-Bouganis, and
          <string-name>
            <surname>Peter Y. K. Cheung</surname>
          </string-name>
          .
          <article-title>High-level power and performance estimation of fpga-based soft processors and its application to design space exploration</article-title>
          .
          <source>J. Syst. Archit.</source>
          ,
          <volume>59</volume>
          (
          <issue>10</issue>
          ):
          <volume>1144</volume>
          {
          <fpage>1156</fpage>
          ,
          <string-name>
            <surname>November</surname>
          </string-name>
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>Nils</given-names>
            <surname>Przigoda</surname>
          </string-name>
          , Christoph Hilken, Robert Wille, Jan Peleska, and
          <string-name>
            <given-names>Rolf</given-names>
            <surname>Drechsler</surname>
          </string-name>
          .
          <article-title>Checking concurrent behavior in UML/OCL models</article-title>
          .
          <source>In International Conference on Model Driven Engineering Languages and Systems (MODELS)</source>
          , pages
          <fpage>176</fpage>
          {
          <fpage>185</fpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>James</surname>
            <given-names>Rumbaugh</given-names>
          </string-name>
          , Ivar Jacobson, and
          <string-name>
            <given-names>Grady</given-names>
            <surname>Booch</surname>
          </string-name>
          . Uni ed Modeling Language Reference Manual,
          <source>The (2nd Edition)</source>
          .
          <source>Pearson Higher Education</source>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Johannes</surname>
            <given-names>Schreiner</given-names>
          </string-name>
          , Rainer Findenig, and
          <string-name>
            <given-names>Wolfgang</given-names>
            <surname>Ecker</surname>
          </string-name>
          .
          <article-title>Design Centric Modeling of Digital Hardware</article-title>
          .
          <source>In IEEE International High Level Design Validation and Test Workshop</source>
          , HLDVT 2016, Santa Cruz, CA, USA, October 7-
          <issue>8</issue>
          ,
          <year>2016</year>
          , pages
          <fpage>46</fpage>
          {
          <fpage>52</fpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Jianlei</surname>
            <given-names>Yang</given-names>
          </string-name>
          , Liwei Ma, Kang Zhao,
          <string-name>
            <given-names>Yici</given-names>
            <surname>Cai</surname>
          </string-name>
          , and
          <string-name>
            <surname>Tin-Fook Ngai</surname>
          </string-name>
          .
          <article-title>Early stage realtime soc power estimation using rtl instrumentation</article-title>
          . pages
          <volume>779</volume>
          {
          <fpage>784</fpage>
          ,
          <string-name>
            <surname>Jan</surname>
          </string-name>
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Leonid</surname>
            <given-names>Yavits</given-names>
          </string-name>
          , Amir Morad, Ran Ginosar, and
          <string-name>
            <given-names>U</given-names>
            <surname>Weiser</surname>
          </string-name>
          .
          <article-title>Convex optimization of real time soc</article-title>
          .
          <source>arXiv preprint arXiv:1601.07815</source>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Elena</surname>
            <given-names>Zennaro</given-names>
          </string-name>
          , Lorenzo Servadei, Keerthikumara Devarajegowda, and
          <string-name>
            <given-names>Wolfgang</given-names>
            <surname>Ecker</surname>
          </string-name>
          .
          <article-title>A Machine Learning Approach for Area Prediction of Hardware Designs from Abstract Speci cations</article-title>
          .
          <source>In Proceedings of the 21st Euromicro Conference on Digital System Design</source>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Ciyou</surname>
            <given-names>Zhu</given-names>
          </string-name>
          , Richard H Byrd, Peihuang Lu, and
          <string-name>
            <given-names>Jorge</given-names>
            <surname>Nocedal</surname>
          </string-name>
          .
          <source>Algorithm</source>
          <volume>778</volume>
          :
          <article-title>Lbfgs-b: Fortran subroutines for large-scale bound-constrained optimization</article-title>
          .
          <source>ACM Transactions on Mathematical Software (TOMS)</source>
          ,
          <volume>23</volume>
          (
          <issue>4</issue>
          ):
          <volume>550</volume>
          {
          <fpage>560</fpage>
          ,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>