=Paper=
{{Paper
|id=Vol-1686/DemoPaper3
|storemode=property
|title=None
|pdfUrl=https://ceur-ws.org/Vol-1686/WSSSPE4_paper_33.pdf
|volume=Vol-1686
}}
==None==
https://ceur-ws.org/Vol-1686/WSSSPE4_paper_33.pdf
Demo: Composing, Reproducing, and Sharing
Simulations
Debashis Ganguly, William C. Garrison III, David Wilkinson,
Bruce R. Childers, Adam J. Lee, and Daniel Mosse
Department of Computer Science
University of Pittsburgh
Pittsburgh, Pennsylvania USA
Every year, research groups around the world contribute Our security simulator, Portuno, conducts cost analyses to
papers and artifacts to the computer science literature. In many explore the suitability of different access control approaches
areas, simulation and modeling play key roles in bringing for a given application workload. Portuno has been used
about these new contributions. Simulation is used to test and in an array of analyses, including evaluating group-centric
validate new ideas prior to their implementation, and thus, approaches to information sharing and exploring the commu-
the artifacts (software, data sets, benchmarks, etc.) used in nication, computation, and administrative overheads associated
simulation are fundamental to the empirical evaluation of a with cryptographic enforcement of role-based access controls
research hypothesis. (RBAC) on untrusted cloud platforms. Portuno uses proba-
Often, the primary focus of a paper is on the validation of a bilistic actor-based models of user, administrator, and system
central hypothesis or building proof-of-concept software, and behaviors to generate application traces (e.g., open file, edit
the details surrounding the artifacts used during and experi- file, share file, modify permissions, etc). These traces are then
ments used for this process are often scarce. Many researchers mapped into traces in concrete access control systems: those
do not intend to build a foolproof software component to share that are candidates for implementing the application. Costs are
with the community. Artifacts may end up limited in scope then aggregated over these candidate system traces. Portuno
or usability, and hidden or unstated assumptions may make supports a wide range of design choices in its actor models,
the artifact difficult (if not impossible) to reuse, extend, or initial system states, and other parameters of an experiment.
compose. Many artifacts take a tremendous amount of effort As such, openly sharing the choices that have been made and
to build and validate and, as such, may remain private to the allowing other researchers to modify these choices can lead
research groups that invested in developing them in the first to a better understanding of the trade-offs among different
place. This limits their availability, increases the difficulty of access controls techniques. Figure 1(a) shows the workflow of
validating claims made in papers based on these artifacts, and the composition of Portuno into OCCAM.
limits the ability of others to build upon prior work. To compose Portuno with other simulations, share the result-
Addressing this situation necessitates sharing and repro- ing infrastructure, and disseminate the experimental outcomes,
ducibility1 . While this problem cuts across most CS disci- Portuno is integrated with SST and incorporated in OCCAM.
plines, the modeling and simulation community has a unique SST acts as the driver of the underlying access control models,
advantage in addressing it. Namely, modeling and simulation which are implemented in Java. This is a novel use of SST as
rely on well-defined artifacts to carry out some activity; a a backbone for probabilistic modeling in an area other than
model, simulation component, initial conditions, input stimuli, computer architecture and computer systems simulation. It also
etc., must be specified and encapsulated in some form as part illustrates interoperability between SST and Java models.
of an evaluation. To this end, our participation at WSSSPE The combination of OCCAM, SST, and Portuno leads to a
2016 will concretely demonstrate our approach to sharing, seamless environment that is more capable than the sum of its
reproducing, and composing simulations toward accelerating parts. This integrated approach offers the capability to quickly
research productivity while also improving accountability and define, run, visualize, and share simulation artifacts and results
credibility. Specifically, we have developed a case study in over a huge design space. It supports an end-to-end workflow
which we compose and share access control simulations in the for modeling and analyzing access controls under a variety of
form of shareable data store units for cloud systems. This case scenarios, making it easier to (a) use Portuno for access control
study is openly hosted in the OCCAM collaborative repository analysis, (b) inspect and augment Portuno experiments done
(http://occam.cs.pitt.edu) and integrated with Sandia’s Struc- by others, and (c) modify Portuno in a contained environment.
tural Simulation Toolkit (http://sst-simulator.org). OCCAM allows for a dynamic environment where a re-
searcher can explore ranges of inputs and simulation results
1 Note that reproducibility in this paper is defined as experimentation that
by (a) specifying ranges and having the system automatically
is both repeatable and modifiable. generate, organize, and tag output results (see Figure 1(b)
This work is licensed under a CC-BY-4.0 license. for a sample of parameters, ranges, and web interface); note
(a) Workflow (b) Interface for parameters and ranges
(c) Error signaled in interface (d) Visualization 1
(e) Visualization 2 (f) Provenance
Fig. 1. Web interface of OCCAM
that Figure 1(c) shows what happens when a parameter is good candidate inputs for HMC simulators that could help us
specified incorrectly when a research is able to specify a range explore the potential benefits of this architectural enhancement
(e.g., number of iterations 4-20 while being a multiple of to speed up the management of files on untrusted infrastruc-
4, OCCAM generates automatically 4, 8, 12, 16, and 20), ture.
(b) visualizing the results of already executed simulations At the workshop, we will show how sharing, composing
(see Figures 1(d) and 1(e) for a sample of automatically and repeating simulations through a collaborative repository
generated visualization of results, which can be manipulated (OCCAM) and a general simulation framework (SST) can
dynamically through our web interface), and (c) requesting the accelerate our efforts as a community. Using our work on
system to extend the simulation runs for different input ranges. access controls as a case study, we will explain our technical
In effect, users of OCCAM can be researchers, developers, approach, how our integrated environment facilitates design
experimentalists, or curious users. exploration, and the potential of composing separate models.
Traditional digital archives for publishing experimentation, In the spirit of this abstract, interactive results obtained from
such as Open Science Framework and Dataverse, focus on Portuno, SST, and OCCAM are available at http://tinyurl.com/
sharing data without directly enabling reproducibility. Some hj2oewn.
archives, such as MyExperiment, specialize their systems fur-
ACKNOWLEDGEMENTS
ther by holding the source code and other offered information,
and introducing a means of visualizing the experimentation The material in this document is based in part upon work
through a workflow of that experiment. In contrast, OCCAM supported by the National Science Foundation (NSF) un-
goes several steps forward by retaining all of the data and code der grant numbers ACI-1535232, CNS-1305220, and CNS-
necessary for an experiment to be executed/run/reproduced, 1228697. Any opinions, findings, and conclusions or recom-
in addition to also giving a consistent means of visualizing mendations expressed in this material are those of the author(s)
the workflow of the experiment, deploying it, and viewing the and do not necessarily reflect the views of the NSF.
results. In the case of OCCAM, the workflow can be composed
of several simulators and not only of steps of a simulation.
Simulation results can be viewed and manipulated in a dy-
namic and interactive analyses, representing the “article of the
future”. Papers currently and recently have been disseminated
as PDFs with limited space, fixed content (e.g., a specific set
of results/graphs), and inadequate or incomplete details (e.g.,
missing setup, limited sweeps, etc.). With OCCAM, the results
will be integrated in the articles, which will be enhanced
to provide greater transparency, actual reproducibility, and
complete provenance of the results. For example, a reader can
click on a graph, and is taken to the digital library repository
of the data used to produce the graph, including the simulator,
the input data, the configuration files for the simulator, etc. See
Figure 1(f) for an example of the output of the provenance.
In addition, the reader will be able to extend a graph beyond
what is shown on the paper, to see trends and other further
results the reader wants to see, not only the (fixed) extended
results on a website provided by the authors.
Our seamless environment also enables the novel compo-
sition of simulators. In particular, we can combine Portuno
with other simulations. For example, we are currently investi-
gating how Microns hybrid memory cube (HMC) can decrease
the overheads associated with enforcing cryptographic access
controls in cloud environments. Recent simulations by our
team show that the administrative costs involved in altering
cryptographically enforced RBAC policies are prohibitive:
e.g., revoking a user from a single role may require thousands
of re-encryptions in even a moderately-sized organization.
The use of HMCs, perhaps combined with trusted execution
environments like Intel’s SGX, would allow users to push the
re-encryption to the data, rather than bringing bulk data to
the processor to re-encrypt. The administrative action traces
generated by various Portuno configurations would serve as