We present a prototype web service that enables researchers to evaluate the performance per watt of semantic web tools. The web service provides access to a hardware platform for collecting power consumption data for a mobile device. Experiments are specified using RDF to define the conditions of the experiment, the operations that compose those conditions, and how they are combined into individual execution plans. Further, experimental descriptions and their provenance are published as linked data, allowing others to easily repeat experiments. We will demonstrate how we have used the system to date, how others can use it, and discuss its potential to revolutionize design and development of semantically enabled mobile applications.
One challenge that semantic technologies face when deployed on mobile platforms like smartphones is the amount of energy available for the device to compute and communicate with other agents. For example, the Google Nexus One, one of the first Android smartphones, had a single core processor operating at 1 GHz and 512 MB of RAM. Samsung's latest offering, the Galaxy S5, has a quad core, 2.5 GHz processor and 2 GB of RAM, more than a 8-fold increase in processing power and 4-fold increase in memory in 5 years. However, the battery capacity of the two phones are 1400 mAh and 2800 mAh, resp., which indicates that battery technology is progressing more slowly than processing technology. Further, application complexity has also increased. Tools are needed to help developers understand how semantic tools consume power so as to identify when they can use local reasoning on mobile platforms or when off-device computation is more practical.
We introduced a broadly reusable methodology [3] motivated by these concerns to evaluate the performance of reasoners relative to the amount of energy consumed during operation. Ultimately, these metrics will provide developers deeper insight into power consumption and enable next-generation applications of semantic technologies for power constrained devices. We present a prototype ontology-driven web service for researchers to use our reference hardware setup to perform analysis of semantic web tools' power consumption.
Our power benchmarking methodology [3] bypasses the removable battery in a Samsung Galaxy S4 to collect power data during reasoning and query answering tasks using three reasoning engines. Because our methodology requires a hardware setup, we are developing and will demonstrate a web service to execute experiments using our existing infrastructure. The web service is based on the Semantic Automated Discovery and Integration (SADI) Framework1 and accepts jobs described using RDF and the ontology we will discuss in Section 3. On completion, it provides a ZIP file containing runtime information, raw and processed power measurements, power and energy consumption statistics, and provenance capturing information about the process. Table 1 shows a sample data point for each of three different reasoners on the Lehigh University Benchmark [2], query 14 (shown in Listing 1.1).
We will demonstrate our experimental ontology for declaratively describing the operational constraints of an experiment, which is then executed on the target device. The experiment description is published as linked data, along with metadata about the experiment output, andprovenance modeled using the PROV-O ontology [1].These metadata are published in a triple store to enable metaanalysis, recombination, and extension of power experiments.
Experiment. Experiment provides the root of an experiment description. Listing 1.2 shows an example experiment. Core to an experiment description are conditions, which are grouped together based on some common dimension. If an experiment defines more than one condition group, the engine performing the experiment can generate specific conditions through the use of a condition-FillStrategy. We are currently investigating two different strategies, CrossJoin and Paired, that evaluate the cross product and paired conditions across groups, respectively. To provide control over what data are returned, the author of the experiment can declare which variables are of interest. Operations. Operations encapsulate actions to be performed on the experimental device. In Listing 1.4, we show an example of how an operation would define tests for the LUBM query set. The measurePowerDuringDownload property can be used to evaluate the performance of communication channels while retrieving the content required for performing the experiment. In addition to loading ontologies, datasets, and executing queries, our ontology supports modeling reasoners, parallel and sequential operations, and randomization of operations.
As semantic technologies become more prevalent, we need to ensure that tools are available to assist in their deployment on a variety of devices including mobile platforms, which are often power constrained. While our initial work provides direction for this effort, we recognize that more widespread adoption requires lower barriers to entry. We described a web service under active development to provide access to a reference implementation of the hardware described in [3] where we found that, while compute time accounts for most energy consumption, significant memory consumption may affect power consumption during reasoning. With this investigation, we are working to enable semantic web researchers and implementers to obtain insight into the power requirements for semantic technology stacks. We will demonstrate a variety of example experiments and discuss broader usage with attendees. In future work we intend to use expand this web service to provide a means of easily repeating experiments as well as further support for modeling the execution of experiments using We also intend to provide example code that present an analysis of more reasoners, e.g. by utilizing the work in [4].
Mr. Patton was funded by an NSF Graduate Research Fellowship. RPIs Tetherless World Constellation is supported in part by Fujitsu, Lockheed Martin, LGS, Microsoft Research, Qualcomm, in addition to sponsored research from DARPA, IARPA, NASA, NIST, NSF, and USGS.