In the rapidly developing space of novel technologies that combine sensing and semantic technologies, research on smart products has the potential of establishing a research eld in itself. In this paper, we synthesize existing work in this area in order to de ne and characterize smart products. We then re ect on a set of challenges that semantic technologies are likely to face in this domain. Finally, in order to initiate discussion in the workshop, we sketch an initial comparison of smart products and semantic sensor networks from the perspective of knowledge technologies.
The increased availability and robustness of sensors, the wide-spread use of the
internet as a communication environment, as well as intensi ed research in the
area of the semantic web, as the ultimate promise for fostering interoperability
of large-scale, heterogeneous data sources have lead to the de nition of various
research trends that draw on the advances in these three technologies. Pervasive
computing is the trend towards increasingly ubiquitous, connected computing
devices in the environment. Ambient Intelligence (AmI) de nes a vision where
distributed services and computing devices, mobile or embedded in almost any
type of physical environment (e.g. home, o ce, cars), all cooperate seamlessly
with one another using information and intelligence to improve user experience
[
In this paper, we investigate the notion of smart products, i.e. a concept that
relies on the above mentioned technologies (sensors, pervasive computing, smart
environments, internet, semantics) and has the goal to in uence the entire
product life-cycle towards innovative economic developments and e ective business
models. Di erent and more focused then the generic view on sensor networks,
and going beyond living spaces and buildings, this promises to be a technology
with obvious industrial value. Therefore, we agree with other authors that the
concept of smart products \has the potential to establish a new research eld
with unique questions from the standpoints of economics, marketing
communications and computer science" [
According to these questions, the major contributions of this paper are threefold. Firstly, we synthesis the most important de nitions and characteristics of smart products that are available in the literature to date (Section 2). Secondly, we distill and re ect upon the challenges that face the application of semantic web technologies in this context. These challenges have been derived as a side-e ect of a major requirements analysis phase in the context of the SmartProducts4 European project, which involves industrial partners from various application domains: EADS Innovation Works, Philips Research, Centro Ricerche FIAT (Section 3). Finally, based on our analysis, we sketch an initial comparison between smart products and semantic sensor networks (Section 4).
Note that our analysis complements the ndings of similar works. For
example, our views are more focused and grounded than the generic research problems
detailed in [
In this section we provide an insight into various views on the de nition of smart products (Section 2.1), we describe an example scenario (Section 2.2), and synthesize a core set of characteristics for these products (Section 2.3). We then use these characteristics to derive concrete technological challenges in Section 3. 4 http://www.smartproducts-project.eu/
The concept of smartness in products has been investigated by several authors. Here we present a synthesis of the most in uential works.
As early as 2005, Allmendinger and Lombreglia investigate the notion of
smartness in a product from a business perspective [
More recently, as part of the AMI'07, [
[
One year later, in 2008, [
The SmartProducts consortium has adopted and modi ed the de nition
given in [
\A smart product is an autonomous object which is designed for selforganized embedding into di erent environments in the course of its life-cycle and which allows for a natural product-to-human interaction. Smart products are able to proactively approach the user by using sensing, input, and output capabilities of the environment thus being self-, situational-, and context-aware. The related knowledge and functionality can be shared by and distributed among multiple smart products and emerges over time." 2.2
We consider the domain of consumer electronics and domestic appliances as a scenario to exemplify the notion of smart products. This domain is characterised by devices that are used for speci c purposes in the home. Often, these devices are designed to work alone and have a limited lifespan, either through feature-based obsolescence (function), or due to changing aesthetics over time (form/fashion). Such devices are unaware of the user that operates them and require external documentation to support the user in learning how to use and maintain them. In contrast to the current status, smart products in this domain are aware of their life-cycle and also of the users that they interact with. They are able to discover and interact with each other, sharing information, resources and services.
Consider, for example, a Smart Kitchen which contains a set of smart products (domestic appliances) such as a Steamer, Measuring Scales and a Toaster. The Steamer and Measuring Scales can communicate with each other, such that when a piece of food is moved from the scales to the steamer, the steamer is \told" how much (and the type of) food that has been put into it. This involves communication between the two devices in a way that the semantics of the measurement (e.g., 115 grams of sh) is consistent between the two devices. When the steaming has nished, the steamer discovers another smart product in its network that has a multimodal display and uses it to communicate its status to the user.
In the case of the toaster, when a user approaches the device and places a croissant on top of its warming rack, the toaster is able to automatically determine the appropriate actuation (warming the croissant at the right heat level) utilizing the user identity and \learned" preferences for warming time and temperature. Furthermore, this context information (the user is near the toaster) is made available to other smart products and services in the home, which are free to act upon it as they see t.
Smart products can also have an in uence on the other stages of their lifecycle besides the actual usage stage. For example, when they are no longer needed by the user, they can be delivered to a recycling centre. Information about the product and its usage, which is available on each smart product, can guide the centre in using the appropriate procedures for their recycling/refurbishment. Furthermore, as part of their operation process, smart products can also provide the manufacturer with usage information associated with their in-service period. This information can be useful when analysing how products are actually used when in-service and can guide future product development processes. 2.3
In this section we synthesize the major characteristics of smart products by
comparing their features that were considered as essential by the above mentioned
works. For example, Maas and colleagues distinguish and explicitly state six
major characteristics for smart products [
The SmartProducts consortium has identi ed the following set of characteristics: Autonomy: Smart products need to be able to operate on their own without relying on a central infrastructure. This is, for example, the case of the smart kitchen devices in our example scenario which interact with each other and the user without the need of central control. Situation- and context-aware: Smart products are able to sense physical information (e.g., via a temperature sensor), virtual information (e.g., about the current state in the cooking process maintained by another smart product) and to infer higher level events from this raw data (e.g., the user has nished cooking). These \higher-level events" are often referred to with the term \situation". Situation and context information allow smart products to adapt their interaction with other products and users accordingly, as well as to infer new knowledge.
uct is able to embed itself into an existing smart product environment and to automatically build a smart product environment. For example, a newly acquired smart product such as a rice boiler should be capable of easily embedding itself into the smart kitchen described above.
Proactively approach the user: The situation information is used to decide when the smart product should proactively approach the user, e.g. for providing additional information or for assisting him in performing a task. Indeed, in our example scenario, when an exceptional situation is detected by a smart product (e.g., it requires some maintenance or cleaning), the smart product can pro-actively interact with the user, potentially through multimodal interaction (see below). Note that proactivity should also characterize the interaction with other products, e.g., the Measuring Scale proactively interacts with the steamer when food is transfered between the two products. Support the user throughout whole life-cycle: The particular life-cycle stage of a product has a major in uence on its behavior. For example, a worker in the production phase needs access to other functionalities (and uses a di erent terminology) than an end-user during the usage phase. In our example scenario, di erent smart product features are relevant for di erent life-cycle stages: the ability to sense the user context is crucial during the usage phase, while providing information about itself and its usage history is needed during the recycling phase.
Multimodal interaction: Smart products should provide a natural interaction, however most smart products have only limited in- and output resources. For that reason, the smart products are able to make use of the di erent input and output capabilities in their smart product environment supporting the usage of various modalities (e.g., speech, pointing). Smart products can discover multimodal user interface services in the network and can make use of them as need be. Examples include networked displays, microphones, speakers, etc. This is, for example, the way in which the steamer communicates its status to the user.
Support procedural knowledge: Many interactions with smart products are based on a procedure, e.g. descaling a co ee machine. Therefore, smart products need to support procedural knowledge, including how the user needs to be involved in the di erent steps and how implicit interaction (e.g., inferred from context information) can be integrated in the procedure, e.g. recognizing when the user has completed a step in the procedure. The supported procedures are thereby not limited to one single smart product, the procedures can also be dynamically composed of procedures provided by several smart products. For example, in the example scenario, a cooking guide could control the overall cooking process, but parts like boiling water can be outsourced to other smart products which are available in the smart kitchen. Emerging knowledge: Smart products learn new knowledge from observing the user, incorporating user feedback and exploring other external knowledge sources like Wikis. They are thus able to gather a more accurate user model and to learn new procedures. Our example scenario illustrates how user preferences are learned and utilized over time, for each individual user (e.g., with the toaster temperature and time when warming the croissant). Distributed storage of knowledge: Many smart products have only limited storage resources, thus they need to outsource their knowledge to other smart products in the environment. The user pro le, as an example, is part of the knowledge that needs to be stored in a distributed way. This enables smart products that just enter a smart product environment to bene t from the information that was gathered so far. Another scenario where distributed storage is required is commissioning, i.e., if one product is broken and has to be replaced by another. The distributed storage enables that the new smart product can be initialized with the knowledge of the old smart product and thus does not need to learn everything from scratch.
Table 1 provides a comparative presentation of the main characteristics of smart products derived from the literature. As can be seen from this alignment, the major characteristics on which all sources agree are: context-awareness (the ability to sense context), proactivity (the ability to make use of this context and other information in order to proactively approach users and peers) and self-organization (the ability to form and join networks with other products). In addition to these characteristics, Muhlhauser and the SmartProducts consortium emphasize the fact that smart products should support their entire life-cycle as well as that special care should be devoted to o ering multimodal interaction with the users, in order to increase the simplicity characteristic of the products.
In addition to these jointly agreed characteristics, Maas and colleagues highlight the need for using context information in order to support personalization and adaptiveness. They also see products as being aware of concrete business and legal constraints. While these characteristics are not stated explicitly in the other two works, they do not contradict the SmartProducts view. Similarly, the SmartProducts consortium identi ed some additional characteristics to those provided by Maas and colleagues. Most importantly, products are seen as capable of acting autonomously (by themselves) without the need of central control. The rest of the characteristics refer to aspects of the knowledge component that enables the smartness of the products. This knowledge has an important procedural component, it should evolve during the life-cycle of the product as a side e ect of its interaction with users and products and, nally, it might need to be stored in a distributed fashion in order to overcome the resource limitations imposed by some products. Maas et Al.
Situatedness
Pro-activity Network ability
Personalization Business-awareness
Adaptiveness
Muhlhauser et Al.
Context-aware
Proactive Behavior Self-organized embedding
Support the entire
life-cycle Multimodal Natural Interfaces
SmartProducts cons.
Situation- and context-aware Proactively approach the user
Self-organized embedding in smart product environments Support the user throughout
whole life-cycle Multimodal interaction
Autonomy Support procedural knowledge
Emerging knowledge Distributed storage of knowledge
Knowledge technologies play a crucial role in the realization of the smart products. In this section we discuss some of the challenges that such technologies are likely to face in this novel context. We derive these challenges based on the main characteristics of smart products presented in the previous section. They are: Dealing with suboptimal data quality: A fundamental characteristic of smart products is that they rely on context information obtained from associated sensors which is then translated into higher level semantic information. While an important part of research focuses on this translation, the resulting semantic information is likely to have a lower quality than manually authored and checked semantic information. For example, the derived data could be incomplete or, on the contrary, contain redundant elements. Therefore it is important to develop semantic techniques that are robust enough to be able to process this data.
Representing a variety of information: Researchers investigating semantic
sensor webs generally agree that semantic models are needed for representing
information about time, space and the domain relevant for the sensors. From
our analysis of smart products and their characteristics, we can conclude that
their representation needs are much richer and more diverse. Indeed, at a
minimum, knowledge associated with smart products should contain user
models, task models (procedural knowledge), models to represent life-cycle
stages and the main users (or communities of practice) involved in each stage,
interaction models. Therefore, the employed semantic technologies should be
able to cover all these representation needs.
Providing complex reasoning algorithms: Smart products use reasoning
mechanisms on their rich knowledge bases in order to adapt to user needs,
to perform personalization and to proactively interact with users and other
products. This complex expected behavior will require sophisticated
reasoning mechanisms such as diagnosis or planning. Such reasoning is much more
ambitious than current work in the area of sensor networks which
primarily relies on subsumption matching (e.g., for matching between available
resources and tasks) [
Dealing with hardware resource limitations: Physical products are heterogeneous in terms of their hardware resources for information storage and processing. However, even the most powerful products will lag behind the resources characteristic to the computer machinery for which semantic technologies are currently built. This requires a considerable e ort in scaling down semantic technologies so that they could run on products with limited resources. This could include reducing the storage space needed for semantic resources as well as optimizing some of the tools for products with limited resource. Additionally to the adaptation of semantic technologies to limited resources, it is important to investigate smart product infrastructures that could \load-balance" between the complex and simple devices (e.g., by the distributed storage of knowledge).
Supporting emergent knowledge: It is envisioned that smart products will continuously update their knowledge bases by deriving knowledge as a side e ect of their interaction with users and other products. Therefore, mechanisms for supporting the derivation of this knowledge need to be built. Ensuring trust and privacy: Given their close interaction with users, smart products need to maintain a considerable amount of information about users including their likes, dislikes, their usage patterns, their personal information etc. It is therefore crucial to implement access rights mechanisms that can ensure the desired level of privacy for user data distributed across multiple products.
experience while working with industry partners is that it is still a challenge to integrate semantic technologies into existing systems used to manage industrial processes and data warehouse systems. Therefore support tools are required for the adoption of semantic technologies in this domain.
geneity of information involved in smart product applications and services requires innovations in the area of scaling data collection, accurate and fast searching and information combining techniques, to be able to meet original application requirements, even with a considerable increase in the number of interacting objects.
How do Smart Products Compare to Semantic Sensor Networks? In this section, we conclude our analysis of smart products by comparing them to the concept of semantic sensor networks.
According to the de nition given in [
Currently there is a signi cant research e ort in the semantic sensor web
community which focuses on developing standards for semantic representation
of raw data. In [
Formally represented knowledge plays a key role both in semantic sensor networks and in smart products. The di erent characteristics of the two systems lead to some core di erences in the processes related to the acquisition, representation and processing of this knowledge.
By knowledge acquisition we mean the process of gathering raw data from sensors and transforming it into semantic representations. This is a core process in both technologies. In the case of some semantic sensor networks, especially those that are composed on the y to meet emergency situation (as is the case of the recently started SemsoGrid project), the mapping between raw data and semantic structures is performed dynamically, at run-time. In contrast, for smart products, this process should be much more controlled as the available sensors and semantic structures are known at design-time. Therefore, their mapping can be performed at the design-time of the product.
The choice of the appropriate knowledge models must be made when representing knowledge. In most semantic sensor network scenarios mentioned above the focus was on obtaining information in a convenient form, and determining actions was assumed to be a separate stage. However, the main goal of smart products is to perform their speci c actions to satisfy the end user's needs (e.g., to cook a dish), and sensing capabilities are just auxiliary means to support their main functionality. In order to achieve that, smart products potentially need to participate in complex work ows consisting of several steps and involving several devices. Thus, the higher-level context of smart products involves not only four types of ontologies used in SSNs (spatial, temporal, thematic, sensor), but also the models of users with their goals and the models of processes and work ows, in which a product can participate. We therefore conclude that smart products will need a higher variety of ontological models than SSNs. At the same time, we think that both research areas can bene t from each other in this aspect. On the one hand, smart products can reuse modelling standards for temporal and spatial information that are researched within SSNs. On the other hand, appropriate ontologies and reasoning mechanisms can potentially be reused in a wider sensor network, in particular, where a sensor network is deployed to monitor a structured process or where information is gathered by multiple autonomous sensor platforms (e.g., unmanned aircrafts).
The ways in which the obtained semantic representations are used (knowledge processing ) can also di er. Firstly, in semantic sensor networks the semantic data is typically aggregated and processed in a centralized fashion. In contrast, in the case of smart products, the storage of obtained semantic data and reasoning about it are mostly performed locally on each product, thus distributed over the members of a smart environment. This leads to the second di erence relating to hardware resource limitations. The central processing node in semantic sensor networks usually provides the hardware resources for which semantic technologies are designed. In contrast, individual smart products have considerably less storage and processing resources. This means that the reuse of existing tools for semantic data processing (e.g., data stores such as Sesame5 and reasoners such as Pellet6) can be complicated. Specially developed tools and intelligent algorithms for distributed data processing are required: e.g., it has to be decided whether a particular observation has to be stored or deleted, when several observations can be aggregated, and which node of the network can perform a reasoning task. These solutions could be bene cial for other sensor network applications where mobile and autonomous sensor platforms are employed. 5
In this paper we investigated the notion of smart products, a novel line of research that combines sensing and semantic technologies.
When synthesizing current de nitions for smart products, we found di erent views of what smart products are. As a common denominator, by dismissing variations introduced by particular views, we conclude that smart products are context-aware (by relying on sensing technology), they have a proactive behavior (ensured by formal reasoning on the represented context data) and are capable of networking with other products (by making use of communication technologies). 5 http://www.openrdf.org/ 6 http://clarkparsia.com/pellet/
We then deduced that these characteristics raise a set of challenges for semantic technologies that are applied in this context. In particular, semantic technologies will need to provide representation support for a wide variety of informations (going well beyond the time, space and thematic ontologies employed in SSNs) and reasoning mechanisms should allow for a sophisticated proactive behavior while being robust enough to deal with potentially low quality data obtained from sensors. Additionally, the resource limitations associated with physical products put further constraints on semantic technologies and require their optimization. Challenges also arise in supporting the emergence of new knowledge as a side e ect of the product's interaction with users and peers, in ensuring trust and privacy for the user's data and in providing the appropriate tool support for integrating these technologies into an industrial setting.
Based on our analysis, we conclude that, while smart products can be seen as a specialized case of semantic sensor networks, from the perspective of knowledge technologies, the two technologies di er in several aspects. We see these di erences as providing a fertile ground for collaboration between the two research directions.
Part of this research has been funded unter the EC 7th Framework Programme, in the context of the SmartProducts project (231204).