The notion of preferences plays an important role in many disciplines including service robotics which is concerned with scenarios in which robots interact with humans. These interactions can be favored by robots taking human preferences into account. This raises the issue of how preferences should be represented to support such preference-aware decision making. Several formal accounts for a notion of preferences exist. However, these approaches fall short on defining the nature and structure of the options that a robot has in a given situation. In this work, we thus investigate a formal model of preferences where options are non-atomic entities that are defined by the complex situations they bring about.
Robots have yet to reach the level of cognitive capabilities required for use-cases in the service sector where they interact with humans, for example as shopkeepers, waiters or general assistants. Through these interactions a course of events ensue that can be more or less suitable to the wishes and intentions of the human. The robot may face different options that would each bring about a different situation where some individual outcome might be preferable over others from the perspective of the human. However, the actual preference of humans over different options in a given situation is highly context-dependent. A human might, for example, prefer cold over hot drinks on summer days, while preferring hot drinks like tea and coffee when eating a cake.
The concept of preferences is relevant in many disciplines ranging from philosophy, economics
and social science to robotics. A unified understanding of preferences across those disciplines
has however yet to emerge. A significant step towards this goal by means of a logical language
is the framework of preference logic [
In this work we, thus, seek to endow our cognitive robotic agents with the conceptual inventory needed to reason about the possible preferences of the involved human agents in the light of the resulting situation for a given action path. To this end, we introduce a formal model of preferences that is purposed as a means for reasoning and ultimately selecting a specific course of actions over alternative ones with respect to the given user preference at hand. The model is designed along a set of competency questions that each corresponds to a robot capability required in the scope of human-robot interaction scenarios. We use the framework of description logics for the formalization of the model which has the advantage of having a trivial implementation in the Web Ontology Language (OWL). The resulting ontology implementation is then used for the validation of the model by showcasing that the competency questions can be implemented in form of queries within a standard language tool.
Summarizing, the contributions of this work are the following ones: • a formal ontological model of user preferences where options are characterized through situations they bring about; and • the alignment of this model with an existing robotics domain ontology.
Taking user preferences into account in an agent’s decision-making system is fundamental for
ensuring an enhanced user experience [
There exist several types of recommender systems, each employing different techniques to
achieve personalized suggestions. Collaborative Filtering [
According to [10] three types of knowledge need to be modeled, including : user knowledge, item knowledge and the contextual knowledge relating the item to the user’s needs. This information can be modeled using ontologies. A robust ontological model is the SOMA model. SOMA provides a structured framework for effectively capturing and representing the contextual knowledge that influence human actions and interactions across diverse everyday situations.
In [11] , a model is proposed to bridge the product design gap between customers’ unconstrained and unstructured high-level preference concepts (e.g. interviews, design logbook, text description) and the specific constrained and structured low-level features (e.g geometric shapes, CAD drawing, final reports) associated with those concepts. The main objective of the paper is focused on retrieving and generating preference concepts. The authors define preference as a subjective property that emerges from customer behavior, representing their desired choices. The model begins by semantically extracting customer preferences from their reviews. These extracted preferences are then subjected to part-of-speech (POS) tagging using semantic rules based on concepts outlined in a customer preference ontology and a preference lexicon.
Another related approach, discussed in [12] , aims to connect a preference model and an ontology-based database. The authors introduce a preference resource concept “property_or_class” within their preference model, which can be linked to the classes or properties defined in the ontology model. Their proposed preference model differentiates between interpreted and noninterpreted preferences. Interpreted preferences can be assigned an interpretation by evaluating their interpretation function. On the other hand, non-interpreted preferences are represented through an enumeration of a set of properties and classes selected from the ontology without any constraints.
In this work, our objective is to develop a preference ontology that can effectively address situation-dependent questions concerning user preferences. It is important to note that our focus lies in establishing the ontology itself, rather than learning the preferences. Therefore, for the purpose of presenting the ontology and its reasoning capabilities, we consider the preferences a-priori knowledge. However, in real-world scenarios, the required preferences could also be derived through learning algorithms. To showcase the supported reasoning of our proposed ontology, we have selected four competency questions that highlight its capabilities. The proposed competency questions are: CQ1 What are a user’s preferences related to a particular situation? (E.g., what information do we have about a user’s ordering of breakfast dishes they might prefer.) CQ2 Which option would the user prefer from those that are about a given situation? (E.g., what would the user prefer to eat for breakfast.) CQ3 Which preferences can be fulfilled by the agent? (e.g. we have no sugar, but other sweeteners that we can offer) CQ4 Can preferences be inferred from other preferences? (E.g., preferring an effect, therefore preferring its cause)
We decided to extend the formal OWL ontology Socio-physical Model of Activities (SOMA) [13]. Our reasons are three-fold. First, SOMA builds upon the foundational ontology DOLCE+DNS Ultralite (DUL) [14], which contains patterns to represent hypothetical worlds conceptualized as Situations and to model Descriptions thereof [15]. As we will explain in more detail in Section 4.2, these will be of utter importance for our model of preferences. Second, SOMA was recently extended to not only model physical experiences, for which it was originally designed, but to also capture metacognitive experiences [16]. It thus already contains concepts such as Decision-Making, Option and Choice, on which preferences are inseparably dependent. Third, SOMA contains an elaborate model of Dispositions. As shown later, this will be useful for the definition of preferences.
DUL, adopting a descriptive world view, consistently distinguishes PhysicalEntities from SocialEntities that exist “for the sake of [. . . ] contextualizing or interpreting existing entities” [14] such as Description and Concept. Accordingly, a physical Event (an objective occurrence) executes an EventType (a subclass of Concept and thereby a subjective interpretation of said event), that isDefinedIn some Description such as an observer’s Narrative or an emerging Affordance. More concretely, a movement may execute a cutting Task.
DUL applies the above principle to other common ontology patterns such as Roles, which it also views as a subclass of Concept and therefore as defined by some Description. Any object x that – objectively – isParticipantIn some Event that executes a Task y may be associated with a subjective Role linking to x via hasRole and to y per hasTask. This allows for contextualizing entities (see, e.g., [17] for details), such as differentiating a cutting tool from the object being cut, or Options from the Performer and the Choices in DecisionMaking.
We first discuss the agentive behaviors in which preferences manifest, i.e., the decision-making processes they guide. SOMA views Decision-making as a MentalTask in which only an agent’s mental representations (InformationObjects that play the roles of, e.g., Premises and Conclusions) are manipulated (c.f. [16]). We can concretize that the InformationObjects behind Premises (Conclusions) contain information about the available Options (selected Choices). For the remainder of this work, however, we will abstract away from this pattern and only consider the Options and Choices as roles played by the considered entities during Decision-Making.
Although some work exists on representing the comparative relationships between different options (c.f. [18]) we are unaware of any models of the options themselves that can answer questions such as, what is the ontological type of options, and how do they relate to other entities. Such a model, if to be applied in flexible scenarios, would have to be of considerable complexity. For illustration purposes, consider the following question:
“Would you like coffee or tea?”
Assuming that neither coffee nor tea is prepared implies that the options are not concretely existing, but hypothetical by nature. Notice also that, as per social norms, the question marks the beginning of a sequence of events in which the questioner prepares the chosen beverage and transfers it to the addressee (along with an exclusive claim of consumption), who eventually drinks it. By these pragmatics, the options are not the hypothetical beverages or even the hypothetical possession thereof, but the hypothetical consumption. This view is consistent with typical characterizations of preferences (c.f. [18]). If this is to be captured by our representation of options, we have to view them as the roles played by hypothetical worlds – in DUL terms, as Descriptions, Situations may satisfy and which play the roles during decision-making. These observations lead to the following definition: Definition 1. Decision-making is a mental task in which the performers select from a set of options, which describe hypothetical worlds, a subset called choices.
DUL Situations can be of arbitrary complexity: Encoded via a relation hasSetting, they can include, e.g., objects, actions, times and even other situations, which themselves can be linked to each other. For example, the situation in which an agent consumes coffee may be modeled as seen in fig. 1.
has setting
Physical Action executes
Consumption Task Situation has setting has setting
Agent Coffee
We build upon this definition of decision-making to define preferences located as a quality of a bearer entity – there is no preference that is not the subjective preference of somebody – and emphasizes its role in decision making tasks performed by that bearer, in a manner similar to functionalist accounts of preference.
Definition 2. A preference is the disposition of its bearer to tend towards a particular subset of choices from a given set of options while performing decision-making tasks.
Notice that the above definition does not make any assumptions on the method by which the options are evaluated and the final choices are selected. Research has shown that human preferences are not necessarily rational [19], i.e, that they can contain cycles, and the cognitive mechanisms of decision-making are not completely understood yet.
To still capture the agents methodology underlying choice selection at least partially, we
introduce a concept called PreferenceDescription that is a Description of a Preference.
In this exploratory paper, we focus on a simple, pragmatic representation of preferences via
PreferenceDescriptions as a partial order, similar to what has been proposed in preference
logic (c.f. [
We argue that any such order underlying preferences is not between situations, but between descriptions thereof: While the input of the above coffee-tea-scenario have been argued to be concrete situations of the decider consuming either, preferences are more general. In our example, preferring tea over coffee is general in that sense that the agent values all situation in which he consumes the one drink over all similar1. Ergo, such an order must be between descriptions (of situations): Definition 3. A preference order is a partial order of descriptions (of situations).
To represent such an order via OWL, we introduce not only the concept PreferenceOrder, but also OrderedElement and the relations orderedBy and ≤ (as well as the latter’s inverse ≥ ). The concept OrderedElement encapsulates Descriptions (of Situations) and orderedBy exactly one PreferenceOrder. The relations ≤ and ≥ have the obvious meaning of defining the order between the OrderedElements. This pattern allows to order the same Description (of Situations) in multiple PreferenceOrders as the ordering information is represented locally between OrderedElements.
Together, the PreferenceOrder and the contextual nature of Situation allows to model very complex, conditional preferences; e.g., that someone prefers their spouse to prepare a meal on days where they work, but otherwise likes to cook themselves. Figure 2 depicts the model as laid out in this section.
Using Konclude [21] on a Ryzen 7 5800X CPU, 32GB RAM, and a GTX 3070 ti graphics card, initial classification of the current SOMA version, including the prefences ontology from section 4, takes about 66ms, and subsequent DL queries take about 1ms each.
We hybridly evaluate the ontology by using the CQs from section 3 and by giving reasoning and querying examples. Following the SABiO guidelines [22], we use the listed formal queries as test-cases for the CQs. The queries are executed over example ABox-data with the preference ontology as the reasoning schema.
Figure 3 shows query A which is associated with CQ 1, asking for all preferences which a user has in a specific situation. Similarly, figure 4 shows query B which is associated with CQ 2, also the user and situation, but only the preference with the highest order, which reflects the user’s preferred option for the situation.
For space reasons, other queries and related machinery such as SWRL rules that are necessary for CQ 3 and 4 are not given in this paper but can be made available on request. 1The compared situations must be similar except for the differences defining the choices: Otherwise, one might infer from an agents preference of coffee over tea that they would also prefer drinking coffee in agony over drinking tea while feeling splendid. Different solutions to this well-known problem exist [20].
defines task Task defines bearer, defines trigger has task Decision Making Legend DUL SOMA is a new Affordance describes
Disposition defines performer
We have introduced a formal ontology-based model for representing preferences. This model serves as a structured framework for capturing and organizing user preferences in the context of robot-human interactions. To evaluate our model, we formulated and examined relevant competency questions for the robot to understand and address user preferences in these interactions.
In future work, we will investigate the usage of learning techniques to incorporate learned user preferences alongside a-priori knowledge of specific user preferences. In previous research, we already implemented a model for interactions in our Ontology with the MOI ontology[16], which provides a framework for logging episodic memories of past interactions. By extracting information from these episodic memories, we can learn user preferences. We will furthermore look at how to automatically match given options to existing preferences, which is non-trivial: E.g., the situation of the user drinking tea is, by common sense, different from the situation in which the user drinks coffee. However, in the light of the open world assumption, such behavior has to be explicitly stated – otherwise, one might come to the conclusion that a user might prefer drinking both simultaneously. It might be worth to look at how non-monotonic reasoning techniques could support such scenarios.
The research reported in this paper has been supported by the German Research Foundation DFG, as part of Collaborative Research Center (Sonderforschungsbereich) 1320 Project-ID 329551904 “EASE - Everyday Activity Science and Engineering”, University of Bremen (http: //www.ease-crc.org/). The research was conducted in subprojects “P01 – Embodied semantics for the language of action and change: Combining analysis, reasoning and simulation”, “P05-N – Principles of Metareasoning for Everyday Activities”, “INF – Research Data Management within EASE and Distribution of Open Research Data” and by the FET-Open Project #951846 ”MUHAI – Meaning and Understanding for Human-centric AI” by the EU Pathfinder and Horizon 2020 Program. affection-based episode ontology for personalization, in: 2013 IEEE RO-MAN, 2013, pp. 155–160. doi:10.1109/ROMAN.2013.6628437. [6] A. I. Schein, A. Popescul, L. H. Ungar, D. M. Pennock, Methods and metrics for cold-start recommendations, in: Proc. of ACM SIGIR, Association for Computing Machinery, 2002, p. 253–260. [7] R. Ali, M. Afzal, M. Sadiq, M. Hussain, T. Ali, S. Lee, A. M. Khattak, Knowledge-based reasoning and recommendation framework for intelligent decision making, Expert Systems 35 (2018) e12242. doi:10.1111/exsy.12242. [8] S. Bouraga, I. Jureta, S. Faulkner, C. Herssens, Knowledge-based recommendation systems: A survey, International Journal of Intelligent Information Technologies 10 (2014) 1–19. doi:10.4018/ijiit.2014040101. [9] M. Martínez-García, A. Valls, A. Moreno, Inferring preferences in ontology-based recommender systems using wowa, Journal of Intelligent Information Systems 52 (2019). doi:10.1007/s10844-018-0532-5. [10] G. Adomavicius, A. Tuzhilin, Context-Aware Recommender Systems, Springer US, Boston,
MA, 2011, pp. 217–253. [11] D. Cao, Z. Li, K. Ramani, Ontology-based customer preference modeling for concept generation, Adv. Eng. Inform. 25 (2011) 162–176. Information mining and retrieval in design. [12] D. Tapucu, G. Diallo, Y. Aït-Ameur, M. Ünalir, Ontology-Based Database Approach for
Handling Preferences, 2009. doi:10.4018/978-1-60566-756-0.ch014. [13] D. Beßler, R. Porzel, M. Pomarlan, A. Vyas, S. Höffner, M. Beetz, R. Malaka, J. Bateman, Foundations of the socio-physical model of activities (SOMA) for autonomous robotic agents, in: Proc. of FOIS, 2021, pp. 159–174. [14] A. Gangemi, The DOLCE+DnS Ultralite ontology (DUL) Version 4.0, 2021. URL: http: //www.ontologydesignpatterns.org/ont/dul/DUL.owl. [15] D. Pisanelli, A. Gangemi, G. Steve, An ontology of descriptions and situations for lyee’s hypothetical world, in: Proc. of Somet, Citeseer, 2003. [16] R. Nolte, M. Pomarlan, D. Beßler, R. Porzel, R. Malaka, J. Bateman, Towards an ontology for robot introspection and metacognition, in: Proc. of FOIS, 2023. Forthcoming. [17] J. Fan, K. Barker, B. Porter, P. Clark, Representing roles and purpose, in: Proc. of K-CAP, 2001, p. 38–43. doi:10.1145/500737.500747. [18] S. O. Hansson, T. Grüne-Yanoff, Preferences, in: E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy, Spring 2022 ed., Metaphysics Research Lab, Stanford University, 2022, p. 1. [19] F. S. M. Inza, M. Ryan, M. Amaya-Amaya, “Irrational” Stated Preferences: A quantitative and qualitative investigation, Springer Netherlands, 2008, pp. 195–215. [20] F. Liu, Reasoning about preference dynamics, volume 354, Springer Science & Business
Media, 2011. [21] A. Steigmiller, T. Liebig, B. Glimm, Konclude: system description, J. Web Semant. 27 (2014) 78–85. [22] R. de Almeida Falbo, Sabio: Systematic approach for building ontologies., Onto.Com/odise@FOIS 1301 (2014).