This paper presents a new approach for exploring a collection of cooking recipes represented as cooking workflows by means of a conversation. Users are guided through the search process by answering posed questions. Thus, they are not required to formulate queries and they do not need to browse a recipe collection by hand. Questions involve ingredients and cooking activities contained in the workflows. The approach is implemented in our CookingCAKE system, extending it with a dialog component.
Nowadays, numerous cooking recipes are available online and various search engines support users in finding suitable recipes. Beside providing a keyword-based search, some engines also support an ingredient-based search by asking the user to specify desired and undesired ingredients. However, in practice, amateur chefs may only have a vague idea of their desired dish or they lack detailed knowledge about required ingredients and thus have difficulties in providing a precise query. In 2010, Yummly1 launched the first semantic search platform for food and recipes. By capturing the semantics of recipe descriptions and ingredients, Yummly is able to handle more vague queries such as general terms in a keyword-based search. For example, if the user starts a search with the keyword meat, Yummly initiates a dialog asking the user which kind of meat she would like. Then, further questions are posed concerning various properties such as the desired type of dish, preparation time, nutritional preferences, and additional ingredients.
This paper follows a similar approach and provides a method to conduct a
conversation with the user to find desired cooking recipes. We focus on structural features of
recipes thus representing them as workflows. More precisely, in addition to considering
the occurance of ingredients and preparation steps, we also analyze their
dependencies. Based on this information, questions concerning the further processing of desired
ingredients can be posed. Thus, the ingredient-based search capabilities provided in
typical search engines for recipes are extended by this approach. Moreover, we
investigate how such features can be constructed automatically from the underlying workflow
1 www.yummly.com
Copyright © 2017 for this paper by its authors. Copying permitted for private and
academic purpose. In Proceedings of the ICCBR 2017 Workshops. Trondheim, Norway
repository and we propose a respective question selection strategy for the
conversation. We consider a recipe search as a problem solving process in which the problem
description specifies the user’s preferences of a desired dish and a possible solution is
a recipe describing its preparation. We apply the methodology of conversational
casebased reasoning (CCBR) [
In the following, section 2 briefly introduces the representation and querying of cooking workflows before section 3 describes our C-POCBR approach. Section 4 concludes the paper and briefly discusses future work. 2
In our approach a cooking recipe is represented as a workflow describing the process to
prepare a particular dish [
In order to retrieve cooking workflows with case-based reasoning, the user’s
preferences must be specified in a query, which in turn must be evaluated against the available
workflows. For this purpose, we proposed a process-oriented query language (POQL)
[
To facilitate the elaboration of a POQL query for workflow retrieval, our approach guides the user through the query process with a sequence of questions about her preferences. The more questions are answered, the more knowledge about desired and undesired properties is available, which is stored in an internal POQL query. A major focus is put on the automatic creation of questions to avoid that they need to be specified manually. For this purpose, we consider workflow fragments as characteristic properties of a workflow, which we refer to as features. The basic idea is to extract features from the workflows stored in the repository (case base) automatically, which are then used as the subject of questions. In order to conduct efficient conversations, we rank features by their ability to distinguish workflows from one another. Furthermore, identified relations between features enable to generate coherent follow-up questions and to infer irrelevant features based on already answered questions. 3.1
In principle, a feature can be any fragment of a cooking workflow. In a workflow, the
smallest possible feature consists of a single workflow item. This can be a single node
such as a data or a task node. More complex features can be created by extracting
partial workflows. To derive questions on a more general level of detail, we apply a
generalization algorithm [
A feature workflow Wd describes structural properties of a workflow W regarding a particular data item d from W . It consists of all (and at least one) tasks connected to d and which are connected by control-flow edges. Moreover, Wd additionally comprises all data items that are connected to those tasks. For instance, regarding the cooking workflow depicted in Figure 1, a feature workflow constructed for the ingredient white bread contains the associated processing steps toast and spread. It further comprises the ingredient butter, which is required for the task spread. lated feature g1 = fcut, meatg is more general and equally large while the feature g2 = fparma-hamg is more specific and smaller. 3.2
In order to obtain the user’s preferences most suitable to determine the best matching cooking workflows, i.e., the candidate workflows, a respective questioning strategy is required. The dialog component iteratively creates and displays questions until the user selects a desired workflow. With each question answered by the user, the set of candidate workflows, which encompasses the whole case base at the beginning of a conversation, is reduced. The dialog starts with an empty query and the set of candidate features, i.e., relevant features to be asked in a question, comprises the full set of features.
In the main loop, the dialog component selects a question based on the candidate features. The selection process considers the previously answered questions as well as the ranking and relationships of features. Each question involves one or in certain cases several candidate features. We provide three major types of questions, which are depicted in Table 1. Based on the ranking of the candidate features, the subject matter of a question is determined. If the user answers that the suggested feature is desired, specific follow-up questions are selected in the subsequent iterations. Those followup questions are derived from related features and aim at further refining the previous question asked. An example is given in Table 1, which presents a sequence of three questions (Q) including the possible answers (A) and the user-selected answers (marked with a box).
At the beginning of a conversation the highest ranked feature from the candidate features is suggested in a feature question (FQ ). This type of question is not related to previously suggested features and it will be asked as long as the user selects the suggested feature as irrelevant or undesired. In the example, the feature meat is the subject matter of the first question and it is selected as desired by the user.
If a feature in a FQ is selected as desired, a first follow-up question, i.e., a specialization question (SQ ), is posed suggesting one or (if available) several equally large but 6 more specific features. The user can choose a specialization, select specializations as undesired, or mark all specializations as irrelevant. This type of question is repeated as long as further specializations exist and are desired by the user. In the example, the user chooses chicken as her desired type of meat.
Following the SQ s, an enlargement question (EQ ) is displayed to the user that suggests, if available, larger features than the previously selected and/or specialized feature. Just as in SQ , the user has three different options: choose an enlargement, select enlargements as undesired, or mark all enlargements as irrelevant. In the given example, we assume that the user does not like shredded or chopped chicken and thus selects those features as undesired. If no more EQ s are available, the next initial FQ is selected, addressing a new and potentially unrelated subject matter.
When the set of candidate features is updated due to an ignored or answered question, irrelevant features are inferred based on the relations between features. If a question is marked as irrelevant, all the related features (e.g., more specific and larger features) are marked as irrelevant, too. If suggested features are selected as undesired, they are added to the restriction part of the current query and related irrelevant features are no longer considered as candidate features, to prevent the system from repetitively asking the user what she does not like. If a feature is marked as desired, also related features such as more general features are removed from the set of candidate features. If a user chooses a specialization or an enlargement, the target feature that is already present in the query is replaced with the new feature. 3.3
Based on the extracted features and the questioning strategy, the conversation is conducted in the dialog component of CookingCAKE4. The graphical user interface is illustrated in Figure 3. It consists of three displays suggesting the best matching workflow (upper part of figure), showing a question (middle of figure), and summarizing the current query (lower part of figure). Figure 3 presents a progressed state in a conversation in which some preferences are already obtained from the user and specified in the internal query by the system. In the given example, the current query contains firm cheese as desired and meat as undesired. The question displayed is a follow-up question targeting the further refinement of the current query. In the example, the question suggests alternative processing steps for firm cheese. The user has two options to react: 1. Ignore the question: In this case, the features being subject of the question as well as related features are ignored and the next best question is displayed. 2. Answer the question: Causes the system to extend the query and to perform a similarity-based retrieval on the current set of candidate workflows. The workflow with the highest similarity is displayed to the user.
In the upper part of the user interface, a solution workflow best fulfilling the current query is suggested to the user. With respect to this suggestion, the user has two additional options to react: 4 Online demo available at cookingcake.wi2.uni-trier.de/conversation 1. Ignore the suggestion: The user can actively ignore the suggested workflow, which causes the system to exclude it from the solution candidates and to trigger a new retrieval for the next best workflow. 2. Select the suggestion: In this event, the conversation terminates successfully.
We presented an approach to retrieve cooking workflows by means of an interactive
dialog with users. To save effort for defining suitable questions, a method for the automatic
creation of questions based on extracted features was described. We recently showed in
an experimental evaluation that those features are meaningful subjects of questions and
that they are suitable to distinguish workflows from one another [
In future work we plan to extend the presentation and explanation of workflows and features. For the sake of simplicity, we used a simplistic representation of cooking recipes and considered basic features, which could be extended in the future. Also, future work should investigate how adaptability of workflows can be considered during a conversation. By this means, interactive retrieval could be combined with interactive adaptation to provide more diverse and customized cooking workflows for users. Acknowledgments. This work was funded by the German Research Foundation (DFG), project number BE 1373/3-3.