International Symposium on Ubiquitous VR 2007 1 Augmented Reality Cooking System Using Tabletop Display Interface Han-byul Jang, Jang-woon Kim, Chil-woo Lee Abstract—Cooking is the theme that causes interest to everyone as element that is essential in life. Everyone try to cook, but fire and knifes are dangerous to handle for children. So children are felt difficult to cook directly. In this paper, we describe about development of cooking system that gives imaginary cooking experience to children. Our cooking system consists of two major technologies (Augmented Reality, Tabletop display) that can interact with user. Using augmented reality technology gives to user more arresting and accessible virtual cooking environment. Figure 1 overview of cooking system Using tabletop display that provides multi-touch can provide interaction between user and the cooking system effectively. B. Interaction system Index Terms— Augmented Reality, Cook, Education, Virtual In the case of general augmented reality system, usually, Reality interaction depends on marker. Generally markers are used as button or manipulator. “3D Pottery Modeling in Augmented Reality [2]” shows such cases. But marker dependent system is I. INTRODUCTION affected by camera view and there is restriction that the user must operate the system using real objects like marker. In this C ooking is subject that excite interest to anyone as action that cook food that is essential in the life. People like delicious food and want to cook with theirs own hands. In the way, state of food changes continuously accords to progress of the cooking in the system, so it is impertinent to give interaction case of children, because it is dangerous to handle fire or knife by markers which display cooking ingredients. We want to in cooking, there is difficulty to cook food with theirs own reduce these problems. We use tabletop display to reduce hands. Augmented reality cooking system that uses tabletop markers dependency and to directly access the augmented display is a virtual cooking application for children. Cooking reality objects. So we are able to intuitionally manipulate system developed in this paper is applying an augmented reality augmented reality objects. Our system analyze hand gesture by technology to overlap 3D graphic objects to real world image. three steps: hand gesture, command and event. In our system, So it is more realistic than a normal 3D virtual environment hand gesture meaning is input that hand touches the tabletop system. Our system uses tabletop display interface that can screen. support multi-touch and multi-users. So we are able to provide Tabletop display can provide rich and intelligent interaction intuitive interface for the users and computers. through analysis of the touch input [3, 4, 5]. We analyze gesture and create interaction system to use special quality of tabletop display. So our system analyzes and distinguishes hand II. COMPOSITION OF COOKING SYSTEM gesture’s meaning in tabletop display and concludes suitable result. Hand gesture analyses have three main steps that are A. Configuration of whole system hand gesture, command and event. In tabletop display, hand Hardware of the system is consisted of the miniature kitchen gesture is the input generated by hand fingertip touch on the set and a tabletop display. The miniature kitchen is the place to tabletop’s screen. Command is specific meaning that is created put cards that express kitchen stuff, ingredients and etc. There by analyzing hand gesture. Event is the result that is produced is a marker used to make augmented reality on Cards and Our by analyzing command in our cooking system. Advantages of system recognize it to demonstrate augmented reality. Tabletop our hand gesture analysis are as following. First, one hand display shows augmented reality image of cooking system on gesture can used for various interaction interfaces because one the screen. The user can interact with both tabletop display and hand gesture can be differentiated and variously analyzed by miniature kitchen set. Figure 1 shows the overview of our various command according to situation. Second, we can cooking system. attempt organic composition of the commands and can compose suitable interaction system. International Symposium on Ubiquitous VR 2007 2 III. FRENCH TOAST COOKING SYSTEM Table 2 necessary hand gesture, command, and events In this paper, we select "French toast" among various kinds Hand gesture command events of recipes and compose system. Because French toast is break egg The egg is popular food for children and its making process is simple. broken Process to cook French toast is just roast bread that soaked in mixed egg and milk liquid. So we thought that French toast is First select egg and than up and suitable for our research. Figure 2 shows French toast. down. Material Egg and mixture milk are mixed Touch the bowl which has material several times. Turn on Fire Fire on and than roast Figure 2 French toast the bread One fingertip moves round after A. Kitchen stuff and ingredients of cooking two fingertip touch the fry pan. Table 1 shows Kitchen stuff and ingredients of French toast. IV. CONCLUSION Table 1 kitchen stuff and ingredients of French toast Our cooking system takes advantage of augmented reality Main material Bread, egg, milk, salt, strawberry jam technology and makes user experience cooking process. By Sub-material Sugar powder, Oil, butter using tabletop display interface, we can provide intuitive Kitchen stuff Fry pan, vessel interaction. Children can imaginarily experience cooking as it is not dangerous to use our cooking system. In present state, we In this way, to make process simple, we omit sugar powder have developed the simple system for cooking. In future that is not important between these materials and butter that is research, we will develop intuitive cooking gesture and handle putted on the fry pan. We use only important materials that are various cooking. bread, egg, and milk. Finally we create markers to represent the materials. ACKNOWLEDGEMENTS B. French toast manufacture by interaction This research is accomplished by research funding of C.N.U French toast is a cooking that is made through process of Culture Technology Institute supported by MCT and KOCCA, roast bread soaked in mixed egg and milk liquid. Necessary Korea hand gesture, command and events are shown in Table 2. The “Break egg” command generates an event that changes the state of the egg object to “broken”. The “Material mixture” REFERENCES command generates an event that changes state of the objects to [1] In the section of “Emerging technologies” of ACM SIGGRAPH2006 Full Conference. “mixed” state. The “Fire on” command generates event that [2] Gabjong Han, Jane Hwang, Seungmoon Choi, Gerard Jounghyun Kim. changes state of the bread on fry pan to “roasted bread”. “3D Pottery Modeling in Augmented Reality”, HCI 2007. Cooking system is run by these gesture commands. [3] Dietz, P. and Leigh, D. 2001. DiamondTouch: A Multi-User Touch Technology. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (Orlando, Florida, November 11 - 14, 2001). UIST ’01. ACM Press, New York, NY, 219-226. [4] Han J. Y. Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, ACM Press, New York, NY, 2005, 15-118 [5] Matsushita, M., Iida, M., Ohguro, T., Shirai, Y., Kakehi, Y., and Naemura, T. 2004. Lumisight Table: A Face-to-face Collaboration Support System That Optimizes Direction of Projected Information to Each Stakeholder. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (Chicago, Illinois, USA, November 06 - 10, 2004), CSCW ’04. ACM Press, New York, NY, 274-283. Figure 3 demonstration of cooking contents