Considerations for a Multi-purpose Agrobot Design Toward Automating Skillful Viticultural Tasks: A Study in Northern Greece Vineyards Eleni Vrochidou 1, Christos Bazinas 1, Efthimia Mavridou 1, Theodore Pachidis 1, Spyridon Mamalis 2, Stefanos Koundouras 3, Theodoros Gkrimpizis 3 and Vassilis G. Kaburlasos 1 1 Human-Machines Interaction (HUMAIN) Lab, Department of Computer Science, International Hellenic University (IHU), 65404 Kavala, Greece 2 Department of Management Science and Technology, School of Economics and Business Administration, International Hellenic University (IHU), 65404 Kavala, Greece 3 Laboratory of Viticulture, Faculty of Agriculture, Forestry and Natural Environment, School of Agriculture, Aristotle University of Thessaloniki (AUTh), 54124 Thessaloniki, Greece Abstract Seasonal labor shortages in agriculture are experienced throughout Europe and beyond especially, but not only, during the harvest time when the demand for hands is high as well as urgent. Agrobots have been established as a sustainable solution to support these growing demands due to their capacity to work incessantly, fast as well as with skillful precision. In the aforementioned context, this work investigates the design of a single multi-purpose robotic system towards automation of a number of skillful viticultural practices. Best practices are recorded thoroughly according to the needs of local northern Greek wineries through interviews with experts. Possibilities and limitations for robotic implementation are discussed and appropriate end-effectors are outlined. Keywords 1 Precision agriculture, agricultural robot, viticultural practices, vineyard, automation 1. Introduction The quality of grapes and produced wine is directly affected by the terroir of wine and both growing and winemaking practices [1,2]. Canopy manipulation practices are traditionally performed by experienced workers. However, the vast cultivation areas due to production growing demands in conjunction to the lack of seasonal labor yell for robotic interventions to automize viticultural seasonal practices in a dexterous and consistent way. Agricultural robots, namely Agrobots [3], have been sparingly used in viticultural automation [4]; for harvesting [5], spraying [6], berry thinning [7], pruning [8], monitoring [9]. However, certain highly skillful tasks, such as shoot thinning or tying, have not yet been automated. Less detailed tasks, such as top/lateral removal, have been automated with appropriate tools mounted on tractors, yet, not altogether by a single muti-purpose agrobot. The challenge is for an agrobot or group of robots [10], to automate a multitude of viticultural practices with minimal adaptations of suitable tools/end-effectors. Towards this end, this work investigates the design of a multi-purpose robotic system for the automated vineyard management practices of harvest, cluster thinning, leaf removal, pruning, shoot thinning, top (topping) and lateral removal, weed removal, spraying, and tying. The focus is on optimal requirements of practices as delineated by expert oenologists and agronomist based on the needs and Proceedings of HAICTA 2022, September 22–25, 2022, Athens, Greece EMAIL: evrochid@cs.ihu.gr (A. 1); chrbazi@cs.ihu.gr (A. 2); emavridou@teiemt.gr (A. 3); pated@cs.ihu.gr (A. 4); mamalis@teiemt.gr (A. 5); skoundou@agro.auth.gr (A. 6); gkrimpiz@agro.auth.gr (A. 7); vgkabs@cs.ihu.gr (A. 8) ORCID: 0000-0002-0148-8592 (A. 1); 0000-0002-3780-5617 (A. 2); 0000-0001-6329-0533 (A. 4); 0000-0003-1868-2081 (A. 6); 0000- 0002-1639-0627 (A. 8) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 45 characteristics of Northern Greek vineyards. For each of the practices, appropriate end-effectors are proposed. The scope is to investigate possibilities and limitations of robotic implementation of a set of basic viticultural practices which in their entirety have never been automated yet by a single robotic system. The remainder of the paper is as follows: optimal viticultural practices are described in Section 2; requirements for appropriate end-effectors and challenges are included; Section 3 concludes the paper. 2. Optimal Viticultural Practices Optimal viticultural practices are extracted through interviews with expert agronomists and oenologists of three wineries of Kavala and Drama, in Northern Greece, under the ongoing project SVtech [11]. Interviews took place in August 2021 and were conducted by teams of engineers and computer scientists who will consequently design the final robotic system. In what follows, all tasks are determined. Suggested end-effectors and challenges towards robotic automation are included in Table 1. 2.1. Cutting Tasks The following subsections describe an array of cutting tasks including harvest and cluster thinning, leaf removal, pruning, shoot thinning, top and lateral shoot removal, and weed removal. 2.1.1. Harvest and Cluster Thinning Harvest is the collection of all fully and equally ripened grape clusters. The process includes cutting grape bunches off the vine and collecting them into harvest crates [12]. Cluster thinning is employed to improve foliage-yield relationship by selecting and removing bunches that are not uniformly ripened, of smaller size, defected or belong on the outer edge of the vine [13]. In both tasks bunches are cut-off from the stem, therefore, they can be automated with the same end- effector. In manual harvest, the harvester grasps the cluster with the non-dominant hand and cuts it free with the dominant hand by using a grape picking scissors. Thus, the automated system needs end- effector(s) able cut the grape cluster from the stem and hold it. 2.1.2. Leaf Removal Leaf removal refers to the modification of the microclimate of foliage, by removing leaves from the base of the shoots in the cluster zone, towards a better exposure to the sun [14]. Leaf removal varies depending on the grape variety; less leaves in white and more in red varieties are removed. Regardless the variety, leaf removal is traditionally performed manually. Leaves are removed easily with a sharp motion pulling the leaf downwards with the hand. Therefore, leaves removal can be automated by using an appropriate end-effector that could grasp and remove leaves. 2.1.3. Pruning Winter pruning is one of the most important viticultural practices due to its physiological and productive effect on plants [15]. There are several ways to prune a vine depending on the variety. Thus, a basic and generic automated approach is extracted; the canes for renewal are selected and cut back to two canes each; the rest of the wood is cut off and removed; manual trimming can be performed afterwards whenever highly specialized pruning techniques are needed. Manual pruning is still prevalent and performed by experienced workers using secateurs with a parrot beak; the worker holds the cane with the non-dominant hand and cuts it with the dominant hand; the cut wood is removed with the non-dominant hand. Thus, the automated system needs to hold the cane, cut it and remove it, by using an electric cutting pruning tool and a holding tool. 46 2.1.4. Shoot Thinning In shoot thinning, it is decided how many buds the whole vine can carry; buds are counted and adjusted according to desired fruit load [16]. Less shoots result in fewer grape clusters with concentrated flavors. This practice requires delicate handling since shoots are very tender and easily cut. Traditionally shoots are manually removed by experienced workers. Therefore, a shoot thinning tool needs to be able to replace a human dexterous hand. 2.1.5. Top and Lateral Shoot Removal Cutting off the shoots that have grown at the sides and above the higher wire at flowering, results in more carbohydrates and nutrient to the grape clusters [17]. Topping is usually done manually with a folding secateur. In commercial vineyards, which are vast and linearly structured, topping is performed mechanically with a tool with cutting bars adapted to a tractor (topping machine). Along with the tops, cutting of the sides is done at the same time. The automated system needs a cutting tool for both top and lateral shoots (Figure 1). (a) (b) (c) Figure 1: Automation of top and lateral shoots removal with (a) a folding secateur tool, (b) a trimming tool and (c) a customized topping machine. 2.1.6. Weed Removal Removing the weed is to limit its competition for water and nutrients with the vine, maintain a good microclimate and soil fertility [18]. In commercial vineyards, weed removal is done mechanically with a weeding machine adjusted to tractors. For the automation of weed removal, a commercial under-row weeding machine can be adjusted to a ground robot. 47 Table 1 Potential end-effectors, requirements, and limitations for the automation of each viticultural practice by autonomous agrobot/s End-effector tool Challenges and Viticultural task Requirements (robotic system) limitations Cutting tool: sharp, durable, • Degree of maturity • Simultaneous cutting- light, with commercial picking determination and-holding tool scissors blades. • Determination of (one arm-one robot) Holding tool: able to hold from Harvest bunches to remove • Cutting tool and large to small objects with and • Stem/bunch detection holding tool adjustable power (e.g., Cluster thinning with environmental (two arms-two commercial gripper or a 5- noise robots or two arms- finger robotic hand simulating • Cutting and holding one robot) the non-dominant hand of a human harvester). without injuring grapes • Grasping of brunches/wires • Applied torque • Removing leaves • Grasping tool without injuring grapes (one arm-one robot) Grasping tool: light, durable, Leaf removal • Determine percentage • 5-finger robotic hand adequate opening. of removed leaves (one arm-one robot) based on variety • Uniform leaf removal • Foliage detection and density determination • Cane determination • Cutting tool and Electric pruning cutting tool: • Cutting without injuring holding tool same as in harvest but with the vine Pruning (two arms-one robot secateurs blades. • Cutting and holding or two arms-two Holding tool: same as in simultaneously robots) harvest. • Removing wood • Adjust the number of buds to remove • Shoot thinning tool Shoot thinning tool: a robotic • Remove bud without Shoot thinning (one arm-one robot) 5-fingers hand. injuring the cane • Bud number determination • Secateur tool (Figure 1(a)) (one arm-one robot) Top Topping tool: sharp rotatable • Adjust the tool to vine • Trimming tool (Figure and blades, durable, light, easily height 1(b)) Lateral shoot adjustable to a ground robot • Remove shoots without (one arm-one robot) removal and to the height of the vine. injuring vine • Topping machine (Figure 1(c)) (one robot) • Under-row weed control Weed removal tool: cut the • Cutting without injuring • Under-row weed weed underneath vines, trunks Weed removal control tool sufficient working width, light, • Adjust the tool to (one robot) height-adjustable, sharp and desired height durable blades. • Trunk/structures detection and avoidance 48 Table 2 (cont.) End-effector tool Viticultural task Requirements Challenges and limitations (robotic system) • Moving spraying tube tool (Figure 2(a)) (one robot) • Moving multiple spray • Adjust the tool to desired nozzle tool (Figure Spraying tool: targeted direction/orientation/height Spraying 2(b)) spraying. • Restriction of pesticide (one robot) diffusion • Customized spraying machine (Figure 2(c)) (one robot) Electric tying tool: quick, light, • Canes detection • Tying tool and holding adjustable tying options • Wires detection tool depending on thickness of • Bend canes without Tying (two arms-one robot canes. breaking/ injuring or two arms-two Holding tool: same as in • Tying the cane and the wire robots) harvest. together 2.2. Spraying For the treatment of diseases and insects of the vine, preventive spraying is performed. Ideally, spraying should be performed in the underside of each leaf first, and then from the top of the vine down to the bottom [19]. Traditionally, vineyards are sprayed with large spray tanks towed by tractors, manual sprayers, or by air. (a) (b) (c) Figure 2: Automation of spraying with (a) a moving spraying tube, (b) a moving multiple spray nozzle and (c) a customized spraying machine. 2.3. Tying After pruning, the two new canes are laid and tied down on the trellis wire, one in each direction [20]. Optimal execution of tying requires skillful handling so as not to injure the canes. Therefore, automation of this practice requires a holding tool to bend and hold the cane close the wire and an electric tying tool to tie them together. 3. Conclusions This work investigates the design of a multi-purpose robotic system towards automation of a set of basic viticultural practices. The aim of this work is to focus on the possibilities regarding these robotic automations, which in their entirety have never been employed to a single robotic system, by proposing 49 end-effector tools for each one of them. Future work will include the overall system structure, in terms of software and hardware technologies. 4. Acknowledgements We acknowledge support of this work by the project “Technology for Skillful Viniculture (SVtech)” (MIS 5046047) which is implemented under the Action “Reinforcement of the Research and Innovation Infrastructure” funded by the Operational Program “Competitiveness, Entrepreneurship and Innovation” (NSRF 2014-2020) and co-financed by Greece and the European Union (European Regional Development Fund). 5. References [1] Cardell, M. F., Amengual, A. and Romero, R., “Future effects of climate change on the suitability of wine grape production across Europe,” Reg. Environ. Chang. 19(8), 2299–2310 (2019). [2] Gutiérrez-Gamboa, G., Zheng, W. and Martínez de Toda, F., “Current viticultural techniques to mitigate the effects of global warming on grape and wine quality: A comprehensive review,” Food Res. Int. 139, 109946 (2021). [3] Mavridou, E., Vrochidou, E., Papakostas, G. A., Pachidis, T. and Kaburlasos, V. G., “Machine Vision Systems in Precision Agriculture for Crop Farming,” J. Imaging 5(12), 89 (2019). [4] Bazinas, C., Vrochidou, E., Lytridis, C. and Kaburlasos, V. G., “Time-Series of Distributions Forecasting in Agricultural Applications: An Intervals’ Numbers Approach,” Eng. Proc. 5(1), 12 (2021). [5] Vrochidou, E., Tziridis, K., Nikolaou, A., Kalampokas, T., Papakostas, G. A., Pachidis, T. P., Mamalis, S., Koundouras, S. and Kaburlasos, V. G., “An Autonomous Grape-Harvester Robot: Integrated System Architecture,” Electronics 10(9), 1056 (2021). [6] Adamides, G., Katsanos, C., Constantinou, I., Christou, G., Xenos, M., Hadzilacos, T. and Edan, Y., “Design and development of a semi-autonomous agricultural vineyard sprayer: Human-robot interaction aspects,” J. F. Robot. 34(8), 1407–1426 (2017). [7] Monta, M., Kondo, N. and Shibano, Y., “Agricultural robot in grape production system,” Proc. 1995 IEEE Int. Conf. Robot. Autom. 3, 2504–2509, IEEE (1995). [8] Botterill, T., Paulin, S., Green, R., Williams, S., Lin, J., Saxton, V., Mills, S., Chen, X. and Corbett- Davies, S., “A Robot System for Pruning Grape Vines,” J. F. Robot. 34(6), 1100–1122 (2017). [9] Roure, F., Moreno, G., Soler, M., Faconti, D., Serrano, D., Astolfi, P., Bardaro, G., Gabrielli, A., Bascetta, L. and Matteucci, M., “GRAPE: Ground Robot for vineyArd Monitoring and ProtEction,” [Advances in Intelligent Systems and Computing], 249–260 (2018). [10] Lytridis, C., Kaburlasos, V. G., Pachidis, T., Manios, M., Vrochidou, E., Kalampokas, T. and Chatzistamatis, S., “An Overview of Cooperative Robotics in Agriculture,” Agronomy 11(9), 1818 (2021). [11] HuMaIN-Lab., “Technology for Skillful Viniculture (SVtech),” 2021, (27 January 2022 ). [12] Varas, M., Basso, F., Maturana, S., Osorio, D. and Pezoa, R., “A multi-objective approach for supporting wine grape harvest operations,” Comput. Ind. Eng. (2020). [13] Carmona-Jiménez, Y., Palma, M., Guillén-Sánchez, D. A. and García-Moreno, M. V., “Study of the Cluster Thinning Grape as a Source of Phenolic Compounds and Evaluation of Its Antioxidant Potential,” Biomolecules 11(2), 227 (2021). [14] Ivanišević, D., Kalajdžić, M., Drenjančević, M., Puškaš, V. and Korać, N., “The impact of cluster thinning and leaf removal timing on the grape quality and concentration of monomeric anthocyanins in Cabernet-Sauvignon and Probus (Vitis vinifera L.) wines,” OENO One 54(1), 63– 74 (2020). [15] Buesa, I., Yeves, A., Sanz, F., Chirivella, C. and Intrigliolo, D. S., “Effect of delaying winter pruning of Bobal and Tempranillo grapevines on vine performance, grape and wine composition,” Aust. J. Grape Wine Res. 27(1), 94–105 (2021). [16] Majeed, Y., Karkee, M., Zhang, Q., Fu, L. and Whiting, M. D., “Determining grapevine cordon 50 shape for automated green shoot thinning using semantic segmentation-based deep learning networks,” Comput. Electron. Agric. 171, 105308 (2020). [17] KORKUTAL, İ., BAHAR, E. and ZİNNİ, A., “Determination the Effects of Leaf Removal and Topping at Different Times on the Grape Berry,” J. Inst. Sci. Technol., 1–9 (2021). [18] Giagnoni, L., Maienza, A., Baronti, S., Vaccari, F. P., Genesio, L., Taiti, C., Martellini, T., Scodellini, R., Cincinelli, A., Costa, C., Mancuso, S. and Renella, G., “Long-term soil biological fertility, volatile organic compounds and chemical properties in a vineyard soil after biochar amendment,” Geoderma 344, 127–136 (2019). [19] Ogawa, Y., Kondo, N., Monta, M. and Shibusawa, S., “Spraying Robot for Grape Production,” [Field and Service Robotics], Springer-Verlag, Berlin/Heidelberg, 539–548 (2006). [20] Jackson, R. S., “Vineyard Practice,” [Wine Science], Elsevier, 143–306 (2014). 51