Robots are increasingly entering agricultural fields to support human labor. Heavy tasks like harvesting are assigned to robots due to their advanced modularity, robustness and accuracy that provide automated solutions to tedious and elaborate tasks. In this paper, the technological requirements of a specialized Agrobot (Agriculture robot) for supporting viticulture tasks such as harvest, green harvest and defoliation, are identified. This robot aims at developing onboard intelligent decision making on-the-spot based on commercial hardware, machine vision and innovative computational intelligence algorithms. Design, structures, methods and sensors are briefly discussed. This study delineates a prototype grape harvesting robot, consisting of a reliable information acquisition system that includes sensor-fusion algorithms and data analysis, adopted to the dynamic conditions of agricultural environments such as vineyards.
Current practices of grape harvest include human involvement, such as monitoring
and specialized manual work, which seems to have reached its limits; young people
abandon viticulture because of inherent work difficulties, the increasing age of grape
harvesters prolongs harvest duration, therefore reduces the quality of the harvested
grapes and of the produced wines. Thus, there is a need to reliably automate grape
harvest
Extensive research has focused on the application of Agrobots to a variety of
vineyard tasks
Simplifying the agricultural tasks and enhancing Agrobots with sensors may
improve their performance, but this will refer to a non-feasibly constructed Agrobot
and it is insufficient for successful implementation in real in-field practices. For this
reason, robotics and non-robotics disciplines need to be identified
The rest of the paper is structured as follows. In Section 2 the requirements of the selected agricultural operations are defined. According to these, in Section 3 the corresponding technological needs of the robot, in terms of hardware and software are identified. Conclusions are presented in Section 4.
First, defoliation will be automated, followed by green harvest in order to optimize grape quality/quantity and prepare vines for the harvest. Finally, homogeneous harvest is automated in the sense that ARG will harvest only grapes of similar degree of ripeness. In what follows, the requirements for the three agricultural tasks are addressed by a group of viticulturists and agronomists.
Selectively removal of leaves, namely defoliation, is suggested in order to improve the microclimate in vineyards (Fig.1). Thus, the quality of the produced wine is increased due to better plant health i.e. better ventilation and due to acquisition of better phenolic characteristics of grapes as a result to their exposure to the sun. The robot must remove carefully and uniformly a percentage of leaves on the crop base, only from the east side of the vineyard.
(a) (b) (c)
In order to improve the phenolic content of grapes, it is necessary to reduce the load on the vine; a number of grapes are removed from each tree. This process, namely green harvest, meliorates taste and aroma of the remaining grapes (Fig.2). The robot must remove a percentage of grape clusters from each vine, in a priority order as follows: sick, malnourished, uneven and immature.
When grapes are ripened, harvesting takes place. At this phase, the robot must remove all ripened grape clusters (Fig.3) and place them in harvesting baskets. Sick/damaged clusters are not collected and either removed or left on the vine.
The in-field robot must be able to understand the physical properties of each encountered object and be able to work under dynamic conditions. The robot first will acquire raw data about the environment from its sensors, will analyze it for reasoning and will operate based on its perception according to its operation plan.
Technically, the ARG will be developed by the integration of a wheeled robot, one robot arm, end-effectors such as a cutter, electronic sensors e.g. cameras, and software that will coordinate the operation of the mechanical and electronic devices. In addition, an aerial drone could contribute to the development of digital maps required for ARG’s autonomous navigation. ARG’s sensing system needs to be equipped with specialized manipulators and end-effectors able to work under varying conditions, since dust, humidity, heat etc. can easily affect the electric circuits and cause corrosion and, therefore, instability to the system. Regarding reasoning and planning, effective software needs to be developed so as to ensure two basic abilities for ARG; robot functionalities (e.g. obstacle avoidance, self-localization, path planning, navigation) and robot applications (e.g. harvesting, defoliating, handling). Algorithms need to be adaptive and able to deal with object and environment variations such as illumination. In what follows, a group of electrical engineers and computer scientists addresses the hardware (Fig.4(a)) and software (Fig.4(b)) needs for developing the ARG.
Drone. An unmanned aerial vehicle (UAV) will acquire multiple RGB imagery of vineyards, resulting in a large fraction of overlapping between them to derive a binary difference dense model (DDM). The DDM will provide the 3-D macro structure of the vineyard which is necessary for the navigation of ARG.
Wheeled Robot. A mobile robotic platform, especially designed for outdoor missions and able to navigate in different types of terrains, will carry all sensors around the field and necessary electronics. It needs to be equipped with sensors such as Global Positioning System (GPS), Light Detection and Ranging (LIDAR), Inertial Measurement Unit (IMU), 3D camera and encoders to facilitate localization, navigation and obstacle avoidance. It needs to be equipped also with an extra battery ensuring the proper operation of the system for a long time, voltage converters and signal processing units.
Manipulator. A robotic arm will perform all agricultural operations. The arm should be lightweight, at least of 6 Degrees-Of-Freedom (DOF) and able to carry the weight of a big grape cluster. The arm will be equipped with adequate end-effectors such as two fingers to hold a stem and a cutter to cut it off.
Sensors. Cameras are the most important external sensors for a harvesting robot as they replace human’s eyes for discrimination, recognition and distance measurements. The two RGB cameras mounted on the robotic arm will provide stereoscopic vision and a near infrared (NIR) and a thermal camera will provide additional information for the spectral characteristics of vines.
Traceability and Geo-positioning. The Agrobot must navigate between the lines of the vineyard and sample. The maps of the vineyards must first be extracted from the drone information (Fig.5(a)). Then, a wireless positioning system based on GPS needs to be developed so as to provide positioning information of the robot and use it to support real-time precision navigation between specific geographic coordinates of the map. Navigation and Guidance. An algorithm must be developed to extract the navigation path in the vineyard (Fig.5(b)), including the headland turns between the vineyard lines. The algorithm needs to combine machine vision and both global and local positioning information from the mounted sensors, so as to determine the navigation paths, and execute path planning along with obstacle avoidance.
(a) (b)
Computer vision. Computer vision algorithms (Fig.6), based on in-field images (Fig.7(a)), need to be developed to support the following tasks: 1) Leaves detection (Fig.7(b)) so as to define the position and percentage of leaves to be removed during the defoliation process. 2) Grape clusters detection (Fig.7(c)) so as to define the position and percentage of grape clusters to be removed during green harvest, giving priority to the malnourished, uneven and immature. 3) Ripeness estimation during harvest. Only fully ripened clusters will be removed.
The rest will remain on the vines or removed separately from the healthy ones. 4) Stem detection (Fig. 7(d)). The stem of leaves and grape clusters needs to be detected, approached and cut-off by the end-effector. 5) Harvesting baskets detection. The baskets need to be located and filled with the collected clusters up to a certain point. 6) Movement of the robotic arm toward a target point. Stereoscopic vision will be used to locate the exact distance of an object. 7) Movement of the robotic arm by avoiding collisions. Cameras mounted on the robotic arm will be used as real-time collision detectors. Manipulators and sensors must be able to cope with various geometries of obstacles and targets. Detection and motion planning algorithms have to generate new manipulator trajectories adapted to every target in a short time horizon. User Interface. A user-friendly interface needs to be developed for establishing the communication between user and robot. The user needs to: have access to the vineyard map, create the navigation path, select exact points on the map where the Agrobot will perform selected tasks and define the tasks. The user interface should permit personalized changes for each agricultural task, e.g. percentage of leaves/clusters to remove. Moreover, it must provide monitoring information for the robot, e.g. battery level, current position, status of sensors and live streaming.
This work addresses the technological needs for the development of an Agrobot that
deals with three vineyard tasks; defoliation, green gravest and harvest. The objective
of this study is to delineate optimally the ARG requirements by a group of experts,
towards mechanizing effectively the aforementioned tasks. Identifying the
technological needs of ARG contributes to a more realistic system design. The
literature indicates that systematic design process techniques can contribute to the
technical and economic feasibility of a robot