With shortage in human workforce and increasing ecological challenges application of Internet of Things and Artificial Intelligence technologies in the agricultural sector, known also as precision farming, has become very popular in recent years. Agriculture is, however, a domain with specific challenges which AI engineers and data scientists usually do not face in other domains, such that healthcare, industry or services, just to name a few. In this paper we collect and summarize the challenges and problems which arose in our honeybee monitoring project from which the most important are trust, seasonality, data labeling issues and sudden events. We believe that, despite not focusing on Machine Learning models nor providing technical details, this paper provides an interesting reading for data scientists and will serve as an aid for planning and executing data science projects in the agricultural domain.
in food industry and agriculture due to honey production
fortunately, honeybee colonies have sufered significant
declines in recent years [
The identification of the health condition of a bee colony is done manually, by opening and inspecting the hive, in the majority of apiaries. By opening the hive the beekeeper disturbs the colony and, thus, introduces a certain level of stress to the bees. Moreover, during
from our project partner, the the Natura mérnökiroda Ltd. from audio data recorded in beehives.
The majority of related works on beehive audio
analysis is focusing on swarming prediction [
Hungary belongs to the most important honey pro- lems important in beekeeping, such that the detection
consequently, results in a reduced amount of honey the
given bee colony produces.
ducers in EU w.r.t. bee density and honey production (in
kg/100 km2) [
Meantime, besides the microphones, we have installed audience of the conference and, also, to the data science other types of sensors, such as temperature, humidity, community, in general. scale and video into the beehives at our test site. The data are being gathered continuously and implementations of baseline approaches for multi-modal data analytics [17] 2. Mutual Understanding and are being implemented. Our recent research consists of Trust analysing these data in order to develop novel approaches for beehive monitoring. First, we have contacted the Hungarian Association of
The permanent research team consists of two authors Beekeepers1 (HAoB) in order to establish a cooperation of this paper, the authors D. T. Várkonyi and T. Horváth, and map beekeepers’ requirements for the concept of a with master students working under their supervision. so-called smart hive (SH).
The data collection, in co-operation with our project part- Since each member of the management of HAoB is a ner (Natura mérnökiroda Ltd.) has started in Spring 2020. practicing beekeeper, mostly with medium sized apiaries Later, at the end of the same year, the apiary in which we (up to a few hundreds of beehives), these sessions were have been collecting the data had to be liquidated, thus, very fruitful and we gained insights into their most imin the Spring 2021 we have installed the sensors in an portant problems (e.g. diseases, irresponsible farming other apiary. practices of crop farmers, changing ecological conditions, bureaucracy and reporting obligations to the authori1.2. Motivation for this Paper ties, lack of man power, etc.) as well as their main work processes and working conditions.
During the course of our project on remote monitoring of On the other hand, beekeepers showed interest in posbeehives, several challenges have popped up which are sibilities of supporting their production using AI and DS specific for the beekeeping domain but one can expect the techniques, however, we have found out that their exmajority of them appearing in other areas of agriculture pectations, based mostly on movies and popular science, as well. Even if we assumed that there would be some were quite unrealistic from our point of view. issues slowing down our work we did not count with Probably arising from the above mentioned distorted these challenges to have such a considerable impact on expectations, we have noticed that those beekeepers who our project. had come into contact with some of the existing SH solu
It is important to mention that our project is pursued tions have felt a relatively strong mistrust towards our within the academic (i.e. public) and agricultural sectors project. This was originating from the fact that the marin an Eastern-European country with its specific opera- ket is flooded with various SH solutions which are quite tional constraints (e.g. public procurement) which might costly2 and, according to these beekeepers, do not work not be so crucial in the private sector. properly3. The usual concern raised by beekeepers re
The main idea behind this paper is to summarize the garding our initiative was that they have payed hundreds challenges we had/have to deal with in our project, pro- of Euros already for various products, however, they viding the community with a kind of a guide (i.e. “what “were useless for them” and what would be the guarantee to expect”) when planning or starting a data science (DS) that our solution would work better. or artificial intelligence (AI) related project in apiculture For such, we realized that the community of beekeepor agricultural domains, in general. It is important to ers, as well as farmers in general, have their “regional stress out that various challenges related to the computer influencers” which play an important role in the process science side of our project, such as hardware and its instal- of acceptance of new technologies. As it turned out, it lation or the complexity of beehive data analytics, have is a well-known fact amongst dealers and sellers of agribeen summarized and described in [18, 19]. While our cultural technologies and products, however, we have approach to technical issues expected within the project been not aware of it. Also, the farming community is was based on the knowledge gained from [19], we have met other issues which are equally important but not yet summarized, according to our best knowledge. Here we will devote a separate section to each of the identified challenges where we will also describe our approach as well as its outcome(s).
Thus, this paper does not contain any technical descriptions of the used models nor formal definitions of the data 1http://www.omme.hu/ (only in Hungarian) 2We have checked the available solutions and could confirm that they are relatively expensive for Hungarian (or Eastern-European) beekeepers. 3For budgetary reasons, we did not tested these solutions. Also, since the related websites and documentations to these products do not uncover enough technical details (e.g. which techniques of AI they utilize, how were they trained, etc.), we are not able to judge these products from the AI or DS point if view. more old-fashioned implying that the acceptance of new name a few, we have been pleasantly surprised regarding technologies is more cumbersome than in case of other the following matter of fact: application domains we have worked in (e.g. healthcare, manufacturing, telecommunication or services).
To sum up, the following two crucial issues, which were impeding the progress in the initial phase of the work, have been identified within our project:
not run into the issues of legal contracts, non-disclosure agreements or data protection regulations (e.g. GDPR) which are common in other application domains and, thus, we have been expecting some delay in getting the Lack of background knowledge Farmers and com- data or some degree of control over our publishing acputer scientists had only some very basic and shallow tivities. The beekeepers, we think that due to their way ideas about the other domain4. To address this challenge, of cooperation with other beekeepers and farmers (e.g. from our side, we have been studying introductory litera- who cultivate the flowering crops), were relying on a ture on beekeeping and research papers in the apiculture “gentleman’s agreement” and all of them were keeping domain. Tomáš Horváth, the second author of this paper, their word. On the other hand, they were/are expecting has absolved an education training on beekeeping (with to keep our side of these agreements as well. the duration of approximately 300 hours). This not only helped us to gain more insights into the area of apiculture, working processes and everyday problems of beekeepers 3. Hardware but also changed the attitude of beekeepers5.
Motivations and Expectations Being researchers in an academic environment, we were expecting to focus in the project on research problems solving which would lead to scientific publications. On the contrary, naturally, beekeepers expected simple solutions supporting their decisions and alleviating their manual work. We have expected that visualizing the gathered sensor data would be enough, however, the majority of beekeepers was not interested in “looking at the data” (i.e. time-series) but rather they would like to have predictions on future events and their possible outcomes based on the current status of their hives. For such, however, we would need lots of feedback and context information provided by the beekeepers, a requirement which is almost impossible to fulfill from beekeepers’ side due to their work overload 6.
We are still seeking for the best way of gathering these information from the beekeepers (e.g. implementing some gaming principles into the application). We also spent considerable time with discussing our (way of) work to the farmers stressing out its usually lengthy time span7.
On the other hand, compared to other domains such that healthcare, finance or telecommunication, just to 4We have also met hobby beekeepers working as IT engineers in their everyday job, however, this is rather an exception. 5One of the instructors of the beekeeping course allowed us to use his apiary as test site for the project. 6It is important to note that for the majority of beekeepers in Hungary, even with around a hundred beehives in their apiary, beekeeping is still a supplementary occupation besides their regular jobs. 7For example, the main beekeeping season, i.e. flowering of the main crops, lasts usually for about 3-4 months while this time is not 8The decision on how much beekeepers are willing to pay for a SH enough for developing and testing the prototype nor an in-depth solution is also influenced by the season. If the season is very bad, research. as was the year 2021, this sum is very low or even zero. There are several issues regarding the hardware which are important when planning or conducting this kind of project. Almost all of these have been covered in [19], however, mainly from the point of view of technical execution, adequacy of current solutions as well as perspectives for future development. It is also worth noticing that the majority of related work in precision beekeeping are either commercial solutions providing a generic data collection and analytics platform or scientific papers focusing on certain aspects of beekeeping. However, in the former case, commercial solutions usually do not consider the diferences in beekeeping practices related to various subspecies of honeybees or regional as well as geographical conditions, as would be optimal and is also stated in [19].
To sum up, the following challenges related to hardware are the most crucial in our project: The price-value trade-of It would be optimal to have all the hardware installed in each hive, however, there are some financial limits Hungarian beekeepers are willing to accept8. On the other hand, various equipments are in diferent price ranges. For example, a weighting scale, quite important for measuring the honey yield, is more expensive than a temperature sensor. We have to experiment with several types of sensors since the sensitivity and precision of these were oscillating on a rather wide range and, thus, it took time to find the precise and afordable equipment for our project. Keeping this in mind, we recommend to acquire more types/brands from the same sensor to not waste time in case a specific type would not work reliably.
Public procurement vs. Seasonality Being a pub- partner, are still working on the question of positioning lic institution, all hardware acquisition is required to be the sensors within the beehive. An other important ismade through public procurement which, usually, takes sue is paying attention on the work ergonomics during time. It is an especially important issue due to the season- beekeeping. Sensors shouldn’t be in the way when the ality of beekeeping where the hives shouldn’t be opened beekeeper is extracting the frames for inspection nor the below 16 ∘C and, thus, the hardware can not be installed installations outside the beehive (see [20], for example) during the whole year. For such, we have decided to should cause any accidents such that, e.g., falling over on buy the cheap equipment, the sensors, from our personal cables or cumbersome removal of the parts of the beehive budget. While the more expensive hardware, e.g. the 4G due to outer parts of the hardware being in the way. It router with the pre-payed SIM card, acquired through is extremely important when the SH solution is being public procurement have arrived9 we have been installing installed in a “non-hobby” apiary when the efectiveness and testing the sensors in laboratory conditions and, af- of the production is essential. ter, used our own 4G router when these sensors have been installed at the apiary.
Beekeeping-specific circumstances When planning a precision farming project one should keep in mind Positioning and Ergonomy There are many types some “farm-specific” conditions which might influence of beehives, as illustrated in Figure 1, each of which the technical infrastructure. For example, weather condihave their specifics and, thus, influences the way sen- tions (humidity, hot, cold) might cause the hardware to sors should be placed within them. Moreover, various break down or go of, domestic or wild animals within parts within the beehive are utilized by the bees for vari- the apiary can damage the equipments and, especially ous purposes such that, for example, honey and pollen in the case of nomadic beekeeping, it might happen that storage, queen’s nest and drone hatchery. Also, it is nat- the connection to electricity or internet is insuficient. ural that bees are regulating the temperature and the Another issue mentioned in [21], specific to beekeeping, humidity within the hive, concentrating more to the part is that bees have the tendency to cover foreign objects inwhere the queen and its nest is located. As a result, it side the hive with propolis (bee glue) and/or seal wholes might happen that there are diferent conditions in dif- with beeswax which might influence or distort sensor ferent parts of the beehive (e.g. more humid and warm measurements. To avoid this, placing sensors in queen near the entrance in a hot or rainy day, respectively). expedition cages, as proposed and done in [21], or some Moreover, during the winter period, the queen’s nest, custom-made covers, as can be seen in the Figure 3, is forming a ball shape the outer temperature of which is desirable. diferent from its inner temperature, is moving slowly towards the remaining storage of honey, usually, drawing away from the entrance of the hive. Thus, it is important that sensors are located in the right places of the beehive. Regarding this, we, together with our beekeeper
especially, of a large quantity of recordings10.
Another information which could be beneficial for deeper data analytics is the production data, such that, the yields and quality of honey from various crops, the history of sanitary and medical treatments of the bees (including the types and dosages of the used substances and agents), the history of feeding11 (indicating the type and amount of nutrition), etc. However, even in larger apiaries, these data are either logged in a paper notebook or not logged at all. Our project partner, the Naturami mérnökiroda Ltd. is currently developing an application for “user-friendly” gathering of such data helping the beekeepers to keep track of inspections in their apiaries, thus, in the future, we will have access to these types of data as well.
(see Figure 2, such that temperature and humidity readings, video as well as audio recordings, from which, so 10Here, some strategies from the area of active learning [22, 23] can far, we have been focusing mainly on the audio data. The 11bBeeeukteileizpeedr,shuoswedevteor,hietlwpotoulmdanienetdaianpthroepceorluonseiersi’ncteornfdacitei.ons by benchmark data we are using consists of 10.000 audio feeding them using various types of nutrients (mainly sugar and ifles of length of 8 seconds, recorded from 10 beehives. some vitamins).
However, the data are not labeled (yet) due to the follow- 12To reach our test site apiary, located 30 kilometres from our working: During the beekeeping season the farmers are too place, we usually drive about half an hour by car, depending on the trafic. busy with their everyday work in the apiary. However, 13Beekeepers usually have a few additional protective clothes which after the season is over and they would have more free are not in use besides their apiary. time, it is a quite cumbersome job to label historical data, 14This disease is not rare in Hungary, nor in the EU, causing lots of harm to beekeepers. One of our beekeeper partner, at the beginning apium), the hives with all of their accessories should be knowledge from the areas of apiculture and electrical burned due to spores of the disease that remain viable for engineering. Thanks goes also to José Luis Seixas Junior up to decades (see Figure 4). Only those equipments can and Wadie Skaf. be kept which are able to withstand the flames (e.g. metal tools), which is not the case of the used sensors. Thus, one “Application Domain Specific Highly Reliable IT Somight count with this possibility as well (including how lutions” project has been implemented with the support to handle such case in the inventory of the university) provided from the National Research, Development and and have some backup hardware acquired. Innovation Fund of Hungary, financed under the Thematic Excellence Programme TKP2020-NKA-06 (National Challenges Subprogramme) funding scheme.
We did not provide any technical details from our research [16] but, rather, were focusing on technical and organizational issues which have delayed our work on the project even if we did not assume them to be so crucial or did not expect them at all. These problems and challenges can be grouped into four groups such that i) mutual understanding and trust between computer scientists and farmers, ii) hardware related issues, iii) lack of information besides the sensor data and iv) sudden events. Where possible we were indicating how did we approach these challenges.
We believe that this work will be useful for computer and data scientists in planning and carrying out their projects related to precision beekeeping or precision farming, in general.
Farkas and Sándor Kerekes for their help and support in our project by providing us with valuable domain of our project, has lost his whole apiary consisting of few hundred beehives. for an iot-based farm service, Sensors 20 (2020) 275 (2019) 012016. URL: https://doi.org/10.1088/ 21–34. 1755-1315/275/1/012016. doi:10.1088/1755-1315/ [10] M. D. Allen, The behaviour of honeybees preparing 275/1/012016.
to swarm, The British Journal of Animal Behaviour [21] S. Gil-Lebrero, F. J. Quiles-Latorre, M. Ortiz4 (1956) 14–22. López, V. Sánchez-Ruiz, V. Gámiz-López, J. J. Luna[11] L. Hord, E. Shook, Determining honey bee behav- Rodríguez, Honey bee colonies remote monitoring iors from audio analysis, National Science Founda- system, Sensors 17 (2017). URL: https://www.mdpi. tion Research Experience for Teachers. Appalachian com/1424-8220/17/1/55. doi:10.3390/s17010055.
State Univ (2013). [22] B. Settles, Active Learning, Synthesis Lectures on [12] R. Michael-Thomas, B. Martin, N. M. Ian, R. Mar- Artificial Intelligence and Machine Learning, Moritza, P. Maryline, C. Didier, D. N. Simon, L. C. gan & Claypool Publishers, 2012. URL: https://doi. Yves, The prediction of swarming in honeybee org/10.2200/S00429ED1V01Y201207AIM018. doi:10. colonies using vibrational spectra, Scientific Re- 2200/S00429ED1V01Y201207AIM018. ports 10 (2020) 9798. doi:https://doi.org/10. [23] S. Zhao, T. Heittola, T. Virtanen, Active learn1038/s41598-020-66115-5. ing for sound event detection, 2020. URL: https: [13] D. Senger, C. Johannsen, T. Kluss, Anomaly detec- //arxiv.org/abs/2002.05033. doi:10.48550/ARXIV. tion at the apiary: predicting state and swarming 2002.05033. preparation activity of honey bee colonies using low-cost sensor technology, in: 2022 IEEE Conference on Technologies for Sustainability (SusTech), 2022, pp. 1–7. doi:10.1109/SusTech53338.2022.
9794223. [14] B. S. Soares, J. S. Luz, V. F. de Macêdo, R. R. V. e Silva,
F. H. D. de Araújo, D. M. V. Magalhães, Mfcc-based descriptor for bee queen presence detection, Expert Systems with Applications 201 (2022) 117104. URL: https://www.sciencedirect.com/ science/article/pii/S0957417422005085. doi:https: //doi.org/10.1016/j.eswa.2022.117104. [15] A. Zgank, Iot-based bee swarm activity acoustic classification using deep neural networks, Sensors 21 (2021) 676–689. doi:https://doi.org/10.
3390/s21030676. [16] D. T. Várkonyi, J. L. J. Seixas, T. Horváth, Dynamic noise filtering for multi-class classification of beehive audio data, Submitted to Expert Systems with
Applications (2022). [17] T. Baltrušaitis, C. Ahuja, L.-P. Morency, Multimodal machine learning: A survey and taxonomy, 2017.
arXiv:1705.09406. [18] H. Hadjur, D. Ammar, L. Lefèvre, Toward an intelligent and eficient beehive: A survey of precision beekeeping systems and services, Computers and Electronics in Agriculture 192 (2022) 106604. doi:https://doi.org/10.1016/j.
compag.2021.106604. [19] A. Zacepins, V. Brusbardis, J. Meitalovs, E. Stalidzans, Challenges in the development of precision beekeeping, Biosystems Engineering 130 (2015) 60–71. [20] S. Cecchi, A. Terenzi, S. Orcioni, S. Spinsante, V. M.
Primiani, F. Moglie, S. Ruschioni, C. Mattei, P. Riolo, N. Isidoro, Multi-sensor platform for real time measurements of honey bee hive parameters, IOP Conference Series: Earth and Environmental Science