=Paper=
{{Paper
|id=Vol-2030/HAICTA_2017_paper91
|storemode=property
|title=Cultivation Optimization Using Augmented Reality
|pdfUrl=https://ceur-ws.org/Vol-2030/HAICTA_2017_paper91.pdf
|volume=Vol-2030
|authors=Alexander Katsaros,Euclid Keramopoulos,Michail Salampasis
|dblpUrl=https://dblp.org/rec/conf/haicta/KatsarosKS17
}}
==Cultivation Optimization Using Augmented Reality==
https://ceur-ws.org/Vol-2030/HAICTA_2017_paper91.pdf
A Prototype Application for
Cultivation Optimization Using Augmented Reality
Alexander Katsaros1, Euclid Keramopoulos2, Michail Salampasis3
1
Yeometry Research & Development, Kerkyra, Greece, e-mail: research@yeometry.com
2
Department of Informatics, Alexander Technology Educational Institute of Thessaloniki,
Greece, e-mail: euclid@it.teithe.gr
3
Department of Informatics, Alexander Technology Educational Institute of Thessaloniki,
Greece, e-mail: cs1msa@it.teithe.gr
Abstract. Rapid development of mobile devices and the emerging trend of
ubiquitous computing has evolved technologies that are always and
everywhere available. Augmented Reality and Internet of Things (IoT) are
such technologies, which either enrich the real-world environment with useful
information or receive from the real-world environment useful information.
The agricultural sector is an area where those technologies can be applied with
tremendous benefits. In this paper, we combine Augmented Reality, IoT with
Semantic Web in order to support the development of knowledge bases and
intelligent applications regarding crops. The ontology in the knowledge base
will be used by our innovative application, in order to advise farmers for the
optimization of their cultivations and at the same time will provide them with
useful information whether the effort to maximize the yield of a particular crop
is economically acceptable in their area.
Keywords: Augmented reality, Agriculture, Sensors, ΙοΤ, Semantic Web
1 Introduction
The optimal growth/performance of a crop, in a small or large area, is affected by the
microclimate of the area (humidity, sunshine, pest growth, etc.). A tomato crop for
example grows very well in hot areas at temperatures between 21 and 24 °C and
requires soil to drain water with a pH between 5.5 and 6.8. However, not all regions
have the optimum conditions required for the development of a particular crop.
Moreover, if the optimal conditions are not present, additional measures (fertilizers,
pesticides) are needed to improve the conditions. When a region's economy depends
on agriculture, then the optimal growth/performance of a crop plays the most
important role in the development of agricultural products trade and affects the local
economy and finally influences the quality of life.
Computer Science supports technologies that can improve the growth/performance
of a crop. In this research, we combine three computer science technologies
(Semantic Web, Augmented Reality and Internet of Things (IoT)) in order to develop
805
an innovative mobile application that will improve the crop efficiency. More
specifically, the contribution of this paper is (a) the specification of an ontology that
will support the development and sharing of knowledge about crop performance in a
specific area, (b) the use of sensors in order to get useful information regarding the
microclimate of the area and (c) the development of an extension to our Augmented
Reality application, FarmAR (Katsaros & Keramopoulos, 2017), in order to enable
the use of the specified ontology and to collect the sensors’ data.
The paper organized as follows. In section 2, we briefly discuss the background
technologies, i.e. Augmented Reality, Internet of Things and Semantic Web. In
section 3, we address the importance of Augmented Reality in agriculture and also we
present relevant research work on this field. Next in section 4, we introduce our
prototype application, FarmAR that based on Augmented Reality, IoT and Semantic
Web Technology to improve crop efficiency. Finally, in section 5, we discuss our
preliminary conclusions.
2 Background
Augmented Reality (Milgam et. al. 1993, Benford et. al. 1998) represents the
technology which allows someone to see more of what others see, listen to more than
what others hear, and maybe touch things that others cannot. Such technology could
have a complete understanding of virtual objects within the real world, even creatures
or constructions that would help in your everyday activities and at the same time be
interactive through the movement or speech.
Augmented Reality is mainly used on mobile devices and enriches the visibility of
the natural world with digital information (texts, sounds and video). Although this
technology appeared several decades ago, it became popular in recent years, because
of the rapid spread of mobile devices and technological advances such as sensors,
cameras, gyroscopes and GPS.
Benford et al. (Benford et. al. 1998) define Augmented Reality, distinguishing it
from Virtual Reality, giving the following characteristics of an Augmented Reality
System:
• Combines virtual and real objects in a real environment.
• Synchronizes virtual objects to work in parallel with real ones.
• Interacts in three dimensions and in real time.
The term ΙοΤ (TechTarget 2017) is collectively known as the technology
allowing a network of objects that are connected to the Internet to collect and
exchange data and interact with other Internet services. It includes technologies such
as RFID, sensors and smart phones. The basic idea of IoT is that almost every
physical object in this world can also become a computer that is connected to the
Internet. To be more precise, these items are not computers, but they can have tiny
computers. The combination of IoT and Augmented Reality can enrich the user's
view with more interesting information that is currently being collected by sensors.
An interesting example is the "digital twin" (TechTarget 2017), which is mainly
found for the digital representation of products.
806
The Semantic Web is designed to be a network of interlinked structured and
semi-structured data that are semantically annotated. Also, the data structure, in this
model, stores the information for the relation and the interconnection between them.
This information is modeled through the semantic annotation, which applied with the
RDF technology at the data interconnection level, while OWL provides the semantics
for further knowledge representation (Berners-Lee et. al. 2001, Shadbolt et. al. 2006,
Bizer et. al. 2009).
Both RDF and OWL formalisms are designed having in mind applications, which
will be able to interpret and understand the information modeled in the semantically
annotated data. The goal is to provide a web of interlinked structured data, where
applications will be acting independently, on behalf of the humans and for the benefit
of the humans.
More specifically, ontologies are the structural element of the Semantic Web.
However, they are also widely used in the field of Artificial Intelligence, applications
related to knowledge management, e-commerce, information retrieval, natural
language processing and many other fields (Fernandez-Lopez & Corcho, 2010). By
ontology we mean the exact description of things and concepts as well as the
relationships that exist between them. The most well-known definition of ontology, in
computer science, on which other definitions were based, was given by Gruber
(Gruber, 2006) and is as follows:
“Ontology is an explicit specification of a conceptualization.”
3 Augmented Reality in Agriculture
Augmented Reality offers a new way of interacting between man and machine in
many areas. One of them is agriculture. In parallel with the evolvement of modern
Agricultural Technology, the Education in Agriculture can help farmers to specialize
in the use of modern agricultural production tools, farm management and improve
cognitive ability in agricultural products and also helping nonprofessional farmers to
understand modern agriculture and the variety of activities it provides. However, the
cost of education in Agriculture is very high and the conditions (generally in the field
of education), training time, place, are not flexible.
Augmented Reality in Agriculture is a very interesting research topic but not many
relevant papers have been published (Katsaros & Keramopoulos 2017, Nigam el. Al.
2011, Neto & Cardoso 2013, Li 2008, Wu et. al. 2013). Neto & Cardoso (Neto &
Cardoso, 2013) presented an interesting augmented reality application regarding
greenhouses. This application uses a network of humidity and temperature sensors to
perceive the conditions for the development of the botrytis cinerea fungus in tomatoes
and warns the farmer through his mobile device. The prototype of the application used
the Layar SDK (Augmented Reality Framework) to real-time visualize the
environmental conditions of the greenhouse microclimate and to identify any
conditions conducive to the growth of the fungus. Neto & Cardoso attempted to create
an application that approaches a Business Intelligence strategy (Neto & Cardoso,
2013), which consists of data sources (sensors), data storage and data analysis, and
807
finally the presentation of the information is developed with a combination of a
mobile device and augmented reality.
4 FarmAR Extension
The FarmAR application (Katsaros & Keramopoulos, 2017) has been developed
using the Android Studio IDE and for minimum API level 19 (Android 4.4). It
consists of two basic modules. The first part uses the Vuforia SDK (PTC Inc, "Vuforia
| Augmented Reality"), in order to identify the subject displayed on the mobile device
camera. In the second part the displayed icon augmented with specific information
based on Apache Jena (The Apache Software Foundation, "Apache Jena"), which
executes SPARQL (DuCharme, 2013) queries using as a service the endpoint of the
Virtuozo Server of the Department of Information Technology of Alexander TEI
Thessaloniki. In this section, we will introduce the FarmAR ontology, we will analyze
how works the FarmAR application and we will show the FarmAR embedded
SPARQL editor.
4.1 FarmAR Ontology
FarmAR ontology was created using the Protégé tool and is available on the
OpenLink Virtuoso Server of the Information Technology Department of the
Alexander TEI of Thessaloniki at the following link http://195.251.123.67/sparql.
FarmAR ontology can be accessed using the PREFIX http://www.semanticweb.org/
katsaros/ontologies/farmar_extented#.
The FarmAR ontology describes the following concepts:
1) Plants (In ontology is defined as Plantae): Each plant is described by data
properties such as its name, scientific name, its use, care information of the plant,
information on sowing and harvesting. Also, there is a "mayDiseased" object that
refers to the concept of "Disease" and correlates it with the possible diseases that
may affect it.
2) Disease (In ontology is defined as Disease): This term describes plant diseases
categorized into categories such as Phosphorous deficiency, Armyworms,
Anthracnose, Bacterial canker, and Barley yellow Dwarf. Each disease is described
by data attributes such as symptoms and treatment.
3) Location (In ontology is defined as Location): describes crop areas and
climatic data for these areas. The most important property of this concept is the
object property "fertile" which refers to the concept "Plants" and associates the area
with a plant and a crop index metric of that plant in that area.
4.2 Identification and Augmentation
The application has three basic functions: plant recognition and knowledge
enhancement, augmented reality of sensor data and access to the knowledge base.
808
The augmented reality service, match the photo of a plant, taken from the mobile
device camera, to a known plant and display relevant information for the identified
plant from the knowledge base (Fig. 1). The service has been implemented using
Vuforia SDK. Plant identification method is as follows: Vuforia SDK detects and
tracks the features that are naturally found in the image itself by comparing these
natural features against a known target resource database.
When the application starts, the location of the user is located. Once the plant is
identified, a SPARQL query is sent to the knowledge base and returns information
about the plant such as its name, common diseases, and crop index specific to that
area. The crop index gets decimal values from 1-10 and presents the effort required to
have the particular crop to have the maximum performance in the specific area where
the user is located. The best performance occurs when the pointer is set to 10.
Moreover, an important service of the application is to augment the reality of a
crop with data derived from sensors (temperature, humidity, soil pH, etc.). This
function helps the farmer to improve the conditions prevailing in the growing area and
at the same time contributes to the calculation of the crop index metric from sensor
data (Fig. 2).
Fig. 2. Augmented Reality using sensor information
Fig. 1. Identification of plant
species and augmentation
4.3 Knowledge Base Access
FarmAR permits the user to run SPARQL queries in the knowledge base. Selecting
the "Knowledge Base" option from the home screen is transferred to the SPARQL
editor. The user can choose between some default queries or write new ones.
809
5 Conclusion
The FarmAR application extension exploits Augmented Reality and IoT
technology to identify a number of plants and augment reality with information such
as common name, frequent plant diseases and crop index metric. Thus, a farmer can
see on the screen of his mobile device all the above information about the plant s/he
has detected with camera of her/his mobile device. The application finds all these
information in an ontology that contains an indicative number of instances and
presents them to the users.
References
1. Katsaros, Α. and Keramopoulos, Ε. (2017) FarmAR, a Farmer’s Augmented
Reality Application based on Semantic Web, ACM/IEEE SEEDA-CECNSM
International Conference, to be published.
2. Milgam, P., Zhai, S., Drascic, D. and Grodski, J.J. (1993) Applications of
augmented reality for human-robot communication, in IEEE/RSJ International
Conference.
3. Benford, S., Greenhalgh, C., Reynard, G., Brown, C. and Koleva, B. (1998)
Understanding and Constructing Shared, ACM Transactions on Computer-
Human Interaction, 5, p.185-223.
4. TechTarget, «digital twin».
Available at: http://searchmanufacturingerp.techtarget.com/definition/digital-twin
[Accessed 14 05 2017]
5. Berners-Lee, T., Hendler, J. and O. Lassila (2001) The Semantic Web. Scientific
American, 284, p.29–37.
6. Shadbolt, N., Hall, W. and Berners-Lee, T. (2006) The Semantic Web Revisited.
IEEE Intelligent Systems, 21, p. 96–101.
7. Bizer, C., Heath,T. and Berners-Lee, T. (2009) Linked data - The story so far.
International Journal on Semantic Web and Information Systems, 5, p.1–22.
8. Fernandez-Lopez, M. and Corcho, O. (2010) Ontological Engineering: with
examples from the areas of Knowledge Management, e-Commerce and the
Semantic Web. London: Springer Publishing Company.
9. Gruber, T. R. (2006) A translation approach to portable ontology specifications,
Knowledge acquisition, 5, p.199-220.
10. Nigam, A. Kabra, P. and Doke, P. (2011) Augmented Reality in agriculture. IEEE
7th International Conference on Wireless and Mobile Computing, Networking
and Communications (WiMob).
11. Neto, M.d.C. and Cardoso, P. (2013) Augmented Reality Greenhouse. EFITA-
WCCA-CIGR Conference “Sustainable Agriculture through ICT Innovation”,
Turin, Italy, 24-27 June 2013.
810
12. Li, H. (2008) Analysis of Virtual Reality Technology Applications in Agriculture.
In Computer And Computing Technologies In Agriculture Volume I, ed. D. Li.,
p.133-139, The International Federation for Information Processing (CCTA)
2007, 258. Boston: Springer.
13. Wu,S., Xiao, B. and Guo, X. (2013) An Interactive Virtual Training System
Based on Augmented Reality. The International Conference on Education
Technology and Information System (ICETIS 2013), p. 1132-1135.
14. PTC Inc, "Vuforia, Augmented Reality", PTC Inc, [Online]. Available:
https://www.vuforia.com/. [Accessed 03 01 2017].
15. The Apache Software Foundation, "Apache Jena", [Online]. Available:
https://jena.apache.org/. [Accessed 03 01 2017].
16. DuCharme, B. (2013) Learning SPARQL: Querying and Updating with SPARQL
1.1 2nd Edition, O'Reilly Media, Inc.
811