This paper presents a Mobile Augmented Reality (MAR) system for coastal erosion visualization based on geographical data. The system is demonstrated at the beach of Georgioupoli in Chania, Crete Grece, in challenging, sunny, outdoors conditions. The main focus of this work is the 3D on-site visualization of the future state of the beach when the shoreline will inevitably progresses in-land based on the impact of severe coastal erosion, taking place across the Mediterranean sea but also worldwide. We feature two future scenarios in three locations of the beach. A 3D sea segment is matched to the user's actual position. The visualization as seen through a smartphone's screen presents an unprecedented seamless view of the 3D sea segment joined with the real-world edge of the sea, achieving accurate registration of the 3D segment with the real-world. Position tracking is performed by utilizing the phone's GPS and the computer vision capabilities of the presented AR framework. A location-aware experience ensures that 3D rendering is space-aware and timely according the user's position at the coast. By combining AR technologies with geo-spatial data we aim to motivate public awareness and action in relation to critical environmental phenomena such as coastal erosion.
• Computing methodologies Mixed / augmented reality; • Applied computing Interactive learning environments; Environmental sciences.
Reality and Virtuality are the two edges of a continuum, the one describes the physical-real world and the other a non physicalvirtual environment. Cross-Reality is the bridging of those two worlds and stands between the two edges. In this presented work, we introduce a cross-reality interface where the users can interact with a virtual environment through their smartphones while they reside on the real location of the environment and witness the scenery to change. The virtual world combines with the real one to provide a cross-reality experience where the user is able to view the future of a speci c coast area using Augmented Reality technology. The virtual content that presented alongside with the real scenery depicts the future state of the coast and manages to visualize the coastline changes on top of the physical world. The virtual environment enhances visually the real world in real time, while the user can interact on both of them. Achieving to provide virtual content without extinguish the real-world factor, is essential on cross-reality interaction. 1
Mobile Augmented Reality (MAR) is an open research area due to
the emergence and widespread uptake of smart-phones that
provide powerful platforms for supporting Augmented Reality on a
mobile device. Littering behavior is a global issue a ecting most
countries, regardless of their development status. Despite the wider
applications of MAR in areas such as cultural heritage [
The phenomenon this paper focuses on, is the erosion of the coastal zone and the tremendous changes of the coastline. Across the Mediterranean sea, coastal erosion will increase provoking disastrous outcomes for the regions. The coastline is the physical line where the land meets the sea. Nowadays, coastline extraction and tracking of its changes has become of high importance because of global warming and rapid growth of population. Our goal was to visualize the shoreline at a speci c spot in Crete, Greece, in its near future state, on the basis of mathematical models that showcase the beach retreat prediction, e.g. the tendency of the beach to erode, without any human corrective measures, enhancing public awareness.
We propose a MAR system for the visualization of coastal erosion on-site (Fig. 1), putting forward successful location-aware recognition outdoors, based on geo-referenced spatial location data, addressing MAR challenges such as occlusions, large variations in lighting, the impossibility of modifying the environment, as well as unpredictable weather conditions and pollution.
The proposed work is based an on-site Mobile Augmented Reality (MAR) system for the 3D on-site visualization of the future shoreline when it will inevitably progresses in-land , o ering a seamless view of the 3D sea segment joined with the real-world edge of the sea (Fig. 1). The system is designed to operate in challenging outdoors, sunny conditions, using a consumer’s handheld smartphone or tablet supporting both Android and iOS devices. The MAR system presented consists of two main phases (Fig. 2). The rst phase guides the user’s navigation on the beach. Once at one of the set Points of Interest (PoIs), the second phase allows the user to experience the visualization, after a brief process of calibration as shown in steps 1-4 in Fig. 3). Over three PoIs, the MAR system visualizes two possible future scenarios of the coastline for sea level elevation by 0.5 meters and 1 meter, where the coastline is estimated to penetrate 3.6 and 7.7 meters inland, respectively (Fig. 4).
The navigation scene was developed using Mapbox SDK for Unity3D, suitable for building systems from real-world map data, enabling interaction with Mapbox web services APIs (Maps, Geocoding and Directions APIs) via a C#-based API. Having access to a device’s GPS, an area map is loaded based on the user’s geo-location. Pointsof-interest (POIs) are added guiding the user to where the AR visualization takes place. The Directions API provides directions from the user’s geo-location to the POI’s geo-location while a script calculates the distance between them at every second.
Planar faces in the real world (the ground, walls etc.) were recognized based on plane detection. The user’s position and orientation in physical space were tracked by the smartphone’s motion tracking. Then, the virtual content appeared on top of the recognized physical world. In order to enable Unity3D’s AR Foundation’s functionality, an ’AR Session’ component controlled the life-cycle and con guration options for the AR session and an ’AR Session Origin’ component represented the device. The user viewed and interacted with the 3D scene using the GameObject that contains this component. Rotation or movements of this object represents rotation and movement of the user in the scene. The ’AR Camera’ GameObject represents what the user sees rendered through the camera.
Attached to the ’AR Session Origin’ object, there were an ’AR Plane Manager’ and an ’AR Point Cloud Manager’ component. The ’AR Plane Manager’ selects the data about the scanned planar surfaces adding each plane that has been detected into a list, creating a GameObject for it. The ’AR Point Cloud Manager’ collects data about feature points scanned by the device and a point cloud is created for depth recognition and tracking to the app. The interaction with the real-world was achieved by ray-casting the tracked planes. A ray is sent from the center of the ’AR Camera’. If a tracked plane is hit by it, then we are able to interact with this speci c plane. Here, this method tracks a plane surface (mostly the ground) and shows an indicator at the point where the ray hits the plane, updated in every frame following the device’s pose. If there is no tracked plane, the indicator disappears. While the indicator is active, the ’Place Here’ button is activated and the user places the virtual sea content at the pointed direction, aligned with the real shoreline. The virtual content is instantiated at the orientation of the indicator (steps 2,3 Fig 3) and the virtual shoreline is placed where the indicator points to. The user relocates the virtual content if the match with the real world is o by pressing the setting button (step 3 of Fig. 3). An intuitive user interface (UI) helps navigation (see Fig. 3). During step 4 of Fig. 3, three UI signs are enabled by pressing the ’info button’ at the top right of the screen. A ray starts from the touched position of the sign on screen. If the ray hits a sign, a window pops up providing information about coastal erosion (Fig. 5).
A terrain was created based on the shape of the beach at the
speci c location [
( = 0.05U 2 + 8.12U
0.46 U : Sea Level Rise (SLR) in meters (1)
The image (Fig. 4) is geo-referenced to the Hellenic Geodetic Reference System 1987 (HGRS87) and is showcasing the retreat of the beach in two scenarios (SLR = 0.5 and SLR = 1). The image was imported in Unity as 1:1 scale so 1 unit in the engine corresponds to 1 meter in the real world. For each possible scenario, an elevation layer was created as well as one corresponding to the current state of the shoreline. A realistic looking water shader was created and attached to a planar surface acquiring water properties such as re ection, transparency, waviness, foam e ects on collision etc. In order to animate the rising of the water, the lerping method gradually moves an object from one position to another during a time window at a given speed.
We received feedback about the functionality of the application and its usefulness, involving users at the beach, using the think aloud usability evaluation methodology. Users involved were either non-experts with AR technology or experts in AR. The non-expert users were fascinated while the experts focused on the functionality. Certain users mentioned that they would prefer an AR head mounted experience while others had no problem with the use of the smartphone. Initially, users found it challenging to accurately place the virtual content depending on the location, mostly due to the physiology of the area. After training, they easily used the app and navigated around. We received enthusiastic feedback concerning the photorealistic 3D water and its seamless integration with the real-world shoreline. The signs and the UI in general were simple and intuitive, communicating the impact of coastal erosion. Certain users preferred the UI to be assigned more visible colours.
The application was hard to use during bad weather (clouds, wind etc.). When the sea was wavy, it was hard to accurately anchor the 3D content as it was drifting in the scene. Tracking and registration in AR are far from solved. Future work could automate shoreline detection, exempting user from the calibration process. Addition of more scenarios and locations. Lighting of the AR digital content can be improved for a stronger feeling of depth and better photorealism.
Concluding, we showcased the design of a mobile Augmented Reality application aimed for consumer-grade mobile phones with the ultimate goal of increasing the environmental awareness of the public audience. By employing AR, we enhance user awareness of coastal erosion, bridging the gap between reality and virtuality of widely available XR technologies.