=Paper=
{{Paper
|id=Vol-3146/PAPER_05
|storemode=property
|title=Method for the Registration and Analysis of Aerial Images, applied to the Architecture of Construction Sites, using Low-Cost Devices
|pdfUrl=https://ceur-ws.org/Vol-3146/PAPER_05.pdf
|volume=Vol-3146
|authors=Aly Auccahuasi,Wilver Auccahuasi,Lucas Herrera,Teddy Esteves,Richard Aguilar Paredes,Sandra Meza,Christian Ovalle,Karin Rojas,Marco Felippe,Pedro Flores Peña,Yuly Montes Osorio,Francisco Hilario,Milner Liendo,Esteban Medina Rafaile
|dblpUrl=https://dblp.org/rec/conf/wai/AuccahuasiAHEPM22
}}
==Method for the Registration and Analysis of Aerial Images, applied to the Architecture of Construction Sites, using Low-Cost Devices==
https://ceur-ws.org/Vol-3146/PAPER_05.pdf
Method for the Registration and Analysis of Aerial Images,
applied to the Architecture of Construction Sites, using Low-
Cost Devices
Aly Auccahuasia, Wilver Auccahuasib, Lucas Herrerac, Teddy Estevesd, Richard Aguilar
Paredese, Sandra Mezaf, Christian Ovalleg, Karin Rojash, Marco Felippei, Pedro Flores Peñaj,
Yuly Montes Osoriok, Francisco Hilariol, Milner Liendom and Esteban Medina Rafailen
a, b, i,
Universidad Privada del Norte, Lima, Perú
c
Universidad Continental, Huancayo, Perú
d, h, l
Universidad César Vallejo, Lima, Perú
e
Universidad Tecnológica del Perú, Lima, Perú
f
Universidad Científica del Sur, Lima, Perú
g
Universidad Autónoma de Ica, Ica, Perú
j
Universidad Nacional Mayor de San Marcos, Lima, Perú
k
Universidad ESAN, Lima, Perú
m
Universidad Privada san Juan Bautista, Lima, Perú
n
Universidad Nacional Santiago Antúnez de Mayolo, Áncash, Perú
Abstract
FPGA-based hardware architectures are being used more frequently in many applications,
thanks to the different programming languages that allow us to access them. The applications
are also being varied, one of the most common areas of work with aerial images, normally
acquired with cameras that are on board drones, the working mode of these configurations, is
focused on being able to use visualize the images online As the flight is carried out, if you
want to carry out some type of processing, it is necessary to download the image from the
camera's memory, this process is already carried out when the drone has finished the flight.
In this work we present a methodology to be able to use the low-cost hardware myRIO, since
it has a processor and an FPGA included, we present the steps to be able to work with the
device, as well as an example of online processing so that the video and images that the drone
camera can process on board and transmitted online, so that images and videos are processed
while the drone is in flight, thereby improving the performance of the drone and can be
applied In special operations, as a result we present the device configuration and the results
of an example.
Keywords 1
Aerial image, online process, FPGA, development, video.
1. Introduction
The use of technology is achieving new ways of working and providing new alternatives achieving
better results as well as reducing processing times, aided by hardware and software that allow them to
be applied to new applications, we find jobs where online processing is performed for the process of
signals and images using low-cost and highly integrated devices [1].
WAI-2022: Workshop on Artificial Intelligence, January 27 – 28, 2022, Chennai, India.
EMAIL: wilver.auccahuasi@upn.edu.pe (Wilver Auccahuasi)
ORCID: 0000-0001-8820-4013 (Wilver Auccahuasi)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
39
Automatic pilots or also called unmanned aerial vehicles (UAV) have critical systems for safety
with reliability and safety requirements, these processes are considered complex, expensive and takes
a lot of time, therefore we present an automatic test platform of the system of automatic pilots with
the aim of making an improvement on the efficiency and safety of UAVs, for which it proposes a
unified modeling of the various types of aerial vehicles which are shared modeling experiences and
existing failure modes, a platform was used of real-time simulation using the automatic code
generation and the method of hardware simulation in the cycle based on FPGA in order to guarantee
the credibility of the simulation at both levels as software and hardware for which a framework of
automatic test with evaluating test cases during flight simulation performed in real time where ev We
will assess the results of the test, then the verification, precision and credibility of the simulation
platform is carried out, obtaining results of the experiment successfully in multicopters that
demonstrate the viability of the proposed platform [2].
Works where complex calculations on image filtering are analyzed, which cannot be operated
efficiently according to the scenario of each image and videos in real time with large size, for which
we present a scheme that was based on the Large Single Image Compute Acceleration Bilateral
Filtering (CABFD), the goal is to defog large video in real time by smart tv, at the same time an
alternative on simplified filtering and FPGA defog architecture has been introduced, After making the
comparison of the proposed method about the fast demisting of large images, it is effective and
practical, this architecture is applied directly for the visualization of Full HD video (1920 × 1080)
with synchronous demisting in real time [3] .
Applications in precision agriculture, with which fertility can be improved through technical
monitoring and cultivation tests, so that the balance will do the job in an essential way with
ingredients essential for humans or other sectors, for which a data analysis through nano and
automatic learning developed through the Internet, with the Contribution to Smart Farming and
Things Internet, which will help to create a viable intelligent agricultural system for the system [4]
[5].
We find jobs using unmanned aerial vehicles (UAV) used in civil construction, for which we will
analyze alternatives for the implementation of image processing algorithms for the detection of cracks
in the facades of buildings, which must be executed on a platform computing integrated and installed
in UAV, 2 algorithms have been selected about image processing about crack detection, one of the
versions runs in Matlab environment on a desktop computer using an image processing approach
which is performed in the ground and the UAV and is only applied for image acquisition taking as a
baseline for comparison with the implementations executed in an integrated processor and with
implementations in a Xilinx FPGA board installed in the UAV, which have defined various scenarios
for the execution of inspection tasks of building facades and the results obtained were presented still
at work [6].
Small-scale helicopter unmanned robotic systems (SRURS), robotic systems unmanned using
manipulation devices, so the objective of the review is to provide an overview about the manipulation
area of the SRURS promoting research, therefore that we provide a review about the literature of the
last 10 years about the SRURS, and details the achievements and challenges, from where the state of
the art, development, classification and challenges of the SRURS were analyzed, I have also reviewed
the Relationship articles which have been organized into 2 categories that are design of mechanical
structures and modeling and control, then a summary and classification is made in the form of tables,
which has been presented in 7 parts, has been compiled and presents trends and challenges which are
used as a resource for researchers who are interested in aerial manipulation of SRURS, considering
that the problem at hand About trends and challenges are described in 3 aspects, allowing conclusions
to be drawn about the effectiveness of the proposed systems [7] [8].
The methods and technologies used in aerial robotics applications were reviewed about unmanned
aerial vehicle platforms, summarizing the different control techniques, where we included control
architectures and control methods, artificial vision techniques were also examined [9]
40
Low bit depth images for image processing applications which offer advantages over full depth,
reduced data transmission, elimination of superfluous details and improved compressibility with
potential reduction of FPGA resources used on small platforms, thus demonstrating descriptors about
the scalable key points considered as binary robust invariables (BRISK) which coincide directly
without modifications between images with different number of gray levels where we evaluate the
performance obtaining the sufficient number of control points for an image registration, which they
can be used to determine the direct equivalence of an unknown, unaligned, reduced palette image as a
reference image called the upper bit depth [10].
This proposal analyzes the use of the low-cost Myrio card, which has an integrated FPGA, the
device is programmable through the LABVIEW software, the proposal allows the device to be
attached to a drone, the video is captured from the drone camera , which can be remotely selected the
image to be analyzed, this image is sent to the device and analyzed, and the result is sent to the
operator, from where the device can operate in this way instead of sending the video to be analyzed ,
the process is done on board and only the final result is sent, which is the processed image.
2. Materials and Methods
The materials and methods are based on Figure 1, which indicates the steps to follow to explain in
detail the proposed methodology, where each of the procedures will be solved so that it can be applied
in other problematic situations.
Figure 1: Block Diagram of the Proposal
2.1. Analysis of the problem situation
In the process of using drones, many solutions arise, depending on the problem and the
application. In the market we have many solutions based on the use of video recording, to be able to
exploit it after the recording, another of the alternatives is the visualization of the online video as the
drone is making the flight, the images are visualized by the operator.
Here we present for the mode of operation of drones in a practical way:
• In image 2, the drone is presented ready to go to flight, so it is necessary the configurations
and flight patterns, as well as the parameters of the recording and transmission of the video.
• In figure 3, the drone is shown in flight, where a night flight is shown in order to test the
transmission of the video.
• In Figure 4, shows the control of the drone, as well as the transmission of the video where the
controls such as selection of recording angles, recording times and recording areas are
presented.
41
Figure 2: Image of the drone ready to fly
Figure 3: Image of the drone in flight
Figure 4: Image of the video sent by the drone
2.2. Video Acquisition
The acquisition of the video is composed of the recording in flight and the download for its
subsequent analysis and interpretation, the present proposal is characterized in being able to analyze
the video in flight and in real time, in such a way that the operator can select the image, this image
can be sent to the FPGA device, the architecture of the myRIO device allows a connection via USB,
42
where the image to be analyzed can be sent, in this sense the device can perform the processing and
returns a resulting image ready to be analyzed and interpreted, with this proposal it is not necessary to
download the video to be able to be analyzed, the entire process is carried out in flight and in real
time.
Figure 5: Aerial view recorded by the drone
The architecture presented in figure 6 is based on the myRIO device, which features a processing
unit, a memory unit, an FPGA unit with its internal memory as well as input and output units, this
architecture allows to perform the reception of images and videos, the internal processing and the
sending of final results. In the application example, the operator captures an image, which is sent to
the device, which performs a search for areas of interest, returning a resulting image, so that it can be
analyzed and used, the architecture of the device allows this operation, where the FPGA unit is used
for image processing.
Figure 6: myRIO device architecture
2.3. Online Processing
The process that is carried out in the drone is characterized by the analysis of the color image, as
can be seen in figure 7, the process begins with the decomposition of the image in its respective color
bands, such as Red, Green and blue. Taking each one of them, the analysis is carried out to obtain the
one that presents the most information, in the case of the aerial image that we have, it corresponds to a
night shot of the train tracks that passes over the city, in the image of color is presented in black,
performing the analysis in each of the color bands, the band that has more information about the train
43
rails is the red band, it is in this band where the binarization process is carried out, in order to
determine the status of the rails in real time, without having to wait for the drone to finish the video.
In figure 7, the diagram of the processes carried out in the image is presented, from the reception
of the color image, passing through the decomposition in their respective bands and ending with the
result of the binarized image.
Figure 7: Image process diagram
2.4. Sending Results
The final result of the proposal is the final image, which is a processed image that the device
returns after having performed the different algorithms, one of the advantages of the myRIO device is
the architecture and ease of programming, thanks to the practicality of its architecture, you can carry
out the programming with Labview and download it to the device, with this method it is possible to
perform different procedures to treat the image and have as a result an image or a set of images
converted into video, this procedure of delivering the result of the procedure is carried out with the
outputs of the device, such as the USB interface.
The sending of the processing result is an important factor with which it can be sent directly to the
drone operator, so that he can make decisions in case of an emergency situation. The architecture of
the device allows a wireless connection through the WIFI protocol, through which the result of the
process can also be transmitted either in image or video format.
44
3. Results
The results that are presented are related to three aspects, first with the procedures to be able to
work with the myRIO device, second with the advantages that the device provides and finally with the
operation and working mode of the device.
Figure 8: Initial device configuration
Figure 9 shows the start of the myRIO device configuration, where it presents the configuration of
two programming modes, if programming is used in the microcontroller, otherwise programming in
the FPGA unit, in our case it is necessary to select the FPGA mode, this choice is very important
since the programming will be available to use the FPGA resources.
Figure 9: Final application image
In figure 9, we present the programming of the program in Labview where the FPGA control
commands appear, as well as the input of the image and the result of the process, this program is the
45
main program, it must be considered that when the program is executed on the computer and the
results are satisfactory, it is downloaded to the device and begins to work online without the need for
the device to be connected to a computer and less with a screen, for this reason it is important that the
main program is working without problems.
4. Conclusion
We come to the conclusion where among the devices based on the FPGA architecture are diverse,
so the device worked with the myRIO is practical both in programming and in the use of the device
for the final application, added to the low cost compared to other architectures, But the most
important part is the use and the multiple applications that must be considered, based mainly on the
programming with Labview, it allows to fully exploit the different input and output units as well as
the universal input that it has where you can connect many devices of storage as well as being able to
connect cameras.
The application that is presented uses the universal USB input, where an application is presented
where it is necessary to process an image online and return a processed image, these applications are
recommended when it is necessary to know the status of some situations considered emergency, in the
search for some patterns, where it is necessary to know their location as well as their existence, the
proposed method is based on the decomposition of a color image, then on the interpretation of each of
the bands and finally the processing in one of them, to finally conclude with the delivery of a
binarized image with the necessary information to make a decision.
Finally we can indicate that the FPGA architecture helps in the processing as well as in the
response time both in the programming level and in the execution, we recommend working with the
device to start programming with FPGA as well as in the use of embedded systems where It is
required to work alongside other platforms such as drones, in our case the energy consumption is
minimal and it receives the power of the drone battery, because the program uses the resources
necessary for the application, the method can be widely used as well as scaling to other applications
where online and real-time processing is required.
5. References
[1] Aiquipa, W. A., Flores, E., Sernaque, F., Fuentes, A., Cueva, J., & Núñez, E. O. (2019, October).
Integrated Low-Cost Platform for the Capture, Processing, Analysis and Control in Real Time of
Signals and Images. In Proceedings of the 2019 2nd International Conference on Sensors, Signal
and Image Processing (pp. 35-39).
[2] Dai, X., Ke, C., Quan, Q., & Cai, K. Y. (2021). RFlySim: Automatic test platform for UAV
autopilot systems with FPGA-based hardware-in-the-loop simulations. Aerospace Science and
Technology, 114. https://doi.org/10.1016/j.ast.2021.106727
[3] Liu, H., Huang, D., Hou, S., & Yue, R. (2017). Large size single image fast defogging and the
real time video defogging FPGA architecture. Neurocomputing, 269, 97–107.
https://doi.org/10.1016/j.neucom.2016.09.139
[4] Yin, L., & Zhang, Y. (2020). Village precision poverty alleviation and smart agriculture based on
FPGA and machine learning. Microprocessors and Microsystems.
https://doi.org/10.1016/j.micpro.2020.103469
[5] Gultekin, G. K., & Saranli, A. (2013). An FPGA based high performance optical flow hardware
design for computer vision applications. Microprocessors and Microsystems, 37(3), 270–286.
https://doi.org/10.1016/j.micpro.2013.01.001
[6] Pereira, F. C., & Pereira, C. E. (2015). Embedded image processing systems for automatic
recognition of cracks using UAVs. In IFAC-PapersOnLine (Vol. 28, pp. 16–21).
https://doi.org/10.1016/j.ifacol.2015.08.101
46
[7] DING, X., GUO, P., XU, K., & YU, Y. (2019, January 1). A review of aerial manipulation of
small-scale rotorcraft unmanned robotic systems. Chinese Journal of Aeronautics. Chinese
Journal of Aeronautics. https://doi.org/10.1016/j.cja.2018.05.012
[8] Jiang, H., Miao, R., Chen, J., Zhang, C., Hu, X., Ouyang, J., … Lu, J. (2019). A resource-
efficient parallel architecture for infrared image stripe noise removal based on the most stable
window. Infrared Physics and Technology, 97, 258–269.
https://doi.org/10.1016/j.infrared.2018.12.035
[9] Ollero, A., & Merino, L. (2004). Control and perception techniques for aerial robotics. Annual
Reviews in Control, 28(2), 167–178. https://doi.org/10.1016/j.arcontrol.2004.05.003
[10] E. J. Griffith, Y. Chi, M. Jump and J. F. Ralph, "Equivalence of BRISK Descriptors for the
Registration of Variable Bit-Depth Aerial Imagery," 2013 IEEE International Conference on
Systems, Man, and Cybernetics, 2013, pp. 2587-2592, doi: 10.1109/SMC.2013.442.
47