This document presents an augmented reality application for mobile devices, as a contribution to education through a technological learning tool that allows the management of industrial robotic arms, implementing advanced control algorithms, which allows the simulation of several selected desired trajectories by the user; and the incorporation of animations that allow to know its operation and to verify the follow-up of the proposed trajectory, as well as the visualization of control errors in each trajectory taken. The application is oriented to the simulation of industrial robotic arms within an intuitive and friendly augmented reality environment, which allows users a great interaction with the robot's structure, providing simulation programs with new immersion technologies, in the educational field. Tests in the augmented reality application demonstrate ease of use and user intuition, providing a better understanding of the operation and structure of programmable manipulators.
Education in recent years has been developing new milestones and ways of teaching,
the use of new technological tools immersed increases the interest and learning
experience [
AR applications are developed for specific tasks [
Second, the entertainment applications [3; 6; 13; 16]. In [
This work aims to assist in learning about industrial robots through the development of an AR application. The application allows the visualization of the components of the manipulators by means of animation and visualization of multimedia files through the 2D recognition of identification codes, as well as the simulation of the behavior of the robot structure through the implementation of advanced control algorithms that allow the mobility of the operating end towards the path desired by the user.
This work is divided into IV sections, the first section includes the Introduction, the second section called Development, details the detection technique used, the characteristics of the mobile application environment and the laws of closed loop control, the third section details the Results obtained when using the AR application through a smartphone with its respective analysis of results and, finally, the fourth section describes the Conclusions of the work done. 2
Development The proposed workflow diagram for the creation and development of the AR application is shown in figure 1, it considers five main stages with a specific task each, plus one or more processes that allow to execute the workflow tasks of the application for the smartphone:
*.fbx
*.fbx
i) Layer 1.1 allows 3D development using Blender software, which is a program
specifically designed for the development of 3D objects, rendering, animation, special
effects, etc. [
ii) Layer 1.2 describes the 2D recognition technique, which allows the detection of images and text. This image is loaded into Vuforia Developer Portal’s Target Manager, which allows the creation of databases and the generation of a Unity-compatible file, as seen in fig 3. Vuforia processes each image and generates characteristic points,
according to these characteristic points the recognition quality is given. Fig. 4 shows the standard image to be used and fig. 5 shows the image characteristics that will determine the recognition quality. iii) Layer 2 this layer describes the characteristics of the mobile application environment and the incorporation of the animations.
Characteristics of the environment, the implementation of a simple but intuitive interface allows a quick handling and navigation in the mobile application, for which Table 1 shows the icons that were used for the realization of the AR application with their respective description; and 3 main scenes are created: the first scene is presented in fig. 6 and contains the main menu of the application, the second scene allows to visualize the introduction on robotic arms through video and animation, as shown in fig. 7, the third scene allows to perform manual and path control its environment is presented in fig. 8.
Control Video Animation Play Graphics Return Exit Name
iv) Layer 3 this layer details the advanced control laws that are implemented in the AR application. To meet this objective, the kinematic model of the robotic arm and its respective stability analysis are described. 2.1
Kinematic model
The diagram proposed in fig. 9 shows a robotic arm with generalized coordinates and
with a position at the operating end, where ℎ is the position of the operating end in
space, is the length of each link and is its rotation angle, is given by:
The first-order differential eℎℎℎq̇u==a=ℎtio(=nℎ,is ,co)nws+++iditehrehd(0(,,,)1=)h.0
The implemented controller is based on numerical methods given in publication [
The use of numerical methods to calculate the evolution of the system is mainly based on the possibility of approximating the state system at the instantaneous moment k+1, if the state and control action over time at the instant k are known, this approach is called Euler’s method.
h(k+1) = h(k) + T0 f(h, u, t) h(k+1) = h(k) + T0 J(q(k)) v(k) h(k+1) = hd(k) – W(hd(k) – h(k)) Thus, the discrete model can be expressed by The following expression is used, so that the tracking error tends to zero. (2) (3) (4) (5) In which, W is the diagonal matrix and its values 0 < diag(whx, why, whz) < 1 are parameters for the proposed controller, hd is the desired path.
Recital (1) and (2), the system can be rewritten as Au = b.
( ) ( ) = ( ) ( ) ( ) ( ) Therefore, the viable solution method is to formulate it as a constrained linear optimization problem.
‖ ‖ = v) Layer 4 this layer allows the rotation angles obtained from the sliders as well as from the position and trajectory controls developed, to be incorporated to the links (extremities) of the 3D model in order to generate the respective movement that complies with the trajectory entered by the user. 3
Results obtained This section shows the augmented reality interface and the usability that it has as a technological tool to manage industrial robots, as well as the simulation of a control algorithm that allows users to follow the desired path. To get started with the app, you need to pre-install the APK on your smartphone.
When the application is run and the QR code is focused, the main scene is shown as shown in fig. 10, to exit the AR application press “exit”. Clicking on “introduction” presents a new environment called introduction, pressing the “animation” icon starts the animation of the constituent parts of a robotic arm, as shown in Fig. 11.
By clicking on “video” an introductory video is presented containing: general characteristics of a robotic arm, constitution and mathematical modeling, as shown in fig. 12.
To control the robotic arm, press “control”, this shows a scene that contains 3 sliders, as shown in fig. 13, these sliders allow to move the rotation angles of the 3 links, thus performing the manual control, as shown in fig. 14.
For the robot to perform path control it is necessary to focus the path target next to the QR image of the robot arm, when the image is recognized its path appears in 3D, fig. 15 shows the circular path. Clicking on “play” starts the trajectory control in which the operating end of the arm follows the circular trajectory as shown in fig. 16.
The graph of the x, y and z position errors in the AR application as in software Matlab can be seen in fig. 17 and Fig. 18, respectively.
The spiral path of the robot in the AR application can be seen in fig. 19. The fig. 20 and fig. 21 show the graph of the x, y and z position errors in the AR application as in the Matlab under the same conditions respectively.
The cylindrical spiral path of the robot in the AR application can be seen in fig. 22.
The fig. 23 and fig. 24 show the graph of the x, y and z position errors in the AR
application as in the Matlab under the same conditions respectively.
Next, the obtained results are shown, that that indicate the validity of the usability of
augmented reality, for handling of industrial robots, in a punctual way; in the one of
handling and training of industrial robots of three degrees of freedom. The SUS
summary evaluation method was used, whose weighting characteristics are described
in the document [
Conclusions The work presents an augmented reality application for smart phones that allows scanning 2D objects and later interacting in the handling of the robotic arm and each of its links, in addition to knowing its parts through animations and multimedia files. Finally, the application allows the visualization of the 3D animation of the robotic arm in the different trajectory control tests established by the user, such as, circular trajectory, spiral and cylindrical spiral, as well as the visualization of the 2D graphics of the control errors, in which it can be seen that the error reaches zero, which indicates that the arm arrives and performs the desired task.
For subsequent works, we will contrast the impact and influence of the application of applied RA in education versus the general teaching methodology, without the implementation of AR technologies; as well as the advantages and disadvantages of learning using this technological tool focused on the assistance and training of industrial robotic arms.
Acknowledgements The authors would like to thanks to Universidad de las Fuerzas Armadas ESPE, for the support to develop this work.