Remote IoT-based Control System of the Mobile Caterpillar Robot Oleksandr S. Gerasin1[0000-0001-5107-9677], Andriy M. Topalov1[0000-0003-2745-7388], Mykyta O. Taranov2[0000-0002-8784-2600], Oleksiy V. Kozlov2[0000-0003-2069-5578], Yuriy P. Kondratenko2[0000-0001-7736-883X] 1 Admiral Makarov National University of Shipbuilding, 9 Heroes of Ukraine Av., Mykolaiv, 54025, Ukraine oleksandr.gerasin@nuos.edu.ua, topalov_ua@ukr.net 2 Petro Mohyla Black Sea National University, 10, 68th Desantnykiv Str., Mykolaiv, 54003, Ukraine Mykyta.Taranov@chmnu.edu.ua, kozlov_ov@ukr.net, y_kondrat2002@yahoo.com Abstract. The paper is dedicated to the development of the remote IoT-based control system of the mobile caterpillar robot (MCR) that able to move on in- clined and vertical ferromagnetic surfaces. Nowadays such industrial robots re- place humans when operating in harmful and dangerous working conditions, they are really needed for cleaning, welding, cutting, painting, inspection in shipbuilding, ship repair and another branches of heavy industry in the world. The authors propose experimental model of the industrial MCR with separate clamping magnets. The general functional structure and the functional features of a hierarchical remote system for the MCR control based on the Internet of Things technology have been considered. The basic software and hardware means of the experimental model of the remote control system of MCR are de- scribed. The hardware includes embedded solution based on non-expensive NodeMCU board with ESP8266 and power units. The program algorithm and Blynk application for Android with cloud service in interaction are made up the software of the robot’s remote IoT-based control system. So, the experimental model of the proposed system for MCR allows to control the spatial movement of the robot in “point-to-point” network and from any place of the world in the presence of access to the Internet. The usage of the given system makes it pos- sible to remotely access to the control processes of outside experts while insuf- ficient qualifications of an attendant. Keywords: IoT-based Control System, Mobile Caterpillar Robot, Android Ap- plication, Cloud Service. 1 Introduction Internet of Things is a rapidly evolving technology. Moreover, it was only a matter of time when the synergy of the Internet of Things (IoT) and Robotics, now known as the Internet of Robotic Things (IoRT), occurs. The IoRT are devices which are able to Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). receive data both from own sensors and external sources or use other tools (e. g., clouds) to process data and make decisions, as well as interact with the physical world [1]. For example, [2] describes the implementation of 5 DoF heterogeneous robotic arm using computations in Cloud Robotics, that allows to use additional libraries, perform analysis and learning, share data between arms. In [3] the social robot is de- scribed, which, in addition to using clouds, analyzes data from Wearables, resolves ways to displace itself, and determines which robotic tasks need to be executed. Many robots are capable of displacing themselves. Some such mobile robots can fall under another type of IoT – Internet of Mobile Things (IoMT), i.e. drones and self-driving cars. They should be able to securely access the Internet through different networks, continue to work if there is no Internet connection, and effectively consume energy [4]. Since, mobile robots often cannot use wired Internet connection and pow- er cable, the following features of IoMT turn to be useful for mobile robots: (a) estab- lishing a secure wireless connection, (b) using energy-efficient transmitters and ap- proaches, (c) protocols suitable for inconstant connection or connection with limited bandwidth, (d) performing heavy computations in clouds to save energy [5]. In [6] considered the ability to create a fire fighting mobile robot. The robot, after it got an alert message from some IoT sensor, reaches the fire location, performs fire- fighting actions, and sends video stream of fire location to fire safety officers. An algorithm developed in [7], allows finding the shortest collision-free path (useful in an automatic warehouse, where numerous mobile robots, coordinated via the cloud, transport goods marked with RFID or QR code). A wireless mobile robot for perform- ing operations on the field, such as moisture sensing, spraying pesticides, scaring birds and animals was designed in [8]. It has a camera as well and uses an IoT ap- proach for remote control. The paper [9] describes the creation of a cherry tomato harvesting robot. To gently harvest tomatoes and correctly define maturity, the au- thors used computer vision in combination with fuzzy logic. As can be seen, some system uses IoT only as service for a reliable Internet connection. According to [10], non-expensive NodeMCU board is powerful enough to stream even video and audio. The papers [11-13] dedicated to developing mobile robots with magnetic caterpil- lars. These robots can move on ferromagnetic surfaces and perform given technologi- cal operations in shipbuilding, which can be dangerous to human health and life [14]. Thus, it is very interesting to use IoT-based approach to mobile caterpillar robots (MCRs) for vertical movement taking into account good practice in industrial IoT- systems for controlling different technical objects [15]. Applying wireless technology to such mobile robots allows to remote control the robots from any point of work- space and even from anywhere in the world with access to the Internet. 2 Generalized Functional Structure of the Hierarchical Remote System for the MCR Control based on the IoT Technology To perform various technical operations by a mobile robot at a considerable distance from the information control center, a computer system for controlling the parameters of the MCR was developed, based on the principles of the Internet of Things technol- ogy for remote monitoring and operator control. The computer system is built in a modular (variable configuration) structure and has a separate remote monitoring sys- tem using cloud technologies. The computer system offered has two levels of monitoring and automatic control: local and remote. The local level, in turn, is divided into three hierarchical levels of monitoring and control: the lower level (level of sensors and actuators), the middle level (control level), the upper level (operator level). The lower level contains devices corresponding to this technological process specialized sensors and actuators (mo- tors). The main task of the sensors is to form and transmit information about the state of the technological process. The task of the actuators is to control the process param- eters. For different processes, the types of measurement and control of the lower level may differ significantly. They can be digital or analog; access can be dynamic (short time intervals) or static with a certain cycle (certain time intervals). The middle level consists of microprocessor-based PLC hardware, cards and I/O modules, as well as hubs. This level receives data from the sensors and issues control commands to the lower level by actuators. Control in the PLC is carried out according to a pre- developed algorithm, which is executed cyclically. Common to this level are: real- time operation with acceptable reaction time, frequent reconfiguration of micropro- cessor devices, the ability to work with a large number of sensors and actuators, working in an industrial interference environment with high-level information trans- mitted to the upper level. An additional communication controller or hub may also be installed to coordinate the operation of the PLC with the top level. The upper level is the level of visualization, dispatching (monitoring) and data collection. It provides the collection and archiving of the most important data from PLCs, maps and I / O mod- ules, as well as visualization of the operating parameters of the process. In accordance with the listed tasks at different levels, appropriate network technol- ogies are used. For the upper level, Ethernet and the TCP / IP family are most widely used. On the middle and lower levels the industrial networks (Fieldbus) are used, which include Profibus, CANbus, Modbus and many others. Remote level is based on cloud technologies for operator control and database ac- cumulation. The main purpose of this level is to give users access to mobile robot settings via a web browser using the Internet/Intranet or wireless (GPRS, Wi-Fi, Blue- tooth, etc.). Moreover, access must be provided in the real time from any computer that uses different operating systems (Windows, Linux, Mac OS, etc.). The functional structure of the generalized IoT-based Control System of the mobile robot [16] is shown in Fig. 1, where the following notations are accepted: PC is the personal computer; PLC is the programmable logic controller; FPGA is the field- programmable gate array; SBC is the single-board computer; SDAM is the sensors data acquisition module; AOM is the analog output module; S – sensor; AM is the actuating mechanism. The output from the sensors, depending on the type of signals are fed to the block of analog input modules or the block of modules of discrete input, and then digitally transmitted to the PLC. The PLC contains a software control unit for actuators. This unit is implemented using specialized SCADA software. Information about the cur- rent values of the mobile robot settings is displayed on the operator's computer screen through a specialized HMI, which also provides controls for actuators. Moreover, information about the main parameters of the MCR is transmitted through the web server to other PCs of the control posts without the ability of the actuators control. Fig. 1. Functional Structure of Generalized IoT-based Control System of the Mobile Robot 3 IoT-based Control System of the Experimental MCR with Separate Clamping Magnets Modern hardware for automatic control systems implementation of mobile robots in industrial design (PLCs, data collection modules, output modules), as well as software environments for the development of control systems projects are very high cost [17]. So, the authors decided to use cheaper development tools for research purposes and proofing the operability of the concept itself at designing the software and hardware of the experimental model of the IoT-based remote control system for the MCR. In this case, the issues of signals lag, safety and security, etc., caused by the peculiarities of controlling via the Internet are not considered. Fig. 2, a shows a cross-sectional drawing of the MCR with separate clamping magnets [18]. The advantages of such caterpillar robot compared with the robot with installed permanent magnets at caterpillar tracks are: good reliability, energy efficien- cy, high dynamics and service life at the large adhesion area to the ferromagnetic surfaces. There is the main clamping magnet 1, spherical joint 2, the frame 3, tracks 4 as the main parts of the experimental robot in Fig. 2, a (δ means the clearance con- cerning ferromagnetic surface 5). The experimental model of the caterpillar mobile robot with individual clamping magnets is shown in Fig. 2, b. As a built-in control module, which is also a mean of communication between the operator and the robot, the NodeMCU development board based on the ESP8266 WiFi module [19] was selected as the popular and inex- pensive hardware tool for the implementation of IoT systems. This board is fully compatible with the NodeMCU Motor Shield (ESP-12E Motor Shield) driver board to power the mobile robot’s motors from a separate source [20]. Free development software Arduino Software (IDE) [21] is used to develop the software for the experimental robot control system, which simplifies the process of working with microcontrollers and development boards, and provides several ad- vantages over other systems through a simple and clear programming environment and large number expansion boards. The basis for the implementation of the concept of IoT and essentially remote con- trol of the robot via the Internet is the Android application Blynk [22] for the smartphone, which has a special interface (Fig. 2, c). The remote control system inter- face for controlling the caterpillar mobile robot includes a virtual moving joystick in the form of a circle, the current position of which corresponds to a special virtual PIN for further processing a given position of the circle of the joystick. A virtual PIN number and its two parameters (x and y coordinates) are set in the application to syn- chronize the application with the main program in the NodeMCU controller. The virtual joystick consists of three circles: outer, working (internal) and mova- ble, which are shown in Fig. 1, d. The movable circle is the border of the joystick circle that is inside the outer circle and can move freely within it. The working circle limits the boundaries of the joystick's working area (center of its circle), so the mova- ble circle reaches the edge of the outer circle, the center of the movable circle moves to the edge of the working circle. Thus, the mobile circle, visually, moves inside the outer circle without leaving it. There are two basic modes (states) of the program: motion and stationary state. The central position of the joystick (point OC1 in Fig. 2, d) corresponds to the stationary state of the robot. Two axes x and y (the coordinate axes in which the center point of the moving circle will move) are selected with the starting points outside the outer circle (fig. 2, d) to eliminate the negative values of the coordinates of the joystick’s center position. Given that the outer circle has a diameter of 1024 pixels, the coordi- nates of the point OC1 is the middle of the diameter of the outer circle (512; 512). The joystick is automatically set to the center (initial) position if it does not interact with it. One of eight directions of movement (courses) can be set: forward, backward, right, left, forward right, forward left, back right, back left (each direction is 45° from the central angle) when moving the joystick across the working area. The position of the center of the joystick in one of the eight formed segments indicates the movement in this direction (direction "Forward" when positioning the center of the joystick at the point OC2 in Fig. 2, d). a) b) c) d) Fig. 2. Schematic diagram (a) and exterior view of the experimental robot (b) with application for remote control (c), as well as the scheme of movement of the joystick in the working area with the outer (1), working (2) and movable (3) circles (d). The Blynk application can work on both wired and wireless lines. WiFi communi- cation has been selected to control the mobile robot. The application and the printed circuit board exchanged data only on condition that both devices (board and device with the application) are connected to a common access point at the initial stage of the development. Then the system was configured to control the robot via the Internet, with the NodeMCU board ‒ via a router and the smartphone with the application ‒ via 4G. The name of the network and the access key are specified in the body of the con- trol program. When the power is applied to the board and the application starts, they are synchronized (the corresponding message notifies about that in the application). Such system allows to carry out remote control of the caterpillar robot from any point of the world in the presence of access to the Internet, for example at insufficient quali- fication of the attendant in non-standard situations. 4 Conclusions The authors developed the IoT-based control system of the MCR that is able to move on inclined and vertical ferromagnetic surfaces to automate the process of moving a working tool through the ferromagnetic surface. The use of the system with the pro- posed functional structure allows to carry out remote control of speed and course as the basic parameters of the mobile robot. The experimental model of the remote con- trol system of the caterpillar mobile robot is developed, which allows to control the spatial movement of the robot from anywhere in the world with access to the Internet. The usage of such a system makes it possible to remotely access the control processes by the third-party experts at the insufficient qualifications of the duty operators. Fur- ther research should be related towards the implementation of advanced con- trol algorithms in the proposed IoT-based control system of the MCR. References 1. Simoens, P., Dragone, M., Saffiotti, A.: The Internet of Robotic Things: A review of con- cept, added value and applications. International Journal of Advanced Robotic Systems 15(1), (2018). 2. Arefin, S. E., Ashrafi Heya, T., Uddin, J.: Real-life Implementation of Internet of Robotic Things Using 5 DoF Heterogeneous Robotic Arm. In: 2018 Joint 7th International Confer- ence on Informatics, Electronics & Vision and 2018 2nd International Conference on Im- aging, Vision & Pattern Recognition, pp. 486-491. Kitakyushu, Japan (2018). 3. Simoens, P., et al.: Internet of Robotic Things: Context-Aware and Personalized Interven- tions of Assistive Social Robots (Short Paper). In: 2016 5th IEEE International Conference on Cloud Networking (Cloudnet), pp. 204-207, Pisa, Italy (2016). 4. Kondratenko, Y. Robotics, Automation and Information Systems: Future Perspectives and Correlation with Culture, Sport and Life Science. In: Decision Making and Knowledge Decision Support Systems, Lecture Notes in Economics and Mathematical Systems. In: A. Gil-Lafuente, C. Zopounidis (Eds.), Vol. 675, pp. 43–56. Springer International Publishing Switzerland (2015). 5. Tkachenko, A., Brovinskaya, N., Kondratenko, Y.: Evolutionary adaptation of control pro- cesses in robots operating in non-stationary environments. Mechanism and Machine Theo- ry 18 (4), 275-278 (1983). 6. Raj, P.A., Srivani, M.: Internet of Robotic Things Based Autonomous Fire Fighting Mo- bile Robot. In: IEEE International Conference On Computational Intelligence And Com- puting Research (IEEE ICCIC 2018), pp. 360-363, Madurai, India (2018). 7. Avila-Alonso, J.L., Lopez-Araujo, D., Alvarez-Jarquin, N.: Planning of collision-free tra- jectories for mobile robots using IoT. In: IEEE International Autumn Meeting on Power Electronics and Computing (ROPEC). Ixtapa, Mexico, USA (2018). 8. Kristina, K.L., Silver, O., Malende, W.F., Anuradha, K.: Internet of Things Application for Implementation of Smart Agriculture System. In: International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud), pp. 54-59. Palladam, India (2017). 9. Biqing, L., Yongfa, L., Hongyan, Zh., Shiyong, Zh.: The Design and Realization of Cherry Tomato Harvesting Robot Based on IOT. International Journal of Online and Biomedical Engineering (iJOE) 12 (12), 23-26 (2016). 10. Singh, D., Nandgaohkar, A.: IOT-Based Wi-Fi Surveillance Robot with Real-Time Audio and Video Streaming. In: Advances in Intelligent Systems and Computing, pp. 639-647. Lonere, India (2019). 11. Huang, H., Li, D., Xue, Z., Chen, X., Liu, S., Leng, J., Wei, Y.: Design and performance analysis of a tracked wall-climbing robot for ship inspection in shipbuilding. In: Ocean Engineering 131, pp. 224-230 (2017). 12. Kermorgant, O.: A magnetic climbing robot to perform autonomous welding in the ship- building industry. Robotics and Computer-Integrated Manufacturing 53, 178-186 (2018). 13. Lee, G., Park, J., Kim, H., Seo, K., Kim, J., Seo, T.: Wall climbing robots with track-wheel mechanism. In: Proceedings of the 3rd International Conference on Machine Learning and Computing (ICMLC 2011), pp. 10-14. Guillin, China (2011). 14. Kondratenko, Y., Zaporozhets, Y., Rudolph, J., Gerasin, O., Topalov, A., Kozlov, O.: Modeling of clamping magnets interaction with ferromagnetic surface for wheel mobile robots. International Journal of Computing 17 (1), 33‒46 (2018). 15. Kondratenko, Y., Kozlov, O., Korobko, O., Topalov, A. Complex Industrial Systems Au- tomation Based on the Internet of Things Implementation. Information and Communica- tion Technologies in Education, Research, and Industrial Applications. In: Bassiliades N. et al. (Eds.), Vol. 826, pp. 164–187. Springer (2018). 16. Mobile robotics complex MRC-27 (Upgraded), http://sktbpr.ru/robot/mrk-27- modernizirovannyy, last accessed 2020/02/23. 17. Kondratenko, Y., Kozlov, O., Gerasin O., Topalov, A., Korobko, O. Automation of Con- trol Processes in Specialized Pyrolysis Complexes Based on Web SCADA Systems. In: Proceedings of the 9th IEEE International Conference Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Vol. 1. pp. 107– 112. Bucharest, Romania (2017). 18. Gerasin, O., Kozlov, O., Kondratenko, G., Rudolph, J., Kondratenko, Y. Neural Controller for Mobile Multipurpose Caterpillar Robot. In: Proceedings of the 10th IEEE International Conference Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Vol. 1. pp. 222–227. Metz, France (2019). 19. ESP8266 NodeMCU WiFi Devkit : user manual v1.2. Handson Technology, https://www.handsontec.com/pdf_learn/esp8266-V10.pdf, last accessed 2020/02/23. 20. ESP-12E Motor Shield : user manual. Shedzhen Doctors of Intelligence and Technology, https://cdn.hackaday.io/files/8856378895104/user-mannual-for-esp-12e-motor-shield.pdf, last accessed 2020/02/23. 21. The open-source Arduino Software (IDE), https://www.arduino.cc/en/Main/Software, last accessed 2020/02/23. 22. Blynk IoT platform, https://blynk.io, last accessed 2020/02/23.