=Paper=
{{Paper
|id=Vol-2516/paper4
|storemode=property
|title=Information Technology of University Class Internet of Things-Module
|pdfUrl=https://ceur-ws.org/Vol-2516/paper4.pdf
|volume=Vol-2516
|authors=Yaroslav Krainyk
|dblpUrl=https://dblp.org/rec/conf/ictes/Krainyk19
}}
==Information Technology of University Class Internet of Things-Module==
https://ceur-ws.org/Vol-2516/paper4.pdf
Information Technology of University Class
Internet-of-Things-Module
Yaroslav Krainyk1[0000-0002-7924-3878]
1 Petro Mohyla Black Sea National University, 10. 68 Desantnykiv str., Mykolaiv, Ukraine
yaroslav.krainyk@chmnu.edu.ua
Abstract. In this paper, new information technology of Internet-of-Things-
module (IoT-module) for university class has been investigated. The IoT-
module is designed to improve user experience in educational process. It allows
lecturer to provide additional content to students and run knowledge testing
immediately during classes. The additional information is presented either in
multimedia (images and video) or text format. The information technology with
the devised module supposes extensive usage of network. Thus, specific com-
munication protocol and has been designed for orchestrating multiple clients us-
ing server software. The module employs Ethernet technology for organization
of communication with server. The class can be equipped with multiple in-
stances of IoT-module. The usage of the module and information technology
provides a scalable solution for modern learning environment enhanced by in-
formation-communication technologies. Information technology has been im-
plemented in Proof-of-concept laboratory stand with two IoT-modules based on
STM32 F7 development boards. The proposed solution has a huge potential for
further improvement and can propose more features for specific learning envi-
ronments.
Keywords: Internet-of-Things, education, technology, module, synchroniza-
tion.
1 Introduction
Influence of Internet-of-Things (IoT) on each field of human activity is getting more
and more notable. New devices are designed to integrate all types of environments
into global network and provide new capabilities to the customers.
Educational environment and specifically educational process that involves usage
of information computer technologies (ICT) should stay at the edge of the new tech-
nologies to match requirements of the time. It includes both learning of modern ad-
vances in the field and employment of contemporary ICT in educational process.
From this perspective, IoT offers almost infinite possibilities for effective usage of
new devices.
IoT-enabled modules can greatly simplify delivery of educational content to stu-
dents. Usage of multimedia means has a great effect on the overall perception and
understanding of materials during lecture or seminar. Providing such multimedia
Copyright © 2019 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
modules are available for every attending student, there is a possibility to complement
main flow of the classes with additional information on the topic or even personalized
content. Another use case for such type of IoT-modules is brief immediate check of
the disseminated materials. Not only it gives an opportunity to evaluate students’
knowledge, but also is a great basis for analysis on how complicated lecture is for
students, what topics are more popular. This is a great source of information for re-
finement of educational materials.
2 Analysis of scientific sources and industrial solutions
IoT can greatly facilitate in development of Smart Learning Environments (SLE) [1].
SLE may leverage various IoT-devices for improvements in educational process,
students experience and involvement, etc. IoT provides multiple enabling means [2]
that simplify integration of modern technologies and new features for SLE.
Wireless technologies [3-6] are dominating in the IoT-solutions because they do
not severely limit location of device and allow changing device position. Numerous
technologies are available in this segment and they can satisfy almost any customer
demands. However, IoT-device does not necessarily need to be wireless one. Wired
connection also can provide access to global network and its usage does not contradict
main ideas of IoT. Ethernet connection provides high-speed reliable channel for data
interchange. In this work, throughput and fidelity of data transmission are highly im-
portant factors because of multimedia data transfer. That is why Ethernet technology
has been chosen as a basis of the network with IoT-modules for education.
Inclusion of ICT in the educational process became a trend from the beginning.
However, possibilities for further development with IoT seem almost unlimited. Its
integration with another information technologies like Big Data, Machine Learning,
Cloud Computing, etc., completes the technology and spreads technical functions
available to students and academic staff. That fact rises a lot of issues for integration
of education with other fields [7-9]. Displays are the most common ICT-component in
the education process. Typically, visual information is perceived much better than
other types of information. Therefore, display is almost must-have component for
modern IoT-device that targets to be used in education. Display is also a part of the
system considered in this paper. Moreover, nowadays, displays are usually accompa-
nied with touch panel or other sensor technology so the user can send commands to
the system without any additional tools. Such systems are in education and many
other fields. For instance, Briolight company (Mykolaiv, Ukraine) has developed
interactive tables for learning anatomy [10]. This very complex system supports dif-
ferent modes and provides great study experience. Nevertheless, it is also oriented on
the specific educational segment and cannot be used simultaneously by the group of
students. Thus, we are developing solution that is deemed as lightweight device that
available to everyone in the class.
The most common base for IoT-devices is embedded systems components. They
include microcontrollers, wireless modules with Wi-Fi, Bluetooth, single-board com-
puters, etc. They are limited in computational power and other resources. However,
modern microcontrollers shows decent results even in multimedia processing. At the
same time, they consume much less energy in comparison with desktop processors.
Therefore, microcontroller with multimedia capabilities can be used in the core of the
proposed module.
In this paper, information technology for IoT-module for university class is pro-
posed. IoT-module aims to perform simple functions like displaying text and image as
well as quite complex task like video demonstration on the display. This work is a
continuation of the results, achieved in [11].
3 Methods and results
The proposed system uses client-server architecture. The server is either pre-installed
machine or portable device with necessary software to communicate with clients. IoT-
modules run as clients. Data transmission inside the network is handled by the dedi-
cated Ethernet-switch placed inside the classroom. It also provides access to the uni-
versity network with possibility to exchange data even with external resources, e.g.
cloud services. Fig. 1 illustrates the general connection scheme for the network.
WAN
University LAN level
Main switch
Classroom level
Switch
Client Client Client … Server
Fig. 1. General scheme for the proposed solution
Devised IoT-module comprises microcontroller, display with sensor touch panel,
external RAM module, networking interface for communicating over Ethernet. This is
the minimal set of components and it can be completed by other parts depending on
the extension demands. Microcontroller manages networking data transmitted or re-
ceived via network module, manipulates graphical data and shows data on the display,
handles user actions on touch panel.
From the educational point of view, the main functions performed by the module
are presentation of text and multimedia content (images and video) and test-based
knowledge checking. It cannot serve as a complete and universal solution for educa-
tional tool but it should be quite effective within specified range of responsibilities.
Placement of the modules in the classroom depends on effective Ethernet cables
connection and desks available in the class. The basic idea is to mount module on the
surface of the desk. The module is stationary and for the power supply of the module
USB-cable can be used or dedicated supply network. Separate interface for microcon-
troller programming should be routed. This ensures that device can be reprogrammed
with adopted software in the future.
Presence of client-server architecture dictates that client software and server soft-
ware applications are required. In this case, client solution is a firmware. It connects
to the server and requests necessary information. At the server side, server software
directs clients to perform necessary actions.
Both Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) are
involved into communication in the network. TCP is used for regulating testing pro-
cess and image transmission. UDP plays a vital role in video data transmission.
Networking level in the microcontroller’s firmware is controlled by lwIP [12] li-
brary. Because of the performance reasons and size of the output file, callback-style
Application Programming Interface (API) has been chosen for implementation. It is
based on usage of Protocol Control Block (PCB) structures. Developer is responsible
to set up callback functions in the necessary moments of initialization, connection,
and data processing. Communication protocol between server and client will be dis-
cussed further.
Most of the work that firmware is responsible for is done automatically after ini-
tialization. Thus, developer is responsible for correct initial configuration and organ-
izing main loop, reaction to the interrupts, and setting callbacks. As peripheral inter-
faces for external memory, display with touch screen, and network interface are con-
figured, a lot of functions related to orchestrating data do not require further control
from the developer, e.g. data from dedicated in memory buffer will be automatically
sent to the screen. To clarify connections among all main comprised parts of the IoT-
module on the firmware level, let us consider them in more details.
Memory is a key part of the IoT-module that is heavily utilized for networking and
multimedia functions. Different types of memory are used for the specified purposes.
Network functions rely on memory buffer placed in RAM while image processing
keeps data in the external DRAM. Thus, when image data are received from network,
transfer from RAM to DRAM takes place. The destination matches buffer that is used
by display controller to output visual data from memory.
It is convenient to organize processing of touch actions via interrupts. It allows
immediately react to the user actions and provide better User eXperience (UX). How-
ever, the main issue with touch sensor is that it tends to react too often on the event.
Therefore, another solution for touch processing is usage of internal counter and its
update (overflow) interrupt. The interrupt occurs in fixed amount of time and does not
overreact on the external influence. In the interrupt handler, state of the sensor is
checked and necessary action is applied according to this information. In our case, it
is used to identify which item is selected as the answer to the test question.
At the server side, the implementation is different, as socket interface has been
used. The server applications launches new thread in which server socket object ac-
cepts incoming connections. When new connection is accepted, new thread is created
and all further communication with this connection are associated with this thread.
This fact indicates that number of communication threads is equal to the number of
clients installed in the classroom. As client is connected, the server software initiates
all further actions. Lecturer starts the testing from the user interface. All multimedia
data transmission begins with the command of lecturer. It greatly simplifies develop-
ment of the firmware because it just need to handle data and process received data.
Most logic functions of the systems are executed at the server side. From the software
point of view, instances of socket interface hold network communications. Each
thread governs single client socket.
As has been mentioned previously, server communicates with clients through mul-
tiple threads. Thus, server application is multi-thread application with the following
thread types:
1. Main thread.
2. Server thread.
3. Client thread.
4. Synchronization thread.
Main thread in the thread that is created first as the program run. It starts server thread
that is listening for incoming connections on the specific port. When the new connec-
tion is available, client thread is launched. However, running the threads is just a part
of the problem to deal with. In the communication process, external resources are
accessed to send data to clients as well as receive data from them. Therefore, syn-
chronization among the threads are required to coordinate them and keep whole sys-
tem in the consistent state. This is the main objective of synchronization thread.
The function that executes when the thread is started (e.g. run in Java) is organized
as the infinite loop. In the infinite loop thread waits for the user action, sends data to
the client, expects reply if necessary. The data to send are identified by user action
and internal state of the client thread.
The idea behind synchronization mechanism for discussed systems is in the follow-
ing. Countdown latch synchronization primitive is supposed to be used for this pur-
pose. Each client thread accesses three latches. The first one is used to postpone exe-
cution of the main thread logic to the moment when lecturer selects some action. Its
initial count value is equal to one. Synchronization threads notifies all client threads
about it by decrementing the latch. Then data interchange section goes where selected
content is being sent to the client and reply is received. Single run of the thread body
is done at that moment. However, different threads finish execution at the different
time. Thus, they decrement another countdown latch to inform synchronization thread
about finished work. The initial value for this counter is equal to the number of con-
nected clients. At the moment when the last thread finishes execution of the loop,
synchronization thread is aware of the fact that all threads have done the assigned
task. Nevertheless, by the moment when any of the threads goes to the next cycle,
new countdown latch should be set. With that intention additional countdown latch
wait operator is placed at the end of the client thread cycle. If all threads are finished,
synchronization thread creates new instance of countdown latch that is checked in the
beginning of the client thread loop and decrements last one in the client threads. This
procedure assures that no client thread starts execution before any other thread.
All the transmission from server to clients goes straightforward with no data sent
back except the case of testing. If lecturer selects testing option, server needs to re-
ceive replies from the clients. Question messages alternate with replies from the user.
Client receives single question with answer variants at a time. Next question is avail-
able only after reply on the current question.
Comparison of networking features implementation in firmware and server soft-
ware indicates that they leverage different approaches. Server contains enough com-
putational resources to bring sockets into work while low-level programming inter-
face in the firmware allows reducing size of the output file. The trade-off for this
attainment is more complicated program code (more lines of code are required, more
structure instances participate in the information exchange, structural complexity of
the program, etc.).
TCP and UDP is the basis for communications on the transport layer. However,
custom application protocols for correct interpretation of data is compulsory in this
case. The following data goes from server to the clients via TCP:
1. Text information to display on screen.
2. Image information.
3. Test information.
4. Scores of the testing.
In opposite direction go only answers on the test questions. As there are only four
variants of how to interpret data, the simplest solution is to excrete single bit in the
first package to identify type of the message. Several following bytes (e.g. four bytes)
should identify total number of data bytes in the message. The rest part of the mes-
sage is useful information that should be processed according to the message type. To
display image on the screen, separate image buffer is available in memory. Image data
are transmitted in the raw form in Alpha-Red-Green-Blue (ARGB) format. Test in-
formation contains question text, options to display as answers. They are separated
from each other with a specific symbol (e.g. #) so corresponding parts can be fluently
extracted from the received string. When question and options are available, they are
displayed on the screen using high-level API (writing into image buffer is involved).
Text information can be transmitted without any additional markers. The last message
type (test score) informs student how successful he or she was during the quiz. It is
also quite simple in implementation as it contains only number of point or percentage
of correct answers that can be represented by single additional byte (e.g. 0-100% cor-
rect answers).
Video information is transmitted using UDP due to the performance reasons. Data
are sent only from server to clients in this mode. Multicast transmission is employed
to send data to solve the problem of delivering all data chunks on the network level.
Although, UDP is not reliable protocol and it cannot prevent possible losses in the
network. In context of presentation of video data on the screen, that means appearance
of artifacts in the reconstructed frame. Regular transmission of synchronization pack-
age is applied to solve this problem. Reception of this package means that writing
position in memory buffer is to be moved to the beginning of the buffer. This package
is sent after each datagram that represents last chunk of data from image frame ex-
tracted from the video at the server side. Synchronization package has fixed width and
can be distinguished among other packages by this parameter. Similar method is used
in the system to collect image information from multiple sensors [13]
Proof-of-concept system that takes advantages of the proposed information tech-
nology has been implemented. STM32F7 Discovery board [14] has been chosen as a
hardware platform for IoT-module. It contains all necessary devices in the single
board and has wide opportunities for extension. Central point of the processing power
for the board is F7-series microcontroller from ST Microelectronics. Firmware for the
board is written in C programming language using assist from tools like
STM32CubeMX and development environment System Workbench for STM32. The
developed laboratory stand includes two boards and Ethernet switch. Two instances
of modules are sufficient to check correctness of multi-thread software and all related
issues. They are connected using Ethernet cables. Almost any device with Ethernet
interface can be used a server. In our case, laptop served as a server machine. Server
software has been deployed on the laptop. The software is developed in Java pro-
gramming language and provides basic user interface to access all described functions
of the system. The main form of server application is demonstrated in Fig. 2.
Fig. 2. Main form of the developed application
Lecturer selects image file to send to the IoT-modules, then it can be sent. The im-
age’s width and height in pixel must match respective dimensions of the screen. Most
popular image formats (PNG, JPG, BMP) are supported. The result of this function
execution is demonstrated in Fig. 3.
Fig. 3. Devices on the stand in the image display mode
The basic testing features are also available to the user. The user prepares file with
test in the specific format. First line contains question, three lines following represent
variants of answer. Then the pattern repeats so file can contain as much questions as it
is necessary. File with answers should be prepared to automatically send score re-
sponse and collect statistics on the students’ performance. In Fig. 4, you can observe
devices with test questions shown on the display during the sample test session for
two devices. The design interface is simple but it provides access to the necessary
functions.
Fig. 4. IoT-modules in knowledge testing mode
The results of passing test for both cases are demonstrated in Fig. 5.
Fig. 5. IoT-modules display results of the test
The proposed information technology and developed IoT-modules based have huge
potential for further improvements and depending on components that included into
the system. However, it should be tested in more rigorous approaches to identify its
reliability and operation conditions [15].
Acknowledgments. The current project has been developed with support of the SEA
company (Kyiv, Ukraine). The author is grateful to company representatives for the
development board that have been used for the experiments and proof-of-concept
exemplar implementation.
4 Conclusions
In this paper, information technology for IoT-module of university classroom has
been established. First, general network topology for the educational environment has
been devised and considered individual roles of IoT-modules and server in such net-
work. Second, comprised parts for the modules has been proposed to implement nec-
essary functionality. The module’s general purpose is to enrich educational experi-
ence for lecturer and for students. It augments typical class flow by making possible
immediate knowledge testing and demonstration of additional multimedia or text
information. Data communication between server and clients (IoT-modules) has been
introduced. In the communication phase, application-level protocol is proposed to
cover all the proposed functions. Consequently, information technology has been
implemented in laboratory stand that includes two clients and allows connection of
device with installed server software. According to the devised statements, firmware
for IoT-modules has been developed. The stand demonstrates all described features
and may be extended during the future investigations on the topic.
References
1. Spector, J.: Smart Learning Environments: Concepts and Issues. In: G. Chamblee & L.
Langub (Eds.), Proceedings of Society for Information Technology & Teacher Education
International Conference (pp. 2728-2737). Savannah, GA, United States: Association for
the Advancement of Computing in Education (AACE) (2016).
2. Alsubaei, F., Abuhussein, A., Shiva, S.: An Overview of Enabling Technologies for the In-
ternet of Things. In: Hassan, Q. (ed.) Internet of Things A to Z: Technologies and Applica-
tions, pp. 79-111, John Wiley & Sons (2018). DOI: 10.1002/9781119456735.ch3
3. Krainyk, Y., Davydenko, Y., Tomas, V.: Configurable Control Node for Wireless Sensor
Network. In: Proceedings of 2019 3rd International Conference on Advanced Information
and Communications Technologies (AICT), Lviv, Ukraine, 2-6 July, pp. 258-262. IEEE
(2019). DOI: 10.1109/AIACT.2019.8847732
4. Krainyk, Y., Davydenko, Y., Starchenko, V.: Message-level Decoding of Error Patterns
for Turbo-Product Codes. In: Proceedings of IEEE 39th International Conference on Elec-
tronics and Nanotechnology (ELNANO), 16-18 April, Kyiv, Ukraine, pp. 660-663. IEEE
(2019). DOI: 10.1109/ELNANO.2019.8783849
5. Kondratenko, Y., Kondratenko, G., Sidenko, I.: Multi-criteria selection of the wireless
communication technology for specialized IoT network. In: 14th Interna-tional Conference
on ICT in Education, Research and Industrial Applications. Inte-gration, Harmonization
and Knowledge Transfer, Workshops, ICTERI, Vol. 2104, pp. 501-516. Kyiv, Ukraine
(2018).
6. Kondratenko, Y., Kondratenko, G., Sidenko, I.: Multi-criteria decision making for select-
ing a rational IoT platform. In: IEEE 9th International Conference on De-pendable Sys-
tems, Services and Technologies, DESSERT, pp. 147-152. Kyiv, Ukraine (2018).
7. Poliakov, M., Larionova, T., Tabunshchyk, G., Parkhomenko, A., Henke, K.: Hybrid
Models of Studied Objects Using Remote Laboratories for Teaching Design of Control
Systems. International Journal of Online and Biomedical Engineering 12(9), 7-13 (2016).
DOI: 10.3991/ijoe.v12i09.6128
8. Ursutiu, D., Kuchirka, Y., Samoila, C., Vytvytska, L.: The Online Course and Laboratory
for Study of Automatic Control Systems. International Journal of Online and Biomedical
Engineering 12(1), 24-27 (2016). DOI: 10.3991/ijoe.v12i01.5106
9. He, J., Lo, D. C.-T., Xie, Y., Lartigue, J.: Integrating Internet of Things (IoT) into STEM
undergraduate education: Case study of a modern technology infused courseware for em-
bedded system course. In: 2016 IEEE Frontiers in Education Conference (FIE), Erie, PA,
USA (2016).
10. Anatomy table | Briolight Innovative electronic solutions,
https://briolight.com/en/anatomy-en/, last accessed 2019/02/03.
11. Krainyk, Y., Razzhyvin, A., Bondarenko, O., Simakova, I.: Internet-of-Things Device Set
Configuration for Connection to Wireless Local Area Network. In: Luengo D., Subbotin
S., Arras P, et. al. (eds.) Computer Modeling and Intelligent Systems (CMIS-2019), Za-
porizhzhia, Ukraine. CEUR-WS, vol. 2353, pp. 885-896 (2019).
12. Dunkels, A.: Design and Implementation of the lwIP TCP/IP Stack. Swedish Institute of
Computer Science (2001).
13. Zhuravska, I., Musyienko, M., Tohoiev, O.: Development the heat leak detection method
for hidden thermal objects by means the information- measuring computer system. In: Lu-
engo D., Subbotin S., Arras P, et. al. (eds.) Computer Modeling and Intelligent Systems
(CMIS-2019), Zaporizhzhia, Ukraine. CEUR-WS, vol. 2353, pp. 350-364 (2019).
14. 32F746GDISCOVERY – Discovery kit with STM32F746NG MCU,
https://www.st.com/en/evaluation-tools/32f746gdiscovery.html, last accessed 2019/02/07.
15. Seniv, M., Yakovyna, V., Symets, I.: Software for visualization of reliability block dia-
gram and automated formulation of operability conditions of technical systems. In: 14th
International Conference on Perspective Technologies and Methods in MEMS Design,
MEMSTECH-2018, pp. 191-195, Lviv, Ukraine (2018).