Thanks to the significant development of the Internet of Things (IoT), measuring instruments have become accessible to everyone, enabling their use for various measurements. However, when people use Internet of Things (IoT) measuring instruments, they often rely on the results of these measurements without justification ,which can lead to incorrect decisions. This article presents the Citizen Science study that analyzes the most commonly used Internet of Things (IoT) measurement instruments, focusing on phonebased systems for conducting indoor lighting measurements using various phone models and software tools. The purpose of the study was ambivalent, as it aimed to involve students in scientific activities through Citizen Science experiments, which use statistical data analysis to assess the ability of phones to perform workplace indoor lighting measurements. During experiments, a measurement study analysis (MSA), a controlled experiment where a sample of items are measured multiple times by different devices and apps allowed us to separate the variation into specific sources - Gage Repeatability and Reproducibility (R&R) and to show differences between Internet of Things (IoT) devices designated for indoor lighting measurements. The conducted study provides knowledge to Internet of Things (IoT) measurement device users on how to perform MSA in order to self-assess the measurements being made and prevent the analysis of poorly collected data.
The performance of the Internet of Things (IoT)-based systems is directly dependent on the quality
of hardware and software. Software Quality Assurance (SQA) is a crucial factor for maintaining the
quality of service of Internet of Things (IoT) based applications. Metrology Evaluation of the Internet
of Things (IoT) Measurement Solutions was done by authors [
Environmental monitoring can be done by any citizen without even leaving their home, so
we have a unique opportunity to involve many people in Citizen Science activities. Citizen Science
is the non-professional involvement of volunteers in the scientific process, whether in the data
collection phase or in other phases of research. The variety of Citizen Science and new sensors and
smart phones make scientific research or environmental monitoring fun and interesting for young
people. For many years, the focus was only on ambient air monitoring, so we can summarize the
impact on our daily life only of Citizen Science for environmental policy projects. The research group
conducted a review of 503 Citizen Science projects [
It is stated that will also continue to support and incentivize the practice of open science by the R&I communities, through: ● improved interoperability and sharing of data (‘as open as possible and as closed as necessary’) and enhanced reproducibility of research results, which will in particular be a focus of several clusters, partnerships and missions; ● the involvement of the general public and other end-users through different participatory formats e.g., co-creation and deliberative exercises, citizen science, and user-led innovation modes of R&I, which will be promoted across the programme; ● the development and consolidation of the European Open Science Cloud (EOSC), the development of appropriate skills, and the diffusion and adoption of open science practices, which will be supported further; ● the availability of Open Research Europe (ORE), the open access peer reviewed publishing venue for Horizon 2020 and Horizon Europe beneficiaries, which will develop further, as well as the support to not-for-profit, scholarly open access publishing models.
This research contributes Open Science in Europe by including the following key aspects: ● Bring Science closer to citizens. To help young people acquire core future skills by participating in scientific activities at the professional and/or Citizen Science level by monitoring indoor lightning condition by using Internet of Things (IoT) devices; ● Improving Data Quality Framework for indoor lightning by standardization of metadata.
Advancing an open science system and aligning national and EU policies to improve the
production of FAIR [
Citizen Science activities inside indoor lightning monitoring. ● Open research data. Datasets with indoor lightning will be uploaded to open access repositories or will be given at any request without restrictions to use it.
Improved traceability of citizen science data uses, both in science and for policy, is important to
appreciate its impact and optimize its uses. This can be achieved by including persistent identifiers
to uniquely identify citizen science datasets or through the development of a tool to track policy
development [
Adherence to FAIR principles is included in the Grand vision of the SI Digital Framework in
metrology at 2021 [
The structure of this article is organized by providing an overview of the theoretical framework concerning Measurement System Analysis (MSA) and its application to Internet of Things (IoT) based on illuminance measurement systems. The experimental design specifies device selection, measurement procedures, and data collection protocol. Two experiments were performed, one involving uncalibrated devices and another incorporating calibrated references and various mobile applications by presenting and analyzing the results.
The data is obtained through measurements, or in other words, the measurement system is
designed to collect data about the objects or environmental parameters that interest us. In practice,
however, the following question arises whether data is reliable and can be trusted. This article
explores the process of collecting data when performing lighting measurements with mobile phones,
one of the most common Internet of Things (IoT) measurement tools. Measurement System is the
combination of people, equipment, materials, methods and environment involved in obtaining
measurements [
Measurement System Analysis (MSA) – series of studies that explains how measurement systems
perform is described in standard ISO 13053-2:2011 Quantitative Methods in Process Improvement —
Six Sigma Part 2: Tools and Techniques [
A measuring system may consist of one or more measuring instruments, the metrological characteristics of which may be as follows: range, bias, repeatability, stability, hysteresis, drift, effects of influencing quantities, resolution, discrimination, error and dead band. All these metrological characteristics contribute to measurement uncertainty.
MSA allows finding possible reasons for the uncertainty of the measurement system due to:
1. Resolution: the smallest increment of measurement variable that devices are capable of
detecting.
2. Measurement accuracy (bias): the differences between what a measurement system 'reads'
and what the true value is.
3. Linearity error: measurement bias across the usable range of the measuring system.
4. Stability: variability in the results given by a measurement system measuring the same
characteristic and the same product over an extended period of time.
5. Repeatability: the difference between results of successive measurements on the same
measurand (with all measurements carried out under identical measurement conditions:
same measurement procedure; same observer, same measuring instrument, used under the
same operating conditions, same location, the repetition over a short period of time.
6. Reproducibility: the difference between results of measurements on the same measure (with
measurements carried out under different measurement conditions) [
In this study, the experiments will be limited to the MSA of different mobile phones and different software mobile applications.
The application of MSA in solving problems in different industry areas has been examined by the
following authors [
During the experiment, the illumination inside the room was measured, i.e. the measurement system was collecting data which is measured on a continuous scale so we used an MSA method called Gauge Repeatability and Reproducibility (Gauge R&R). Regardless of the type of MSA being used, there are two key things we are usually interested in when we analyze a measurement system: • If the same person or piece of equipment measures the same item over and over again, do we consistently get the same data? Repeatability assesses whether each person can measure the same item multiple times with the same measurement device and get the same value when measuring continuous data. • If different people or pieces of equipment measure the same item, would they each get the same result? Reproducibility assesses whether different people can measure the same item multiple times with the same measurement device and get the same average value.
Measurement System Analysis works by setting up and running a controlled experiment to check
the repeatability and reproducibility of the within appraiser and between appraiser agreement of the
system [
After the Gauge R&R MSA is performed the usual decision criteria are: • GRR< 10%: the measurement system is acceptable; • 10%<GRR<30%: the measurement system needs improvement; • GRR> 30 %: the measurement system is unsuitable.
The following section provides descriptions of the experiment and data that will then be analyzed
using the MSA Gauge R&R method. The main document on which the research was based is the
Reference Manual of Measurement System Analysis 4th edition [
During the experiments, lighting measurements were carried out using mobile phones. The aim of the study was to find out: • the extent to which measurements of non-calibrated Internet of Things (IoT) measuring devices can be relied upon; • whether different software, illumination measurement programs influence the measurement results.
Lighting measurements were carried out in a room with covered windows, under artificial
lighting, so that the measurement system would be exposed to as few environmental factors as
possible. Mobile phones were randomly selected for measurements. Before taking measurements, it
was made sure that the charge level of mobile phone batteries was between 80-90%, in addition,
phone sensors and light-capturing cameras were wiped with a special cloth so that the optical
measurements would not be distorted due to light damping by the detector's dirty surface. The data
was fed into a specially prepared spreadsheet which allows the user to perform a complete Gauge
R&R study (Figure 3). At the end of experiments Analysis of Variance (ANOVA) and Xbar/Range
calculations are performed and conclusions formulated. ANOVA is a statistical test for Analysis of
Variance for detecting differences in group means when there is one parametric dependent variable
and one or more independent variables [
A common standard is to use a minimum of 10 parts for a Gauge R&R study, so we choose 10 different lighting levels which were adjusted by a dimmable lamp. The Lucas Nuelle Photovoltaic board CO4203-2A was used in the experiments. The experiment's setup is presented at Figure 2 and includes a dimmable 120W reflector lamp, integrated solar irradiance meter, Labsoft software with ability to measure irradiance.
Detailed specification of UniTrain Interface with virtual instruments are provided by the producer
of equipment [
In this experiment 3 different phones were randomly selected from the class. The phone models and applications used during the experiment are next: 1. Iphone 15+ / Lux Light Meter & Exponometer for iPhone by Mindateq Sp.z o.o. ver 1.0.9 2. Iphone 13 / Lux Light Meter & Exponometer for iPhone by Mindateq Sp.z o.o. ver 1.0.9 3. Xiaomi 11T/ Phyphox Light for Android Version: 1.1.16
During the experiment the setup depicted in Figure 2 was used. The scale of dimmable lamp potentiometer was divided into 10 equal parts. At each position the irradiance of the photovoltaic board was recorded by built-in sensors and results are presented in the Table 1 below. The spreadsheet allows the user to perform a complete Gauge R&R study. The ANOVA and Xbar/Range calculations were performed. During the experiment one operator was using three phones (Appraiser) and three trials (Trial#) accommodated at each measuring point (PART). The percentage of equipment variation, appraiser variation, part variation, and the number of distinct categories that can be distinguished are shown in Figure 4. The numerical summary showed the results of all calculations used in the ANOVA and Xbar/Range method of analysis. The Variance, Standard Deviation and Percent of Variance for each variation component were calculated. At the end the Gauge R&R for Percent of Study Variation and Percent of Tolerance were calculated using 5.15 standard deviations (99% of the variation) and 6.0 standard deviations (99,7% of the variation) and ANOVA table is displayed in Figure 5. After the Gauge R&R MSA was performed, study variation by using ANOVA and Xbar/Range was more than 30%, so we can come to the conclusion that the measurement system is unsuitable and correction actions should be taken.
After the first experiment we took corrective actions. We found that not all apps allow calibrating the phone illuminance sensors, so we selected the phones, which are close to a reference measuring device or true value. For this action the reference device Metrel Eurotest 61157 with type B lux meter sensor was used. The same phones which were used in the first experiment and software were used: Experiment was performed on the same setup depicted in Figure 2.
During the first experiment we noticed that the scale of the dimmable lamp potentiometer is nonlinear, so the next measurement points were selected in Table 2. All results were collected and written to the same spreadsheet depicted in Figure 3 and MSA was performed and ANOVA and Xbar/Range analysis was performed and graphically visualized (Figure 7).
This study demonstrated the value and limitations of using non-calibrated Internet of Things (IoT) devices, specifically mobile phones and various mobile applications for indoor illuminance measurement. Through a series of controlled experiments, the Gauge R&R method enabled us to evaluate the repeatability and reproducibility of measurements across different devices and software platforms. The findings confirm that while some mobile phones and apps provide fairly consistent readings, significant variability still exists between devices, however we did not find a big difference while using different software versions on the same phone. This variability underlines the importance of performing MSA prior to relying on IoT devices for scientific or policy-related decisions. Furthermore, our Citizen Science approach helped involve students and young researchers in real-world data collection and analysis, promoting open science principles and digital literacy. The results also highlight the necessity for harmonized metrology standards in Internet of Things (IoT)based measurements. Future work should focus on creating unified calibration protocols, expanding the range of tested Internet of Things (IoT) sensors, and integrating automated MSA tools into commercial apps to improve data reliability and user trust.
The study was carried out in the framework of cooperation activities of the Uninovis University Alliance in order to attract young researchers to scientific activities for creation of interdisciplinary projects.
The author(s) have not employed any Generative AI tools.