Smartphones are key devices in the Internet of Things paradigm. Social networking services on the Internet can use smartphones applications as data providers. The data gathered from sensors and data harvested from social networking services can be used by di erent applications for providing context-aware services. However, the excellence of the data oriented services depends on the Data quality (DQ). DQ is critical for decision making mechanisms. We present the problem related to DQ when dealing with social and sensor data. Also, we present and explore a framework whose objective is to evaluate and control DQ aspects when dealing with social and sensor data.
The Internet of Things (IoT) paradigm embraces several types of smart devices
which are composed by sensors, actuators and other devices with networking
capabilities [
Huge data repositories are produced in this scenario. Services can use data
fusion [
However, the excellence of the aforementioned services depends on Data
Quality (DQ) aspects [
In this chapter we brie y discuss some important Data Quality (DQ) concepts.
DQ is deemed as a critical requirement for decision making mechanisms,
applications and services on the IoT. There are di erent de nitions of DQ. In this work
DQ can be de ned as a concept that refers to how well the data corresponds to
the necessities of data processing mechanisms [
The (i) intrinsic data quality semantic aspect is related to the quality of
the data in relation to itself. As examples of dimensions there are: accuracy,
objectivity, believability and reputation [
This section presents the framework for social and sensor DQ standardizing. Standardizing DQ is making the data conforms certain DQ requirements. This framework receives data as input and transforms the input data to conform the DQ requirements of a given application or system that will receive such data as the output of the framework. The framework has two components: (i) social DQ component and (ii) sensor DQ component.
The (i) Social DQ component is responsible for standardizing social-originated
data according to DQ requirements. The (ii) Sensor DQ component is
responsible for standardizing sensor-originated data according to DQ requirements. Both
the (i) and (ii) components present two subcomponents: a rst subcomponent
is responsible for DQ assessment, while the second subcomponent is responsible
for DQ enhancement. DQ assessment subcomponent is responsible for the DQ
evaluation according to given DQ requirements. DQ enhancement subcomponent
is responsible for enhancing DQ to a given DQ requirement. The framework is
illustrated at Figure 1.
Social DQ component is responsible for social DQ standardizing. The rst
challenge for the Social DQ component is that social-originated data present
significant di erences according to the social media that originated the data. Di
erences can be if the data is structured or if it is unstructured [
These di erences in uence the development of Social DQ analysis solutions. It is important to notice that di erent Social platforms may request a priori specialized DQ analysis solutions. The social DQ component is composed by two subcomponents: (i) Social DQ assessment subcomponent and (ii) Social DQ enhancement subcomponent.
The Social DQ assessment subcomponent aims at assessing the DQ
from data originated from social networks. According to [
The Social DQ enhancement subcomponent aims at processing social data to enhance the social DQ to correspond a DQ requirement of a data consumer. Given the result of the Social DQ assessment subcomponent, the Social DQ enhancement subcomponent performs the following actions: (i) accept the data to be given to the data consumer, (ii) enhance the data to the corresponding DQ requirement of the data consumer or (iii) discard the data. 3.2
Sensor DQ Component
Sensor DQ component is responsible for standardizing sensor-originated data.
However, assessing and enhancing DQ for sensor data is not trivial. The
sensory platforms are heterogeneous and resource-constrained. Sensor data can be
originated from di erent kinds of sensor devices (e.g. smartphones, smart sensor
networks, etc. [
Regarding the sensed environment, sensory devices are placed in uncontrolled environments. In such case, misplacement, communication errors, power failure, sensor malfunction, human error or intentional misuse can potentially degrade sensor-originated DQ. The sensor DQ component is composed by two subcomponents: (i) Sensor DQ assessment subcomponent and (ii) Sensor DQ enhancement subcomponent.
The Sensor DQ assessment subcomponent has the objective of
assessing DQ for data-collected by sensors. Much of the sensor DQ assessment can be
performed on sensor data for assessing DQ according to the dimensions
mentioned in the session 2. Particularly, the dimensions of believability (comparison
with the correct operating bounds), completeness (missing values), free-of-error
(erroneous values), consistency (over time), timeliness (delay), accuracy
(deviation from true value) and precision (granularity of readings) are all important
aspects of high-quality sensor data [
The Sensor DQ enhancement subcomponent aims at the enhancement
of sensor-originated DQ. Since the sensor data is constantly being integrated by
data fusion procedures, it is important to perform DQ enhancement on the y, as
the data is being collected and processed. The on-the- y data processing avoid
erroneous and low quality data to propagate on fusion procedures. Also, after
data fusion procedures, applying well de ned DQ enhancement procedures can
avoid the production of low quality data. According to [
(i) Outlier detection helps to improve the overall quality of datasets by making them more consistent. Moreover, outlier detection is concerned about handling instances of the unreliable datasets. Metrics used in outlier detection techniques focus on enhancing the di erence between data values in order to identify outliers. (ii) Interpolation consists of inferring missing values based on other (available) values. Missing values represent gaps in available data about a certain entity or phenomena of interest for the user. As knowledge deriving processes use these datasets as input, these gaps could also lead to incomplete knowledge or wrong decisions which means that missing values could lead to a decrease in DQ. (iii) Data integration is important since social and sensor data come from di erent sensing platforms and di erent environments. In order to be used, these data need to overcome their structure di erences and inconsistencies to become truly bene cial for the various services. Data integration solutions mainly focus on resolving the inconsistencies between the various data streams. (iv) Data deduplication is a data compression mechanism aiming to reduce data handling's resources consumption by reducing the amount of available data through removing of duplicate data items and replacing them with a pointer to the unique remaining copy. Data deduplication is quite simply a removal process of redundant data items. (v) Data cleaning is a process composed of 3 main phases: (i) Determination of error types, (ii) Identi cation of potential errors and (iii) the correction of identi ed (potential) errors 4
In this work we presented the problem related to DQ when dealing with social and sensor data standardization. In this work we de ned that standardizing DQ is making the data conforms certain DQ requirements. Also, we presented and explored a framework for social and sensor DQ standardizing. The framework we presented has the primary objective of standardizing social and sensor DQ according to an application or system DQ requirements. The proposed framework has two components: social DQ and sensor DQ components. Social DQ component is responsible for standardizing social-originated data according to DQ requirements, while Sensor DQ component is responsible for standardizing sensor-originated data according to DQ requirements. Also, each component is composed by two subcomponents: DQ assessment and DQ enhancement subcomponents. DQ assessment subcomponent is responsible for the DQ evaluation according to given DQ requirements. DQ enhancement subcomponent is responsible for enhancing DQ to a given DQ requirement.
For the realization of a framework for social and sensor DQ standardizing, a future work is to systematically study the DQ requirements for di erent types of applications that deal with social and sensor data. This future work aims at directing the research for solving DQ standardizing problems. Another future works that directs solutions for the problem of DQ standardizing are related to the development of techniques for assessing and enhancing sensor and social DQ aspects. We direct future works to create techniques to assess and enhance DQ considering the diverse social and sensor data inputs.