The problem of creation and application of methods and means of information technologies of electronic commerce for various subject areas and applications has been studied, as the problems of developing mathematical models, solution methods and instrumental means for the integration of information resources and the functioning of intelligent electronic commerce systems with the use of effective intelligent models have been solved. Processes of modelling and design of business analytics tools for processing heterogeneous information resources based on ontologies are described. To solve the problem, several scientific tasks were performed, in particular, a classification of intelligent electronic commerce systems and means of processing heterogeneous distributed information resources of business analytics was proposed, a formal model of intelligent electronic commerce systems is developed using ontologies, its components, a structural model of information resources, methods and algorithms for designing intelligent electronic commerce systems based on the apparatus of ontologies and integration of information resources.
The processes of data integration have a fairly wide scope of practical applications. In particular,
in areas such as construction of DS of various types and directions, development of corporate
management systems, information Web systems, electronic business systems, computer
monitoring, etc. The information resources of such systems provide for the simultaneous use of a
significant number of various forms, structures, content, methods of presentation and application
of data [
Eliminating the listed contradictions and conflicts between input data is one of the tasks of the data integration method. The multi-level data integration method is based on the multi-level data model developed in the previous section and involves the decomposition of the overall process into sub-processes of value integration, data syntax, structure and semantics. A key element of this approach to integration processes is the possibility of their implementation at the level of data meta-schemas, which allows to reduce the number of references to, in fact, data, the volumes of which can be significant. Due to this execution of data integration procedures, they are transferred to the meta-level, operating instead of data with their formalized description. Similar principles of replacing operations on information resources with operations on metadata that specify them are used in the concept of the "semantic web", which is part of the general concept of Web 2.0, data spaces [8] and DS of the second generation [9].
The purpose of the method of multi-level data integration is to determine the principles,
composition and content of actions for the formation of the information resource of open
information systems and the order of their implementation. Since the object of application of the
method is information resources, it is advisable to organize the process of its development
according to a set of requirements [9-10], which are applied to the design processes of
information systems. The most acceptable is the application of the popular FURPS+ requirements
model [11] defined according to RUP (Rational Unified Process) specifications and IEEE Std
1233a-1998 [12], IEEE Std 610.12-1990 [13] standards, which today are typical in the field of
creating information systems and their components. Such a model provides for the formulation
of basic and additional requirements for the final result of the development process. The main
requirements that must be met by the method of multi-level data integration, according to the
chosen approach, form a set that will be formulated as follows [
1. Functionality is compliance of the functionality of the multi-level data integration method with the requirements and needs of users of the final result. 2. Usability is the possibility of applying the method for implementation using an open information system. 3. Reliability is the ability of the method to provide the appropriate level of quality indicators of the results under the specified conditions during the time of its application. 4. Performance is the ratio between the level of costs for the implementation of the actions provided for within the method and the weight of the results obtained. 5. Supportability is the ability of the method to be applied in all situations and conditions in which the means of the corresponding information system function.
In addition to the set of basic requirements, the FURPS+ model provides for the formulation
of additional requirements, which, unlike the basic set of FURPS requirements, are not unified
and are formulated to reflect the specifics of the area and subject of the application. For the
method of multi-level data integration, focused on open information systems, additional
requirements are as follows [
6. Portability is the ability to move the tools that implement this method from one application environment to another without rebuilding them. 7. Interoperability is the ability to jointly apply the method and means that implement it with other methods and means of forming IRP (information resource processing) of open systems. 8. Unification is the use of typical concepts, objects and tools and the formation of results, by the uniform requirements of IS (intelligent system).
The fulfilment of such a set of requirements aims to ensure the appropriate level of quality of the method being developed and advantages over other known methods of data integration.
The problem of data syntax integration (syntactic integration) is fundamental to the
integration of other components of their general description. Solving the problems of building a
generalized structure and semantics of data is possible only based on a single agreed system of
notation. The concept of data syntax itself is complex and takes into account various aspects of its
representation in documents, DB, DS data repositories, etc. [14]. Taking this into account, the data
syntax is presented as a combination of three components G=<A, T, R>, where A is an alphabet, T
is a set of data types, and R is a set of syntactic restrictions [
Integration of data
syntax Integration of constraints Integration of types Integration of the
alphabet Integration of values An integrated set of
constraints An integrated set of
types Integrated alphabet
Integrated set of values
According to this scheme, the syntax of the image of the values of the integrated data set GI is
presented as a combination of three components GI =<AI, TI, RI>, where AI =IA(A1, A2, …, AN) is the
alphabet of the integrated data set, formed by integrating the input alphabets data sets A1, A2, …,
AN ; TI =IT(T1,T2,…,TN) is the set of data types used in the integrated set, obtained as a result of the
integration of the data types defined for the input data; RI =IR(R1,R2,…,RN) is the set of constraints
of the integrated data set formed by the integration of the constraints applied to the input data;
IA, IT, IR are integration operators, respectively, of alphabets, data types, and constraints. Each of
these operators describes the mapping, respectively, of IA is sets of input alphabets into the output
global alphabet of the integrated data set, IT is sets of local input sets of data types into the output
global set of data types of the integrated set, IR is sets of local input sets of syntactic constraints
into the output global a set of syntactic restrictions of data of an integrated set [
3.1. Basic principles of the extended data integration model
Further development of the concept of modelling data integration processes is possible due to
the transition in the formal model from the concept of a scheme as an object of integration to the
concept of a data set. Each data set is a combination of a scheme, as some formalized description
of the composition and structure of data and a set of values (constants) formed according to the
requirements of the scheme. In this way, the formal objects of the model are a set of input (local)
data sets, an output (global) set of integrated data and a mapping that establishes correspondence
between the elements of the input and output sets (Fig. 2a). Formally, such a model is presented
as a triple of the form [
Global scheme of integrated data D2 2 . .
.
DN N . .
.
* *
D N N L T E
DI I
Data source 1 Data source 2 Data source N Figure 2: According to the improved model and data consolidation
A feature of the data consolidation method is the application of data extraction, transformation
and loading procedures as the basis of the data integration process. The result of consolidation is
a global set of integrated data, which has its scheme, which summarizes the composition and
content of the schemes of the input sets. A description of the process of data consolidation
according to the proposed generalized model is given in Fig. 2b [
The method of data federalization differs in the way of forming a set of integrated values (Fig.
3a) [
<{<Di,Σi > | i=1,…, N}, View(<Di,Σi >) | i=1,…, N, <DI,ΣI>>, (1)
where < {<Di, Σi > | i=1,…N} is a set of input data sets, each of which is given by a scheme Σi and
a set of values Di; <DI, ΣI> is a global set of integrated data, with the scheme ΣI and a virtual set of
DI values; View(<Di, Σi >) is mapping of the scheme of the input data set to the global scheme of
integrated data. The key principle of such mapping is the formation of a description of a subset of
data of the local input set in terms and composition that meets the requirements of the global
scheme, while the set of values described by the new scheme Σ*i is a subset of the input local set
of values <Di, Σi>. The result of the mapping is a global scheme of integrated data, formed as a
union of mappings of local schemes [
* RM M
DI I
Data integration using the replication method involves the formation of a certain mapping
(projection) of the local input data set according to a given mechanism, similar to the
federalization method. The fundamental difference is that the result of displaying input data is
not a virtual set of values, but some intermediate set of data that has its physical image formed
according to some scheme, as in the case of data consolidation. But at the same time, the data set
created in this way - a replica, cannot be moved to a specially defined storage environment. An
advantageous global set of integrated data is formed as a union of a set of replicas. The general
scheme of data integration by the replication method is shown in Fig. 3b [
A feature of data integration using the hybrid method (Fig. 4) is the combination of the
possibilities of the three methods described above – consolidation, federalization and replication
– in one process. In this case, the global initial set of integrated data is formed as a heterogeneous
entity that combines several segments, each of which is formed based on different methods and
technologies [
DC C
L T E
D*C *C R DR . . .
F DF
D*F *F
DC DI
*F
R* *R I
Collage (mashup), as a method of integration, is most often used in Web-systems to combine
in a single presentation of data received from different sources, different in form, structure, and
methods of representation, but combined by a common content/application. The peculiarity of
collage is the absence of a permanent scheme of integrated data and the dynamic formation of a
set of values with each access to resources of this type. At the same time, the initial data are
combined in various ways, forming, as a result, arbitrarily structured hybrid data. The general
scheme of the data collage process is shown in Fig. 5a [
D1 1 D2 2 . .
.
DN N r e v r e s p u h s a M
{<Di,Σi > | i=1,…N}, Mashupi(<Di,Σi >), <DI,ΣI> >, (2)
where {<Di, Σi > | i=1,…N} is the input local data set, with scheme Σi and set of values Di; <DI, ΣI>
is the initial global set of integrated data; Mashupi(<Di, Σi>) is a mapping that forms a data collage
element for further combining parts of input local data sets into a single view. In the collage
process, some subset of D*i values is selected from each input local set, which is described by the
scheme Σ*i. From these parts, by combining and superimposing different types of data and
forming a global scheme as a combination of schemes, a single integrated data set <DI, ΣI> is
formed for presentation to the user. The difference between integration by collage and other
methods is the absence of physical storage of integration results and the dynamic formation of a
global scheme of integrated data upon user request [
3.7. Formal modelling of data integration processes and results
The analysis of the results of modelling data integration processes using various methods and
methods using an extended formal model allows the following conclusions to be drawn [
The general conclusion regarding the modelling of data integration processes is that as a result
of integration, new data values, new forms and presentation formats, new data structures, new
content and new purpose of data are created [
The results of the analysis of formal models of data integration using various methods show
that in the process of integration, significant transformations of the composition, content and
form of data occur. This means generating, based on input sets, a set of new final data that have
fundamentally different properties. This creates the basis for further development and
improvement of the formal model of the data integration process by introducing into its
composition elements that describe the main properties of the data and the order of their change.
According to the concept of presenting data as a formal system, the data form some formal
language that is used to denote a set of values and concepts from a certain SA in the environment
of the information system [
DS=I(DS1, DS2, …, DSN), (3)
where I is the data integration operator, DS1, DS2, …, DSN is the set of input initial data sets, and
N is the number of data sets participating in the integration process. In general, such data sets
may contain repeated values, i.e. [
D1∩ D2 ∩ … ∩ DN ≠. (4)
Given the data model, which is based on the specification of their syntax, semantics and
structure DS=<D,>=<D, G, S, H>, the formal definition of the integration process can be reduced
to actions on these components, replacing the DSI value with a detailed description all
components of data definition as follows [
<DI,I>=<DI, GI, SI, HI>=I(<Di,i> | i=1,…N)= (5) =I(<D1, G1, S1, H1>,<D2, G2, S2, H2>, …, <DN, GN, SN, HN>), where <Di, Gi, Si, Hi>, i=1,2, …, N is the detailed formal representation of the ith data set.
In this way, the problem of data integration can be decomposed into separate problems of data
value integration, syntax integration, structure integration, and semantic integration. The general
data integration operator I is presented as a combination I=<ID, IG, IS, IH>, where IV is the value
integration operator, IG is the syntax integration operator, IS is the data structure integration
operator, and IH is the semantics integration operator. At the same time, the integration process
will be decomposed into corresponding sub-processes, which can be described by a formal
scheme of the form [
<D, G, H, S> =<ID(D1, D2, …, DN), IG(G1, G2, …, GN), IS(S1, S2, …, SN), IH(H1, H2, …, HN)>. (6)
The mutual relationship of these processes and their classification by levels are shown in Fig.
5b. According to such a scheme, each subsequent level of integration is based on the results of the
previous one. Thus, the semantic integration of data is possible only after the integration of their
structure, which, in turn, requires the construction of an integrated syntax that defines the
methods of data representation and the integrated set [
The integration of alphabets at the stage of designing a unified integrated data processing
environment consists in creating a consistent set of symbols for representing values from the
resulting data set - the integrated AI alphabet, such that for each symbol of the input alphabet Ai,
which is used to represent the value of the input data set Di (i=1,2, …, N), there is a unique mapping
αi: Ai→AI, which matches each symbol of the input alphabet of the ith data set σi(Ai) with a symbol
of the integrated alphabet – σ(AI). The following ratios of input and integrated data alphabets are
possible (Fig. 6) [
Integrated alphabet AI
Input alphabet A1
Input alphabet A2
Input alphabet A4
Input alphabet A7
Input alphabet A5
Input alphabet A8
Input alphabet A3
Input alphabet A6
We present the process of building an integrated alphabet as a sequence of solving
interconnected problems according to the following scheme [
1. Let A0 be some initial set of symbols of the integrated alphabet.
2. For each of the input alphabets Ai, i=1,2,…, N, the ratio Ai A0 is checked. If it is fulfilled,
then all characters of the alphabet Ai used to represent the values of the data set Di are also
acceptable for representing the corresponding values in the integrated data set D. Therefore,
it can be assumed that the input data can be included in the integrated set without changing
the form of their presentation.
3. In this case, the phenomenon of polysemy of symbols is possible. We will call polysemous
symbols that have the same shape and reflect different meanings, for example, the Ukrainian
letter "І", the Latin letter "I", the Roman numeral "І" (1), the Latin letters A-F, which are used
to represent both letters and numbers in the 16th number system, etc. Such a phenomenon
may cause an ambiguous interpretation of data values and their content in the future. The
problem of polysemic characters has the following solutions:
banning the use of the same symbols to denote different concepts - this method involves
defining a single image for all symbols that have the same shape; this option for solving the
problem of polysemic symbols is possible in cases when they are used for formal meanings
that do not have additional (phonetic, lexical or substantive) interpretation (for example, the
same type of use of Latin and Cyrillic letters that match the spelling in car registration
numbers); in this case, problems are possible when interpreting, reading or phonetizing data
values;
the use of polysemic symbols without restrictions - for each of the symbols that have the
same form, they retain their method of application; in this case, the problem of polysemy of
symbols of the integrated alphabet is not solved in the process of integration, but is transferred
to the level of data application;
replacement of identically shaped symbols with an alternative image - transliteration; this
transformation allows you to eliminate the polysemy of characters without narrowing the
possibilities of data presentation.
4. If the set of characters of the input alphabet is not a subset of the integrated alphabet – Ai
A0, then it is divided into two subsets – Ai1=AiA0 and Ai2=Ai\A0. The first includes symbols
that are elements of the integrated alphabet, the process of integration in this case is described
above. The set Ai2 includes characters of the input alphabet that are not elements in the current
state of the integrated alphabet A0. In such a situation, character polymorphism is possible. We
will consider polymorphic symbols to be symbols that differ in image form and are used to
denote the same concepts. For example, upper/lower case letters in words represent the same
sounds, numerical values can be represented in different number systems, using Arabic, Latin
numbers or letters, etc. The appearance of polymorphic symbols in alphabets is a possible
cause of ambiguous perception and interpretation of data values during their processing. As
for solving the problem of processing polymorphic symbols, the following solutions are
possible [
Regarding the set of symbols Ai2 = Ai \ A0, the following options are possible prohibiting the use of symbols from the Ai2 set to represent integrated data; expansion of the integrated alphabet due to the inclusion of a set of symbols Ai2 in its structure, with the formation of the next version A1 = A0Ai2; transliteration – replacing symbols that are not elements of the integrated alphabet with symbols from the A0 alphabet. 5. As a result of iterative repetition of the described sequence of actions, an integrated alphabet AI=IA(A1, A2, …, AN) is formed, which defines a set of symbols for presenting data values in an integrated set.
The integration of data types in the construction of the output integrated set consists in the
formation of a set of data types TI, such that for each of the types applied in the input data sets
there is a mapping τ: Ti → T, which establishes a one-to-one correspondence between the data
types t(Ti) applied in the input set Di (i=1,2, …, N) and t(TI) are the data types used in the
integrated data set DI. The mutual relationship of different sets of data types in the process of
integration is shown in Fig. 7. As can be seen from the diagram, similar to the process of
integration of alphabets, input sets of data types may or may not have full or partial intersections
with the integrated, and may or may not have mutual intersections with each other. The process
of forming an integrated set of types involves the sequential execution of such actions [
1. Let T0={ t1(T0), t2(T0), …, tn(T0)} be some initial set of types that are defined in the integrated
data set forming the initial state of the TI type set.
2. For each set of input data Di (i=1,2, …, N), check the ratio Ti T0, which determines the
agreement of the types of the input set with the types of the integrated data set DI.
3. Fulfilment of this condition does not guarantee that only types allowed for use in the
integrated set are used to represent the data of the input set Di since there is a possibility
of type polysemy. We will call polysemous types that have the same designation and differ
in implementation methods. For example, a value of the "date/time" type can be
represented by both numeric and character values, the "text" type in some applications
represents character strings, in others - notes, values of the logical type are represented
as numeric or bit, etc. Discrepancies of this nature are a potential factor in possible errors
in data processing, ambiguous interpretation and obtaining incorrect results. This kind of
inconsistency of data types has, in particular, the following solutions [
Integrated set TI of data types Input set of types Т1 ВInput set of types Т2
Input set of
types Т4 Input set of types Т8
Input set of
types Т3
Input set of Input set of types Т5 types Т7
Input set of types Т6 replacement of homonymous data types with new ones, which by definition do not coincide with others; reformatting the values of the input data set to the format of the corresponding types defined in the integrated data set; 4. If the set of data types Ti, which are applied in the input set Di, exceeds the set of data types of the integrated set D, i.e. Ti T0, this indicates the presence in the input set of data belonging to such types that are not valid data types of the original integrated set.
Therefore, the data types of the input set are divided into 2 such subsets: a subset of types Ti1 = Ti T0, which are included in the set of types of the integrated data set; a subset of types Ti2= Ti \ T0, which are not included in the set of types of the integrated data set.
For the subset of Ti1 types, the type-matching procedure is as described above. In the case
when the data of some input set Di belong to types that are not supported in the original
integrated data set D1, the following variants of further transformations are possible [
The first way is easier to implement, and the second - expands the possibilities for describing and manipulating data in an integrated set.
6. The result of the steps described above is a generalized and agreed list of data types that
are used in determining the units of the integrated set TI =IT(T1, T2, …, TN) [
The integration of restrictions, which are used when forming data values of some input sets,
involves the formation of such a set of restrictions RI=(r1(RI), r2(RI), …, rm(RI)) that for each
restriction r(Ri) Ri, applied to some input data set Di (i=1,2, …, N) there is a one-to-one
correspondence given by the mapping ρ:r(Ri) → r(RI). However, unlike alphabets and data types,
the restrictions are not free elements, they are formulated and applied only to specific data types,
categories or values. Therefore, each restriction that is applied to a certain set of data Di is defined
as a condition of the form r(Ri, t(Ti), Dji), which is determined by such factors as belonging to a set
of restrictions Ri, binding to a certain type of data – t(Ti), and the scope is some subset of the data
set Dji Di. Therefore, the problem of integrating the set of constraints of the input data sets into
a single set of constraints of the integrated set can be solved only after performing the integration
of the alphabet and data types. The ratio of the sets of input data constraints and the integrated
data set is shown in Fig. 8. The input sets of constraints can have partial intersections with each
other, be subsets of each other, be completely independent, be fully or partially part of the
integrated set formed by their integration, or not have an intersection and not be part of it. The
general sequence of the process of integration of input syntactic constraints, the purpose of which
is to create a single, consistent and complete set of constraints applied to values from the
integrated data set, is presented in the form of a scheme of actions [
Integrated set of constraints RI
Input Constraint
Set R1
Input Constraint
Set R5
Input Constraint
Set R8
Input Constraint
Set R2
Input Constraint
Set R3
Input Constraint
Set R4
Input Constraint
Set R6
3. The fulfilment of this condition means that each of the constraints of the input data set
takes place in the integrated set. But for the final determination of the possibility of
applying restrictions to integrated data, the following factors are additionally checked
[12, 15-18]:
the presence among the data types of an integrated set of types for which restrictions are
defined, i.e. for each of the restrictions rj(Ri) Ri there is tj(Ti) TI, where tj(Ti) is the data type
to which the restriction is applied, TI is multiple types of integrated dataset;
the presence among the set of values of an integrated data set of values for which
restrictions are defined, that is, for each of the restrictions r(Ri) Ri, Dji DI is performed,
where Dji is a subset of values of the data set Di to which the restriction is applied, DI is an
integrated data set;
the fulfilment of these requirements ensures the possibility of applying the restriction to
the data of the integrated set, and the lack of appropriate data types and/or values makes it
impossible to apply this restriction to the integrated data set.
4. If among the set of constraints Ri of some input set Di some are not constraints of the
integrated data set DI, i.e. Ri R0, the set of constraints is divided into subsets Ri1 = Ri R0
and Ri2= Ri \ R0.
5. The set of restrictions Ri1 is consistent with the set R0 and the process of its integration is
performed as described above, but the set of restrictions Ri2 has the following integration
options [
In the first case, each of the restrictions that does not have an object of application can be
removed without loss from the set of restrictions of the integrated set. In the second, the
procedure for matching additional constraints Ri2 and a set of constraints R0 of the integrated data
set is applied. The reconciliation of these sets of restrictions is achieved due to [
By performing a sequence of actions on the integration of alphabets, data types and syntactic
constraints according to the scheme described above, a complete and consistent set of elements
of the integrated syntax of GI data is formed, which is used as a method and means of displaying
integrated data obtained as a result of the processes of data extraction, transformation and
loading in DS, as well as with their dynamic integration in operational systems. At the same time,
the problem of detecting and correcting the elimination of contradictions between the local
syntax of the input data sets to be integrated is solved. The general order of syntactic integration
describes such a sequence of steps [
Step 1. Constructing an integrated alphabet as a complete and consistent set of characters to represent data values in the original integrated set. Performing this step involves the implementation of the following procedures.
1. Detection and elimination of contradictions of input alphabets in the process of syntax integration. This is done according to the following rules. Elimination of character polymorphism – the alphabet of the resulting integrated data set cannot contain different characters with the same interpretation. Such a rule for matching the representation of the symbols of the alphabets Ai and Aj is described by an expression of the form
Alph1(Ai, Aj): αAi,Aj | inti(α) = intj(), (7) where Alph1 is the rule identifier, α, are symbols of the input alphabets Ai and Aj; inti(α), intj() are symbol interpretation functions.
Elimination of character polysemy - the output alphabet of the resulting integrated data set cannot contain the same characters with different interpretations. The rule for matching the interpretation of the symbols of the alphabets Ai and Aj is described by an expression of the form
Alph2(Ai, Aj):αAi,Aj | inti(α) intj(α), (8) where Alph2 is the rule identifier; α is a symbol included simultaneously in the input alphabets Ai and Aj; inti(α), intj(α) are functions for interpreting symbols in the alphabets Ai and Aj. 2. Construction of an integrated AI alphabet by combining agreed local input alphabets according to the rules defined in clause 1:
AI =IA(A1, A2, …, AN) = A1 A2 … AN | (9)
Alph1(A1, A2, …, AN)=true ^ Alph2(A1, A2, …, AN) =true, where IA is the alphabet integration operator; A1, A2, …, AN is a set of input local alphabets; Alph1(A1, A2, …, AN), Alph2(A1, A2, …, AN) are the rules for matching alphabets defined in paragraph 1. a) and 1. b).
Step 2. Construction of a single, consistent list of data types that are used in the original integrated set. The process of integrating data types involves the following actions. 3. Identification and elimination of inconsistencies in data typing methods from input sets.
The following rules apply to this. Elimination of polymorphism of types - the data types used in the original integrated data set cannot contain different types that have the same interpretation. Such a matching rule for sets of data types Ti and Tj is described by an expression of the form
Type1(Ti, Tj):t1Ti,t2Tj |inti(t1)= inti(t2), (10) where Type1 is the rule identifier; t1, and t2 are data types of input resources, which are included in the sets of types Ti and Tj, respectively; inti(t1), intj(t1) are interpretations of types t1 and t2, respectively, in sets Ti and Tj.
Elimination of polysemy of types - in the composition of types used for data typing of the resulting integrated data set, there cannot be identically defined types with different interpretations. The rule for matching the interpretation of sets of types Ti and Tj is described by an expression of the form Type2(Ti, Tj): tTi, Tj | inti(t) intj(t), where Type2 is the rule identifier; t is a type included simultaneously in the input local sets of types Ti and Tj; inti(t), intj(t) are type interpretation functions, respectively, in the sets Ti and Tj. 4. Construction of a set of types of the original integrated resource TI by harmonizing and combining local input sets of types according to the rules defined in clause 1:
TI =IT(T1, T2, …, TN) = T1 T2 … TN | (11)
Type1(A1, A2, …, AN)=true ^ Type2(A1, A2, …, AN) =true, where IT is the integration operator of input sets of data types; T1, T2, …, TN are sets of input local data types; Type1(T1, T2, …, TN), Type2(T1, T2, …, TN) are the rules for matching alphabets defined in clauses 1. a) and 1. b).
Step 3. Formation of a single consistent set of syntactic constraints by merging and matching local constraint sets of input datasets.
Detection and elimination of contradictions in the syntactic constraint sets of input datasets. The following rules apply to this.
a. Elimination of polymorphism of constraints - in the composition of the set of syntactic constraints, which are applied in the resulting integrated data set, there cannot be different constraints that have the same interpretation. Such a rule for matching the sets of constraints Ri and Rj is described by the expression
Restrict1(Ri, Rj):r1Ri,t2Rj |inti(t1)= inti(t2), (12) where Restrict1 is the rule identifier; r1, r2 are syntactic restrictions applied to input resources, which are included in the sets of restrictions Ri and Rj, respectively; inti(r1), intj(r1) are an interpretation of syntactic constraints r1 and r2 in sets Ri and Rj.
b. Elimination of polysemy of constraints – in the set of syntactic constraints, which are applied to the data of the resulting integrated set, there cannot be identically defined constraints that have different interpretations. The rule for matching the interpretation of the sets of constraints Ri and Rj is described
Restrict2(Ri, Rj): rRi,Rj | inti(r) intj(r), (13) where Restrict2 is the rule identifier; r is a syntactic restriction that is simultaneously included in the input local sets of restrictions Ri and Rj; inti(r), intj(r) are functions for interpreting constraints in the sets Ri and Rj.
Construction of a set of syntactic restrictions of the original integrated RI resource by harmonizing and combining local input sets of application types defined in clause 1, rules:
RI =IR(R1, R2, …, RN) = R1 R2 … RN | (14)
Restrict1(R1, R2, …, RN) = true ^ Restrict2(R1, R2, …, RN)=true, where IR is the syntactic constraint integration operator; R1, R2, …, RN are sets of input local syntactic constraints; Restrict2(A1, A2, …, AN), Restrict2(A1, A2, …, AN) are syntactic restriction matching rules defined, respectively, in clauses 1. a) and 1. b).
Step 4. Constructing an output syntax for representing data in an integrated set. The output integrated data syntax GI is formed based on the integrated consistent alphabet AI, the integrated consistent set of data types TI and the integrated output set of syntactic constraints RI
GI =<AI, TI, RI>. (15)
The syntax formed in this way provides a correct, consistent and unambiguous representation of the data values in the data set, which is created as a result of their integration.
The problems of creating and maintaining heterogeneous structures of integrated information
resources, in general, go beyond the functional capabilities of traditional data storage
environments implemented by DB servers and DBMS [15-19]. Today, many other abstractions
and management methods are known, which either confirm their suitability or are removed from
the management environment of integrated heterogeneous content [
CI=<R, NR1, NR2, …, NRk , JR, JN, JRN>, (16) where СI is a description of the structure of the integrated information resource; R is description of the structure of the relational component, which is formed by structured data presented in the form of database tables; NR1, NR2, …, NRk are description of non-relational components of various types; JR is a set of connections between relational elements; JN is a set of connections between non-relational elements; JRN is a set of relations between relational and nonrelational elements.
R1 R
R2 .. .
Rn Relational component
NR2
NRm
NR1 ...
NR
The integration of data structures (structural integration) of data is defined as the process of
a coordinated combination of structured (relational) data stored in databases and non-relational
data stored in formats other than DB. The relational component in this sense is the central
element of integration since modern DBs and DBMSs [15-18] provide a sufficiently wide range of
opportunities for joint and coordinated processing of not only structured data, but also
information resources specified by other data methods [
The main problems of DB integration with other types of data and directions and principles of
their solution are defined in [21-23]. Typical approaches to the integration of structured
relational, weakly structured/self-structured data are described by models [
a1 a2 ... an
XML1 ...
XMLm 3. Integration into databases of triggers and data processing procedures. The use of such elements ensures the implementation of the concept of an active database. Such a DB, together with the values, stores a description of certain rules and actions that are performed when the state of the database changes. Tables, which include traditional data and active elements, are described by a structure model of the following type R(A1, A2, …, Ak, T1, T2, …, Tm), where A1, A2, …, Ak are table columns that represent ordinary typed values, T1, T2, …, Tm are a set of triggers that describe the actions associated with changing the state of the table (Fig. 11b).
M1
M1 ...
Mn A1
A2 ...
An
A1
A2 ...
An
T1 T1 ...
Tn
4. Integration of static data with streams and data queues. Stream data is a set of values that
is not stored on the system media, but exists only at the time of application of this data. An
example of streaming data is monitoring, stock exchange information, broadcast news in
a standardized format, etc. The data flow is formed as a result of the execution of requests,
forwarding or selection of data. A queue is a special type of data stream in which each unit
is given an ordinal character. The structure of the data stream S at a certain time t is
described by an expression of the form [
St=<S(Rt), S(Xt), S(St`), S(Wt)>, (17)
where S(Rt) is the structure of a set of values obtained as a result of selection operations from
relational database tables; S(Xt) is the structure of a set of data obtained as a result of selection
from sets of weakly structured data; S(St`) is the structure of a set of data obtained as a result of
selection from other data streams; S(Wt) is the structure of a set of data obtained from web
resources. The result of the integration of static and flow components is a semi-dynamic structure
(Fig. 12), which combines data stored in databases and data formed in the form of a time-varying
flow [
The structure of the integrated set, which combines relational and stream data CI, is described as CI=<Rt, St, JRt, JSt, JRSt>, where Rt is the structure of the relational component determined at time t; St is the structure of the flow component determined at time t; JRt is the scheme of connections between the elements of the relational component at the moment of time t; JSt is a diagram of connections between elements of the flow component at time t; JRSt is the scheme of connections between the elements of the relational and flow component at the moment of time t [15-18]. R
R2 ...
Rn
m
S 1. Integration of information resources in DS structures and data spaces (Fig. 13) [15-18].
The significant volumes and variety of data used in the activities of large businesses and other formations require the use of special approaches to their organization and processing, which ensure their availability and efficiency. Today, DS and data space technologies are such ways. These information resources provide the ability to perceive and apply a large number of values obtained from tens or hundreds of operational DBs, due to their aggregation and fusion. Modern DSs support two methods of integration: asynchronous - "extraction-transformation-loading" (ETL) and operational (OnLine) integration of sets and streams and other units of data received from different sources. The first provides for the formation of a stable and time-invariant DS for long-term, repeated use, and the second - is the formation of an integrated resource that reflects the state of the data at a certain point in time and provides one-time access to the IR of the corporate system. DS, as a means of integrated presentation and processing of data, provides for the formation of some global data structure, which reflects the structures of local input resources. Depending on the method of integration, such a scheme can be static (in the case of using "extraction-transformation-loading" procedures) or created dynamically (for DS based on On-Line integration) [15-18].
Users
In both cases, the global CG structure is presented as a union of the mappings of n local
structures onto the global one, built according to a predefined procedure [
CG=Str1(CG) Str2(CG) … Strn(CG), (18) where CG is the global structure of DS; Stri(CG) is the mapping of the local structure of the ith input resource to the global structure (i=1,2, ..., n). The structure formed in this way is the result of the integration of a set of input resources and provides the possibility of their joint application, management and access.
2. Integration of sensor data and sensor networks. This relatively new direction in data
integration makes it possible to realize the possibilities and advantages of cloud (network)
computing [15-18]. A sensor network is a network of distributed devices, each of which is
a source of certain data [23]. Examples of such networks are a monitoring sensors
network, a network of points of goods or services sale, terminals for customer service,
meteorological or geo-informational distributed measuring complexes, etc. From the point
of view of integration, the sensor network is a distributed database that can function both
independently and is compatible with other databases as part of integrated data
processing systems (Fig. 14) [
R1 R
R2 ...
Rn
A network of sensor data can include both structured objects and loosely structured data sets,
or data sets without a predefined structure. In addition, sensor network data can be both static
and dynamic, that is, it can be presented both in the form of sets and streams of data. In this case,
the sensor network is divided into two parts - static and dynamic, between which connections
and the order of interaction are defined [15-18]. The general structure of the data obtained as a
result of the integration of relational and sensor network data: CI=<R, SN, SNt, JR, JSN, JSNt, JRSNt>,
where R is the structure of the relational component; SN is the structure of the static component
of the sensor network; SNt is the structure of the dynamic component of the sensor network at
time t; JR is the scheme of connections between elements of the relational component; JSN is a
scheme of static connections between sensor network elements; JSNt is the scheme of dynamic
connections between elements of the sensor network at time t; JRSNt is a scheme of connections
between the elements of the relational and static and dynamic components of the sensor network
at the moment of time t [15-18]. The integrated data structure formed in this way provides access
and management of an integrated information resource, which includes relational structured
data and sensor network data [
The models of structural integration, which are described above, reflect the peculiarities of the
formation, processing and application of heterogeneous content of open information systems of
various directions. This approach allows to formalize and generalize methods of structuring and
maintaining access to integrated information resources of open systems regardless of their
content, purpose and means of implementation. The general order of integration of
heterogeneous data structures is based on the following scheme (Fig. 15) [
Step 1. Formation of a single agreed description of structured (relational) data. The structure of the relational component is part of the original structure and describes a set of tables/images from the DB of the input local resources, which form a subset of the data participating in the formation of the original integrated resource. It is described by the expression of the species
RI=(DB1.(R1, R2, …, Rk1) , DB2.(R1, R2, …, Rk2), …, DBN.(R1, R2, …, RkN ), JIR), (19) where RI is the scheme of the relational component of the original integrated data set; DB1, DB2, …, DBN are databases of local input resources; N is the number of local input resources; DBi.(R1, R2, …, Rki) is relational data structure in the database of the ith input structured resource (i=1,2, …, N), k is the number of tables or images of the input resource; JIR is a diagram of relationships between relational components in an integrated source dataset. 1 S
S2 .
. .
m
S Streaming
data
Procedural data M1
M1 ...
Mn
R a1
S2
Sm
S Weakly structured
component a2 ...
XML is a component an
XML1 ... XMLm
Step 2. Formation of an agreed description of non-relational resources. Performing this step involves the following steps.
1. Specification of composition, properties of units of weakly structured resources and connections between them. As a result, they form a general description of resources in XML, HTML, formatted documents, etc. formats. 2. Specification of the composition and properties of resource units without a predetermined structure and the relationships between them. The result is a list and description of the structural features of text, graphic, and multimedia data sets. 3. Formation of an agreed description of procedural and active data and their connections. 4. Building a general description of the stream data structure. 5. Determination of relationships between various components of the non-relational component of the original integrated data set.
Step 3. Formation of a general description of the elements of the relational and non-relational components of the original integrated data set.
Step 4. Determination of the scheme of mutual relations between the elements of the relational/non-relational component of the original integrated data set.
Step 5. Construction of a description of a single global CI structure as a description of the
composition and properties of the original integrated data set, formed by integrating the
structures of input local data sets of different natures [
The result of performing the assigned tasks is the specification of protocol and language tools
for creating metadata that describe the syntax, semantics, and structure of data in integration
processes, as an interoperable service of an intermediate level [
Some of today's popular approaches, principles and concepts [
The main principles of XMI application are [
All the approaches described above, to a certain extent, have an impact on data integration processes. Therefore, in the integration service concept formulation, the features of each of them were used [52-63].
The data integration service implements a common unified data access model through a
special layer of standardized reusable data abstractions [
... Managers
Consumers ...
USERS
Analysts external level APPLICATION Electronic business ... Businessanalytics
Monitoring
Corporative management applied level
Social ...
resources
The functioning of data integration services as part of the information system is built
according to the following scheme (Fig. 17) [
request metaschema
results application
integration services ...
Data repositories results Access service N Repository 1
Repository 2
Repository N 1. Unification - the use of protocol-level tools makes it possible to reduce the processes of data integration to the manipulation of typical concepts, objects, properties and procedures for their processing. 2. Interoperability - provides the possibility of joint use of means of maintaining the application protocol of data integration with any other means of data processing. 3. Mobility - the implementation of protocol means of data integration at the application level makes them independent of the specifics of the platforms and the implementation environment, which ensures the possibility of their free movement. 4. Processing formats and procedures standardization - XML use as the basis of the language means of describing objects and processing metadata makes it possible to process the necessary resources by standard means and according to standardized procedures. 5. Compliance with the principles of SOA - provides easy access and use of data integration protocol tools for a wide range of information system users. 6. Implementation ease – such means use of integration does not require restructuring business processes, platforms or other means of maintaining the open IS environment, as it involves the additional, relatively autonomous intermediate layer formation of data abstraction. 7. Insignificant cost - the absence of the need to rebuild information systems, and develop, purchase and implement complex and expensive data integration tools and platforms makes projects based on protocol tools relatively inexpensive.
The Data Integration Service Protocol (DISP) is an application-level protocol that defines the
methods, order and means of implementing the data integration service at the request of the user
of the open information system. The main purpose of using the protocol is the organization of the
service for the user of the open information system, which forms the data set he needs by
integrating the data located in a set of disparate local information resources. The protocol does
not depend on protocols of lower (presentation and session) levels and provides for user
interaction with open information system services using standardized interfaces. The goals
achieved by the protocol are as follows [
Based on the performed analysis and generalizations, the main methods of building information resources of business analytics systems are determined - homogenization, distribution, and integration. The approach based on data integration is the most appropriate to the features of business intelligence systems. Together with the other resources integration of business intelligence systems, data integration forms a single coordinated set of actions for the design, construction and support of business intelligence systems.
Based on the conducted analysis, it is possible to conclude that there are several relevant today's interrelated problems in the e-commerce field.
1. The problem of forming a high-quality information resource of business analytics systems that is relevant to the system goals, adequate to its tasks and appropriate to the needs and requirements of users. 2. The problem of heterogeneous form, content and properties integration of input information resources into a single agreed common resource of business analytics systems. 3. The problem of developing and substantiating a single generalized theoretical concept of data integration for the unified effective methods, tools and modern technologies creation to solve the problem of forming information resources of business analytics systems in various industries and fields of application.
The development of methods of multi-level data integration in business analytics systems allows us to draw the following conclusions: It is advisable to base the method on a multi-level data model that takes into account various aspects of this data (syntax, semantics, meaning); The key point of the integration process is its implementation based on data metaschemas, which made it possible to reduce access to the actual data and reduce the time required for integration; The general process of integration is carried out as a set of sub-processes of integration of values, syntax and semantics of data; The use of ontologies in the processes of semantic data integration made it possible to carry out integration at the level of data content, to achieve the same interpretation of data by both people and machines.
In the process of developing knowledge presentation methods for intelligent business analytics systems, the algebraic theory of types was laid as its basis. An algebraic system has been built in which there is an unambiguous mapping between ontology entities and abstract data types. The development of presentation methods and the architecture of intelligent networks of business processes allows us to draw the following conclusions: IS is based on the ontology of business processes. Given the changing nature of SA and the incomplete knowledge of the ontology developer, it is necessary to use ontologies that can adapt to changes. Intelligent networks of business processes should be presented as a set of interacting executable ontological models.
As a result of the work, a specification of language tools was developed for creating metadata that describes the syntax, semantics, and structure of data in integration processes, as an interoperable service of an intermediate level. The advantages of linguistic means of data integration over instrumental ones are due to the following factors.
1. Unification - the use of protocol-level language tools makes it possible to reduce the processes of data integration to the manipulation of typical concepts, objects, properties and procedures for their processing. 2. Interoperability – provides the possibility of joint use of means of maintaining the application protocol of data integration with any other means of data processing. 3. Mobility – the implementation of protocol means of data integration at the application level makes them independent of the specifics of the platforms and the implementation environment, which ensures the possibility of their free movement. 4. Standardization of processing formats and procedures – the use of XML as the basis of the language means of describing objects and processing metadata makes it possible to process the necessary resources by standard means and according to standardized procedures. 5. Compliance with the principles of SOA – provides easy access and use of data integration protocol tools for a wide range of information system users. 6. Ease of implementation – such means use of integration does not require restructuring of BP, platforms or other means of maintaining the open information system environment, as it involves the formation of an additional, relatively autonomous intermediate layer of data abstraction. 7. Insignificant cost – the need absence to rebuild information systems, and develop, purchase and implement complex and expensive data integration tools and platforms makes projects based on protocol tools relatively inexpensive. [8] F. K. Bruder, R. Hagen, T. Rölle, M. S. Weiser, T. Fäcke, From the surface to volume: concepts for the next generation of optical–holographic data‐storage materials, Angewandte Chemie International Edition 50(20) (2011) 4552-4573. [9] M. L. Brodie, Data integration at scale: From relational data integration to information ecosystems, in: 24th IEEE International Conference on Advanced Information Networking and Applications, 2010, April, pp. 2-3. [10] M. Stonebraker, U. Çetintemel, S. Zdonik, The 8 requirements of real-time stream processing,
ACM Sigmod Record 34(4) (2005) 42-47. [11] IBM Rational Unified Process, Rational Process Library, URL: https://www.ibm.com/support/pages/rational-unified-process-rup-plug-ins-rationalmethod-composer-751. [12] IEEE Guide for Developing System Requirements Specifications: IEEE Std 1233a-1998.
Institute of Electrical and Electronics Engineers, Inc., 1998. [13] IEEE Standard Glossary of Software Engineering Terminology: IEEE STD 610.12-1990.
Institute of Electrical and Electronics Engineers, Inc., 1990. [14] P. Zdebskyi, A. Berko, V. Vysotska, Investigation of Transitivity Relation in Natural Language
Inference, CEUR Workshop Proceedings 3396 (2023) 334-345. [15] P., Zdebskyi, et al., Intelligent System for Semantically Similar Sentences Identification and Generation Based on Machine Learning Methods, CEUR Workshop Proceedings 2604 (2020) 317-346. [16] V. Lytvyn, S. Kubinska, A. Berko, T. Shestakevych, L. Demkiv, Y. Shcherbyna, Peculiarities of Generation of Semantics of Natural Language Speech by Helping Unlimited and ContextDependent Grammar, CEUR Workshop Proceedings 2604 (2020) 536-551. [17] V. Lytvyn, et al., Methods and Models of Intellectual Processing of Texts for Building Ontologies of Software for Medical Terms Identification in Content Classification, CEUR Workshop Proceedings 2488 (2019). 354-368. [18] M. Fedorov, et al., Decision Support System for Formation and Implementing Orders Based on Cross Programming and Cloud Computing, CEUR Workshop Proceedings 2917 (2021) 714-748. [19] E. E. Schadt, M. D. Linderman, J. Sorenson, L. Lee, G. P. Nolan, Computational solutions to large-scale data management and analysis, Nature reviews genetics 11(9) (2010) 647-657. [20] J. Hamilton, A Conversation with Pat Selinger: Leading the way to manage the world’s information, Queue 3(3) (2005) 18-28. [21] P. Bernstein, et al., The asilomar report on database research, ACM Sigmod record 27(4) (1998) 74-80. [22] R. Agrawal, et al., The claremont report on database research, ACM Sigmod Record 37(3) (2008) 9-19. [23] S. Abiteboul, et al., The Lowell database research self-assessment, Communications of the
ACM 48(5) (2005) 111-118. [24] Y. Burov, V. Vysotska, V. Lytvyn, L. Chyrun, Software Based on Ontological Tasks Models, in; International Scientific Conference on Intellectual Systems of Decision Making and Problem of Computational Intelligence, 2022, May, pp. 608-638. Cham: Springer Int. Publishing. [25] Y. Burov, Knowledge Based Situation Awareness Process Based on Ontologies, CEUR
Workshop Proceedings 2870 (2021) 413-423. [26] I. Pelekh, A. Berko, V. Andrunyk, L. Chyrun, I. Dyyak, Design of a system for dynamic integration of weakly structured data based on mash-up technology, in: IEEE Third International Conference on Data Stream Mining & Processing, 2020, August, pp. 420-425. [27] A. Berko, I. Pelekh, L. Chyrun, I. Dyyak, Information resources analysis system of dynamic integration semi-structured data in a web environment, in: IEEE Third International Conference on Data Stream Mining & Processing (DSMP), 2020, August, pp. 414-419. [28] A. Berko, Y. Matseliukh, Y. Ivaniv, L. Chyrun, V. Schuchmann, The text classification based on Big Data analysis for keyword definition using stemming, in: IEEE 16th International Conference on Computer Sciences and Information Technologies (CSIT), Vol. 1, 2021, September, pp. 184-188.