The convergence of semantic web techniques with web service technologies has enabled the emergence of so-called semantic web services. This new kind of services enacts the automatic manipulation of services by software programs, to perform tasks such as automatic service location, composition, and invocation. In this paper, we propose methods and techniques that enable the semi-automatic generation, deployment, semantic annotation and classification of web services. semantic web services; web service generation; web service classification; web service ontologies; reasoning; WSDL; OWL
Since the emergence of the semantic web
Nevertheless, semantic web services are facing today some problems and
limitations that restrain their advancement and evolution. From our point of
view there are two main problems to be solved:
• There is no consensus on which language has to be used when
describing the semantic information about web services, although
there are two main proposals that are gaining momentum at the present
time, OWL-S
The research presented here is set forth to help overcome these problems. More specifically, our work proposes: • A realistic approach to the semantic information that can be obtained and used from a web service. • A set of automatisms that will allow web service developers to perform some tasks related to web services. More precisely: creation, deployment, testing, semantic annotation, and classification.
The solutions developed in this work have been implemented and integrated in a web platform called Federica. Besides describing this platform in some detail, the focus of this paper will be to explain our research ideas and discuss the results achieved so far.
The rest of the paper is structured as follows. Section 2 shows all the details of our work towards the semi-automatic generation of web services. Next we explain our approach to the semi-automatic classification of web services. Finally, on Section 4, we provide some conclusions, as well as the next foreseen steps. 2.
The critical mass of available web services, let alone semantic ones, is still quite limited today. This is an important practical barrier for the advancement of research and innovation in this field, which is difficult to achieve without a sufficient testbed to try and evaluate the innovations. Artificial examples (i.e. built by the innovators themselves) hardly provide an objective basis for measuring the usefulness and performance of new proposals, not to mention the considerable cost implied in building the testbed, just for experimentation purposes.
The semi-automatic generation of web services, from such a widespread commodity as are web applications, can help with this necessity, and is an interesting research problem by itself. Of course, the expected quality of automatically generated services should not be the same as that of manually defined ones, but we aim at achieving a sufficient quality for the services to be useful for a variety of purposes, where of course, if needed, the generated web services can be completed or refined by a programmer. Moreover, such a facility as we are proposing here can be helpful in the transition from the current World Wide Web of applications to a (Semantic) Web of services.
The idea of the automatic generation of web services from web
applications has already been addressed in former research works. Because
of its relevance for our research, it is worth citing the work developed by
It is also interesting to cite the early work by
The process flow of our approach to the generation of web services is shown in Figure 1. The input to the automatic generation system is the entry web page of an application. The page is parsed (“HTML parser” box in the figure) into a easier to process in-memory data structure, which is analyzed in order to produce a WSDL description (WSDL Generator module) for the service to be generated, plus some additional semantic descriptions (Semantics Generator module). An implementation of the WSDL service is automatically generated (Service Implementor module) and deployed into a web service support platform (Axis and Tomcat have been used). Once the service is deployed, it can be invoked from any web service client that adheres to the generated WSDL description. One such client is automatically generated for testing purposes. Calls to the generated web service (SOAP requests) are automatically deferred to the original web application (HTTP request) by the generated service implementation. The result (a web page source code) is returned to the service client (SOAP response).
The steps enumerated above will be explained in more detail in each of the following subsections. More precisely, in section 2.1, we explain how WSDL descriptions are extracted from the web interface of applications. In section 2.2, the linkage of web services with web application functionality, and the deployment of the service, are described. Then, in section 2.3, we show how the execution of the generated web services is managed.
Web Server
Web App Web Service Client
HTML
H T T P R e q u e HTML ts
SOAP Request SOAP Response
Federica
HTML Parser Tomcat Server
Deployed Service
Deployed
Service
Page Description WSDL Generator
Semantic Generator WSDL
Service
Implementor Axis Server
Semantic Descriptions
D e vcyeeopS l ir
For the automatic analysis and description of the functionality provided by a web application, the system we propose takes as input a) the URL of the web page which gives access to the application, and b) a name to identify the service to be generated. Our approach for the generation of WSDL descriptions is based on the inspection of the source code of the entry web page to the application, more specifically the HTML forms, as the basic UI construct for web applications. Of course web application UIs are being implemented today using other client-side technologies (JavaScript, Flash, etc.) beyond plain HTML, but as a first approach to the problem, we have circumscribed it to HTML forms.
In addition to the UI elements to interact with the application, the source code of a web page through which a web application is accessed includes all kinds of additional content, just as any other web page does: purely informative contents (titles, instructions, logos, advertisements, etc.), navigation elements (hyperlinks), plus page style and layout details. Although all these elements surrounding UI controls could be exploited to find cues which help in the analysis of forms, in our current approach we only inspect the UI elements themselves.
From the analysis of the web UI, our system determines the operations of the web service to be generated, their inputs, and the type of these, and this information is retained in a WSDL description. The generation procedure works through the following steps: 1. A service with a unique wsdl:portType is defined for the whole application. 2. The application page source is read from its URL, and the HTML forms that the page contains are extracted using an HTML parser. 3. A wsdl:operation is created for each of the forms found in the page. 4. An output message and an input message are defined for each wsdl:operation. 5. The “input”, “select” and “textarea” HTLM components are identified as inputs for the service (that is to say, as wsdl:part elements in the “in” wsdl:message) for each of the forms. 6. Depending on the HTML control type and attributes, a different data type is assigned to each service input, a new XML schema type being defined in some cases. Table 1 shows the correspondence between UI controls and service message part data type. 7. The wsdl:output message will contain only one wsdl:part of xsd:string type which, once the service is invoked, will return the source code for the web page that is returned from the web application form.
HTML control text password textarea submit radio checkbox hidden select (simple selection) inputs (same name) inputs (with value and readOnly attributes) select (multiple selection)
XML Schema data type
an
WSDL is generated, our implementation of the service, consisting system of an provides an
automatic application wrapper from which calls to the service are transferred as requests to the initial application. To achieve this, a link engine has been implemented using for sending and receiving
messages
the communication with web services.
Using the Axis WSDL2Java tool, a set of empty Java classes is generated from a WSDL description, which once implemented will define the desired web service functionality. Service deployment is also done with
engine of which processes the SOAP messages received by the service.
The result of this process is a fully functional web service, whose operations carry out the same tasks as those the web application provides. 2.3
In order to try the generated services, our environment provides the automatic generation of a client and a web interface to invoke the generated services. This facility is an invaluable tool for development and testing. Furthermore, we plan to integrate this capability with a sophisticated editor with which developers can refine the service response description, and make it more precise, e.g. as an XML Schema data type, rather than a plain web page fragment source. Furthermore, this option opens further possibilities to augment descriptions with richer semantics, in the spirit of the Semantic Web Services vision. These would also be useful, in particular, for the automatic classification of the service, which is addressed in the next section. We have started the development of this edition tools, but, at the present time they are not yet integrated with the rest of the platform.
With the techniques described in previous sections, we have the capacity to generate functional web services almost automatically and have them deployed (and stored) in a repository. Nevertheless, those services are stored in a chaotic way, without any type of structure. This lack of structure in the service storage can be seen as a problem because tasks such as, for instance, searching for a concrete service, gain much complexity. In addition, if we analyze the current situation of web services we see that classification is already an important concern by itself.
Nowadays, UDDI
What we are proposing here is to use a software agent to help service publishers in the classification task. In order to meet this goal, in the context and spirit of the generation process described in previous sections, our basic starting point for any automated processing of the generated services are WSDL descriptions. Nevertheless, the information contained in a WSDL description is not sufficient to perform any type of classification. So, we need more information about services, their parameters, their operations, etc. In conclusion, we need a higher level description of the service including some semantic information that enables a software agent to help publishers in the classification task.
In addition, as we are trying to classify web services in a taxonomy, we will need, of course, to define that taxonomy. For this purpose, our system can use any available taxonomy, such as the standards currently recommended by UDDI registries, e.g. UNSPSC or NAICS.
Now starting from a taxonomy and a means to represent the semantic
information of a web service, the classification process is based on
comparing the new generated web services with some services previously
classified in the taxonomy. From this comparison, we obtain a similarity
measure representing the probability that a service should be placed under a
certain classification category in the taxonomy. The definition of similarity
measures between taxonomy concepts has been already addressed, for
example, by
All the details about each issue taking place in the classification process can be found in the following subsections. More precisely, in section 3.1, all the information about the representation of web service semantics can be found. Next, in section 3.2, we will explain some details related to the taxonomy we are going to use and the previously classified services we will use for comparisons. Finally, in section 3.3, we will show the complete process we will apply to obtain the similarity measure between services and taxonomy classes. 3.1
Our first step towards the achievement of the classification mechanism is to obtain, define and represent all the extra needed semantic information for a service. As it has been already said, there are mainly, at the present time, two initiatives on semantic description languages for web services: OWL-S and WSMO. Both of these languages provide a highly generic way to describe semantic information about a web service. Because of this, they are considerably complex and difficult to use, even for very simple services. Therefore, we have decided to define a simpler ontology that fits better with our purposes. We do not discard using one of these initiatives in the future, as we gain more generality in our own service descriptions.
We have chosen OWL
Using the aforementioned ontology we have an OWL representation of the service in terms of the information described in WSDL. The “extra semantics” in this description with respect to plain WSDL is held: • In the “serviceType” property of the “WebService” class. This allows linking the description of the service to a type of service defined in other ontologies (or taxonomies), i.e. the one explained on the section 3.2. • In the “parameterType” property of the “Parameter” class. This allows linking the description of a data type to a type defined in other data type ontologies. label wsdlName po rtType wsdlName
With this ontology we add to WSDL the possibility of representing the classification of a web service and a higher-level description of data types. Regarding the latter, and in order to enrich service parameter descriptions, we have defined a data type ontology to which parameter data types will be mapped. This mapping means that the range of the “parameterType” property of the web service ontology described above is conformed by the classes of the data type ontology, the main elements of which are shown in Figure 4.
label wsdlName parameterType
Input
Output SubClassOf Property
ServiceType As can be seen, the ontology defines three kinds of data types: • Simple data types: This is an ontological representation of XML Schema data types. The subclasses represent the different kind of data types that can be used in XML Schema (although the classes represented each data type have not been shown in the figure). • Complex data types: This is an ontological representation of XML Schema complex types, i.e. structures, which could appear in a WSDL. Obviously, as we do not know, a priori, which structures will be in the analyzed WSDL descriptions, this class is initially empty. • Enumerated data types: Despite these data types could be seen as restricted simple types, we have decided to place them under a separate class. This decision is based on the fact that, due to the basis of the generated web services (this is, web applications), the appearance of enumerated data types is very common (i.e. every combo box of an HTML form).
DataType
Thus, with this ontology we have an OWL representation of the different data types used by a service. These representations are very useful when trying to classify a service, as they can be used as source for a data type similarity comparison between two services.
Then, with the union of both ontologies we have all the service information needed represented in an ontological way and the possibility of using software agents (i.e. based on OWL reasoners) to perform the classification of the service. 3.2
Given a set of classified WSDL descriptions under some classification taxonomy, and a newly generated web service description, based on the service ontology and techniques described in previous sections, in order to deploy the new service, and make it available in a service registry, an appropriate classification category should be assigned to the new service, e.g. for easy retrieval.
As it has been already said, we use the classification standards used by UDDI registries (such as UNSPSC or NAICS) as the taxonomy where our web services will be classified, but our techniques are independent from the standard used. To facilitate our development, and in anticipation of future semantic-web-oriented extensions of our work, we are using an OWL representation of the taxonomy. This has the advantage that the same tools, e.g. OWL reasoners, can be used to process service descriptions and classification hierarchies in a uniform way (e.g. for the computation of similarity measures between services and categories, reasoning about class hierarchies, etc.).
We have developed a heuristic for automated service classification, based
on the comparison of an unclassified service with available classified
services, whereby a measure of the likelihood that the service can be
correctly classified under some category is computed. This problem is
related but different from the ones addressed by other service comparison
techniques found in the literature
Our heuristic works as follows. Let S be the set of all web services, and let C be a classification taxonomy. Since C is used to classify the services in S, we may define C as a subset of the parts of S, P(S), i.e. each c∈C is a labelled set of services c ⊂ S (note that it is possible that c = ∅). Given a new service s∈S, we want to find the category c∈C that maximizes its similarity with s. We measure the similarity between s and c is by the following formula: sim(s, c) = ∑ (−1) c' +1∏ sim(s, s ') c'⊂c s'∈c' whereby the similarity between a service and a category is computed in terms of the similarity between the service and the services classified under that category. We have chosen the above formula because it meets the following desired properties: • sim(s,c) ∈ [0,1] provided that sim(s,s’) ∈ [0,1] ∀s’∈c • sim(s,c) ≥ sim(s,s’) ∀s’∈ c (in particular this means that sim(s,c) = 1 if sim(s,s’) = 1 for some s’). • sim(s,c) increases monotonically with respect to sim(s,s’) ∀s’∈c, • Since ∀s’∈c, sim(s,c) = sim(s,s’) + (1 – sim(s,s’)) sim(s, c – {s}), sim(s,c) can be computed efficiently (i.e. with linear Θ(|c|) complexity).
Now, the similarity between two services is measured in terms of the similarity of their operations and parameters. If we denote by Ps the set of the parameters of service s, and by OPs the set of its operations, the similarity of two services is defined as: sim(s, s ') = f (sim(Ps , Ps ' ), sim(OPs , OPs ' ))
Developing a measure for comparing service operations is still work in progress in our research as of this writing, so in the meantime we are working with f (x, y) = x .
The similarity between two parameter sets P and P’ is computed as the average of the best possible pairwise similarities obtained by an optimal pairing of the elements from the two sets. This can be formalized as follows. We define top(P,P’) as the pair (p,p’) ∈ P × P’ that maximizes sim(p,p’). Then let P1 = P, P1’ = P’, Pk = Pk-1 – {pk-1} and Pk’ = P’k-1 – {p’k-1}, where (pk-1,p’k-1) = top(Pk-1,P’k-1). With these definitions, the similarity between two parameter sets is given by: sim(P, P ') = min( P , P ' ) ∑ i=1
sim ( top ( Pi , Pi ')) max ( P , P ' )
In our current model, the similarity between two parameters is defined as the similarity between their respective types. Let T denote the set of all types (i.e. the set of domain ontology classes). We define the similarity between two types t∈T, t’∈T as: ⎧1 if t = t ' ⎪ ⎪ t ∩ t ' ⎪⎪ t ∪ t ' sim(t,t ') = ⎨ ⎪0.5h(t,t ') if t subtype of t ' ⎪0.5h(t ',t) if t ' subtype of t ⎪ ⎪⎩0 otherwise
if t and t ' are subclasses of EnumeratedDataType where h(t, t’) denotes the distance between t and t’ in the type hierarchy (e.g. the distance between a type and its immediate supertype is 1).
With this process, we can have finally a vector of similarity measures between a new analyzed service and all the taxonomy classes. Then it is time for service publishers to decide on which class (from a ranked list) the service best fits.
Our work shows that automatic web service generation is feasible. In fact, we have implemented all the ideas shown in this document and have developed a first version of the Federica platform (accessible at http://rhadamanthis.ii.uam.es:8080/federica). In that platform it can be seen that tasks such as generation and deployment of web services are done automatically. It also includes the tools needed for web service execution and testing. Nevertheless, the edition tools that have been discussed in section 2 are not yet integrated with the rest of the platform.
Our work on semi-automatic classification has some commonalities with
the research on web services matchmaking
Meanwhile, the goal of this paper is to show it is possible to develop automated assistance capabilities for service generation and classification, using only syntactic descriptions (WSDL), service taxonomies, and data type ontologies (in OWL), which are already in use in the current state of semantic web and web service technology development and adoption, or which can be created or completed with little effort. Some current gaps, that are requiring an effort from our part in compensation, include the construction of a large-enough repository of manually classified WSDL service descriptions (say, by the hundreds), to serve as testbed for our techniques, and some current limitations of available semantic web tools, such as the OWL reasoners. It is to be expected that our workarounds can be removed as these tools keep reaching maturity.
Rather than aiming at, or starting from, a maximum semantic web service representation expressiveness, as in OWL-S and WSMO, we have approached our research objectives in a bottom-up approach, starting from consolidated, ready-to-use technology, such as WSDL and OWL, already being adopted by industry as of today. From this standpoint, our long-term goals are the same as those of OWL-S and WSMO, namely to bridge the gap between current web service technology, and the vision of Semantic Web Services.
Our next steps in this direction include: growing the collection of services generated by our platform, stored in our repository; extend the semantic information represented in our service ontology, that can be obtained in an automated way; complete the development of the edition/administration tools to customize and improve the quality of the generated services; further testing, evaluation, and tuning of the automatic web service classification techniques.
We would like to thank Ruben Lara for his thorough revision, comments, and suggestions on the early drafts of this paper, which greatly helped to improve our work.