The Web is fastly becoming the predominant source for news and information for many people. In the past few years, a new delivery system has emerged in the form of RSS feeds. Such feeds normally provide a brief of a larger news posted on the Web. RSS feeds, collected to form “channels” according to some thematic criteria, can be accessed using Web browsers or specialized software called “news aggregators”. Even so, the amount of information available on the Web still exceeds human possibilities. In order to allow more selective and precise user choice, we developed a Web Cocoon-based platform which selects and publishes news gathered from various news Web sites. The selection is done submitting XQuery queries to a local repository and exploits the intrinsically semantic nature of RSS feeds.
The World Wide Web (the Web) has become the predominant source for news
and information for many people. To address the vast amount of content and
the high frequency of news publication, a new delivery system has emerged in
the form of “channels” or “feeds.” These feeds, which are supplied by Websites
such as CNN and BBC News, can be read using traditional Web browsers
or specialized software, called “news aggregators.” The two main formats for
these feeds are RSS [
The instruments we used for the development of our project (the
graphical interface of the portal has been written in XHTML[
Cocoon was born as a Java servlet [
Cocoon is based on the Avalon model[
Cocoon’s most important innovation is SoC-based design. During Web development, programmers often need to interfere with graphical designers’ work and vice-versa, often resulting in a reduction of productivity. The purpose of Cocoon is to separate productive contexts to maximize the effectiveness of each team; style construction develops in parallel with logic design, improving productivity, quality and maintenability.
As per Web applications, the idea introduced by Cocoon is to use a pipeline to manage requests. A pipeline is a series of steps to process a particular kind of content. Usually, a Cocoon pipeline consists of a set of steps that specify generation, transformation and serialization of SAX events composing generated content.
After being processed, requests move through pipeline stages. Each stage
is responsible of a part of generation or transformation of contents. Cocoon
allows to define all parts of a pipeline. SAX[
A pipeline can be composed of four or more components always executed in the same order. For an example of pipeline see Fig. 1.
The sitemap is the heart of Cocoon. Here the developer configures Cocoon components and defines the client/server interactions in the pipeline. Cocoon matches each HTTP request to relative content in the sitemap, so that every part of the application (e.g. an XSLT file) is submitted to the appropriate component; each of them carries out a precise task and communicates with the precedent and/or the successive one by means of a stream of SAX events, activated when documents to be manipulated are submitted to the parser. SAX model consists of a set of classes and interfaces; it concerns two components placed in succession inside a pipeline; the first one sends a set of events, the following one pays attention waiting for these informations.
Transformations may be very demanding in terms of resources from servlet
engine. Text parsing and transformations application require, in fact, a large
quantity of processor resources. As to memory management, the situation has
improved since Cocoon has adopted SAX in place of DOM[
For assigns to a variable a list of elements, extracted from a XPath expression, involved in the XQuery query; Let operates a generic assignation (e.g. variable function value); Where establishes the condition to satisfy in the query; Order by establishes how results will be ordered;
Return indicates the result of the query.
The argument of a clause is an expression in which function and XPath[
Each XSP page is processed by ServerPages generator, which represents in Cocoon the starting point of elaboration by means of pipelines. The ServerPagesGenerator transforms tags in a Java class which implements the Generator interface. XSP page is only compiled after first creation of the Generator; following executions will use the generator already available.
Each XSP pages starts with the < xsp : page > tag; on its interior we declare the embedded programming language and the namespaces used to import tags from logicsheets. XSP supports programming language such as Java, Javascript and Python. The rest of page comprises tags extracted from libraries and one or more < xsp : logic > elements containing embedded code. XSP default library provides a further top-level element, called < xsp : structure >, in which declarations inherent to the used embedded language can be enclosed. Generally, it is used to declare the import of external modules as, for example, classes package. Being both logic and structure top-level elements, it’s impossible to include one into the other.
Summarizing, an XSP page with only elements from default logicsheet introduces the following structure: a < xsp : page > node, one or more < xsp : structure > nodes, and one or more < xsp : logic > nodes. The elements taken from this library don’t allow a fluid XSP programming as they leave the development of dynamic content to embedded code, thus weighing down source code remarkably. Besides, it’s advisable to divide code in syntactic markup blocks, each of them having its own function (session management, parameters management, etc.) and to commit what cannot be manipulated with these blocks to embedded logic or through creation of new specific logicsheets.
The idea at the heart of our project [
Indeed, each external URL access involves latency periods related to the nature of the connection. They grow linearly with the number of resources accessed.
A cache that memorizes the examined resource and the search parameters, has been used in order to eliminate this bottleneck. A deadline is assigned to each temporary version of the resources. It is defined as the parameter inside the pipeline, at the end of which the resource is considered stole. An additional Cocoon component (written in Java and inserted as a JAR) has been created in order to schedule cache updates. This “daemon” is like an Action component inside the site-map. Moreover, the parameter concerning the duration of the cache is transferred.
Resources are served from an internal applicative pipeline which returns them through redirection, following a matching strategy studied for URLs that are corresponding to RSS and Atom files. This solution allows the storage and access to temporary copies of the requested resources, without causing modifications to the portal structure. Thus, together with the site-map, it defines an interface between our application and the Web.
A Java module has been implemented in order to schedule the access and then the storage of all resources in the cache through the creation of a connection and the request of an URL like http://www.feeding.it/allresources, which makes reference to a XQuery, forcing the update. During the first updating request a thread is created. It is kept in memory in order to satisfy the following updating requests and executed in parallel to both Java modules and searches. The response time is close to zero. If a search is executed during the updating operation, previously cached copies would be served.
A sequence of tests has been executed in order to study the speed of resources retrieval. The tests were made without using the cache, to better understand the updating times, with particular attention to differences of performance among searches inside and outside Italy. Each test has been made with 25, 50, 100, 200 resources and has been repeated ten times using the word “Iraq”, first over Italian resources and then on non-Italian sites. Figures 2 and 3 illustrate the results of our test.
The max semi-dispersion here is represented by intervals of uncertainty enclosed within the upper and lower extremities which are respectively green and red. The rising of the resources coincides with the rising of intervals of semi-dispersion and a sub-linear growth of response times in the case of searches within Italy. We can note, moreover, how the response time for search done within Italy is extremely lower than that over the whole Web. This shows that latency times within the server determine performances.
The portal has been called Feeding from the noun “feed” associated with the English suffix “ing” used to indicate action in progress, thus reflecting the nature of the project that handles data in continuous evolution. With the exception of thematic pages and search engine written in Xquery, the rest of the dynamic pages which compose the portal have been realized in XSP and acquire the respective contents through SQL query on HSQL-DB, a RDBMS integrated in Cocoon. Queries are embedded in the XSP tags, then transformed through XSLT and XSL-FO style sheets. The native XML-DB, Exist, offers the Xquery support used to query RSS/Atom feeds. The graphic interface is written in CSS 2.0, XHTML-Transitional 1.0 and AJAX.
The search engine has been entirely written in Xquery. Feeding uses an XML file with the URLs of each feeds. Two attributes, which indicate the “language” and the “topic”, have been assigned to each URL, respectively. When search is done, the selected key-words and the radio-button index are transmitted by URL-rewriting to the page of interest. The news source acts as a filter in order to select the XML-path used in the query. It is in fact the result of a Xquery function that uses the above parameters as arguments. The main function of the engine uses the search keys obtained and returns all the occurrences within the elements item/title and item/description of the feed. This procedure is iterated for all RSS/Atom resources of interest. The news of each feed are then listed ordered by publication date.
Feeding allows the use of advanced functions in order to obtain a highly selective search. Selection criteria may be specified in one of these forms: •
Basic search: Universita` Messina. Looks for the first OR the second word occurrence in the feed. • Pattern search: “Universita` di Messina”. Looks for the exact pattern occurrence in the feed. • Exclusion search: Universita` -Messina. Looks for Universita` without Messina matching. • Inclusive search: Universita` +Messina. Looks for both Universita` AND
Messina, meaning that the feed has to have both words at once. • Search with date: 2007-04-07. Looks for all news published in the specified data. The data has to be written in the form YYYY-MM-DD, MM and DD can be optionally excluded.
One or more of the listed search forms can be used at the same time, thus allowing the users to make complex searches as: Universita` -Messina 200703-07. It looks for all news published in the specified data with the matching word “Universita´” and without “Messina”. Even better, a user can search for terms appearing in the title field, other terms appearing in the description field while limiting the feeds only to those published within a time period, say a couple of days.
The content of each thematic page is generated through a query which acquires
its parameters through URL rewriting and uses it to select the resources from
which the news will be extracted. We notice that each URL inside the XML file
is equipped with an attribute that specifies the topic of the feed. The content of
the page is generated through a query which associates the acquired parameter
during the request according to the value of the attribute mentioned above.
The result of the query includes the latest news published for each feed.
Apache Cocoon is a successful framework and by now it has been deployed at
several sites1; some of which exploit its main feature, that is the separation
of content, logic and presentation. In the few last years also some scientific
projects adopted Apache Cocoon as a framework for their applications, even
if their field of interest differs from ours. See [
As to the main goal of our project, that is management of repositories of
RSS/Atom feeds and the subsequent extraction of relevant information, we
found a correspondence in the works on information extraction tools. These,
whose aim is to convert semi-structured or structured Web content into a
structured, i.e. XML, format, have been thoroughly surveyed from a number
of authors. See for example [
We presented a new platform for retrieval and querying of RSS/Atom feeds by means of a powerful XQuery engine, which fully exploits the structure of XML documents. Selected RSS/Atom news sites are frequently queried and newly produced feeds are retrieved and stored in a local XML database for future queries. Although our project is still in a early development stage, its first results seem promising and the emphasis on Xquery queries are unique among various feeds portal on the Web. We planned, as our next achievings, to better manage feeds polling to minimize the number of unnecessary feed retrievals and to publish our platform on the Web. 1See http://cocoon.apache.org/link/ for an updated list 38. Chia-Hui Chang, Kayed M., Girgis M.R., Shaalan K. (2006) A Survey of Web Information Extraction Systems IEEE Transactions on Knowledge and Data Engineering, TKDE-0475-1104.R3