Creating good User Interfaces (UIs) to render Linked Data visually is a complex task, often involving both UI and Linked Data specialists. The resulting solutions are typically application-dependent and di cult to adapt or reuse in a di erent context. To tackle this problem, we propose Uduvudu, a exible, open-source engine to visualize Linked Data. Our engine is built in JavaScript and runs in the browser natively. Non-specialist users can use Uduvudu to describe recurring subgraph patterns occurring in their data. They can then exibly and automatically extract, transform, and visually render such patterns in multiple ways depending of the usage context. Uduvudu is intuitive, exible, and e cient and makes it possible to jump-start the development of complex user interfaces based on Linked Data without the need of data specialists.
Companies are progressively adopting Linked Data in their internal IT ecosystems. As a result, they are increasingly interested to leverage Linked Data to build better User Interfaces (UIs), both for their employees and for their customers. Ideally, there should be as little information loss as possible when exporting the Linked Data to the UI. Often, however, Linked Data is transformed into less expressive data formats that are then rendered in a simplistic fashion in the interface. The exported Linked Data typically looses some of its structure, its semantics and/or some of its links. Ultimately, it becomes impossible to use the representation in the interface to write data back into the graph. Also, handling arbitrary Linked Data in order to render it properly in
Uduvudu
Subgraph underscore.js
Y Y Y Y
Callimachus
Application RDFa Templates
N N N Y Most of the related work focused so far on the exploration and visualization of Linked Data. The focus of many systems is hence either on how to e ectively explore large quantities of Linked Data, or on how to compactly aggregate large data to visualize it. Less attention has been given to the case where a data publisher already knows exactly which parts of the data he/she wants to publish.
Brunetti et al. [
Fresnel [
We experienced further problems when working with
Fresnel. First, ontologies typically do not provide a way to
indicate if an attribute might be optional. While this is expected
for RDF data processing, this poses problems at the
presentation layer where it is crucial to know precisely which
information is available. The rendering of a complete address
(with street, city, and country values) can for instance be
quite di erent from an address containing only the country
or city name. Other researchers have also struggled with
this limitation of Fresnel [
Through a mechanism called lenses, users can de ne which properties have to be selected in an RDF graph. The user has to specify which lenses shall be used for which graph as well as to indicate all the corresponding properties. From our perspective, the selection of which information is to be rendered should be part of the application logic instead, e.g., by providing a declarative representation (e.g., a SPARQL query) of the targeted data upfront.
The formatting step in Fresnel uses Formats to do simple
string manipulations and Cascading Style Sheets to describe
the rendering. While this approach makes it possible to
create fairly complex renderings, it is also in our opinion
intimidating for the designers, who most often do not have
a deep understanding of the OWL syntax. Also, it is not
possible to inject or attach transformation code to create
complex renderings and visualizations. Finally, the output
generation step of Fresnel is using the prepared information
from the formatting step to render the visualization. Hence,
it it not possible to change the representation based on the
context, which is a very important requirement in practice
and has been one of the original ideas behind the Semantic
Web [
In [
The use of templates for presentation of semantic data was
introduced through Lens Templates in Exhibit [
Templates used to render RDF data directly were introduced
through LESS [
Both frameworks allow to recursively include other templates. Uduvudu enforces the possibility of creating negrained templates at all levels, without requiring any ontology or schema beforehand. We think that the potential reuse of these smaller templates is higher than coarsergrained mechanisms.
Callimachus [
Balloon Synopsis [
Our framework extends the aforementioned approaches in several ways, most prominently by the automatic selection of the templates based on the input data, by the creation of a more exible rendering process, and by templates that can match incomplete data.
In addition, the ability of Uduvudu to exibly combine templates to create bigger structures makes it possible to adapt the presentation according to the availability of data facts.
In order to tackle the challenges described above, we adapt a process work ow to streamline the creation of UIs for Linked Data. Our process is driven by three di erent actors: The data expert is the person (or group) having knowledge on the Linked Data that needs to be rendered. The UX specialist is the user experience person (or group) knowing how to best transform and restructure the data for the application at hand.
The graphical designer, nally, is the person (or group) specialized in creating e ective visual renderings of data structures.
Those three roles might be assumed by the same person in small settings, but will typically be assumed by distinct persons in larger companies. Our UI creation process is given in Figure 1. The overall process can be summarized as follows:
First, the data expert gathers and exports the data that needs to be rendered.
The selected data is then passed to the UX specialist, who restructures it for rendering purposes.
Finally, the graphical designer takes the various data structures built by the UX designer and builds the visual rendering of the data.
UX Specialist
Graph File App. Logic Asks for needed
data fields.
Assembles graph
G Creates matchers M1
F
S
The Linked Data UI creation process we propose has a
number of distinct advantages, including:
Clear separation of roles: involving the data expert|or
the other specialists|throughout the entire process
can be very costly in practice [
Iterative development process: new elements can be added to each task without blocking the other tasks. That way, the experts can iterate on their own design autonomously without consulting the other experts. Highly reusable outcome: as the the three tasks are partially decoupled, all the individual pieces of data, structures and templates created during the process can be reused and adapted later to another context, data, or application exibly.
Zero-input fallback: any valid Linked Data provided by the data specialist can be rendered without any additional processing from the UX specialist or from the graphical designer (in that case, the data is simply rendered as lists of strings displayed at the bottom of the rendering).
The architecture we adopted for our framework was directly based on the creation pipeline described above, as well as on a series of prerequisites we elicited from data experts, UX specialists and graphical designers. It is composed of three main components: the selector choosing the input, the matcher which discovers known structures in the selected input, and the renderer which incorporates the visualization context to render. Figure 2 gives an overview of our architecture. We start below by introducing the prerequisites of our architecture, before giving some detail on each of its three main components.
Through our own experience and interviews with Linked Data specialists, UX experts and graphical designers, we came up with a set of pre-requisites for our architectures: Never show URIs to non-technical end-users: They are systematically confused when confronted to data URIs displayed in a graphical interface. While data URIs are essential to uniquely identify entities and properties on the Web of data, they should not be used in mundane graphical interfaces where plain literals and labels should be preferred.
Leverage the extracted data structure for the rendering: the way data is extracted from the original data typically follows long deliberations and some decision process. We think that the resulting structures should be leveraged as much as possible for the creation of the nal user interface.
Matcher Renderer
User ? ? ? <b>?</b> <p> ? - ? </p> ? ? ? ? <span>?</span> <div> <i>?</i> </div> <script> a = ?; b = ?; </script>
Handle language transparently, until needed speci cally: speci c languages add a level of complexity to every application. Hence, we feel that for our process, if literals are available in multiple languages, only the one matching the context language should be presented, while still keeping the possibility to ne-tune and control the language if necessary depending on the application at hand (e.g., showing multiple languages). Maximize the dynamic composition of information: our goal is not to come up with highly-complex renderings composed of thousands of distinct elements. Rather, we target usual business interfaces composed of a few dozens elements at a time maximum. Hence, our goal is not to come up with a highly-optimized and scalable rendering engine but rather to propose a dynamic composition model where Linked Data and templates can by dynamically recycled and integrated in order to create the nal rendering.
We now turn to the description of the three main components of our architecture. First, we start with the Data Selector, which decides which data that will be transformed and rendered. Based on the use-case at hand, one needs to select and combine the relevant Linked Data from one or several sources. The decision on which part to select from the available data sources is typically an internal process, which is outside the scope of the present paper. Any selector which supports classical Linked Data input, e.g., through a SPARQL query, an RDF/XML dump or triples serialized in one or several text les can be used. Uduvudu does not expect any inherent structure from the data (though it must be well-formed), which does not need to comply to any speci c ontology or structure at this stage. Hence, the selector takes as input a superset of all informations that need to be shown to the end-user, and trim them to an input graph G containing exactly the information that needs to be rendered. This step is typically carried out by a data specialist. Figure 3 gives a simple example input graph corresponding to Figure 4. <corky> a vcard:Individual; vcard:fn "Corky Crystal"; vcard:hasAddress <corky#address>; vcard:hasTelephone <corky#telephone>; vcard:nickname "Corks". <corky#telephone> a vcard:Home, a vcard:Voice; vcard:hasValue "tel:+61755555555". <corky#address> a vcard:Home; vcard:country-name "Australia"; vcard:locality "WonderCity"; vcard:postal-code "5555"; vcard:street-address "111 Lake Drive".
The second component in our architecture, the Structure Matcher, holds a catalogue M of known structures (matchers) and tries to match parts of this catalogue onto the input graph G.
The catalogue contains on one hand custom matchers that describe dedicated processings for speci c data patterns, and prede ned matchers on the other hand that act as a fall-back solution in case no custom matcher can be matched to some node in the input graph. Figure 7 gives an example of a structure matcher.
The custom structures in the catalogue are typically provided by the UX specialist. They are serialized as JSON structures in our implementation (see below), and can be generated through a GUI or can be based on an RDF processor. In order to make the de nitions in the catalogue as reusable as possible, they are all built hierarchically as tree structures. This gives the possibility to attach a template at every step and also to regroup already matched data structures into bigger structures, without touching the inner-working of the smaller structures.
The matcher takes as input an input graph G and one or several corresponding known structures from its catalogue and returns as output a tree structure (see Algorithm 1) with at least one pointer to a rendering structure from the Renderer (see below Section 4.4). The output is split into a set of multiple tree-structured sub-graphs with additional information about the template and a precedence number. All input data is eventually consumed through this process. At this step, we like to mention that a pointer pointing back to the input graph is attached to the tree for every literal. This makes it possible to provide a simple form of data lineage, as well as to create applications which can write back to the input graph.
During this process, all information is treated equally except identical properties available in multiple languages. To provide language information in a transparent manner, the literals which are provided in multiple languages are grouped in the internal data structures. This enables the renderer component to select the correct language based on the context, without analyzing the graph again in more detail. Catalogue of Matchers (Known Structures). We adopt a simple yet generic way to match RDF subgraphs and to generate trees. To initiate the matching process, providing an entry point in the RDF data is preferable. This start resource is a URI which denotes the node in the graph which will be used to begin the matching process. If no start resource is provided, the matcher falls back to try all nodes available to start the matching. Even though it is more exible, this fall back approach needs more processing power compared to the former.
The matching process is implemented as a guided graph traversal. Based on the current start resource, all matching functions which are able to proceed through a predicate edge to the next node are listed. The set of functions which incorporates the highest amount of nodes is selected to be added to the set of trees. After this step, the matched nodes are deleted from the input graph and the same procedure starts over on the remaining nodes. The current solution is based on three types of matching functions: else end Data: Graph G, Start Resource URIs S, Matcher
Functions M1 Result: Forest with References to Template Sets F Function Matcher() foreach Matcher Function M1 do
Process G with Function; if Result then Add to Proposals P ; end if Proposals P available then
Find best P add to F ; Delete Nodes covered by P from G; Call Matcher() with remaining G; Add remaining Nodes each as Tree to F ;
Algorithm 1: Matcher Algorithm
e nf:dracv rd:nicknam a c v vcard:hasValue
tel:+61755555555 http://example.com/me/corky#telephone http://example.com/me/corky vcard:haA s d d rs e s
ard:street-address vc vcard:locality
vcard:postal-code vcard:country-name http://example.com/me/corky#address 111 Lake Drive
WonderCity
5555 Australia LinkMatcher: this matching function supports simple navigation inside an RDF graph. It allows to hop from a start resource to another URI, which is then de ned as a new start resource. For this function also, it is possible, like for the PredicateMatcher, to specify if the resource URI is in the subject or object position.
The adaptive renderer takes as input the tree structure given by the matcher (F , see for instance Figure 5), the available context variables (language, user, device) and the provided templates (T , see for instance Figure 8) to nally render the output. The templates are written in HTML and access the tree structure through escaped variable de nitions (see Table 2 for details).
Before the input is combined with the template, the structure is prepared by a language subcomponent, which determines|based on the language context|which literals to use.
For each template pointer, multiple templates can be available. When no speci c template is speci ed, a combination of the deepest underlying templates available in the hierarchy will be combined together. The templates in the deeper levels of the hierarchy will be chained in place. As a set of simple fallback templates are de ned for all literals, it is hence assured that all provided data will be rendered at least in its simplest form.
The di erent templates have di erent ags attached denoting the context in which they have been prepared. It is the duty of the renderer to decide which is the best suitable template for the context at hand. The di erent ags describing the contexts can be on the input / output (device), the usecase, the targeted user, or the language. Those ags allow to
</div> </div> <span class="glyphicon
glyphicon-envelope" style="font-size:48px;
float:left; cessed inside the variable blocks (denoted through <%- and %>.) easily change the rendering based on the context provided. Context information is not always guaranteed to be available.
The output device and language are usually the most readily available types of context information, typically followed by the user at hand.
The decision on which template to use is solved by giving the output device the highest priority and enforcing it if available.
The remaining available context variables are used to nd the best match within the available templates.
The template with the most matching ags is chosen by the system. If there is still some ambiguity, the priority is set as follows: Use-case, user, and nally language. 4.5 We
Extensibility mentioned in the introduction that our architecture was designed to be highly extensible. In the short introduction of the architecture, we saw
how the behavior of the pipeline can be extended by adding new matcher de nitions and, accordingly, template de nitions. For the template de nitions, the system can be extended
dynamically, as multiple templates can be de
ned for the context variables per matcher de nition.
Thus, by adding new templates for new kinds of devices or for new users or user groups, the behavior can also be dynamically adapted based on the context. On the source code level, it is possible to extend the di erent types of matchers by providing new matcher factories.
A matcher factory gets initialized according to the matcher de nitions that in turn populate the tree which is handed to the renderer. 4.6
Uduvudu does not enable laypersons to create stunning visualizations
through Rather, it provides a prede holistic ned
o -the-shelf framework for templates. developers, which in turn have the possibility to delegate data domain, UX and design tasks to the respective experts.
Furthermore, the framework proposes no functionality to enhance or reason on the input data.
With the creation of user interfaces for RDF centric applications in mind, the pre-processing of the input graph needs to be provided by the application logic. 5.
Our framework also supports a simple yet intuitive and powerful editor.
As it is possible to render an output at any time, the editor is simply based on the results incorporating the available templates. Matchers and their corresponding templates can
be added iteratively to create templates for bigger structures. tchomementt:emplate was changed toi re e+ nder the literal as an HTML Friburgo (en francés Fribourg/Fribourg en Nuithonie, en alemán Freiburg/Freiburg im Üechtland, tein titalliaeno.Friburgo y en romanche Friburg) es una pequeña ciudad de Suiza occidental, capital del cantón de Friburgo y del distrito de Sarine. Friburgo es una de las tres ciudades bilingües de Suiza, unas 21.000 personas hablan francés, mientras que unas 8.300 hablan alemán. Se ubica en la región l amada Üechtland.
Tcomhmernto:ugh the same mechanis mie (+using the + icon), templates Фрибу́ р (фр. Fribourg) или Фра́ йбург (нем. Freiburg) — город на западе Швейцарии, cфaранnцузскиb,22e% — cпо-oнемmецки, b5%i—nпоe-иdтальянсtкиo.Прgи нeасеtлеhнииeв 3r7 200tчеoловекcreate more expressive temстолица кантона Фрибур.Город многоязычен: около 58 % населения говорят по(2007) свыше 10 000 — студенты здешнего университета.Святые покровители города — Николай Чудотворец и Екатерина Александрийская. ht p:/ www.w3.org/2000/01/rdf-schema#label edit Creates a basic predicate matcher. matcher Predicate: Matcher Name: Template Variable: label title Order : 100000 template Template ID : h1_title <div class="uv"> <h1><%-title.u%></h1> </div> Add Matcher & Template Combine multiple matchers to a new matcher. edit matcher Matcher Name: Template Variable: head head Order : 100000 tpl_6.label tpl_6.abstract template Template ID : head <div class="uv"> <h1><%=head.label.u%></h1> <%=template(head.abstract)%> </div> Add Matcher & Template This proof of concept application shows UDUVUDU rendering entries from DBpedia. It further makes uses of linked data fragments, rdf-ext and also bootstrap and typeaheadjs.
Please start searching for an entry in DBpedia by typing at least 3 letters. To confirm your decision select a proposition in the list. en Fribourg fourNgeiegnhbeorriicngtemmunpiclaiptaelistieusseCdatteogogrieesnerate the
Düdingen 1157 establishments
Givisiez Canton of Fribourg PictureGrawngeist-PhaccTot itle and AbCsanttornaalccatp:italTsohfe rst template exMtarraly,cFtrisboutrhge picture andSwtitzheerlanldabels of the entity, and
Pierrafortscha Cities in Switzerland crSeta.Utresesn a rendering wiCtuhlturablporotphertyeolf enamtioenanl ts integrated (wTiatfehrs the title overlappisniggnifitcahncee ipnithcetcuanrteon).of Furthermore, either a Fribourg
Villars-sur-Gclâonmement or absFrtibroaucrgt is shown to the right of the picture.
Imperial free cities
Municipalities of the canton of Selected Facts: Elevation, AFribroeuarg and the Website of the
University towns selected city are put together in a small facts box. Each o fWtikhipeedfaiacts could also be rendered independently. This ID: 1m64e06a5ns that the maximum of all available elements are Revision: 602735683
Show remaining literals Show remaining links rendered in all cases.
Map of Location: Coordinates that are attached to an instance are shown using a map widget. To initialize the widget, some JavaScript code gets injected on the y. Graphs of Humidity and Temperatures: The average mean, high and low temperature as well as the humidity per month are attached to the main instance as a set of 12 properties. They get combined by matcher and rendered by a template which loads a charting library.
Below the templates, we then see the list of non-matched properties. Each of them is internally represented as a tree with only one leaf. The rendering of such simple trees is dened by the built-in templates. By overriding this template, it is also possible to customize the fall-back representations. Furthermore, it is possible|on the base of those fall-back templates|to create di erent implementations of user interfaces which help in de ning new matcher and template de nitions.
Within the EU research project Fusepool1, a Uduvudu component is used to render particular data views selected by the user in a dashboard. Fusepool integrates multiple, potentially large data sources using RDF as their data model. Example data sources used in this context are patents, Pubmed articles or user-contributed datasets like logs of parliamentary discussions or collections of beers and breweries. Among other things, Fusepool applies natural language processing on plain text content to extract entities, which can then be used for more speci c search queries.
Fusepool provides a basic user interface, which makes it possible to support faceted search on top of the data. Based on the search results, the user might want to read the detailed content of a particular patent, pubmed article or speci c
Entities CognitiveAgent
SIEMENS AG (14) SIEMENS AKTIENGESELLSCHAFT (7) BOSCH GMBH ROBERT (6) ROBERT BOSCH GMBH (6) SIEMENS, GERHARD (9)
SIEMENS GERHARD (7) Facet types
Document (45) A1Document (676) A3Document (276) PatentPublication (1845) B1Document (470) GrantedPatent (478)
B2Document (3) A2Document (309) A4Document (105)
Entity (1845)
Device Documents (1890)
Classify:
Label prediction:
Preview Device for cooling a motor in a climate control system <div xmlns="ht p:/ www.w3.org/1999/xhtml" xml:lang="en"> <p>A conduit for providing air flow from an air conditioning housing (44) to a motor (28) for driving a climate control system air blower (26) is disclosed. The blower (26) is located in the air conditioning housing (44). The air conditioning motor (28) includes a motor … Labels: Add label POURING DEVICE DISPOSITIF VERSEUR POURING DEVICE AUSGIESSVORRICHTUNG Labels: Add label beer. This particular selection is then shown using Uduvudu. In the example shown in Figure 14, the title, abstract, and license properties have a matcher and template provided. The rest of the data properties are still presented in a generic way. If new data types are added to Fusepool, one just needs to provide speci c matchers and templates to make sure Fusepool can then render the new data in a useful way. Besides providing the matcher and template, no interaction with a programmer who is familiar with the Fusepool code base is necessary. Both the matcher and the template can be stored in RDF as well as in the Fusepool graph store.
In this paper, we presented Uduvudu, a graph-aware, exible and user-friendly UI Engine for Linked Data. Based on user input, Uduvudu captures and transforms inherent features of graph-structured data, which are then leveraged in the data-driven UI renderings. We described the uniquer features of our system, its architecture, and two of its recent deployments. By o ering multiple ways of de ning extensions based on a few basic templates, our solution is kept both user-friendly and generic enough to solve a wide variety of use-cases when rendering Linked Data. As the matchers and their accompanying template de nitions are tailored to data structures and not to a particular UI use-case, the probability of reusing parts of the rendering de nitions can be increased.
Furthermore, our architecture takes into account the fact that such rendering projects often involve multiple parties. The clear separation of tasks during the UI creation process and the possibility to work on the sub-tasks independently allows more agile work ows.
One of the main features of the architecture we propose is the ability to easily create context-aware renderings. Inherent support is given in order to optimize UIs for di erent devices, users, or languages. This functionality is mostly transparent to the developers and can also be leveraged after a contextinsensitive solution is provided.
We provide an open-source implementation of our architecture as a complete framework on http://www.uduvudu.org/. The resulting system runs in a browser directly. For further implementation details, please refer to the provided website. We discussed two applications leveraging our implementation. The DBpedia front-end shows how our framework can be used to provide all kinds of rendering modalities. This was done either by leveraging the templates themselves or by injecting rendering libraries through the template system. In the context of the Fusepool application, our system was used to create the rendering of various heterogeneous pieces of data. The main requirement for this application was the ability to adapt the presentation layer without extending the application code.
Finally, we note that for certain applications it might be necessary to dynamically update the user interface based on changes made to the underlying Linked Data itself. This dynamic update of the presentation layer can be realized on top of the pipeline approach of Uduvudu. We can apply the paradigm known as reactive programming in that sense, where the implementation is built on data ow architectures. Reactivity can be directly achieved in our framework by re-submitting the input graph to Uduvudu on every change. Using networked-clients and the ability of Uduvudu to propagate changes made through the visual rendering of the data directly back to the data itself, one could build exciting new applications leveraging reactivity, where the graph representation on the client-side propagates updates to the data server, which in turn automatically updates all other user interfaces.
We thank the teams in Fribourg and Biel for many fruitful discussions; special mentions go to Bart van Leeuwen, Pascal Mainini and Thomas Bergwinkl for their contributions to the architecture and development of Uduvudu. Uduvudu was partly funded by the EU Framework Program for Innovation under grant 296192.