Current web technologies have been leaping forward, especially since the introduction of HTML5. The web browsers of the day implement various APIs for the client-side scripts, such as elaborate data storage or advanced network connectivity. We propose, how to combine these technologies to create distributed data storage using the web environment. We have implemented a prototype framework as a proof of concept and explored the most problematic issues which require to be researched further. Systems that would use this storage may bene t from the fact that the users do not need to install or con gure any type of client application, since the only thing required is the web browser. Web-based distributed data storage can be used as an extension of server storage, for data caching, and in various other applications.
World wide web played a role of open interactive platform for information
exchange, business, or entertainment for over twenty years. It begun as a simple
technology for presenting text documents, but it has evolved signi cantly in the
past decades. In the last few years, its technologies leaped forward as HTML5
[
The ability of executing scripts in the browser raises many security issues, since the user have virtually no way how to verify that these scripts do not perform malicious routines and will not pose a threat. The client scripting environments in the browsers must nd a very ne balance between security and provided functionality, which is required by complex client-server applications. Furthermore, the browsers must implement these speci cations correctly to ensure at least some level of protection.
The HTML5 technologies are designed to support more complex client-side scripts for more elaborated client-server web applications. Particularly, HTML5 advanced these browsers capabilities:
GUI design, graphics, and multimedia support, communication capabilities (Server-sent events, WebSockets, WebRTC), client-side data and le management (File API, Web Storage, Indexed DB), client-side computations (Web Workers, WebCL).
The line between legitimate resource utilization and their exploitation is quite thin and not formally de ned. Therefore, it is reasonable to assume that these technologies will remain in the browsers for the perceivable future time, despite the possibility they might be misused by parasitic applications. 1.1
Our particular interest turns towards data storage capabilities and communication capabilities of the browsers. Many current users of the web have a lot of free disk space in their computers and su cient connectivity. This space may be utilized to store or cache data of the web applications they use. It may also be exploited by a web application, or its embedded component such as ad banner, to secretly infest the users hard drive with their own data.
This paper explores the limitations of current HTML5 technologies and propose a way how these technologies may be used or misused to create a distributed data storage. We have implemented a prototype framework that tests the feasibility of this idea. The prototype let us identify the most problematic areas of this domain and outline possible solutions for them. We have also proposed possible applications which may use these technologies for legitimate reasons and in a parasitic way { i.e., stealing the disk space from the users.
The paper is organized as follows. Section 2 revises related work. The HTML5 technologies relevant to our work are presented in Section 3. Section 4 proposes a design of distributed data storage and outlines possible problems that will require further research. Possible applications are presented in Section 5 and Section 6 concludes the paper. 2
Utilizing unused resources from desktop computers is not a new idea. Many
systems, such as Entropia [
In this work, we focus on utilizing the spare storage capacities of ordinary
computers to form a distributed data storage. Distributed storage systems of the
day have evolved from providing basic means of remote data, to o ering services
like high availability, anonymity, redundancy, and archival [
Several projects already looked into scavenging unused storage on desktop
computers. Farsite [
As a peer-to-peer storage system, CFS [
Although all these storage systems proved their usability and applicability, they su er from one substantial drawback from a user's point of view: they require to install a client software. To our best knowledge, no distributed storage system operates without the need of such additional client component. 3
In this section, we will brie y introduce the HTML5 technologies, which have emerged in the past few years and which may be used to create distributed data storage using web browsers of common users. These technologies are implemented in the leading desktop browsers, thus available for most web users. 3.1
The HTTP was designed as strictly stateless protocol, with no user session
support. This aw was partially remedied by introducing cookies { a simple
mechanism, that allows server to register session data at client side, which are resent
automatically with each request. Cookies do not satisfy complex needs of modern
web applications, thus more elaborate solutions were devised in HTML5.
Web Storage. The web storage [
The session storage is designed for scenarios when the application needs to store session data, but the user may run multiple sessions in di erent windows. Therefore, the storage is tied to a speci c site and speci c opened window/tab.
The local storage is designed for scenarios when the application needs to store data at the client side, but these data span throughout the opened windows. It is also particularly useful when the data needs to overlive the current user session. The storage is tied to a speci c site and the data are persistent.
Both storages use simple API that allows saving key-value pairs, where both
the key and the value are simple strings. The keys are treated as unique
identi ers and the API allows the application to iterate over all stored keys. The
speci cation also includes some security restriction. The most important
restriction in our case is the disk quotas for the stored data, i.e., the browser does not
permit the script to store more data then speci ed by the quota.
Indexed Database. More complex data management problems require more
elaborate solutions. For this purpose, the W3 consortium presented the Index
Database API [
The indexed database API allows the JavaScript at client side to create indexed data stores and operate them in a transaction-safe way. The data are usually stored in e cient persistent data structures (such as B-trees), which support e cient insertion, deletion, key-based searching, and deterministic (ordered) enumeration. 3.2
The rst direct communication capabilities were introduced into client-side web scripting with the asynchronous HTTP API (XMLHttpRequest). This API allowed the client code to perform its own requests on the background, thus synchronize client and server application status without explicit user actions. However, the communication must still be initiated from the client side, which complicates situations when the server needs to notify clients.
One of the rst techniques designed for server-initiated noti cations (also
denoted AJAX-push or reverse AJAX) was Comet [
An extension of the Comet approach was presented by the Server-sent events API. This API uses also long-held HTTP connections, but it allows multiple events to be sent over an opened HTTP request and standardizes both the message format and the client-side API that process these messages. WebSocket. The WebSocket protocol is a HTTP-compatible protocol, which allows upgrading regular HTTP connections into a WebSocket connections. The WebSocket connection reuses underlying TCP (or SSL/TLS) channel of the original connection to transfer data frames in either direction. Both sides (former HTTP client and server) become equal communication partners, i.e., they can send a message over to their peer at any time.
The WebSocket client API [
The browsers rst create a RTCPeerConnection, which is a logical representation of the connection. This connection is usually routed using ICE technologies for NAT traversal, since most browsers are connected to the internet from a private network. The RTC connection allows the transfer of one or multiple streams. A stream may be either media stream, which can be directly captured from a web camera and directly displayed in an appropriate HTML5 element, or a data stream, which has the same behaviour and API as a WebSocket connection. 4
A distributed data storage system (as almost any database system) usually employs layered architecture. Lower layers are responsible for raw data operations while upper layers add user-friendly functionality, such as transparent format transcoding, management of logical data entities, integrity constraints, transaction support, or security. These high-level features are quite well understood and beyond the scope of this work. Hence, we focus on the lowest level { the utilization of HTML5 technologies for raw data storage and transfer. 4.1
The data storage infrastructure can be divided into three logical components: the master (running mostly on a server or on multiple servers), the clients (the connected web browsers), and the communication protocol they use.
The master controls the entire storage. It integrates all data distribution logic and initiates all data transactions. It can be implemented as one centralized server, as a small cluster of servers located in one server room, or even as a large distributed system. Theoretically, some parts of the master functionality can be performed in the browsers; however, these details are not important from the data storage point of view. Hence, we will present the master as a single server.
The most important role of the clients is to provide their storage space for the system. Some of these clients may also use the system functions (i.e., retrieve/store user data); however, we primarily focus on the data management. The clients require to execute only a minimalist script which connects to the master and waits for commands. All operations are issued from the master and the client simply process them. There are three basic kinds of operations: metadata operations (client identi cation, capabilities assessment), data retrieval (list, read), and data modi cations (insert, update, delete).
The protocol is designed to work as a simple RPC for the client operations. The data transfers are bidirectional and server-initiated, thus we selected the WebSockets as the underlying protocol. Each operation has its own request message and response message. The client process the messages sequentially and for each incoming request generates one response. Figure 1 illustrates this model. Optimizing Data Transfers. Even though the basic concept may suggest that the major data ow will be between master and clients, there are many situations, when the data needs to be transferred between clients directly. For instance, when a data block needs to be replicated from one nodes to another or when a client requests data, which are stored on adjacent nodes.
The WebRTC peer-to-peer connections may be used to optimize data transfers between the clients (browsers). The WebRTC creates data streams, which are operated by the same API as the WebSockets. Hence, it will be easy to adapt the implementation to accomodate this feature. Even though the WebRTC may reduce the master-client communication, the uplink to the master is still required, since the master controls the performed operations.
Figure 2 depicts the augmentation of centralized data storage where
peerto-peer data channels are used to optimize the data tra c between clients. The
peer-to-peer channels are created on demand only between those clients that
need to communicate and they may be closed when no longer needed.
We have implemented the prototype of the data storage layer. The client was
implemented as a simple web page which connects to the master using
WebSocket [
Addressing all technical details is way beyond the scope of this work, but we
have identi ed the most problematic issues and proposed solutions for them.
Resource Limitations. One of the key issues is the the limitation of user
resources, especially the disk space. Current browser use the default quota of 5 MB
per site for local storage [
Another resource that might limit the system is the network connectivity. The desktop computers are usually connected by some form of wired connection, such as xDSL (phone lines) or FTTx (dedicated metallic or optical lines). According to various resources, the average connection speeds are oscillating between 1 and 100 MBps depending on the location. Furthermore, we have to consider that many technologies (like the ADSL) use asymmetric connections, where the upload is several times slower than the download.
The storage and the data transfer issues are even more serious if we consider portable devices such as tablets or smartphones. Regular user hardly notices if we allocate one gigabyte on a desktop computer, but the same amount of data cannot be easily taken from a smartphone. Furthermore, mobile devices are usually connected via cellular networks which are much slower and often limit the data transfers by fair user policies. Volatility of the Clients. Most of the distributed systems expect that the participating nodes are devoting their e orts to ful ll the objectives of the system. Although there are methods of dealing with many forms of failures, the system usually expects that the nodes are available most of the time. The web users, on the other hand, connect to the system at their own discretion without any regards to the system requirements.
The system must employ speci c measures to ensure data availability. Prehaps the most straightforward solution is to select su cient data replication level. More elaborate approach would be to study individual behaviour of the users and data requests. These strategies will require intensive future research. Security. The system must provide some guarantees regarding the stored data, such as persistence, integrity, or privacy. Our distributed data store does not ful ll any of these requirements, since the clients do not provide any guarantees. The data may be easily corrupted, erased, or accessed by unauthorized users. Some of the security requirements may be ful lled by employing check sums, security hashes, cryptography, and data replication. However, this topic is too broad and beyond the scope of this paper. 5
We have projected initial estimates concerning the web storage capacity based on existing web application and outlined a few possible use cases to illustrate the applicability of the storage. 5.1
We have based our analysis on the statistical data of Facebook [
Utilizing this raw capacity e ectively will be quite a challenging task. Even the users of such popular application as the Facebook exhibit very irregular behaviour in visiting the web page. The active users spend about 640 million minutes each month, which is only one minute per user in average. On the other hand, 48% of all users connect to the web page on a daily basis.
These statistics suggest that the matter of data replication cannot be solved by a simple random approach, but careful study of user behaviour has to be conducted. Furthermore, the application may employ additional methods like bene ts to encourage the users connect and stay on the web page. 5.2
The data storage and sharing systems usually require some form of a client application. If such systems are built on the top of web technologies, they may provide clients which can be directly used in the web browser without explicit installation or con guration.
Existing applications, in which the users share some content, can use the distributed storage as a cache to reduce the data transfers of their internal servers. For instance, Facebook can use this cache for photographs, Google Doc for documents, etc. When the data are requested by a user, they may be downloaded from a peer who is online instead of downloading them from a server.
A di erent approach can be used by applications which have many users, but do not require storing or caching large amounts of data. These applications may o er the capacity of the distributed storage to third parties. The concept is similar to web advertisement where a web page shows ad banners, however in this case, the users support the application by donating their own disk space. 5.3
The data storage can be also misused by malicious applications to steal the disk space from the users. For instance, the idea of acquiring the data storage at clients by the means of ad banners can be used both legitimately and parasitically, depending on whether the user is informed about the intent or not. However, when used parasitically, the application needs to use quite elaborate way of user targeting in order to create at least somewhat persistent data storage.
To acquire even more clients, the parasitic storage malware may resort to
exploit vulnerabilities of other web applications. When a victim application is not
correctly protected against script injection attacks [
We have proposed a novel idea how existing HTML5 technologies may be used or even misused for a distributed data storage. The storage clients require only modern web browser, which is a piece of software that is present on virtually every desktop computer. We have implemented prototype framework as a proof of concept and outlined additional problems that require our attention.
In the future work, we would like to address the remaining problem, especially analyze the user behaviour. If we are able to predict client up times based on behavioural models, we will be able to distribute data among the client nodes in more e cient way. Furthermore, we would like to explore possibilities of creating distributed database system, that will also distribute the query evaluation plans and execute them partially on the client nodes.