The distribution of data sources has formed a classical challenge for data management. The LINDO framework is an open system that manages the indexing, storage and retrieval of multimedia contents that are distributed in different remote servers and generated in a real time basis. The main objective of this framework is to provide efficient information retrieval with minimal processing costs. This was achieved through the proposal of an efficient decentralized content indexing mechanism. When considering the pervasive and mobile access to the managed content, the need of an access control becomes essential. In this paper, we apply an access control layer on top of the LINDO architecture that manages access based on the RBAC model and realizes decision making using the XACML standard. We explore the challenges that face the system in processing access requests showing how an access denial could influence the system's usability especially when returned to a user facing an important situation. Thus, we propose to apply flexible decision making that searches for alternative resources. This operation is performed using PSQRS, a query rewriting system that aims to provide users with pervasive accessibility where they could access any needed multimedia source at anytime, anywhere and anyhow.
The necessity of handling a huge quantity of multimedia content created by multiple sources in a distributed environment emerges and raises new challenges concerning the indexing and access to the multimedia content, such as: distributed storage and decentralized processing, choice of the indexing algorithms, real time information retrieval and location-aware retrieval. On top of that we have to consider also that the users are more and more mobile and they need to access the system from anywhere. In such mobile and pervasive contexts, privacy and security management is a central issue.
In this paper, we present a new layer on top of the architecture proposed by the LINDO project1 in order to tackle the abovementioned challenges. The objective of the LINDO project was to build a distributed system for multimedia content management, and to ensure effective indexing and storage of data acquired in real time. The project didn’t address the issues linked to data privacy and security.
Knowing that ensuring the protection of multimedia content is a key issue in certain application domains (e.g., video surveillance, 1 http://www.lindo-itea.eu medical domain, etc.), the access control management should be taken into consideration at the different levels of data processing and should take into account the user’s mobility. Meanwhile, these security constraints should not affect the user’s accessibility needs especially in important situations.
Our objective is to include the access control within the query
processing and enrich it within the LINDO framework in order to
attain a pervasive accessibility that enables the user to access
multimedia sources at anytime, anywhere and anyhow. To achieve
this goal, we have employed PSQRS – Pervasive Situation-aware
Query Rewriting System – that offers adaptive context and
situation-aware access solutions. The decision making within the
system is based on the RBAC model [
The solution overcomes the access denials taking place in real time mobile situations by modifying the query processing mechanism of the LINDO framework and by providing adaptive solutions that can bypass the access control constraints. Next in section 2, we introduce a state of the art covering the different systems managing distributed multimedia content in 2.1, the basic standards for distributed access control management in 2.2 and some research about multimedia access control in 2.3. The LINDO approach for efficient multimedia distributed content management is described in Section 3 through its architecture, as well as its indexing and querying mechanisms. In section 4, we apply an access control layer on top of the LINDO architecture. In section 5, the adaptive access control solution is illustrated through a video surveillance use case. Finally, conclusions and future work directions are provided in section 6.
The constant growing dimension of the multimedia collections that are generated every day brings to the light problems of efficient indexing and retrieval. The solution to these issues passes through the generation and management of the metadata associated to the multimedia content.
These metadata are obtained through the application of indexing
algorithms, which have different performances, purposes and
constraints. Besides, a great heterogeneity of indexing algorithms
has been defined in the state of the art (e.g., [
In the following, we present some distributed systems that manage multimedia contents by emphasizing the architectural choice and the adopted solution for multimedia indexing. A distributed management of the multimedia is used by many projects due to the mobile acquisition context of these contents. An advantage of this kind of systems is that they benefit from the distributed storage and processing of the multimedia content and thus, the performances of the system can be improved. The distributed systems that handle multimedia contents employ peer-to-peer or service-oriented architectures. The major problem that these systems encounter is the heterogeneity of indexing algorithms and of the generated metadata. The following projects addressed this problem in different manners.
The SAPIR (Search on Audio-visual content using Peer-to-peer
Information Retrieval) project [
The DISCO (Distributed Indexing and Search by Content)
project2 has chosen a structured peer-to-peer architecture for the
management of multimedia contents [
The CANDELA (Content Analysis and Network DELivery
Architectures) project3 is focused on the video content analysis
and retrieval into a Service Oriented Architecture, where the
content is stored and indexed on the distributed servers. The
proposed solution was implemented for several use cases:
personal mobile multimedia management [
The WebLab project4 proposes an integration infrastructure that
enables the management of indexing algorithms as Web Services
in order to be used in the development of multimedia processing
applications [
The VITALAS (Video & image Indexing and retrieval in the
Large Scale) project5 capitalizes the WebLab infrastructure in a
distributed multimedia environment [
A comparative study of these systems shows that no matter what the architectural choice is, the content indexing is usually done on dedicated servers (the content and the associated resulting metadata are transferred over the network) using a pre-defined set of indexing algorithms. These algorithms are executed on all ingested multimedia. Thus, the resource consumption is not optimal. This important consumption problem was addressed by the LINDO project, which proposes a distributed architecture for the management of multimedia contents, which is favoring reduced resource consumption, in terms of data transfers over the network, storage and CPU utilization.
Access control and privacy protection are key issues nowadays, especially in the context of distributed systems. In this section, we present two main standards that are widely employed for managing access control within distributed environments: the RBAC model and the XACML standard.
The principal motivation behind the proposal of the RBAC (Role
Based Access Control) model [
The main concept of the RBAC model was to group users within roles that reflect their organizational positions then, simply distribute permissions to these roles instead of repeating the process for each individual.
As illustrated in Figure 1, the role is placed at the heart of the
RBAC model and is seen as an intermediary element that connects
between the users and permissions as it attributes a set of
privileges to those users based on their roles. These permissions
(PRMS) allow the users to perform operations (OPS) on system
sources expressed as objects (OBS).
2.2.2 XACML
The RBAC model managed to solve the challenge of
administrating access permissions to distributed data sources by
providing centralized management for permissions through roles.
With the evolution of service-oriented architectures and web
services, new challenges has arisen and the problem of managing
access becomes more complicated as the access control policies
are also being distributed and more dynamic since they’re
managed by different administrating authorities. To resolve this
problem, the XACML standard was introduced by [
The projects mentioned in Section 2.1 were focused on the
indexing and retrieval of multimedia contents, but none of them
took into consideration problems related to the privacy and access
control management of the contents and systems resources.
Meanwhile, many solutions have been proposed in order to secure
the access to multimedia databases and systems. While some
authors were interested in the security of the connection to the
systems and on the distribution of the contents [
The main goal of the LINDO project (Large scale distributed
INDexation of multimedia Objects) is to define a distributed
system for multimedia content management, while focusing on the
efficient use of the resources in the indexing and query processes.
Thus, not only the multimedia contents storage is distributed but
also the indexing process. The originality of this solution is that:
(a) the content is not moved to indexing servers, but indexing
algorithms are deployed on the servers where the content is
acquired; (b) the indexing process is accomplished in two steps: a
generic indexing at ingest time (i.e., implicit indexing) and a more
detailed one at query time (i.e., explicit indexing). The Figure 3
illustrates an example of the distributed architecture proposed
within LINDO project. A more detailed presentation of the
LINDO architecture can be found in [
- Presence of people - Number of people - Main color of the upper part of the people
- Presence of people & vehicles - Number of people, number of vehicles - Main color of the people upper part. - Main color of vehicles
The functionality adopted within the previously presented system architecture goes as follows: the content is acquired and stored on the remote servers, and the collection of indexing algorithms is stored and managed on the central server. This collection is variable; at any moment we can integrate new algorithms with different functionalities, execution constraints and performances.
In order to reduce resource consumption, the architecture allows the indexing of multimedia contents to be accomplished at acquisition time (i.e., implicit indexing) with some generic algorithms (e.g., person detection, dominant color detection) and on demand (i.e., explicit indexing) with some algorithms that will analyze the contents more in detail (e.g., person recognition, register plate detection). This avoids executing all the indexing algorithms at once and producing metadata that might never be used but raises access rights issues concerning the explicit indexing. The Figure 4 and Figure 5 offer some indexing algorithms examples that illustrate the difference of the level of detail attained by the implicit and explicit indexing. These algorithms differentiate between two types of context acquisition (indoor and outdoor).
The query processing (illustrated in Figure 6) begins with the query specification on the central server. First, the query is processed and executed on the metadata collection on the central server (which is a summary of the metadata collections from remote servers) in order to select the remote servers that could provide answers to the query and it is sent for execution to the selected servers. Among the servers that were not selected at the first step, there could be some servers that contain relevant information that has not been indexed with the right algorithms. For this reason, the LINDO solution detects such supplementary algorithms [7] and starts their execution (i.e., explicit indexing) on a sub-collection of multimedia contents. All the results obtained from the remote servers are sent to the central server, where they are combined and displayed to the user.
The sensitivity of the multimedia content and the privacy protection law that imposes anonymity constraints justify the need of applying an access control scheme on top of the LINDO architecture. The proposed layer customizes access based on the user’s role (RBAC model) and is responsible for managing: 1. The access rights granted to users or services demanding access to the multimedia sources (e.g., video surveillance, medical domain, etc.) that vary not only according to their role but also in terms of their context (time, location, etc.). 2. The access rights for executing queries that employ the explicit indexing algorithms: the risk of disclosing personal or confidential information arises with the level of detail sought and provided by the indexing algorithm increases.
We highlight that in the context of adding this access control layer, the lack of responses returned to a user’s query might not only be due to the lack of results existing within the system but also due to access restrictions imposed by the security layer.
Our goal is to apply the access control layer and to balance between the security constraints and the user needs to find solutions that can ensure seamless accessibility to the requested resources at any time, from anywhere and anyhow.
The pervasive accessibility that we aim to provide matches with the pervasive characteristics of the LINDO system, which are: • The distribution of multimedia sources. • The variation of the entities managing these resources. • The evolutive nature of these resources (generated and indexed in real time). • The sensitivity and confidentiality of their content. • The diversity of contextual information. • The distribution of the indexing process performed by a variety of indexing algorithms. • The execution of access requests in real time. • The importance level of obtaining reactive solutions in important consultations or critical situations.
Managing access requests becomes more challenging within pervasive environments due to the dynamicity of contextual and situational information. Our objective is to ensure an efficient information retrieval process despite the security challenges. In order to achieve this objective, we employ PSQRS (Pervasive Situation-aware Query Rewriting System) - an adaptive decisionmaking system that confronts access denials taking place in realtime consulting situations by rewriting access requests in order to offer alternative-based access solutions.
The access control relaxation that we propose to carry out respects the access rights defined to protect the multimedia content and applies the adaptive decision-making at two functionalities:
The choice of using the explicit indexing algorithms (located on remote servers). 2.
The presentation of the video contents (the identity of filmed persons in a video surveillance system is protected by privacy laws that assure their anonymity). Next, we introduce the detailed functionality of the PSQRS architecture.
As illustrated in Figure 7, the PSQRS (Pervasive Situation-aware Query Rewriting System) architecture contains several components and the sequence of its functionality starts from the user, who enters the system through an authentication portal (step 1) and launches an access request to a certain element (step 2). This request will be interpreted by our Query Interpreter that will translate the request into an XACML request and send it to the Query Analyzer (step3). The request (R) will be analyzed in consideration with the user’s profile - automatically extracted at the sign in process - and according to his context (XACML flow chart, Figure 2). As the analysis finishes, the Query Analyzer would send the result directly to the user if it’s a Permit (step 4a) or back to the Query Interpreter, if it’s a deny (step 4b). In a deny situation the adaptive situation-aware query rewriting mechanism will take place and function as follows: the Query Interpreter will check the sensitivity of the consulting situation with the help of the Sensitivity Analyzer component (steps 5, 6) and according to the importance level of the situation, the Query Interpreter will search for similar or alternative resources through the Similarity Provider component (steps 7, 8) and employ them to rewrite the XACML request (R’) and send it again to the Query Analyzer that will analyze the request and transfer the result back to the user (steps 10a,10b).
In this section, we present an example where the implementation of our proposal is used to overcome the lack of answers provided by the system. As we will illustrate next, the system will modify the query processing and will adapt access decisions according to the level of importance of the querying situation.
Scenario: Taking the metro from « Trocadéro » station to « Place d'Italie » station at 14:15, Helen has forgotten her red bag on a bench at the waiting line. As soon as she realized, she went out to report the problem at the information counter.
A typical treatment of such situations goes through the customer service agent who would open a lost object file, take the descriptions and transmit them to the security officer on site. The security agent will follow different steps in order to find the object; he will check if the object has already been found or returned to the lost and found office by someone. Otherwise, he will try to see the video surveillance system to check if the object is still in the same location.
Figure 8 shows the typical interpretation performed by the information retrieval system provided by LINDO. The launched request will be processed and parsed to extract the main keywords that are then reformulated in the form of an XML user query. Query: Find all videos containing a red bag, forgotten in
2:00pm and now (3:00pm). <UserQuery>
<QueryInText> find all videos containing a red bag, forgotten in Trocadéro, Paris metro station, on 2 February, between 2:00pm and 3:00pm. </QueryInText> <MediaLocation>metro station, Paris, Trocadéro </MediaLocation> <MediaFormat>Video</MediaFormat> <TimeSpan> <From>2012-02-02T14:00:00</From> <To> 2012-02-02T15:00:00</To> </TimeSpan> </UserQuery>
After keyword extraction [
In this scenario, the requested information are generic thus, the query processing will perform the search on the metadata generated by the implicit indexing algorithms and placed at the central server. The system will continue the search to find a red object in the retrieved list of metadata describing the chosen segments.
A filtering process is applied to take into account access control rules. Analyzing the access rights assigned to the security agent, we find that he is not authorized to access the videos containing passenger faces nor to use the personalized search options that employ the explicit indexing algorithms existing at remote servers. Therefore, considering these access restrictions, the system will perform another filtering step to eliminate the segments that contain people faces and finally return to the user the list of segments that contain a red object (if available).
The search results returned to the security agent in this case might be insufficient especially that the red bag might be present in the unauthorized segments containing passenger faces. Our proposal can take place at this level as a step towards ensuring a better quality of service by offering a wider subset of resources to the user while respecting the access rights defined on the consultation of the video surveillance data sources.
Through the usage of our proposed PS-RBAC model, the system would be able to offer more accessibility and adapt the permissions assigned to the security agent according to his contextual attributes and to the importance level of the situation of the consultation.
This adaptive solution can be employed when the system identifies access challenges related to the user’s context or at an important situation. In this scenario, the « lost object » situation identification can be obtained from the file number.
The implementation of the adaptive solutions is performed by the PSQRS that adapts decision-making by rewriting the XACML queries. The solution proves its effectiveness due to its ability to achieve decision making to access video surveillance sources that are distributed and administrated by different authorities. <Request xmlns="urn:oasis:names:tc:xacml:2.0:context:schema:os" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:oasis:names:tc:xacml:2.0:context:schema:os http://docs.oasisopen.org/xacml/access_control-xacml-2.0-context-schema-os.xsd"> <Subject> <Attribute AttributeId="urn:oasis:names:tc:xacml:2.0:subject:subject-id" DataType="http://www.w3.org/2001/XMLSchema#string"> <AttributeValue>John Smith</AttributeValue> </Attribute> <Attribute AttributeId="urn:oasis:names:tc:xacml:2.0:subject:role"
DataType="http://www.w3.org/2001/XMLSchema#anyURI"> <AttributeValue>Security Agent</AttributeValue> </Attribute> <Attribute AttributeId="urn:oasis:names:tc:xacml: 2.0:example:attribute:securityAgent-id" DataType="http://www.w3.org/2001/XMLSchema#string" > <AttributeValue>sa2023</AttributeValue> </Attribute> </Subject> <Resource> <ResourceContent> <UserQuery> <QueryInText> find all videos containing a red bag, forgotten in Trocadéro, Paris metro station, on Thursday, 2 Febuary, between 2:00pm and 3:00pm).</QueryInText> <MediaLocation>metro station, Paris, Trocadéro </MediaLocation> <MediaFormat>Video</MediaFormat> <TimeSpan> <From>2012-02-02T14:00:00</From> <To> 2012-02-02T15:00:00</To> </TimeSpan> </UserQuery> </ResourceContent> </Resource> <Action> <Attribute AttributeId="urn:oasis:names:tc:xacml:2.0:action:action-id" DataType="http://www.w3.org/2001/XMLSchema#string"> <AttributeValue>Read</AttributeValue> </Attribute> </Action> <Environment> <Attribute AttributeId="urn:oasis:names:tc:xacml:2.0:environment:environment-id" DataType="http://www.w3.org/2001/XMLSchema#string"> <AttributeValue>Situation</AttributeValue> </Attribute> <Attribute
AttributeId="urn:oasis:names:tc:xacml:2.0:environment:situation-id" DataType="http://www.w3.org/2001/XMLSchema#string"> <AttributeValue>Forgotten Object</AttributeValue> </Attribute> <Attribute AttributeId="urn:oasis:names:tc:xacml:2.0:environment:sitLevel-id" DataType="http://www.w3.org/2001/XMLSchema#string"> <AttributeValue>1</AttributeValue> </Attribute> </Environment> </Request> As shown in Figure 8, the richness of the elements that we can embed within an XACML query enables it to describe the contextual attributes characterizing: (i) the requested source in the « resource » tag, (ii) the user launching the request in the « subject » tag and (iii) the situation at which the user has launched the access request in the « environment » tag.
The importance level of the situation will determine the level of adaptation to be realized. The activation of the adaptive search mode will be communicated from the XACML response in the form of an « obligation » that accompanies the resulting access decision, see Figure 10.
<Response> <Result> <Decision>Deny</Decision> <Status>
<StatusCode Value="urn:oasis:names:tc:xacml:2.0:status:ok"/> </Status> <Obligation FulfillOn="Deny"
ObligationId="ApplyAdaptiveQueryingMode"> <AttributeAssignment AttributeId="AQM"
DataType="http://www.w3.org/2001/XMLSchema#string">
On </AttributeAssignment> </Obligation> </Result> </Response> As the adaptive querying mode is triggered, the query processing mechanism will change to ensure the success of the search by providing a variety of adaptive solutions in correspondence with the situation’s sensitivity level.
This adaptive search solution is realized by the PSQRS that detects the situation sensitivity through the Situation Analyzer component and turns to the Similarity Provider component to find similar resources that will guide the query rewriting process (see Figure 7).
In the case where the search didn’t retrieve satisfactory results to
the user and the consultation is taking place in a normal situation
(Sit_Lvl = 0), the system will perform the adaptive query
rewriting step through semantic similarity. The keywords of the
user query will be reformulated using similar words or more
generic concepts offered by the Similarity Provider. Similar works
have been introduced in [
At the other hand, the adaptation process in the mentioned scenario will follow another scheme since the lost object situation is judged to be of higher importance (Sit_Lvl = 1). Hence, the Similarity Provider component will be replaced by an Adaptive Solutions Provider. This component will provide some predefined solutions that could bypass the access control challenge or would assist the user in adapting and reformulating his query by pointing out the access challenge and offering him adaptive solutions that would suit his context, the solutions are often saved in a predefined database. Table 1 shows examples of the solutions that the system can offer.
New solutions can also be inserted to the adaptive solutions database through a learning mechanism that detects the solutions that users employ when encountered with access challenges in real time.
The success of the adaptive solutions suggested by the users would eventually be more efficient if they knew the reason behind the access denial. The error messages that often accompany the returned access denial responses can serve as indicators to help the users in finding alternative solutions.
Therefore, the adaptive solution for this example will modify the treatment process and will: (i) neglect the filtering step responsible for imposing the access control constraints and (ii) replace it with an adaptive step-related to the presentation of resources with unauthorized content.
By applying this process to the scenario described above, the system will return the video segments taken from the Trocadéro station between 14:00 and 15:00 and containing a red object. These results will be filtered in order to detect the unauthorized segments (containing passenger faces). This is where the system will apply the adaptation process that would filter the display to conform with the access restrictions imposed by the system. The adaptation will be performed through a face detection step and the use of an algorithm that applies a “blur function” to protect the privacy of passengers appearing in these segments in order to return to the user a list of pertinent results that respect the access rules.
In this paper, we have presented an adaptive approach for access
control management within multimedia distributed systems. Our
solution overcomes the access denials that take place in real time
access demands by modifying the query processing mechanism
and by providing adaptive solutions to bypass the access control
constraints. The proposed solution has been validated within the
LINDO framework in the context of a video surveillance use case.
We applied and validated the same access control approach for
other use cases, such as Healthcare Systems [
The adaptive and alternative based situation-aware solution can increase the complexity of processing the request, but if we consider the usefulness of the results provided in real time and the fact they do not violate the access rights defined by the privacy law, this complexity seems quite acceptable.
In future works, we aim to extend our proposal by taking into account different contextual elements that might also influence the accessibility to multimedia content (e.g., hardware, network bandwidth, etc.) and to apply the adaptive process not only at the presentation level but also at the choice of the explicit indexing algorithms that are protected by RBAC constraints.
This work has been supported by the EUREKA project LINDO (ITEA2-06011).