Service mesh technologies have emerged as a powerful tool for managing communications in microservicesoriented architectures. However, enforcing complex access control policies often requires stateful mechanisms, which are not directly supported by policy languages like Rego. To address this limitation, we propose the OPA Wrapper State Manager (OWSM). OWSM maintains a separate state store that can be accessed during policy evaluation. This enables the specification and enforcement of stateful access control policies using Rego's declarative syntax. We evaluate the performance and overhead of OWSM through experiments, demonstrating its efectiveness in enhancing the capabilities of service mesh environments.
In microservice-oriented architectures, large monolithic applications are split into smaller, independent
services, often implemented using virtual machines or containers. This approach ofers numerous
benefits, including scalability, resilience, and faster development cycles. However, it also introduces
significant complexity, especially when managing inter-service communication and security. Hardcoding
these functionalities into each service implementation introduces redundancy, complicates maintenance,
and increases the risk of misconfigurations [
To address these challenges, Service Mesh (SM) technologies have emerged [
While service meshes ofer powerful capabilities, they demand efective governance mechanisms
to maintain consistency, security, and compliance. To streamline administration and automate policy
enforcement, policy-based approaches have gained traction [
Among various domain-specific languages for policy authoring, Rego has emerged as a popular
choice for service mesh environments [
Controller Data Plane
...
Service A Sidecar Proxy
Service Z Sidecar Proxy
Service
Rego’s functional and declarative nature allows it to access a data store for policy evaluation, but it is limited to read-only operations. This restriction can hinder the specification of access policies that require maintaining and updating state. For instance, a policy enforcing a rate limit of 10 requests per hour from service to service necessitates tracking access counts and timestamps. Similarly, as in Bell-LaPadula model, a policy granting access to service based on ’s lack of access to service requires remembering the “ to ” access history.
These scenarios highlight the need for stateful policy enforcement, which is not directly supported by Rego’s core capabilities. In fact, in these cases the state must be updated by the services, thus violating the separation between business logic and policy specification.
To address the limitations of Rego’s stateless nature, in this paper we introduce the Open policy agent Wrapper State Manager (OWSM). OWSM maintains a separate state store to track policy-specific information, which can be accessed by Rego engine during policy evaluation. From Rego’s JSON response, OWSM extracts state update instructions and forwards the final authorization decision to the OPA agent, sidecar to the real service. By leveraging OWSM, we can write stateful policies directly in Rego without modifying its syntax or evaluation engine. In this way, services do not need to update the state with policy-specific information, thus keeping business logic and policy specification well separated. Moreover, thanks to controlled access to avoid inconsistencies, the data store can be shared across multiple Rego engines, allowing for eficient concurrent access to policy-specific information. Synopsis. In Section 2, we provide an overview of service meshes, OPA and Rego. To address its limitations, we introduce the OPA Wrapper State Manager (OWSM) in Section 3. In Section 4, we present the results of experiments conducted to evaluate the impact of OWSM. Finally, we conclude in Section 5, summarizing our findings and outlining directions for future work.
A Service Mesh typically consists of two main components: a controller and a set of sidecar proxies (Figure 1a). The sidecar proxies form the data plane network, which directly handles the flow of requests between microservices. Each proxy is responsible for intercepting all incoming and outgoing trafic to its service, enforcing the policies received from the controller on a separate control plane.
SMs can have multiple goals, here we focus on their role in authorization and authentication. Authorization determines which services or users are permitted to perform specific actions on resources, while authentication ensures that only legitimate services can communicate with each other. Compared to traditional orchestrator policies, SMs enable more fine-grained access control and identity management. A key advantage of SMs is their centralized policy enforcement, which simplifies the management of security policies across a distributed environment. Policies are defined centrally and propagated to sidecar proxies, which locally enforce the policies by intercepting and evaluating requests, thus ensuring decentralized decision-making. For each intercepted request, a proxy evaluates the authorization policy against the received data and metadata, such as source service identity, user attributes, request headers, and requested actions. Based on the policy evaluation, the sidecar proxy decides whether to allow or deny the request. This enforcement occurs transparently to the services, ensuring security without requiring modifications to application code.
The challenge of expressing security policies in distributed applications has been a major focus of research and development in recent years. Existing solutions can be categorized into two approaches: Policy via Configuration Files and Policy-as-Code.
Configuration files are collections of key-value pairs used for defining system configurations and
behaviors. This approach is widely adopted in cloud computing, including service meshes. Notable
examples include Istio [
On the other hand, the Policy-as-Code paradigm [
One of the most widespread language of this category is Rego [
A Rego policy is a collection of rules (see Listing 1). Each rule comprises a head, which defines the decision or value to be computed, and a body, which consists of a set of conditions or queries that must evaluate to true for the rule to be applicable. To make decisions, OPA can access the information provided in the request (Listing 3) and possibly additional data (Listing 2) from the evaluation context.
As Figure 1b shows, OPA performs the following steps to evaluate a policy: 1. OPA accepts JSONformatted inputs representing the request; 2. OPA interprets Rego rules to compute a decision based on the request and data; 3. OPA returns the evaluation outcome to the requester as a JSON response, which contains the outputs of all the evaluated rules.
Despite the versatility of Rego, certain use cases remain challenging due to its stateless nature. The inability to manage or persist data across requests hinders Rego’s applicability in scenarios that require stateful paradigms. In fact, in these situations the update of the data object is on the programmer of the services, thus violating the separation principle between policy specification and business logic. Listing 1: Example of Rego policy for RBAC.
Listing 3: Input for the RBAC example.
In this section we present OPA Wrapper State Manager (OWSM), whose aim is to extended OPA with state management capabilities. By using OWSM, developers are freed from the burden of managing state directly within their application’s business logic in scenarios where persistent state is crucial.
An example of an inherently stateful policy is the one regulating access to a service API where each call costs a token; each user is given a certain number of tokens at the beginning of the month. The policy has to keep track of token expenditure, and to reset token counters once a month. This can not be expressed in traditional policy engines without the introduction of additional elements that modify and maintain the number of available tokens.
Another example comes from security concerns, such as those investigated in [
Our solution relies on wrapping the OPA engine with a custom API that interacts with a datastore in order to maintain and modify a state at runtime. Comparing with Figure 1a, a sidecar proxy will interact with an OWSM instance, permitting or not a request access. There can be multiple OWSM instances, one for each sidecar proxy in the service mesh.
A core requirement for our system is the inclusion of primitives for reading from and writing to a state that persists across multiple policy queries. These primitives enable a policy decision engine to dynamically update the data upon which decisions are made.
The state must be accessible by multiple instances of the decision engine. Consider a service mesh where decision engines are deployed as sidecar proxies alongside multiple replicas of a web server ofering an API. If we aim to protect this API with the aforementioned “limited-token” middleware, the state must be maintained consistently across all replicas. This centralized state management is crucial to ensure that the policy “a user can access the API up to times, where is the number of tokens available to the user” is enforced uniformly across all replicas.
To achieve this, our system should consist of two components: (1) a policy decision engine that can interact with (2) a datastore. We choose OPA as the underlying policy engine and Rego as the policy language. This choice is based on the fact that its output is a JSON object, rendering it flexible and aiding the integration with our system.
We avoid modifying the decision engine directly by wrapping it with an API. This API receives queries, retrieves the latest state from the datastore, and forwards both to OPA. Our implementation is depicted in Figure 2. The datastore must implement some locking mechanism to ensure consistency in presence of concurrent requests. The minimal functional API for the store must thus contain four endpoints: two to respectively get and set the state, and two to interact with the locking subsystem.
The modification of the state is achieved by reserving a special policy name, i.e., state; this policy can be defined by Rego rules (as any other policy), yielding a standard JSON dictionary. This output is interpreted by the wrapper, which extracts the keys that need modification in the datastore, removing them from the JSON. The datastore is updated accordingly, and then unlocked, before returning the decision result to the requesting service.
From an implementation point of view, we decided to use the Go programming language to implement both the wrapper and the datastore. This choice has been made due to the fact that OPA is written in Go and ofers a native Go library to interact with its internals.
The API of the datastore is ofered via gRPC, an open source, high performance RPC framework. More specifically, the datastore exposes a gRPC service, called Store, with four procedures: Get, to retrieve the state; Put, to update a key with a new value; Lock, to guarantee mutually exclusive access to concurrent clients; and Unlock, to release the lock. The wrapper communicates with the datastore through gRPC and, finally, exposes to the final user an HTTP endpoint to allow for submission of decision queries, as does OPA when used as an HTTP API. We use /query as the endpoint name. 2 OPA
4 State updater policy .rego 5 3
To validate our system, we implemented the “token counter” and “three microservices” use cases described above, and tested them using OWSM yielding a null error rate, thus proving the higher expressiveness of OWSM compared to OPA. See Figure 3 for the corresponding code.
In this section, we perform an experimental evaluation of OWSM to asses the eficiency and overhead
introduced in comparison to pure OPA. To this end, we consider use cases that can be expressed in both
pure OPA and OWSM, i.e., without any stateful information. These use cases are taken from the Access
Control section of the Rego Playground [
Use case 1 considers an RBAC model for the Pet Store API [
Use case 2 follows an Attribute-based Access Control (ABAC) model for the same API: users, resources, and actions are associated with attributes, over which access decisions are made.
Use case 3 aims to mitigate the “Role Explosion” problem in RBAC through hierarchical roles. Role Explosion occurs when the number of roles in a RBAC grows exponentially as the number of users and permissions increases, leading to a complex and unmanageable set of roles. This example shows how to implement a simple hierarchical access control policy using a graph of related roles. Hierarchical roles help address this issue by allowing roles to inherit permissions from other roles, reducing redundancy and simplifying role management. Specifically, users submit requests with one or more roles, and the policy checks if the user has the required permission by traversing the role hierarchy.
RQ1 [Time performance] What are the characteristics of the overhead introduced by OWSM when handling sequences of requests?
Understanding the overhead introduced by OWSM is important to evaluate its impact on maintaining a good time performance during request handling. This overhead includes both the computational and latency costs of managing state modification and wrapping pure OPA.
RQ2 [Access scalability] What is the behaviour of OWSM as the number of concurrent requests increases?
Understanding the behaviour of OWSM under increasing concurrency (i.e., number of concurrent requests to the datastore) is essential for evaluating its concurrent scalability. This requires analysing response times as the number of simultaneous requests grows.
To evaluate response timing, we rely on “Apache Benchmark” (ab) version 2.4.62, a standard tool for benchmarking web servers. We patched the source code of ab to support outputting timings in microseconds, instead of rounding them to the nearest millisecond. Our patch modifies the ap_round_ms macro to remove the rounding operation and return the raw runtime time value (already in microseconds) and does not modify the functionality of ab.
For comparing OPA and OWSM, we run experiments on both for each use case. At the beginning of each experiment, we perform a warmup of OPA and OWSM by running 10 non-concurrent requests. This avoids spurious high times from the first system query.
For RQ1 we run batches of 100, 200, 400, 800, 1600, 3200, 6400, 12800, 25600, 51200 and 102400 requests to both OPA and OWSM, and we calculate the mean and the interquartile range (IQR) of the response times for each batch.
For RQ2 we run 50000 total requests with increasing levels of concurrency (100, 650, 1200, 1750, 2300, 2850, 3400, 3950, 4500, 5050, 5600, 6150, 6700, 7250, 7800, 8350, 8900, 9450 and 10000 parallel requests). We then calculate the mean and IQR of the response times for each concurrency level.
We use a server with Debian GNU/Linux 12 at kernel version 6.10-28 with a Intel(R) Core(TM) i9-10900 CPU @ 2.80GHz (20 threads) and 128 GB of RAM.
To account for the inherent randomness in the measurements, we performed multiple requests in each configuration and assessed the statistical significance of the comparison between the IQRs of OPA and OWSM by using the Mann–Whitney U test.This test provides a measure of whether one group tends to have larger values than the other. Specifically, we calculated the U statistic and its corresponding p-value to determine if the observed diferences between the systems were statistically significant at a significance level of = 0.05.
Our assessment of the observed trends is supported by fitting a linear regression model to the data and analysing the coeficient of determination ( 2).
We mitigate the threats to external validity (i.e. generalization) by considering three representative and diverse use cases directly drawn from the oficial OPA playground.
(a) Time performance (use case 1) (b) Time performance (use case 2) (c) Time performance (use case 3)
Figures 4a to 4c show the overhead introduced by OWSM over OPA in the three stateless use cases under consideration. Some variability in response times for individual queries can be introduced by the fact that OWSM consists of two separate parts (i.e. the wrapper and the datastore) communicating over an API. This is reflected in the significantly higher IQR shown by OWSM (p-value < 0.05). (a) Concurrent scalability (use case 1) (b) Concurrent scalability (use case 2) (c) Concurrent scalability (use case 3)
However, the overhead remains nearly constant (|slope| < 0.001, p-value > 0.05) across all measured points. This is a desirable outcome, as it indicates that the overhead introduced by OWSM does not increase as the sequence of consecutive requests progresses. A constant overhead means that OWSM can handle multiple consecutive requests without accumulating additional delays, ensuring consistent performance over time. This is important for maintaining predictability and reliability in real-world use cases, where repeated requests are common.
Answer to RQ1: OWSM introduces an overhead of ∼ 1 millisecond, which remains constant thorough the whole sequence of requests.
Figures 5a to 5c show that the overhead for concurrent requests introduced by OWSM follows a linear trend (2 > 0.999 , p-value < 0.05). This is due to the fact that the implemented locking mechanism in the datastore allows only one request to be processed at a time, even when the store is not modified. This result is consistent with the previous experiment, as for 10000 concurrent requests the elapsed time per request is ∼ 10000 milliseconds.
Remarkably, OWSM demonstrates much more predictable response times in comparison to OPA, with a significantly lower IQR. In fact, the mean response time for OWSM is ∼ 10× lower than for OPA, highlighting its improved consistency in handling concurrent requests.
Answer to RQ2: At increasing concurrency level, the response time for OWSM increases linearly, with higher temporal stability.
Our experiments quantify the expected performance overhead of OWSM compared to OPA, a consequence of OWSM’s added functionality. The consistent overhead observed across request sequences and the stable performance under increasing concurrency suggest that OWSM ofers a promising solution to OPA’s lack of state management primitives. While our prototype’s basic locking mechanism efectively maintains temporal stability, future work could explore more sophisticated concurrency control mechanisms to further enhance performance.
In this paper we introduced the OPA Wrapper State Manager (OWSM), a novel solution designed to extend the capabilities of the OPA authorization engine by incorporating state management for Rego policies. To validate the practical applicability of OWSM, we have successfully applied it to various scenarios that demand stateful access control. Our solution allows to maintain the definition of (stateful) access policies just as Rego rules, without the need of modifying the business logic of services. To assess the performance impact of OWSM, we conducted empirical evaluations comparing it to pure OPA. Our findings demonstrate that OWSM maintains temporal stability even under increasing concurrency levels, while the introduced overhead remains relatively low, making it a viable solution for service mesh environments.
Future work includes generalizing our findings to a broader range of real-world use cases, also derived from software repositories. We also plan to investigate advanced locking mechanisms to improve eficiency and concurrency while maintaining temporal stability; this includes exploring techniques like caching data subsets within the wrapper to reduce RPC accesses. Furthermore, we aim to statically verify properties of Rego policies to prevent errors and vulnerabilities. Finally, we are interested in integrating NLP to bridge the gap between informal natural language descriptions and the definition and validation of stateful Rego security policies.
This work was partially supported by the Department Strategic Project on Artificial Intelligence of the University of Udine (2020-25), and the M4C2 I1.3 “SEcurity and RIghts In the CyberSpace - SERICS” (PE00000014 - CUP H73C2200089001, D33C22001300002), under the National Recovery and Resilience Plan (NRRP) funded by the European Union - NextGenerationEU.
During the preparation of this work, the authors used ChatGPT, Reverso Context in order to: Grammar and spelling check, paraphrase and reword. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.