Research on security as code approach for cloud-native applications based on Kubernetes clusters⋆ Oleksandr Vakhula1,† and Ivan Opirskyy1,*,† 1 Lviv Polytechnic National University, 12 Stepan Bandera str., 79000 Lviv, Ukraine Abstract The fast evolution of cloud-native applications and the widespread adoption of Kubernetes clusters have revolutionized how modern software is developed, deployed, and managed. However, this paradigm shift has introduced new security challenges that require innovative solutions. This research explores the “Security as Code” (SaC) approach, which integrates security policies and practices into the development and deployment pipelines of cloud-native applications on Kubernetes clusters. The study begins by outlining the theoretical foundations of the SaC approach, emphasizing the need for automated and consistent security measures across all stages of the software development lifecycle. We then explore the implementation of the policy engine and its gatekeeper component, as core tools for enforcing security policies within Kubernetes environments. The research details the setup process on AWS using a cost- effective configuration, augmented with GitOps tool for continuous deployment and container image vulnerability scanner. Our methodology includes configuring OPA Gatekeeper for admission control, defining and applying constraint templates, and integrating FluxCD to automate policy deployment and enforcement. We provide a step-by-step guide for setting up the environment, ensuring that the approach is practical and reproducible. The findings demonstrate that the SaC approach significantly improves security management in cloud-native environments, offering a scalable and flexible framework for integrating security into DevOps workflows. This research contributes to the broader understanding of how security can be codified and automated, paving the way for more secure and resilient cloud-native applications. Keywords security-as-code, cloud-native applications, Kubernetes clusters, open policy agent, gatekeeper, GitOps, continuous deployment, container security, DevOps, policy-as-code, service mesh, shift-left security1 1. Introduction automatically enforced across all environments, from development to production. By integrating security into the The introduction of cloud-native applications has marked a DevOps pipeline, organizations can achieve continuous significant shift in the landscape of software development security and compliance, reducing the risk of vulnerabilities and deployment. These applications, designed to leverage and misconfigurations. the advantages of cloud computing, offer unparalleled This research focuses on implementing the SaC scalability, flexibility, and resilience. At the heart of this approach within Kubernetes clusters, leveraging Open transformation is Kubernetes, an open-source container Policy Agent (OPA) and its Gatekeeper component. OPA is orchestration platform that has become the de facto a general-purpose policy engine that enables the standard for deploying, scaling, and managing enforcement of fine-grained, context-aware policies. containerized applications. Gatekeeper extends OPA's capabilities by integrating with While Kubernetes simplifies many aspects of Kubernetes admission controllers, allowing policies to be application management, it also introduces new security enforced at the time of resource creation and modification. challenges. Traditional security practices often struggle to Additionally, this study incorporates FluxCD, a keep pace with the dynamic and ephemeral nature of cloud- continuous delivery tool for Kubernetes, and Trivy, a native environments. This gap has led to the emergence of comprehensive vulnerability scanner for container images. the “Security as Code” (SaC) paradigm, which aims to By combining these tools, we aim to create a robust embed security directly into the development and framework for automating security policy enforcement and operational processes through code. continuous monitoring of container vulnerabilities. Security as Code involves defining security policies and The goal of this research is to demonstrate the practical controls as code, allowing them to be versioned, reviewed, implementation and effectiveness of the Security as Code and deployed alongside application code. This approach approach in Kubernetes environments. Specifically, we aim ensures that security measures are consistently applied and to integrate and automate security policies, ensuring CSDP-2024: Cyber Security and Data Protection, June 30, 2024, Lviv, 0009-0008-5367-3344 (O. Vakhula); 0000-0002-8461-8996 (I. Opirskyy) Ukraine © 2024 Copyright for this paper by its authors. Use permitted under ∗ Corresponding author. Creative Commons License Attribution 4.0 International (CC BY 4.0). † These authors contributed equally. oleksandr.p.vakhula@lpnu.ua (O. Vakhula); ivan.r.opirskyi@lpnu.ua (I. Opirskyy) CEUR Workshop ceur-ws.org ISSN 1613-0073 58 Proceedings continuous security and compliance throughout the emphasizing key innovations such as Borg, the initial development lifecycle of cloud-native applications. system developed to manage both long-running services and batch jobs. Borg introduced resource sharing between  Define Security Policies: Create and codify different types of applications, significantly improving security policies that can be enforced within resource utilization. Omega, built to improve the software Kubernetes clusters using OPA Gatekeeper. engineering of Borg, introduced a more consistent and  Automate Policy Enforcement: Integrate security principled architecture, using a centralized Paxos-based policies into the CI/CD pipeline to ensure transaction-oriented store. Kubernetes was designed for a automated and consistent enforcement. broader developer audience, emphasizing ease of use for  Implement Continuous Deployment: Use FluxCD deploying and managing distributed systems, leveraging a to manage and automate the deployment of shared persistent store accessed through a REST API. Kubernetes manifests, ensuring the desired state of Kubernetes’ architecture is designed to support scalability the cluster is maintained. and flexibility while enforcing consistent security policies.  Vulnerability Scanning: Incorporate Trivy to scan This is achieved through a centralized API server that container images for vulnerabilities, adding a layer ensures all state changes are validated, defaulted, and of security. versioned, providing a robust foundation for enforcing  Cost-Effective Setup: Establish a cost-effective policies and maintaining system invariants. Reconciliation environment for testing and deploying the SaC controllers improve resiliency by continuously aligning the solutions, particularly in cloud environments like desired and observed states, a concept shared with Borg and AWS. Omega [1]. “Kubernetes Security: Securing Microservices and The swift adoption of Kubernetes as a container Applications in the Cloud” by O’Reilly Media is a orchestration platform has revolutionized the deployment comprehensive guide that explores various security and management of cloud-native applications. However, challenges and best practices for securing Kubernetes this shift has also introduced significant security challenges clusters. It covers topics such as securing the Kubernetes that traditional security approaches are ill-equipped to API server, controlling access with RBAC, network security address. The need for dynamic, automated, and scalable policies, and monitoring and auditing Kubernetes clusters. security measures has become crucial. The book emphasizes the importance of integrating security The primary problem this research addresses is the throughout the development lifecycle and provides implementation of a “Security as Code” (SaC) approach in practical examples of implementing security measures [2]. Kubernetes-based cloud environments. Specifically, the Practical guides and best practices for Kubernetes challenges include вynamic and ephemeral nature of security, such as those by AquaSec and Red Hat, provide containers, containers are inherently ephemeral and comprehensive overviews of necessary security measures. dynamic; сomplexity of Kubernetes, Kubernetes, while These include network policies to control traffic between powerful, introduces significant complexity with its pods, secret management to securely manage sensitive numerous components (e.g., API server, etcd, scheduler, information, and admission controllers to enforce security controllers); multi-tenancy and Isolation; integration with policies at the point of deployment, ensuring that only existing security tools; continuous security and compliance; compliant configurations and container images are visibility and monitoring. deployed [3–4]. Through this research, we aim to explore and address The concept of Policy as Code, particularly with tools these challenges by providing a detailed implementation like Open Policy Agent (OPA), has gained traction for guide, evaluating the effectiveness of automated security automating and enforcing security policies within policies, and offering practical insights into integrating Kubernetes. Works such as those by O'Reilly and industry security as code into Kubernetes-based workflows. By doing blogs discuss implementing OPA for dynamic policy so, we contribute to the broader discourse on enhancing the enforcement, highlighting its flexibility and power in security of cloud-native applications through innovative managing complex security policies programmatically [5]. code-centric approaches. The article “XI Commandments of Kubernetes Security: A Systematization of Knowledge Related to Kubernetes 2. Related works Security Practices” provides a systematic approach to securing Kubernetes environments by identifying eleven The field of container orchestration and security has critical security practices. These practices, derived from a evolved significantly over the past decade, with numerous comprehensive analysis of internet artifacts, include Role- contributions from both academia and industry. This Based Access Control (RBAC), network policies, and regular section reviews some of the seminal works and current security patching. The systematic approach offers a research related to the “Security as Code” approach in structured framework for practitioners to enhance the Kubernetes environments. security posture of their Kubernetes deployments [6]. The article “Borg, Omega, and Kubernetes” by Burns et The research article “Comprehensive Approach for al. from Google Inc. provides an in-depth look at the Developing an Enterprise Cloud Infrastructure” by Khoma evolution of container management systems within Google, et al. emphasizes the need for a multilevel security approach starting with Borg, moving to Omega, and finally to in cloud environments. The article outlines the limitations Kubernetes. This progression highlights the increasing of existing “Security as Code” practices and proposes a sophistication and scalability of container orchestration, 59 comprehensive framework to enhance cloud infrastructure organizations can achieve continuous security and security. Key aspects include effective access and privilege compliance, ensuring that their Kubernetes environments management, logical isolation of network resources, remain secure and resilient. These resources provide continuous monitoring, and automated response to valuable insights, practical examples, and best practices that anomalies. This comprehensive approach aligns well with can guide practitioners in enhancing the security of their the principles of “Security as Code” by integrating security cloud-native applications. into every layer of the cloud infrastructure, thus providing The field of container orchestration and security has a robust foundation for secure cloud-native applications [7]. evolved significantly over the past decade, with numerous The paper “Cloud Container Technologies: A State-of- contributions from both academia and industry. This the-Art Review” by Pahl et al. provides a systematic section reviews some of the seminal works and current mapping study of container technologies and their research related to the “Security as Code” approach in orchestration, particularly in cloud environments. The Kubernetes environments. study identifies and classifies 46 selected studies on container technologies, highlighting the key concerns and 3. General overview of container trends in the field. It reveals that container technologies positively impact both development and deployment cluster and Kubernetes aspects, supporting continuous development and orchestration deployment pipelines. However, the study also notes the Container clusters and orchestration are fundamental to lack of tool support to automate and facilitate container modern application deployment and management. management and orchestration, particularly in clustered Containers encapsulate an application and its dependencies, cloud architectures. The findings underscore the need for providing a consistent environment across development, advanced orchestration support and the importance of testing, and production. Orchestration is crucial for container-based orchestration techniques in balancing managing these containers at scale, ensuring efficient optimized resource utilization and performance in the cloud resource utilization, high availability, and automated [8]. workflows. Kubernetes has emerged as the leading The paper “Expanding DevSecOps Practices and orchestration platform, offering robust tools for deploying, Clarifying the Concepts within Kubernetes Ecosystem” by scaling, and operating containerized applications across Alawneh and Abbadi discusses the integration of clusters of machines. DevSecOps principles within Kubernetes environments. Kubernetes clusters are highly versatile and can be The authors highlight the importance of incorporating utilized to manage and orchestrate a range of innovative security by design within organizational processes, technologies. In blockchain [10], Kubernetes supports including development, deployment, and operational platforms like Hyperledger Fabric and Ethereum, ensuring management. The paper outlines several real-life examples scalable and resilient node deployment. For machine that illustrate the integration of security into each practice, learning and AI, tools like TensorFlow Serving and emphasizing how DevSecOps practices can enhance Kubeflow benefit from Kubernetes’ scalability and application delivery, resilience, elasticity, availability, and automated management. In big data, Kubernetes efficiently reliability. The paper also addresses the challenges of manages Apache Spark and Elasticsearch clusters. establishing robust mechanisms for integrating security Kubernetes is also pivotal in IoT with edge computing within existing DevOps practices and provides insights into solutions like KubeEdge and in CI/CD with Jenkins X and the roles of DevSecOps practices in securing the Kubernetes Argo CD. Additionally, it supports microservices and ecosystem. This work aligns with the Security as Code serverless architectures through Istio and Knative, and approach by demonstrating how security can be seamlessly manages databases such as Cassandra and PostgreSQL, integrated into the Kubernetes lifecycle, thereby enhancing making it an essential tool for modern, cloud-native the overall security posture of cloud-native applications [9]. applications. The literature review highlights the importance of Core Concepts adopting a security-as-code approach in modern cloud- Containers are lightweight, portable, and consistent native environments. By automating and codifying security units of software that include everything needed to run an policies, organizations can achieve continuous security, application. Unlike traditional virtual machines, containers maintain compliance, and build more resilient systems. The share the host system’s kernel but operate in isolated user articles and studies reviewed provide valuable insights, spaces. A container cluster is a group of interconnected practical examples, and best practices that can guide nodes that work together to provide a scalable and resilient practitioners in enhancing the security of their Kubernetes environment for running containerized applications. This deployments. These resources emphasize the critical role of clustering allows for efficient resource sharing, load policy automation, continuous monitoring, and integration balancing, and fault tolerance, making it possible to manage of security into DevOps practices in building secure and thousands of containers seamlessly. compliant cloud-native applications. Kubernetes Architecture (Fig. 1): The research and resources analyzed above underscore Kubernetes architecture consists of a master node and the critical importance of integrating security into the multiple worker nodes. The master node controls the development and operational processes of cloud-native cluster, housing components such as the API server, etcd (a applications. By adopting a Security as Code approach and key-value store for cluster data), the scheduler, and the leveraging tools like OPA Gatekeeper, FluxCD, and Trivy, controller manager. The API server serves as the main interface for interaction with the cluster. Etcd stores all 60 cluster configuration data persistently. The scheduler actual applications in containers, managed within pods, assigns workloads to nodes based on resource availability, which are the smallest deployable units in Kubernetes [11– and the controller manager handles routine tasks like 12]. replication and state management. Worker nodes run the Figure 1: Kubernetes architecture Operational Features: 4. Problem statement: Challenges in Kubernetes provides several critical operational features to enhance the management and resilience of implementing security for containerized applications. Self-healing capabilities containerized services in cloud automatically replace failed containers, ensuring continuous availability. Load balancing distributes network environments traffic evenly across all running containers, optimizing resource usage and performance. Automated rollouts and rollbacks allow for seamless updates and rollbacks of applications without downtime, ensuring that deployments are both reliable and consistent. These features collectively contribute to the robustness and efficiency of Kubernetes as an orchestration platform. Declarative Configuration: Kubernetes employs a declarative approach to configuration management, where the desired state of the system is defined in configuration files using YAML or JSON. Users specify what the end state should be, and Kubernetes takes responsibility for achieving and maintaining that state. This approach simplifies management, as Kubernetes continuously monitors the current state and makes necessary adjustments to align it with the desired state. It ensures consistency, and repeatability, and reduces the complexity of managing configurations manually. In summary, Kubernetes has revolutionized the orchestration of container clusters, providing robust tools Figure 2: Main problems of ensuring a needed level of for managing containerized applications at scale. By security for containerized services leveraging a sophisticated architecture, operational features, and a declarative configuration approach, Let’s dive deeper into each of them: Kubernetes ensures efficient, reliable, and consistent application deployment and management. [13] This 1. Dynamic and Ephemeral Nature of Containers: overview sets the stage for understanding the complexities Containers are inherently ephemeral and dynamic, and benefits of Kubernetes orchestration. often created and destroyed within seconds. This transient In the next chapter, we will dive into the challenges nature makes it difficult to maintain consistent security associated with implementing security in Kubernetes policies and apply traditional security measures. Ensuring environments, exploring the issues that must be addressed that security policies are consistently applied to every to maintain robust security postures. instance of a container can be challenging, leading to potential security gaps. 61 2. Complexity of Kubernetes: microservices deployment, playing an important role in Kubernetes, while powerful, introduces significant cloud computing emerging fields such as service meshes, complexity with its numerous components (e.g., API server, market surveys show that container security is the main etcd, scheduler, controllers). Securing each component and concern and adoption barrier for many companies. The ensuring secure communication between them requires a literature on container security identifies four generalized deep understanding of the Kubernetes architecture. use cases that cover security requirements within the host- Misconfigurations and overlooked security settings can lead container threat landscape: to vulnerabilities, making the cluster susceptible to attacks. 1. Protecting a container from applications inside it. 3. Multi-Tenancy and Isolation: 2. Inter-container protection. In a multi-tenant environment, ensuring proper 3. Protecting the host from containers. isolation between different tenants’ workloads is crucial to 4. Protecting containers from a malicious or semi- prevent unauthorized access and data leakage. Achieving honest host. strong multi-tenancy security requires robust network policies, resource quotas, and effective namespace The first three use cases utilize software-based solutions management, which can be complex to implement and that mainly rely on Linux kernel features and Linux security manage. modules, the last use case relies on hardware-based solutions such as trusted platform modules [17]. 4. Integration with existing Security Tools: The swift adoption of Kubernetes as a container Integrating Kubernetes with existing security tools and orchestration platform has revolutionized the deployment processes can be difficult due to differences in how these and management of cloud-native applications. However, tools are designed to operate. Organizations may struggle to this shift has also introduced significant security challenges leverage their existing security investments, leading to that traditional security approaches are ill-equipped to potential gaps or redundant efforts in securing Kubernetes address. The need for dynamic, automated, and scalable environments [14]. security measures has become crucial. The primary problem this research addresses is the 5. Continuous Security and Compliance: implementation of a “Security as Code” (SaC) approach in Maintaining continuous security and compliance in a Kubernetes-based cloud environments. fast-paced, CI/CD-driven development environment is challenging. Automated pipelines need to incorporate security checks without hindering development velocity. 5. Review security as a code concept Ensuring that security checks are seamlessly integrated into The SaC paradigm aims to embed security policies and the CI/CD pipeline is essential to catch vulnerabilities early practices into the development and deployment pipelines, and maintain compliance without slowing down ensuring consistent and automated enforcement across all development [15]. stages of the application lifecycle. The “Security as Code” approach in cloud environments involves embedding 6. Visibility and Monitoring (Telemetry): security measures directly into the software development Achieving comprehensive visibility and monitoring of and deployment process. This method enables the containerized applications across a distributed cloud automation of various security tasks, enhancing consistency environment is challenging. Traditional monitoring tools and effectiveness. It is especially crucial in cloud may not provide the granularity needed for container environments, where rapid and flexible responses to environments. Lack of visibility can hinder the detection changes and emerging security challenges are required. and response to security incidents, making it difficult to “Security as Code” helps in the early identification of enforce security policies effectively. potential vulnerabilities and ensures compliance with A critical issue highlighting these challenges is the fact regulatory and security standards [18]. Despite its potential, that only 0.79% of Kubernetes commits are security-related, the practical implementation of SaC in Kubernetes suggesting that security-related defects are under-reported environments faces several challenges, which we should and could lead to large-scale security breaches. This statistic take into account: underscores the need for a more proactive and integrated approach to security within the Kubernetes ecosystem. By 1. Defining and Enforcing Security Policies: addressing these problems, the implementation of a robust How can organizations define and enforce security “Security as Code” framework can ensure that Kubernetes policies in a dynamic and scalable manner that aligns with environments are secure, compliant, and resilient, the ephemeral nature of containers? protecting them against the evolving threat landscape [16]. 2. Automation and Integration: Container security is a major concern for companies, How can security policies be automated and integrated with four generalized use cases and solutions relying on into existing CI/CD pipelines to ensure continuous security software-based and hardware-based solutions. Containers without hindering development velocity? emerged as a lightweight alternative to virtual machines that offer better microservice architecture support. The 3. Tooling and Best Practices: value of the container market is expected to reach $2.7 What are the best practices and tools (e.g., OPA billion in 2020 compared to $762 million in 2016. Although Gatekeeper, FluxCD, Trivy) for implementing SaC in they are considered the standardized method for 62 Kubernetes environments, and how can they be effectively 6. Overview of solution based on configured and managed? security as a code in the context 4. Cost-Effective Deployment: of containerized cloud-native How can organizations set up a cost-effective environment for testing and deploying SaC solutions, application particularly in cloud environments like AWS? Open Policy Agent (OPA) is a general-purpose policy engine that enables unified, context-aware policy This research aims to explore and address these enforcement across the stack. OPA decouples policy challenges by providing a detailed implementation guide, decisions from the application logic, allowing evaluating the effectiveness of automated security policies, administrators to manage policies centrally [19]. and offering practical insights into integrating security as OPA uses a high-level declarative language called Rego code into Kubernetes-based workflows. By doing so, it to write policies. Rego allows users to define policies based contributes to the broader discourse on enhancing the on various data inputs, supporting complex logic and security of cloud-native applications through innovative, queries to determine policy compliance [20]. code-centric approaches. Figure 3: Interaction between Kubernetes components and admission control with the Open Policy Agent (OPA) OPA can be integrated with a variety of systems, including Gatekeeper is an admission controller for Kubernetes Kubernetes, CI/CD pipelines, microservices, and more (Fig. that uses OPA policies to enforce security and operational 3). In Kubernetes, OPA can enforce policies on resources rules within the cluster. It provides a framework for policy such as pods, deployments, and services. It can also enforcement and auditing, ensuring that all changes comply integrate with CI/CD pipelines to ensure compliance during with predefined policies before being accepted by the the build and deployment phases. Kubernetes API server. Figure 4: Integration of the Open Policy Agent (OPA) with Kubernetes through Gatekeeper 63 Fig. 4 shows the integration of the Open Policy Agent (OPA) the Git repository. This process is known as GitOps. The with Kubernetes through Gatekeeper, detailing how policies EC2 instance runs Minikube, which sets up the Kubernetes are enforced and managed. cluster. Within this cluster, OPA Gatekeeper, Trivy, and FluxCD are deployed as separate services. OPA Gatekeeper 1. Policy Templates and Instances: enforces security and compliance policies by validating – Policy Template Custom Resource Definitions resources during the admission process. Trivy scans the (CRDs) define reusable policy templates. container images used within the cluster for vulnerabilities, – Policy Instance CRDs apply specific policies ensuring that only secure images are deployed. FluxCD using these templates. watches the Git repository for changes and updates the 2. Kubernetes API Server: – Handles resources like Pods, Services, and Kubernetes cluster configuration accordingly, automating Configurations. the deployment process and maintaining the desired state. – Uses Admission Controllers and AuthZ This guide provides general steps for integrating Webhooks for authorization and admission control. FluxCD into a Kubernetes environment set up with 3. OPA and Gatekeeper: Minikube on an EC2 instance, along with OPA Gatekeeper – Gatekeeper replicates policies to OPA. and Trivy for security scanning of container images. The – OPA evaluates AdmissionReview requests steps include: against these policies. – Results are audited and enforced through the 1. Set Up Minikube on EC2: Install and configure API server, ensuring compliance with defined Minikube on an EC2 instance to create a local policies. Kubernetes cluster. Policies in Gatekeeper are written using the Rego 2. Install OPA Gatekeeper: Deploy OPA Gatekeeper language and configured as ConstraintTemplates. These to enforce policies within the Kubernetes cluster. templates define the policy logic and the constraints that 3. Integrate Trivy: Set up Trivy to scan container must be met. Examples of common policies include images for vulnerabilities. restricting certain container images, enforcing namespace- 4. Deploy FluxCD: Install FluxCD to manage specific policies, and ensuring resource quotas. Kubernetes manifests and automate deployments Gatekeeper offers auditing and monitoring capabilities, based on changes in a Git repository. providing visibility into policy violations and historical data Step 1: Launch an EC2 Instance for compliance audits. It helps identify non-compliant resources and offers detailed reports on policy enforcement 1. Create an EC2 Instance: across the cluster. - Open the AWS Management Console and navigate Adding FluxCD to the setup enables continuous to the EC2 service. deployment for the Kubernetes environment. - Click on “Launch Instance.” - Choose an Amazon Machine Image (AMI), such as 7. Practical implementation Amazon Linux 2 AMI (HVM). The foundation of the architecture is an Amazon EC2 - Select an instance type, such as t3.small, for cost- instance. This instance serves as the host for Minikube, effectiveness. which is used to create a local Kubernetes cluster. Minikube - Configure the instance details, including network is installed and configured on the EC2 instance. It creates a settings. local Kubernetes cluster within the EC2 environment, - Add storage (default settings are typically sufficient). enabling Kubernetes functionalities in a contained setup. - Configure security groups to allow SSH (port 22) The Kubernetes cluster orchestrated by Minikube access. consists of multiple nodes that manage containerized - Review and launch the instance. applications. These nodes handle the deployment, scaling, 2. Connect to the EC2 Instance: and operation of application containers. OPA (Open Policy - Use an SSH client to connect to your EC2 instance. Agent) Gatekeeper is deployed within the Kubernetes ```sh cluster. It acts as a policy enforcement tool, ensuring that all ssh -i /path/to/your-key-pair.pem ec2- resources and configurations within the cluster comply user@ with predefined policies. It intercepts admission requests `` and validates them against the policies before allowing them into the cluster. [21–22] Trivy is integrated into the Step 2: Install Minikube and Kubernetes Tools Kubernetes environment to scan container images for vulnerabilities. It runs security scans on images either 1. Install Docker: before they are deployed or continuously as part of the - Update the package database and install Docker. CI/CD pipeline. [23] Trivy helps in identifying and ```sh mitigating potential security risks in container images. sudo yum update -y FluxCD is installed in the Kubernetes cluster to manage sudo amazon-linux-extras install docker -y Kubernetes manifests and automate deployments based on sudo service docker start changes in a Git repository. FluxCD continuously monitors sudo usermod -aG docker ec2-user the repository for changes and applies them to the cluster, ``` ensuring that the cluster state matches the declared state in 64 2. Install Minikube: Step 5: Configure Admission Control Policies - Download and install Minikube. ```sh 1. Create a ConstraintTemplate: curl -Lo minikube - Define a custom constraint template YAML file. For https://storage.googleapis.com/minikube/releases/latest/mi example, `k8srequiredlabels.yaml`: nikube-linux-amd64 ```yaml chmod +x minikube apiVersion: templates.gatekeeper.sh/v1beta1 sudo mv minikube /usr/local/bin/ kind: ConstraintTemplate ``` metadata: name: k8srequiredlabels 3. Install kubectl: spec: - Download and install kubectl. crd: ```sh spec: curl -LO “https://storage.googleapis.com/kubernetes- names: release/release/$(curl -s kind: K8sRequiredLabels https://storage.googleapis.com/kubernetes- ... release/release/stable.txt)/bin/linux/amd64/kubectl” targets: chmod +x kubectl - target: admission.k8s.gatekeeper.sh sudo mv kubectl /usr/local/bin/ rego: | ``` package k8srequiredlabels 4. Start Minikube: violation[{“msg”: msg}] { - Start Minikube with a specific driver (e.g., Docker). ... ```sh } minikube start --driver=docker ``` ``` Step 3: Install OPA Gatekeeper 2. Apply the ConstraintTemplate: ```sh 1. Deploy Gatekeeper: kubectl apply -f k8srequiredlabels.yaml - Apply the Gatekeeper manifest to deploy OPA ``` Gatekeeper in your Minikube cluster. ```sh 3. Create a Constraint: kubectl apply -f - Define a constraint to enforce the policy, e.g., https://raw.githubusercontent.com/open-policy- `requiredlabels.yaml`: agent/gatekeeper/master/deploy/gatekeeper.yaml ```yaml ``` apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sRequiredLabels 2. Verify Installation: metadata: - Check the Gatekeeper pods to ensure they are name: ns-must-have-gk running. spec: ```sh match: kubectl get pods -n gatekeeper-system kinds: ``` - apiGroups: [“”] Step 4: Install FluxCD kinds: [“Namespace”] parameters: 1. Install Flux CLI: labels: [“gatekeeper”] - Download and install the Flux CLI. ``` ```sh curl -s https://fluxcd.io/install.sh | sudo bash 4. Apply the Constraint: ``` ```sh kubectl apply -f requiredlabels.yaml 2. Bootstrap FluxCD with GitHub: ``` - Bootstrap FluxCD with your GitHub repository. ```sh Step 6: Check Container Image Vulnerabilities flux bootstrap github \ --owner= \ 1. Install Trivy: - Trivy is a popular tool for scanning container images --repository= \ for vulnerabilities. Install it on your EC2 instance. --branch=main \ ```sh --path=clusters/my-cluster \ wget --personal https://github.com/aquasecurity/trivy/releases/download/v ``` 0.28.0/trivy_0.28.0_Linux-64bit.deb 65 sudo dpkg -i trivy_0.28.0_Linux-64bit.deb 3. Effective Vulnerability Management: ``` Incorporating Trivy for container image vulnerability scanning provides an additional layer of security by 2. Scan an Image: identifying and mitigating potential vulnerabilities before - Use Trivy to scan an image, e.g., `nginx:latest`. they can be exploited. This proactive approach helps ```sh maintain a secure application environment. trivy image nginx:latest ``` 4. Cost-Effective Setup: Using Minikube on an EC2 instance as a testing 3. Automate Scanning in CI/CD: environment offers a cost-effective alternative to managed - Integrate Trivy into your CI/CD pipeline to automate Kubernetes services like EKS. This setup allows for image scanning. For example, add a step in your GitHub comprehensive testing and validation of security policies in Actions workflow: a controlled, affordable manner. ```yaml name: Scan Docker image for vulnerabilities 8. Comparison of approaches for on: [push] jobs: Kubernetes cluster security scan: This table compares three approaches to security in runs-on: ubuntu-latest Kubernetes environments: Security as Code with OPA steps: Gatekeeper, Traditional Security Methods, and Admission - name: Checkout code Controllers. The comparison is made based on various uses: actions/checkout@v2 criteria, highlighting the strengths and weaknesses of each - name: Set up Trivy approach based on the aforementioned research. run: | sudo apt-get install wget apt-transport-https 9. Implications in practice gnupg lsb-release -y wget -qO - https://aquasecurity.github.io/trivy- While this research provides a robust framework for repo/deb/public.key | sudo apt-key add - implementing SaC in Kubernetes environments, several echo deb https://aquasecurity.github.io/trivy- areas warrant further investigation: repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list  Service Mesh Integration: Exploring the sudo apt-get update integration of service mesh solutions like Istio to sudo apt-get install trivy -y enhance security, observability, and traffic - name: Scan image management within Kubernetes clusters [26–27]. run: trivy image nginx:latest The topic of my next research will be centered around ``` the benefits of fine-grained access control by integrating Through our study, we demonstrated the practical Open Policy Agent (OPA) with Istio. This integration allows implementation of SaC using Open Policy Agent (OPA) and for a more detailed and context-aware approach to access its Gatekeeper component, integrated with FluxCD for control within a Service Mesh architecture. Unlike continuous deployment and Trivy for container image traditional access control mechanisms, fine-grained access vulnerability scanning. The research provided a detailed, control enables policies that consider multiple attributes, step-by-step guide for setting up a cost-effective testing including request context, user roles, resource types, and environment on AWS using Minikube, making it accessible more. This level of granularity is crucial for implementing for practitioners and researchers alike [24–25]. robust security measures, especially in complex Key findings microservices environments [28]. The research underscores the importance of adopting a 1. Automated and Consistent Security Policies: security-as-code approach in modern cloud-native By defining security policies as code, organizations can environments. By embedding security directly into the ensure consistent enforcement across all stages of the development and deployment processes, organizations can application lifecycle. OPA Gatekeeper enables fine-grained achieve a higher level of security automation, reduce the control over Kubernetes resources, preventing attack surface, and improve overall resilience. The findings misconfigurations and unauthorized changes. provide practical insights and best practices that can be leveraged by DevOps teams, security engineers, and IT 2. Seamless Integration with CI/CD Pipelines: professionals to enhance the security of their Kubernetes Integrating security checks into CI/CD pipelines using deployments. FluxCD ensures that security is continuously maintained without hindering development velocity. Automated deployments and policy enforcement reduce the risk of human error and accelerate the development process. 66 Table 1 SaC with OPA Gatekeeper vs. Traditional Security Methods vs. Admission Controllers in Kubernetes Clusters Security as Code with OPA Criteria Traditional Security Methods Admission Controllers Gatekeeper Policy Management Centralized and consistent policy Often decentralized, policies may Centralized management, management through code be managed manually or with predefined rules scripts Automation The high degree of automation in Varies, and often requires Automated enforcement of policy enforcement and compliance manual intervention or custom predefined policies checks automation scripts Integration with Seamless integration with CI/CD Integration may require Limited integration, primarily CI/CD pipelines for continuous security additional effort and custom focused on runtime tooling enforcement Flexibility Highly flexible, supports complex Limited flexibility, often Moderate flexibility, and fine-grained policies using Rego constrained by predefined rules dependent on the admission language and configurations controller's capabilities Scalability Scales well with Kubernetes clusters, Scalability varies and may face Generally scales well, but can suitable for large environments challenges in large or dynamic add latency environments Visibility and Enhanced visibility and auditing Basic visibility often lacks Basic visibility, some support Monitoring capabilities for policy violations detailed auditing features for logging violations Complexity Initial setup and policy definition can Generally simpler setup but may Moderate complexity, setup be complex lack advanced features and configuration can vary Vendor Lock-in Open-source and vendor-neutral Varies, some methods may Generally open-source or involve vendor-specific solutions native to Kubernetes Community and Strong community support, Varies, proprietary solutions Strong community support Support extensive documentation and may have limited community for popular admission tutorials support controllers Real-time Enforces policies at admission Enforcement may be reactive, Real-time enforcement at Enforcement control, ensuring compliance before relying on post-deployment resource creation and deployment scans and checks modification Resource Overhead Low to moderate, depending on the Varies, can be high depending on Low to moderate, depending complexity of policies the security tools used on the controller and policies enforced Adaptability Easily adaptable to new security Adaptability depends on the Moderate adaptability, requirements and evolving threats flexibility of the chosen security predefined rules may need method updates for new threats Table 2 Pros and Cons of SaC with OPA Gatekeeper Pros Cons Centralized and consistent policy management through code Initial setup and policy definition can be complex The high degree of automation in policy enforcement and compliance Requires learning Rego language for policy writing checks Seamless integration with CI/CD pipelines for continuous security Potential performance impact on admission control with complex policies Highly flexible, supports complex and fine-grained policies using Rego May require significant time investment for initial language configuration and policy creation Enhanced visibility and auditing capabilities for policy violations Debugging policy violations can be challenging Strong community support, extensive documentation, and tutorials Need for ongoing maintenance to keep policies up to date 10. Conclusion with the dynamic and ephemeral nature of cloud-native environments. This gap has led to the emergence of the The introduction of cloud-native applications has marked a “Security as Code” (SaC) paradigm, which aims to embed significant shift in the landscape of software development security directly into the development and operational and deployment. These applications, designed to leverage processes through code. the advantages of cloud computing, offer unparalleled Security as Code involves defining security policies and scalability, flexibility, and resilience. At the heart of this controls as code, allowing them to be versioned, reviewed, transformation is Kubernetes, an open-source container and deployed alongside application code. This approach orchestration platform that has become the de facto ensures that security measures are consistently applied and standard for deploying, scaling, and managing containerized automatically enforced across all environments, from applications. development to production. By integrating security into the While Kubernetes simplifies many aspects of application DevOps pipeline, organizations can achieve continuous management, it also introduces new security challenges. security and compliance, reducing the risk of vulnerabilities Traditional security practices often struggle to keep pace and misconfigurations. 67 This research focuses on implementing the SaC approach continuous enforcement of up-to-date security measures. within Kubernetes clusters, leveraging Open Policy Agent This capability is essential for maintaining a strong security (OPA) and its Gatekeeper component. OPA is a general- posture in an ever-evolving threat landscape. Through this purpose policy engine that enables the enforcement of fine- research, we have demonstrated the practical grained, context-aware policies. Gatekeeper extends OPA’s implementation and effectiveness of the Security as Code capabilities by integrating with Kubernetes admission approach in Kubernetes environments, offering insights into controllers, allowing policies to be enforced at the time of best practices and potential challenges. resource creation and modification. Additionally, this study By integrating OPA Gatekeeper, FluxCD, and Trivy, we incorporates FluxCD, a continuous delivery tool for have established a comprehensive framework for Kubernetes, and Trivy, a comprehensive vulnerability automating security policy enforcement and continuous scanner for container images. By combining these tools, we monitoring of container vulnerabilities. This integrated aim to create a robust framework for automating security approach not only addresses the inherent challenges of policy enforcement and continuous monitoring of container dynamic and ephemeral container environments but also vulnerabilities. ensures continuous security and compliance throughout the The swift adoption of Kubernetes as a container development lifecycle of cloud-native applications. Our orchestration platform has revolutionized the deployment findings contribute to the advancement of secure and management of cloud-native applications. However, Kubernetes deployments and provide a robust foundation this shift has also introduced significant security challenges for future research and practical implementations in the that traditional security approaches are ill-equipped to field of cloud-native security. address. The need for dynamic, automated, and scalable security measures has become crucial. The primary problem References this research addresses is the implementation of a “Security as Code” (SaC) approach in Kubernetes-based cloud [1] B. Burns, et al., Borg, Omega, and Kubernetes, Queue environments. 14(1) (2016) 70–93. doi: 10.1145/2898442.2898444. Through this research, we aim to explore and address [2] B. Creane, A. Gupta, Kubernetes Security and these challenges by providing a detailed implementation Observability, O'Reilly Media (2016). guide, evaluating the effectiveness of automated security [3] R. Osnat, Kubernetes Security Basics and 10 Essential policies, and offering practical insights into integrating Best Practices (2020). URL: security as code into Kubernetes-based workflows. By doing https://www.aquasec.com/cloud-native- so, we contribute to the broader discourse on enhancing the academy/kubernetes-in-production/kubernetes- security of cloud-native applications through innovative security-best-practices-10-steps-to-securing-k8s/ code-centric approaches. [4] M. Isberner, 11 Kubernetes Admission Controller Best The arrival of cloud-native applications and the Practices for Security (2019). URL: widespread adoption of Kubernetes have ushered in a new https://www.redhat.com/en/blog/11-kubernetes- era of software development and deployment. While these admission-controller-best-practices-for-security technologies offer significant benefits in terms of scalability, [5] J. Ray, Policy as Code (2018). URL: flexibility, and resilience, they also present unique security https://www.oreilly.com/library/view/policy-as- challenges that traditional security practices are often ill- code/9781098139179/ch04. html equipped to handle. This research aimed to explore and [6] Md S. Shamim, F. A. Bhuiyan, A. Rahman, XI implement a “Security as Code” (SaC) approach within Commandments of Kubernetes Security: A Kubernetes environments to address these challenges Systematization of Knowledge Related to Kubernetes effectively. Security Practices, IEEE Secure Development (SecDev) By leveraging OPA’s powerful policy language, Rego, (2020) 58–64. doi: 10.1109/SecDev45635.2020.00025. organizations can define precise access control policies that [7] V. Khoma, et al., Comprehensive Approach for are dynamically enforced across the Kubernetes ecosystem. Developing an Enterprise Cloud Infrastructure, in: This not only enhances security by ensuring that only Cybersecurity Providing in Information and authorized requests are permitted but also allows for the rapid Telecommunication Systems, vol. 3654, (2024) 201– adaptation of policies in response to emerging threats or 215. changes in compliance requirements. Additionally, the [8] C. Pahl, et al., Cloud Container Technologies: A State- separation of policy logic from application code simplifies the of-the-Art Review, IEEE Transactions on Cloud development process, allowing developers to focus on Computing 7 (2024) 677–692. doi: /10.1109/TCC.201 business functionality while security teams manage access 7.2702586. policies independently. [9] M. Alawneh, I. Abbadi, Expanding DevSecOps Furthermore, the ability to update policies without Practices and Clarifying the Concepts within redeploying services ensures minimal disruption and Kubernetes Ecosystem, Ninth International 68 Conference on Software Defined Systems (SDS) (2022) [23] G. Olaoye, A. Luz, DevSecOps and Integrating Security 1–7. doi: 10.1109/SDS57574.2022.1006 2874. into the Cloud Development Lifecycle (2024). URL: [10] S. Vasylyshyn, et al., A Model of Decoy System Based https://www.researchgate.net/publication/378233643_ on Dynamic Attributes for Cybercrime Investigation, DevSecOps_and_integrating_security_into_the_cloud Eastern-European J. Enterp. Technol. 1 (9(121)) (2023) _development_lifecycle 6–20. doi: 10.15587/1729-4061.2023.273363. [24] G. Sayfan, Policy as Code and the Open Policy Agent [11] The Linux Foundation, Official Kubernetes (2022). URL: documentation (2024). URL: https://blogs.cisco.com/developer/policyascode01 https://kubernetes.io/docs/concepts/overview/compo [25] SecureFlag blog, Securing Kubernetes: Using nents/#control-plane-components Gatekeeper to Enforce Effective Security Policies [12] Veritis, An Advanced Approach for Deploying (2024). URL: Containerized Applications in a Cloud Environment https://blog.secureflag.com/2024/03/13/security- (2024). URL: policy-enforcement-in-kubernetes/ https://www.veritis.com/solutions/devops/kubernetes/ [26] Taikun, A Beginner’s Guide to Istio: A Service Mesh [13] The Linux Foundation, Official Kubernetes for Kubernetes (2024). URL: documentation (2024). URL: https://taikun.cloud/beginner-guide-to-istio-service- https://kubernetes.io/docs/concepts/security/ mesh-for-kubernetes/ [14] The Linux Foundation, Cloud Native Security and [27] AquaSecurity, Service Mesh: Architec-ture, Concepts, Kubernetes (2024). URL: and Top 4 Frameworks (2021). URL: https://kubernetes.io/docs/concepts/security/cloud- https://www.aquasec.com/cloud-native- native-security/ academy/container-security/service-mesh/ [15] Y. Martseniuk, Automated Conformity Verification [28] R. Chandramouli, Z. Butcher, A Zero Trust Concept for Cloud Security, in: Cybersecurity Architecture Model for Access Control in Cloud- Providing in Information and Telecommunication Native Applications in Multi-Location Environments, Systems, vol. 3654 (2024) 25–37. NIST (2023). doi: 10.6028/NIST.SP.800-207A. [16] D. Bose, A. Rahman, M. Shamim, ‘Under-reported’ Security Defects in Kubernetes Manifests, IEEE/ACM 2nd International Workshop on Engineering and Cybersecurity of Critical Systems (EnCyCriS) (2021) 9–12. doi: /10.1109/EnCyCriS52570.2021.00009. [17] Sultan, S., Ahmad, I., & Dimitriou, T. “Container Security: Issues, Challenges, and the Road Ahead”, (2019), IEEE Access, 7, 52976-52996. doi: 10.1109/ACCESS.2019.2911732. [18] O. Vakhula, et al., Security-As-Code Concept for Fulfilling ISO/IEC 27001:2022 Requirements, in: Cybersecurity Providing in Information and Telecommunication Systems, vol. 3654 (2024) 59–72. [19] O. Vakhula, I. Opirskyy, O. Mykhaylova, Research on Security Challenges in Cloud Environments and Solutions based on the “Security-As-Code” Approach, in: Cybersecurity Providing in Information and Telecommunication Systems, vol. 3550 (2023) 55–69. [20] Guest Expert on GitGuardian blog, What is Policy-as- Code? An Introduction to Open Policy Agent (2020). URL: https://blog.gitguardian.com/what-is-policy-as- code-an-introduction-to-open-policy-agent/ [21] OPA Official documentation (2024). URL: https://www.openpolicyagent.org/docs/latest/kubern etes-tutorial/ [22] S. Ragonessi, Secure your Kubernetes environment with OPA and Gatekeeper (2023). URL: https://www.cncf.io/blog/ 2023/10/09/secure-your- kubernetes-environment-with-opa-and-gatekeeper/ 69