Skip to content

Overview

This reference design and implementation will help users provision a RBAC controlled, dedicated operating environment aka project for an application team/business unit based on one or many Kubernetes clusters (EKS, AKS, GKE or Upstream Kubernetes in datacenters). We will then run a number of day-two operations on our cluster.

The lifecycle of all resources will be managed using Terraform Infrastructure as Code based on Rafay's Terraform Provider. 

sequenceDiagram
    participant pipe as External Pipeline <br> OR <br> Rafay Environment Manager 
    participant tf as Rafay Terraform <br>Provider
    participant rafay as Rafay Org
    participant csp as Infra Provider

    pipe->>tf: New Operating Environment 
    tf-->>rafay: Create Project 
    tf-->>rafay: Create User Group
    tf-->>rafay: Map Roles to Group (RBAC)
    tf-->>rafay: Create Cloud Credential
    tf-->>rafay: Add Git/Helm Repositories <br> (for Add-Ons)
    tf-->>rafay: Create Namespaces <br> (for Add-Ons)
    tf-->>rafay: Create Add-ons <br> (nginx, cert-manager)
    tf-->>rafay: Create OPA Constraint Templates
    tf-->>rafay: Create OPA Constraints
    tf-->>rafay: Create OPA Policy
    tf-->>rafay: Create Custom Blueprint <br> (Managed and Custom Add-ons)
    tf-->>rafay: Add Cluster Overrides 
    rafay->>csp: Provision Cluster
    rafay->>csp: Apply Custom Blueprint <br> (with Cluster Override)
    csp-->>rafay: Blueprint Successful 
    rafay->>tf: Cluster Ready
    tf->>pipe: Operating Environment Ready

Background

It is a relatively common pattern for platform teams to provide downstream application teams with a dedicated operating environment.

Day-1

  • This environment maps to a project in a Rafay Org
  • It is common for organizations to require "separation of duties" esp. between infrastructure operations and application operations. So, we will configure user access to the project for specific users/groups based on RBAC and/or ABAC rules
  • Cloud credentials are automatically created in the project which is then used to provision at least one Kubernetes cluster in the specified infrastructure provider
  • It is common for organizations to standardize their clusters with a required baseline (aka cluster blueprint) comprising several software addons. We will create add-ons and assemble a blueprint based on the add-ons. We will then apply the blueprint to the cluster will provision.
  • Corporate policies are typically put in place to provide security, operational, and reliability guardrails. OPA constraints and constraint templates will be defined to deploy a profile that will validate all cluster operations and manifests are operating within policy.

Day-2

  • The cluster may need to be scaled up/down as per the needs of the application team
  • The cluster may need to be upgraded to a new Kubernetes version to ensure that it is supported by the provider

Items Out of Scope

Although possible using Rafay, this reference design does not include the following items. We have provided pointers below to content that can be used to address this.

  • The user is expected to embed the working Terraform code in their pipeline.
  • Creation and management of namespaces and/or workspaces for application teams
  • How application teams can deploy/operate their containerized applications on the Kubernetes cluster.

Roadmap

This reference design is constantly evolving. We plan to progressively enhance the design with additional functionality based on our roadmap and customer feedback. Please watch this space or our product blogs for updates.