Workload Identity

Overview

Workload identity allows Kubernetes workloads to securely access Azure Active Directory (Azure AD) protected resources such as Azure Key Vault, Azure Storage, Azure SQL Database, Azure Monitor, and Azure Cosmos DB. This identity solution leverages Azure AD-based managed identities via a workload identity workflow. Azure AD Workload Identity overcomes the limitations of the deprecated Azure AD Pod Identity, such as scale and performance issues with identity assignment.

Users can create a workload identity during cluster provisioning (Day 0). For existing clusters, the workload identity can be created on Day 2 to integrate with the existing infrastructure.

Did you know ?

Workload Identity can be managed through various methods:

• CLI - V3 and V1
• UI (Save & Customize option)
• Terraform - V3 and V1
• GitOps
• Rafay-hub APIs

Important

Currently, there is no UI support. UI integration is planned for an upcoming release

---

Pre-requisite

• A Kubernetes cluster with version ≥ v1.20
• Support for AKS Workload Identity is available only for clusters with api_version ≥ 2023-03-01 in the cluster spec or for newly created clusters

---

User vs. Rafay-Managed Identity and Role Assignment

Flow 1: User-Provided Identity and Role Assignments

In this flow, users interact with Rafay by providing managed identity and role-related details. There are several scenarios to consider:

Scenario 1 User Provides Identity Name and Status Attributes - createIdentity=False is set: Rafay does not create the identity. Instead, it works with the provided identity

Scenario 2 User Shares Identity Specification Parameters - createIdentity=True is set: Rafay creates the identity and works with the newly created identity

Scenario 3 User provides Role Assignment specification list - Rafay performs the necessary role assignments on behalf of the user - Details Provided: Role assignment specification list, including: - Role ID and Role Name: Specific identifiers or names of the roles - Scope: The resource that the managed identity needs to access from within the cluster using the service account. This defines the permissions and boundaries applicable for the resource

Scenario 4 User does not provide Role Assignment list - Rafay assumes that the identity already has the necessary role assignments and does not perform any role assignments

Scenario 5

User provides Service Account as specification parameters - createAccount=True is set: Rafay creates the service account.

Scenario 6 User provides Service Account as reference attributes - createAccount=False is set: Rafay does not create the service account but references the existing service account.

---

Flow 2: Rafay-Created Identity, Role Assignments and Service Accounts

In this flow, Rafay is responsible for creating the identity, assigning roles, and managing service accounts. There are a few scenarios here as well:

Scenario 1 Rafay creates Workload Identity and assigns roles - Identity Creation: Rafay establishes a new identity - Role Assignments: Rafay applies roles as part of the identity creation process

Scenario 2 Service Account details provided by users - Users supply detailed information for service accounts, and Rafay uses these details to create the service accounts as needed. - createAccount=True is set: Rafay creates the service account.

Important

Workload Identity supports the creation of service accounts. However, automount service accounts are not supported

---

Workload Identity Via RCTL

V3 Config Spec (Recommended)

Below is an AKS configuration specification with workload identity parameters, role assignments, and a service account.

apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
  name: demo-aks-cluster
  project: defaultproject
spec:
  blueprintConfig:
    name: minimal
  cloudCredentials: demo-azure-creds
  type: aks
  config:
    kind: aksClusterConfig
    metadata:
      name: demo-aks-cluster
    spec:
      managedCluster:
        apiVersion: '2024-01-01'
        identity:
          type: SystemAssigned
        location: centralindia
        properties:
          apiServerAccessProfile:
            enablePrivateCluster: true
            privateDNSZone: System
          dnsPrefix: demo-test-dns
          enableRBAC: true
          kubernetesVersion: 1.28.9
          networkProfile:
            loadBalancerSku: Standard
            networkPlugin: kubenet
          powerState:
            code: Stopped
          securityProfile:
            workloadIdentity:
              enabled: true
          oidcIssuerProfile:
            enabled: true
        type: Microsoft.ContainerService/managedClusters
      nodePools:
      - type: Microsoft.ContainerService/managedClusters/agentPools
        apiVersion: '2023-11-01'
        name: primary
        location: centralindia
        properties:
          count: 2
          enableAutoScaling: true
          maxCount: 2
          maxPods: 40
          minCount: 1
          mode: System
          orchestratorVersion: 1.28.9
          osType: Linux
          type: VirtualMachineScaleSets
          vmSize: Standard_DS2_v2
      resourceGroupName: demo-rg-ci
      workloadIdentities:
      - createIdentity: true
        metadata:
          clientId: 27514982-9cca-43f6-996d-100b5a641beb
          principalId: 72308b4b-5f25-4359-b52d-07c9b0891141
          name: demo-uai-1
          location: centralindia
          resourceGroup: demo-rg-ci
          tags:
            example.com/owner: "demo"
            example.com/department: "demo"
        roleAssignments:
        - name: "Key Vault Secrets User"
          roleDefinitionId: "8e3af657-a8ff-443c-a75c-2fe8c4bcb635"
          scope: "/subscriptions/a2252eb2-7a25-432b-a5ec-e18eba6f26b1/resourceGroups/demo-rg-ci/providers/Microsoft.KeyVault/vaults/demo-keyvault"
        serviceAccounts:
        - createAccount: true
          metadata:
            name: demo-aks-cluster-sa
            namespace: default
            annotations:
              example.com/role: "demo-aks-cluster-sa"
            labels:
              example.com/owner: "demo"
              example.com/department: "demo"

V1 Config Spec

apiVersion: rafay.io/v1alpha1
kind: Cluster
metadata:
  name: demo-aks-cluster
  project: demo
spec:
  blueprint: minimal
  cloudprovider: demo-azure-creds
  clusterConfig:
    apiVersion: rafay.io/v1alpha1
    kind: aksClusterConfig
    metadata:
      name: demo-aks-cluster
    spec:
      managedCluster:
        apiVersion: "2022-07-01"
        identity:
          type: SystemAssigned
        location: centralindia
        properties:
          apiServerAccessProfile:
            enablePrivateCluster: true
          dnsPrefix: demo-aks-cluster-dns
          enableRBAC: true
          kubernetesVersion: 1.27.7
          networkProfile:
            loadBalancerSku: standard
            networkPlugin: kubenet
          powerState:
            code: Running
          oidcIssuerProfile:
            enabled: true
          securityProfile:
            workloadIdentity:
              enabled: true
        sku:
          name: Basic
          tier: Free
        type: Microsoft.ContainerService/managedClusters
      nodePools:
      - apiVersion: "2022-07-01"
        location: centralindia
        name: primary
        properties:
          count: 1
          enableAutoScaling: true
          maxCount: 1
          maxPods: 110
          minCount: 1
          mode: System
          orchestratorVersion: 1.27.7
          osType: Linux
          type: VirtualMachineScaleSets
          vmSize: Standard_DS2_v2
        type: Microsoft.ContainerService/managedClusters/agentPools
      resourceGroupName: demo-rg-ci
      workloadIdentities:
      - createIdentity: true
        metadata:
          clientId: 27514982-9cca-43f6-996d-100b5a641beb
          principalId: 72308b4b-5f25-4359-b52d-07c9b0891141
          name: demo-uai-1
          location: centralindia
          resourceGroup: demo-rg-ci
          tags:
            example.com/owner: "demo"
            example.com/department: "demo"
        roleAssignments:
        - name: "Key Vault Secrets User"
          roleDefinitionId: "8e3af657-a8ff-443c-a75c-2fe8c4bcb635"
          scope: "/subscriptions/a2252eb2-7a25-432b-a5ec-e18eba6f26b1/resourceGroups/demo-rg-ci/providers/Microsoft.KeyVault/vaults/demo-keyvault"
        serviceAccounts:
        - createAccount: true
          metadata:
            name: demo-aks-cluster-sa
            namespace: default
            annotations:
              example.com/role: "demo-aks-cluster-sa"
            labels:
              example.com/owner: "demo"
              example.com/department: "demo"
  type: aks

---

Workload Identity Via UI

Users can configure the Workload Identity using the Save & Customize option. During the cluster creation process, enter the required details and click the Save & Customize button. After providing the configuration spec, click Save Changes.

Important

• Once workload identity is deployed, changes to the workload identity configuration will not automatically reflect in existing workloads. To apply the new identity settings and ensure that all configurations are up-to-date, affected workloads will need to be redeployed. This process involves updating and redeploying K8s pods or deployments to apply the new workload identity settings.

• Updating the labels or annotations of the existing service accounts is not supported.

---

Workload Identity Takeover as Part of AKS Cluster Takeover

Workload identity takeover ensures seamless access and continuity by transferring workload identities between AKS clusters during migration or conversion, preserving configurations and permissions.

• Currently, workload identity takeover is supported only if the associated service accounts have been discovered
• Workload identity Day 2 operations will function on a cluster that has undergone a takeover

Once workload identity is deployed, changes to the workload identity configuration will not automatically reflect in existing workloads. To apply the new identity settings and ensure that all configurations are up-to-date, affected workloads will need to be redeployed. This process involves updating and redeploying Kubernetes pods or deployments to apply the new workload identity settings.

Note: Support for service account takeover and workload identity takeover is planned for an upcoming release.

Alternative Path for Users

There is an alternative path currently available that ensures customers can take part of transition:

• Take Over the Cluster with Workload Identity Feature Flags: Users can take over the cluster with the workload identity (WI) feature flags either enabled or disabled.
• Enable Workload Identity Feature Post Takeover: After taking over the cluster, users can enable the WI feature if it was previously disabled.
• Soft Create Workload Identities: Users can soft create the workload identities using metadata from the existing workload identities (WIs).
• Day 2 Operations Availability: All Day 2 operations that are available with Rafay created clusters are also available with the taken over cluster.

This alternative path allows for flexibility and continuity in managing workload identities during cluster transition.

To learn more about the use cases and best practices, you can refer to the Workload Identity Blog.

---