Workload Identity Overview & Configuration

Overview¶

Workload identity allows Kubernetes workloads to securely access Azure Active Directory (Azure AD) protected resources such as Azure Key Vault, Azure Storage, Azure SQL Database, Azure Monitor, and Azure Cosmos DB. This identity solution leverages Azure AD-based managed identities via a workload identity workflow. Azure AD Workload Identity overcomes the limitations of the deprecated Azure AD Pod Identity, such as scale and performance issues with identity assignment.

Users can create a workload identity during cluster provisioning (Day 0). For existing clusters, the workload identity can be created on Day 2 to integrate with the existing infrastructure.

Did you know ?

Workload Identity can be managed through various methods:

Important

Currently, there is no UI support. UI integration is planned for an upcoming release

Pre-requisite¶

A Kubernetes cluster with version ≥ v1.20
AKS Workload Identity is supported for existing clusters (Day 2 operations) with an API version of 2023-03-01 or earlier that are already present on the platform. However, for new clusters (Day 0 operations) requiring Workload Identity support, an API version of 2023-03-01 or later is mandatory, as clusters with Workload Identity or OIDC feature flags enabled and an API version earlier than 2023-03-01 are not supported
For existing clusters with an older API version, users can still leverage it by using the workload identity configuration provided below

User vs. Rafay-Managed Identity and Role Assignment¶

Flow 1: User-Provided Identity and Role Assignments¶

In this flow, users interact with Rafay by providing managed identity and role-related details. There are several scenarios to consider:

Scenario 1 User Provides Identity Name and Status Attributes - createIdentity=False is set: Rafay does not create the identity. Instead, it works with the provided identity

Scenario 2 User Shares Identity Specification Parameters - createIdentity=True is set: Rafay creates the identity and works with the newly created identity

Scenario 3 User provides Role Assignment specification list - Rafay performs the necessary role assignments on behalf of the user, provided that the relevant permissions (Role Assignment Write and Delete Permissions) are assigned to the Service Principal Name (SPN) used to create the cloud credentials for the request. This is crucial for enabling role assignment under Workload Identity support - Details Provided: Role assignment specification list, including: - Role ID and Role Name: Either the specific identifier (Role ID) or the name of the role - Scope: The resource that the managed identity needs to access from within the cluster using the service account. This defines the permissions and boundaries applicable for the resource

Scenario 4 User does not provide Role Assignment list - Rafay assumes that the identity already has the necessary role assignments and does not perform any role assignments

Scenario 5

User provides Service Account as specification parameters - createAccount=True is set: Rafay creates the service account.

Scenario 6 User provides Service Account as reference attributes - createAccount=False is set: Rafay does not create the service account but references the existing service account.

Important

A single workload identity can be associated with multiple service accounts, but one service account cannot be associated with more than one workload identity

Flow 2: Rafay-Created Identity, Role Assignments and Service Accounts¶

In this flow, Rafay is responsible for creating the identity, assigning roles, and managing service accounts. There are a few scenarios here as well:

Scenario 1 Rafay creates Workload Identity and assigns roles - Identity Creation: Rafay establishes a new identity - Role Assignments: Rafay applies roles as part of the identity creation process

Scenario 2 Service Account details provided by users - Users supply detailed information for service accounts, and Rafay uses these details to create the service accounts as needed. - createAccount=True is set: Rafay creates the service account.

Important

Workload Identity supports the creation of service accounts. However, automount service accounts are not supported

Workload Identity Via RCTL¶

V3 Config Spec (Recommended)

Below is an AKS configuration specification with workload identity parameters, role assignments, and a service account.

apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
  name: demo-aks-cluster
  project: defaultproject
spec:
  blueprintConfig:
    name: minimal
  cloudCredentials: demo-azure-creds
  type: aks
  config:
    kind: aksClusterConfig
    metadata:
      name: demo-aks-cluster
    spec:
      managedCluster:
        apiVersion: '2024-01-01'
        identity:
          type: SystemAssigned
        location: centralindia
        properties:
          apiServerAccessProfile:
            enablePrivateCluster: true
            privateDNSZone: System
          dnsPrefix: demo-test-dns
          enableRBAC: true
          kubernetesVersion: 1.28.9
          networkProfile:
            loadBalancerSku: Standard
            networkPlugin: kubenet
          powerState:
            code: Stopped
          securityProfile:
            workloadIdentity:
              enabled: true
          oidcIssuerProfile:
            enabled: true
        type: Microsoft.ContainerService/managedClusters
      nodePools:
      - type: Microsoft.ContainerService/managedClusters/agentPools
        apiVersion: '2023-11-01'
        name: primary
        location: centralindia
        properties:
          count: 2
          enableAutoScaling: true
          maxCount: 2
          maxPods: 40
          minCount: 1
          mode: System
          orchestratorVersion: 1.28.9
          osType: Linux
          type: VirtualMachineScaleSets
          vmSize: Standard_DS2_v2
      resourceGroupName: demo-rg-ci
      workloadIdentities:
      - createIdentity: false
        metadata:
          clientId: 27514982-9cca-43f6-996d-100b5a641beb
          principalId: 72308b4b-5f25-4359-b52d-07c9b0891141
          name: demo-uai-1
          location: centralindia
          resourceGroup: demo-rg-ci
          tags:
            example.com/owner: "demo"
            example.com/department: "demo"
        roleAssignments:
        - name: "Key Vault Secrets User"
          roleDefinitionId: "8e3af657-a8ff-443c-a75c-2fe8c4bcb635"
          scope: "/subscriptions/a2252eb2-7a25-432b-a5ec-e18eba6f26b1/resourceGroups/demo-rg-ci/providers/Microsoft.KeyVault/vaults/demo-keyvault"
        serviceAccounts:
        - createAccount: false
          metadata:
            name: demo-aks-cluster-sa
            namespace: default
            annotations:
              example.com/role: "demo-aks-cluster-sa"
            labels:
              example.com/owner: "demo"
              example.com/department: "demo"

V1 Config Spec

apiVersion: rafay.io/v1alpha1
kind: Cluster
metadata:
  name: demo-aks-cluster
  project: demo
spec:
  blueprint: minimal
  cloudprovider: demo-azure-creds
  clusterConfig:
    apiVersion: rafay.io/v1alpha1
    kind: aksClusterConfig
    metadata:
      name: demo-aks-cluster
    spec:
      managedCluster:
        apiVersion: "2022-07-01"
        identity:
          type: SystemAssigned
        location: centralindia
        properties:
          apiServerAccessProfile:
            enablePrivateCluster: true
          dnsPrefix: demo-aks-cluster-dns
          enableRBAC: true
          kubernetesVersion: 1.27.7
          networkProfile:
            loadBalancerSku: standard
            networkPlugin: kubenet
          powerState:
            code: Running
          oidcIssuerProfile:
            enabled: true
          securityProfile:
            workloadIdentity:
              enabled: true
        sku:
          name: Basic
          tier: Free
        type: Microsoft.ContainerService/managedClusters
      nodePools:
      - apiVersion: "2022-07-01"
        location: centralindia
        name: primary
        properties:
          count: 1
          enableAutoScaling: true
          maxCount: 1
          maxPods: 110
          minCount: 1
          mode: System
          orchestratorVersion: 1.27.7
          osType: Linux
          type: VirtualMachineScaleSets
          vmSize: Standard_DS2_v2
        type: Microsoft.ContainerService/managedClusters/agentPools
      resourceGroupName: demo-rg-ci
      workloadIdentities:
      - createIdentity: false
        metadata:
          clientId: 27514982-9cca-43f6-996d-100b5a641beb
          principalId: 72308b4b-5f25-4359-b52d-07c9b0891141
          name: demo-uai-1
          location: centralindia
          resourceGroup: demo-rg-ci
          tags:
            example.com/owner: "demo"
            example.com/department: "demo"
        roleAssignments:
        - name: "Key Vault Secrets User"
          roleDefinitionId: "8e3af657-a8ff-443c-a75c-2fe8c4bcb635"
          scope: "/subscriptions/a2252eb2-7a25-432b-a5ec-e18eba6f26b1/resourceGroups/demo-rg-ci/providers/Microsoft.KeyVault/vaults/demo-keyvault"
        serviceAccounts:
        - createAccount: false
          metadata:
            name: demo-aks-cluster-sa
            namespace: default
            annotations:
              example.com/role: "demo-aks-cluster-sa"
            labels:
              example.com/owner: "demo"
              example.com/department: "demo"
  type: aks

Workload Identity Via UI¶

Users can configure the Workload Identity using the Save & Customize option. During the cluster creation process, enter the required details and click the Save & Customize button. After providing the configuration spec, click Save Changes.

Create AKS Cluster

Important

Once workload identity is deployed, changes to the workload identity configuration will not automatically reflect in existing workloads. To apply the new identity settings and ensure that all configurations are up-to-date, affected workloads will need to be redeployed. This process involves updating and redeploying K8s pods or deployments to apply the new workload identity settings.
Updating the labels or annotations of the existing service accounts is not supported.

Workload Identity Takeover as Part of AKS Cluster Takeover¶

Workload identity takeover ensures seamless access and continuity by transferring workload identities between AKS clusters during migration or conversion, preserving configurations and permissions.

Currently, workload identity takeover is supported only if the associated service accounts have been discovered
Workload identity Day 2 operations will function on a cluster that has undergone a takeover

Once workload identity is deployed, changes to the workload identity configuration will not automatically reflect in existing workloads. To apply the new identity settings and ensure that all configurations are up-to-date, affected workloads will need to be redeployed. This process involves updating and redeploying Kubernetes pods or deployments to apply the new workload identity settings.

Note: Support for service account takeover and workload identity takeover is planned for an upcoming release.

Alternative Path for Users¶

There is an alternative path currently available that ensures customers can take part of transition:

Take Over the Cluster with Workload Identity Feature Flags: Users can take over the cluster with the workload identity (WI) feature flags either enabled or disabled.
Enable Workload Identity Feature Post Takeover: After taking over the cluster, users can enable the WI feature if it was previously disabled.
Soft Create Workload Identities: Users can soft create the workload identities using metadata from the existing workload identities (WIs).
Day 2 Operations Availability: All Day 2 operations that are available with Rafay created clusters are also available with the taken over cluster.

This alternative path allows for flexibility and continuity in managing workload identities during cluster transition.

To learn more about the use cases and best practices, you can refer to the Workload Identity Blog.

Known Issues¶

To view the known issues for the 2.10 release of the Workload Identity feature, please refer to Known Issues