Skip to content

Restricted Roles & Identities

Restricted Roles and Identities Required for Creating an AKS Cluster

In Azure Kubernetes Service (AKS), clusters require an identity to access Azure resources. This identity can either be a managed identity (recommended) or a service principal.

In this discussion, we will delve into bringing your own managed identities and outline the minimum restricted roles and identities needed to create an AKS cluster with Azure Container Registry (ACR) and custom Virtual Network (VNET) configuration.

Another example is if you want to use Azure Monitor to get logs and metrics from your cluster. The pods responsible for this task need to collect logs from the cluster and send them to Azure Monitor. To do this, they need to be authenticated and authorized. One solution is to implement another managed identity to authorize the sending of data to Azure Monitor.

Restricted Roles and Identities

When creating a cluster through Rafay Platform with a restricted set of roles, we need the following identities and roles:

  • Rafay's Service Principal (SPN): Requires the Contributor role specifically for the cluster resource group.

In addition to this, we need to create managed identities.

For example, if you need to create an AKS cluster with ACR and a custom VNET, you need the following managed identities:

  • Managed Identities:

    • Kubelet Identity: Assign ACR Pull permission to this scoped to the ACRs from which the cluster would pull images.This will be used by kubelet Component, which is why it's called Kubelet identity.

    • Cluster Managed Identity:

      • Assign Network Contributor Role to this scoped to the virtual network where the cluster is deployed (this can also be scoped to individual subnets).
      • Assign Managed Identity Operator Role to this scoped to the Kubelet Identity that we created previously.

AKS Identity

After successfully creating the roles and identities on Azure, we need to utilize them in the cluster when it is created through our Rafay controller.

Here is the cluster configuration that you can use as a reference, where a User Assigned identity along with kubelet identity is configured.

  1. aks-identity: Used for the cluster resource group.
  2. aks-kubelet-identity: Required for kubelet to Pull from ACR.
apiVersion: rafay.io/v1alpha1
kind: Cluster
metadata:
  name: aks-demo-test2
  project: ankurp
spec:
  blueprint: default-aks
  cloudprovider: demo-ankurp-app
  clusterConfig:
    apiVersion: rafay.io/v1alpha1
    kind: aksClusterConfig
    metadata:
      name: aks-demo-test2
    spec:
      managedCluster:
        apiVersion: "2022-07-01"
        identity:
          type: UserAssigned
          userAssignedIdentities:
            ? /subscriptions/a2252eb2-7a25-432b-a5ec-e18eba6f26b1/resourceGroups/ankurpmanaged/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-identity
            : {}
        location: centralindia
        properties:
          identityProfile:
            kubeletidentity:
               resourceId: /subscriptions/a2252eb2-7a25-432b-a5ec-e18eba6f26b1/resourceGroups/ankurpmanaged/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-kubelet-identity
          apiServerAccessProfile:
            enablePrivateCluster: true
          dnsPrefix: aks-demo-test2-dns
          kubernetesVersion: 1.27.1
          networkProfile:
            dnsServiceIP: 10.0.0.10
            dockerBridgeCidr: 172.17.0.1/16
            loadBalancerSku: standard
            networkPlugin: azure
            networkPolicy: azure
            outboundType: loadBalancer
            serviceCidr: 10.0.0.0/16
        sku:
          name: Basic
          tier: Free
        type: Microsoft.ContainerService/managedClusters
      nodePools:
      - apiVersion: "2022-07-01"
        location: centralindia
        name: primary
        properties:
          count: 1
          enableAutoScaling: true
          maxCount: 6
          maxPods: 110
          minCount: 1
          mode: System
          orchestratorVersion: 1.27.1
          osType: Linux
          type: VirtualMachineScaleSets
          vmSize: Standard_B4ms
          vnetSubnetID: /subscriptions/a2252eb2-7a25-432b-a5ec-e18eba6f26b1/resourceGroups/networkingrg/providers/Microsoft.Network/virtualNetworks/vnet-network-ap/subnets/default1
        type: Microsoft.ContainerService/managedClusters/agentPools
      resourceGroupName: ankurpmanaged
  proxyconfig: {}
  type: aks

With the above identities and roles, you will be able to create an AKS cluster with a custom VNET and an ACR, like this: Depending on the specific resources you intend to use, you will need to create the necessary managed identities as outlined in the Azure documentation: https://learn.microsoft.com/en-us/azure/aks/use-managed-identity.