KOP Cluster Template - CLI
For purposes of automation, it is strongly recommended that users create and manage version controlled "cluster templates" to provision and manage the lifecycle of clusters. This is well suited for scenarios where the multiple cluster creation is required with the same configuration
Step 1: Create Cluster Template¶
Command to create Cluster Template:
./rctl apply -f <clustertemplate.yaml>
Below is an example of yaml file to create a cluster template named demotest with the configuration details provided in Step 1
kind: ClusterTemplate
metadata:
name: demotest
description: 'Creating demo-template'
clusterType: aks
To create this cluster template, use the command
./rctl apply -f demo-template.yaml
Output
{
"id": "yemp028",
"name": "demotest",
"description": "Creating demo-template",
"cluster_type": "aks",
"created_at": "2022-01-25T09:52:19.061555Z",
"modified_at": "2022-01-25T09:52:19.061555Z",
"organization_id": "9dk3emn",
"partner_id": "rx28oml",
"project_id": "pkv40kn",
"request_type": "TEMPLATE_FILE",
"input_template": "kind: ClusterTemplate\nmetadata:\n clusterType: aks\n description: Creating demo-template\n name: demotest\n"
}
Step 2: Create Cluster Template Snapshot¶
Command to create cluster Template Snapshot:
./rctl apply -f <clustertemplatesnapshot.yaml>
Below is an example of a cluster template snapshot yaml file named "demo_template.yaml". This configuration file is to set the required parameters on the cluster template. Users can modify these parameters when required and create multiple versions
apiVersion: rafay.io/v1alpha1
kind: ClusterTemplateSnapshot
metadata:
description: desc
name: v0
project: defaultproject
templateName: aks-template
parameters:
aks_cluster_params_dns_prefix:
default: aks-dns
override:
allow: true
aks_cluster_params_location:
default: northcentralus
override:
allow: true
values:
- northcentralus
- southcentralus
aks_nodepool_params_1_name:
default: value1
override:
allow: true
values:
- value1
- value2
aks_cluster_params_kubernetes_version:
default: 1.22.11
override:
allow: true
values:
- 1.22.6
- 1.23.5
- 1.22.11
spec:
blueprint: minimal
blueprintversion: ""
cloudprovider: AKS-provider
cniprovider: aws-cni
managedCluster:
apiVersion: "2021-05-01"
identity:
type: SystemAssigned
location: "{{ aks_cluster_params_location }}"
properties:
apiServerAccessProfile:
enablePrivateCluster: true
dnsPrefix: "{{ aks_cluster_params_dns_prefix }}"
kubernetesVersion: 1.22.11
networkProfile:
loadBalancerSku: standard
networkPlugin: kubenet
sku:
name: Basic
tier: Free
type: Microsoft.ContainerService/managedClusters
nodePools:
- apiVersion: "2021-05-01"
location: northcentralus
name: primary
properties:
count: 1
enableAutoScaling: true
maxCount: 1
maxPods: 40
minCount: 1
mode: System
orchestratorVersion: 1.22.11
osType: Linux
type: VirtualMachineScaleSets
vmSize: Standard_DS2_v2
type: Microsoft.ContainerService/managedClusters/agentPools
- apiVersion: "2021-05-01"
location: northcentralus
name: "{{ aks_nodepool_params_1_name }}"
properties:
count: 1
enableAutoScaling: true
maxCount: 1
maxPods: 40
minCount: 1
mode: System
orchestratorVersion: 1.22.11
osType: Linux
type: VirtualMachineScaleSets
vmSize: Standard_B2ms
type: Microsoft.ContainerService/managedClusters/agentPools
resourceGroupName: resource-grp
type: azure-aks
Important
Users can parameterize the fields in the template with overrides and restricted values. In the above config file, the parameterized fields are maxSize, instanceType, region within the double curly braces
To create this cluster template snapshot, use the command
./rctl apply -f demo-template.yaml
Output
{
"id": "pkv40kn",
"template_id": "z247n2y",
"template_name": "aks-template",
"name": "v0",
"cluster_type": "azure-aks",
"created_at": "2022-09-13T10:13:32.797026Z",
"modified_at": "2022-09-13T10:13:32.797026Z",
"project_id": "rx28oml",
"partner_id": "rx28oml",
"organization_id": "7w2lnkp",
"request_type": "TEMPLATE_FILE",
"cloud_credential": "AKS-provider",
"blueprint_name": "minimal",
"gen_template": "apiVersion: rafay.io/v1alpha1\nkind: ClusterTemplateSnapshot\nmetadata:\n description: desc\n name: v0\n project: defaultproject\n templateName: aks-template\nparameters:\n aks_cluster_params_dns_prefix:\n default: aks-dns\n override:\n allow: true\n aks_cluster_params_kubernetes_version:\n default: 1.22.11\n override:\n allow: true\n values:\n - 1.22.6\n - 1.23.5\n - 1.22.11\n aks_cluster_params_location:\n default: northcentralus\n override:\n allow: true\n values:\n - northcentralus\n - southcentralus\n aks_nodepool_params_1_name:\n default: value1\n override:\n allow: true\n values:\n - value1\n - value2\nspec:\n blueprint: minimal\n blueprintversion: \"\"\n cloudprovider: AKS-provider\n cniprovider: aws-cni\n managedCluster:\n apiVersion: \"2021-05-01\"\n identity:\n type: SystemAssigned\n location: '{{ aks_cluster_params_location }}'\n properties:\n apiServerAccessProfile:\n enablePrivateCluster: true\n dnsPrefix: '{{ aks_cluster_params_dns_prefix }}'\n kubernetesVersion: 1.22.11\n networkProfile:\n loadBalancerSku: standard\n networkPlugin: kubenet\n sku:\n name: Basic\n tier: Free\n type: Microsoft.ContainerService/managedClusters\n nodePools:\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: primary\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_DS2_v2\n type: Microsoft.ContainerService/managedClusters/agentPools\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: '{{ aks_nodepool_params_1_name }}'\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_B2ms\n type: Microsoft.ContainerService/managedClusters/agentPools\n resourceGroupName: resource-grp\n type: azure-aks\n",
"input_template": "apiVersion: rafay.io/v1alpha1\nkind: ClusterTemplateSnapshot\nmetadata:\n description: desc\n name: v0\n project: defaultproject\n templateName: aks-template\nparameters:\n aks_cluster_params_dns_prefix:\n default: aks-dns\n override:\n allow: true\n aks_cluster_params_kubernetes_version:\n default: 1.22.11\n override:\n allow: true\n values:\n - 1.22.6\n - 1.23.5\n - 1.22.11\n aks_cluster_params_location:\n default: northcentralus\n override:\n allow: true\n values:\n - northcentralus\n - southcentralus\n aks_nodepool_params_1_name:\n default: value1\n override:\n allow: true\n values:\n - value1\n - value2\nspec:\n blueprint: minimal\n blueprintversion: \"\"\n cloudprovider: AKS-provider\n cniprovider: aws-cni\n managedCluster:\n apiVersion: \"2021-05-01\"\n identity:\n type: SystemAssigned\n location: '{{ aks_cluster_params_location }}'\n properties:\n apiServerAccessProfile:\n enablePrivateCluster: true\n dnsPrefix: '{{ aks_cluster_params_dns_prefix }}'\n kubernetesVersion: 1.22.11\n networkProfile:\n loadBalancerSku: standard\n networkPlugin: kubenet\n sku:\n name: Basic\n tier: Free\n type: Microsoft.ContainerService/managedClusters\n nodePools:\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: primary\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_DS2_v2\n type: Microsoft.ContainerService/managedClusters/agentPools\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: '{{ aks_nodepool_params_1_name }}'\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_B2ms\n type: Microsoft.ContainerService/managedClusters/agentPools\n resourceGroupName: resource-grp\n type: azure-aks\n",
"publish": true
}
Step 3: Create Cluster out of template¶
Command to create cluster out of template:
./rctl apply -f <cluster.yaml>
An example of a yaml file named demo-cluster.yaml to create a cluster using the template created in Step 2:
kind: Cluster
metadata:
description: "creating a demo cluster"
name: demo-cluster
project: defaultproject
spec:
clusterTemplateRef:
name: aks-template
version: v0
values:
aks_cluster_params_location: northcentralus
where name: demo-template is the selected template for the new cluster
To create a cluster out of template, use the command
./rctl apply -f demo-cluster.yaml
Output
{
"taskset_id": "dkgv8kx",
"operations": [
{
"operation": "NodegroupCreation",
"resource_name": "name1",
"status": "PROVISION_TASK_STATUS_PENDING"
},
{
"operation": "NodegroupCreation",
"resource_name": "name2",
"status": "PROVISION_TASK_STATUS_PENDING"
},
{
"operation": "ClusterCreation",
"resource_name": "demo-cluster",
"status": "PROVISION_TASK_STATUS_PENDING"
}
],
"comments": "The status of the operations can be fetched using taskset_id",
"status": "PROVISION_TASKSET_STATUS_PENDING"
}
To know the status of the cluster template apply operation, enter the below command with the generated task id dkgv8kx
./rctl status apply dkgv8kx
Share Cluster Template¶
- Command to block sharing the cluster template to any project
./rctl share clustertemplate <template-name> --none
- Command to share the cluster template to all the projects
./rctl share clustertemplate <template-name> --all-projects
- Command to share the cluster template to a specific projects
./rctl share clustertemplate <template-name> --projects <project1>,<project2>
- Command to assign/unassign the cluster template from/to one or more project(s)
./rctl share clustertemplate <template-name> --unassign-projects <project1>,<project2>
Get Cluster Templates¶
- Command to get the list of cluster template details:
./rctl get clustertemplate
./rctl get clustertemplate <template-name>
Example:
./rctl get clustertemplate demotest
Output
{
"count": 1,
"results": [
{
"id": "yemp028",
"name": "demotest",
"cluster_type": "aks",
"created_at": "2022-01-25T09:52:19.061555Z",
"modified_at": "2022-01-25T09:52:19.061555Z",
"organization_id": "9dk3emn",
"partner_id": "rx28oml",
"project_id": "pkv40kn",
"request_type": "TEMPLATE_FILE",
"input_template": "kind: ClusterTemplate\nmetadata:\n clusterType: aks\n description: Creating demo-template\n name: demotest\n"
}
]
}
- Command to get all Cluster Template Snapshots:
./rctl get clustertemplatesnapshot <template-name>
- Command to get a Cluster Template Snapshot with a specific version
./rctl get clustertemplatesnapshot <template-name> <versionname>
Example:
./rctl get clustertemplatesnapshot aks-template v1
Output
{
"count": 1,
"results": [
{
"id": "pkv40kn",
"template_id": "z247n2y",
"template_name": "aks-template",
"name": "v0",
"cluster_type": "azure-aks",
"created_at": "2022-09-13T10:13:32.797026Z",
"modified_at": "2022-09-13T10:13:32.797026Z",
"organization_id": "7w2lnkp",
"partner_id": "rx28oml",
"project_id": "rx28oml",
"request_type": "TEMPLATE_FILE",
"cloud_credential": "AKS-provider",
"blueprint_name": "minimal",
"blueprint_version": "",
"gen_template": "apiVersion: rafay.io/v1alpha1\nkind: ClusterTemplateSnapshot\nmetadata:\n description: desc\n name: v0\n project: defaultproject\n templateName: aks-template\nparameters:\n aks_cluster_params_dns_prefix:\n default: aks-dns\n override:\n allow: true\n aks_cluster_params_kubernetes_version:\n default: 1.22.11\n override:\n allow: true\n values:\n - 1.22.6\n - 1.23.5\n - 1.22.11\n aks_cluster_params_location:\n default: northcentralus\n override:\n allow: true\n values:\n - northcentralus\n - southcentralus\n aks_nodepool_params_1_name:\n default: value1\n override:\n allow: true\n values:\n - value1\n - value2\nspec:\n blueprint: minimal\n blueprintversion: \"\"\n cloudprovider: AKS-provider\n cniprovider: aws-cni\n managedCluster:\n apiVersion: \"2021-05-01\"\n identity:\n type: SystemAssigned\n location: '{{ aks_cluster_params_location }}'\n properties:\n apiServerAccessProfile:\n enablePrivateCluster: true\n dnsPrefix: '{{ aks_cluster_params_dns_prefix }}'\n kubernetesVersion: 1.22.11\n networkProfile:\n loadBalancerSku: standard\n networkPlugin: kubenet\n sku:\n name: Basic\n tier: Free\n type: Microsoft.ContainerService/managedClusters\n nodePools:\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: primary\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_DS2_v2\n type: Microsoft.ContainerService/managedClusters/agentPools\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: '{{ aks_nodepool_params_1_name }}'\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_B2ms\n type: Microsoft.ContainerService/managedClusters/agentPools\n resourceGroupName: resource-grp\n type: azure-aks\n",
"input_template": "apiVersion: rafay.io/v1alpha1\nkind: ClusterTemplateSnapshot\nmetadata:\n description: desc\n name: v0\n project: defaultproject\n templateName: aks-template\nparameters:\n aks_cluster_params_dns_prefix:\n default: aks-dns\n override:\n allow: true\n aks_cluster_params_kubernetes_version:\n default: 1.22.11\n override:\n allow: true\n values:\n - 1.22.6\n - 1.23.5\n - 1.22.11\n aks_cluster_params_location:\n default: northcentralus\n override:\n allow: true\n values:\n - northcentralus\n - southcentralus\n aks_nodepool_params_1_name:\n default: value1\n override:\n allow: true\n values:\n - value1\n - value2\nspec:\n blueprint: minimal\n blueprintversion: \"\"\n cloudprovider: AKS-provider\n cniprovider: aws-cni\n managedCluster:\n apiVersion: \"2021-05-01\"\n identity:\n type: SystemAssigned\n location: '{{ aks_cluster_params_location }}'\n properties:\n apiServerAccessProfile:\n enablePrivateCluster: true\n dnsPrefix: '{{ aks_cluster_params_dns_prefix }}'\n kubernetesVersion: 1.22.11\n networkProfile:\n loadBalancerSku: standard\n networkPlugin: kubenet\n sku:\n name: Basic\n tier: Free\n type: Microsoft.ContainerService/managedClusters\n nodePools:\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: primary\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_DS2_v2\n type: Microsoft.ContainerService/managedClusters/agentPools\n - apiVersion: \"2021-05-01\"\n location: northcentralus\n name: '{{ aks_nodepool_params_1_name }}'\n properties:\n count: 1\n enableAutoScaling: true\n maxCount: 1\n maxPods: 40\n minCount: 1\n mode: System\n orchestratorVersion: 1.22.11\n osType: Linux\n type: VirtualMachineScaleSets\n vmSize: Standard_B2ms\n type: Microsoft.ContainerService/managedClusters/agentPools\n resourceGroupName: resource-grp\n type: azure-aks\n",
"publish": true
}
]
}
Delete Cluster Template and Snapshot¶
- Command to delete a cluster template
./rctl delete clustertemplate <template-name>
Example:
./rctl delete clustertemplate demotest
Output
Deleted demotest
Important
Deleting a cluster template deletes all the cluster template snapshots under this specific template
- Command to delete a cluster template snapshot
./rctl delete clustertemplatesnapshot <template-name> <clustersnapshot-name>
Example:
./rctl delete clustertemplatesnapshot demotest v1
Output
Deleted v1
Note: Users cannot delete a cluster template which is associated with a cluster