Part 2: Provision
What Will You Do¶
In this part of the self-paced exercise, you will provision an Amazon EKS cluster based on a declarative cluster specification using the minimal blueprint
Step 1: Cluster Spec¶
- Open Terminal (on macOS/Linux) or Command Prompt (Windows) and navigate to the folder where you forked the Git repository
- Navigate to the folder "
/getstarted/karpenter/cluster"
The "cluster.yaml" file contains the declarative specification for our Amazon EKS Cluster.
Cluster Details¶
In the cluster spec file, we define a tag with the cluster name that will automatically be applied to the AWS cluster resources during cluster creation. This tag will be used by the Karpenter provisioner to identify associated resources.
'cluster-name': 'karpenter-cluster'
The following line must be updated to match the ARN of the previously created instance profile. This can be acheived by updating the ACCOUNT-NUMBER
- arn: "arn:aws:iam::<ACCOUNT-NUMBER>:role/KarpenterNodeRole-Rafay"
- name: "karpenter-cluster"
- project: "defaultproject"
- cloudprovider: "aws-cloud-credential"
- name: "karpenter"
- namespace: "karpenter"
- name: "karpenter-cluster"
- region: "us-west-2"
- cluster-name: "karpenter-cluster"
apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
name: karpenter-cluster
project: defaultproject
spec:
blueprintConfig:
name: minimal
version: latest
cloudCredentials: aws-cloud-credential
config:
iam:
withOIDC: true
serviceAccounts:
- attachPolicy:
Statement:
- Action:
# Write Operations
- "ec2:CreateLaunchTemplate"
- "ec2:CreateFleet"
- "ec2:RunInstances"
- "ec2:CreateTags"
- "iam:PassRole"
- "ec2:TerminateInstances"
# Read Operations
- "ec2:DescribeLaunchTemplates"
- "ec2:DescribeInstances"
- "ec2:DescribeSecurityGroups"
- "ec2:DescribeSubnets"
- "ec2:DescribeInstanceTypes"
- "ec2:DescribeInstanceTypeOfferings"
- "ec2:DescribeAvailabilityZones"
- "ssm:GetParameter"
- "eks:DescribeCluster"
Effect: Allow
Resource: "*"
Version: "2012-10-17"
metadata:
name: karpenter
namespace: karpenter
identityMappings:
arns:
- arn: "arn:aws:iam::<ACCOUNT-NUMBER>:role/KarpenterNodeRole-Rafay"
username: system:node:{{EC2PrivateDNSName}}
group:
- system:bootstrappers
- system:nodes
managedNodeGroups:
- amiFamily: AmazonLinux2
desiredCapacity: 2
instanceTypes:
- t3.medium
- t3.large
maxSize: 4
minSize: 2
name: managed-spot
spot: true
- amiFamily: AmazonLinux2
desiredCapacity: 1
instanceType: t3.large
labels:
nodes: system
maxSize: 2
minSize: 1
name: managed-system
taints:
- effect: NoSchedule
key: components
value: system
metadata:
name: karpenter-cluster
region: us-west-2
tags:
'cluster-name': 'karpenter-cluster'
email: tim@rafay.co
env: qa
version: "1.23"
network:
cni:
name: aws-cni
vpc:
autoAllocateIPv6: false
clusterEndpoints:
privateAccess: true
publicAccess: false
cidr: 192.168.0.0/16
systemComponentsPlacement:
nodeSelector:
nodes: system
tolerations:
- effect: NoSchedule
key: components
operator: Equal
value: system
type: aws-eks
Step 2: Provision Cluster¶
- Type the command below to provision the EKS cluster
rctl apply -f cluster.yaml
If there are no errors, you will be presented with a "Task ID" that you can use to check progress/status. Note that this step requires creation of infrastructure in your AWS account and can take ~20-30 minutes to complete.
{
"taskset_id": "6kn1dlm",
"operations": [
{
"operation": "ClusterCreation",
"resource_name": "karpenter-cluster",
"status": "PROVISION_TASK_STATUS_PENDING"
},
{
"operation": "NodegroupCreation",
"resource_name": "managed-spot",
"status": "PROVISION_TASK_STATUS_PENDING"
},
{
"operation": "NodegroupCreation",
"resource_name": "managed-system",
"status": "PROVISION_TASK_STATUS_PENDING"
},
{
"operation": "BlueprintSync",
"resource_name": "karpenter-cluster",
"status": "PROVISION_TASK_STATUS_PENDING"
}
],
"comments": "The status of the operations can be fetched using taskset_id",
"status": "PROVISION_TASKSET_STATUS_PENDING"
}
- Navigate to the "defaultproject" project in your Org
- Click on Infrastructure -> Clusters. You should see something like the following
- Click on the cluster name to monitor progress
Step 3: Verify Cluster¶
Once provisioning is complete, you should see the cluster in the web console
- Click on the kubectl link and type the following command
kubectl get nodes
You should see something like the following
NAME STATUS ROLES AGE VERSION
ip-192-168-0-191.us-west-2.compute.internal Ready <none> 16m v1.23.15-eks-49d8fe8
ip-192-168-28-126.us-west-2.compute.internal Ready <none> 16m v1.23.15-eks-49d8fe8
ip-192-168-80-236.us-west-2.compute.internal Ready <none> 16m v1.23.15-eks-49d8fe8
Recap¶
Congratulations! At this point, you have successfully provisioned an Amazon EKS cluster with a managed node group and spot instances managed by Karpenter in your AWS account using the RCTL CLI.