Part 1: Provision
What Will You Do¶
In this part of the self-paced exercise, you will provision an Amazon EKS cluster based on a declarative cluster specification which contains two (2) managed nodes groups. The first managed node group will be comprised of on-demand compute instances and run the system level resources while the second managed node group will be comprised of spot instances and run application workloads. The cluster will use the minimal blueprint.
Step 1: Provision Cluster¶
In this step, we will create the declarative cluster specificiation file and use the RCTL CLI to provision the cluster from the specification file.
- Save the below specification file to your computer as "eks-spot-cluster.yaml". Note, the highlighted sections in the spec will need to be updated to match your environment.
apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
name: eks-spot-cluster
project: defaultproject
spec:
blueprintConfig:
name: minimal
version: latest
cloudCredentials: aws-cloud-credential
config:
managedNodeGroups:
- amiFamily: AmazonLinux2
desiredCapacity: 2
iam:
withAddonPolicies:
autoScaler: true
instanceTypes:
- t3.large
- t2.large
maxSize: 4
minSize: 2
name: managed-spot
spot: true
- amiFamily: AmazonLinux2
desiredCapacity: 1
iam:
withAddonPolicies:
autoScaler: true
instanceType: t3.large
labels:
nodes: system
maxSize: 2
minSize: 1
name: managed-system
taints:
- effect: NoSchedule
key: components
value: system
metadata:
name: eks-spot-cluster
region: us-west-2
version: "1.23"
network:
cni:
name: aws-cni
vpc:
autoAllocateIPv6: false
clusterEndpoints:
privateAccess: true
publicAccess: false
cidr: 192.168.0.0/16
systemComponentsPlacement:
nodeSelector:
nodes: system
tolerations:
- effect: NoSchedule
key: components
operator: Equal
value: system
type: aws-eks
Update the following sections of the specification file with details to match your environment
-
Update the name section with the name of the cluster to be created and the project section with the name of the Rafay project you previously created
name: eks-spot-cluster project: defaultproject
-
Update the cloudCredentials section with the name of the AWS cloud credential that was previously created
cloudCredentials: aws-cloud-credential
-
Update the name andregion sections with the cluster name and the AWS region where the cluster will be located
metadata: name: eks-spot-cluster region: us-west-2
-
Save the updates that were made to the file
- Open Terminal (on macOS/Linux) or Command Prompt (Windows) and navigate to the folder where you saved the file
-
Execute the following command to provision the cluster from the specification file previously saved
./rctl apply -f eks-spot-cluster.yaml
-
Login to the web console
- Navigate to your project
- Select Infrastructure -> Clusters
- Click on the cluster name to monitor progress
Provisioning the infrastructure will take approximately 45 minutes to complete. The final step in the process is the blueprint sync.
Step 2: Verify Cluster¶
Once provisioning is complete, you should see the cluster in the web console
- Click on the kubectl link and type the following command
kubectl get nodes
You should see something like the following
NAME STATUS ROLES AGE VERSION
ip-192-168-34-232.us-west-2.compute.internal Ready <none> 8m46s v1.23.13-eks-fb459a0
ip-192-168-4-152.us-west-2.compute.internal Ready <none> 8m10s v1.23.13-eks-fb459a0
ip-192-168-69-188.us-west-2.compute.internal Ready <none> 8m9s v1.23.13-eks-fb459a0
Now, we will verify that the one node group is using spot instances.
- Select Infrastructure -> Clusters
- Click on the cluster name
- Click on the "Node Group" tab
You will see the "managed-spot" node group shows the use of spot instances and the spot instance types that can be used.
Next, we will verify the nodes in the node group are labeled to use spot instances.
- Click on the "Nodes" tab
- Click on "Overview" on one of the nodes in the "managed-spot" node group
You will see the capacity type is set to "SPOT"
Now we will verify the system resources are running on the on-demand node group.
- Click "Nodes" in the tree at the top of the page to return to the nodes tab
- Locate the "Node ID" of the node in the "managed-system" node group
- Click on the "Resources" tab
- Click on "Pods" in the left side window
- Select "rafay-system" from the namespace drop down menu
- Click the gear icon on the right side of the page and select "Node"
You will see that all of the system components are running on the "managed-system" node that was previously identified.
Recap¶
Congratulations! At this point, you have successfully provisioned an Amazon EKS cluster with a managed spot instance node group in your AWS account using the RCTL CLI.