Skip to content

In Place Upgrades

Performing upgrades of Amazon EKS Clusters by yourself can be a complex, nuanced, time-consuming, manual activity. We have made With the Controller, users can perform this in just a few clicks.


Upgrade Strategies

Depending on the nature and criticality of their workloads, different strategies can be employed for upgrades of Amazon EKS Clusters.

Migrate to New Cluster

This approach is well suited for scenarios where a low blast radius is required and AWS costs for parallel infrastructure is not a concern. In many cases, this is performed over an extended time period (Blue/Green type) providing cluster administrators the option to switch back and forth between the old and new clusters if required.

Step 1 - Provision a replacement EKS cluster and node groups with new k8s version

Step 2 - Publish and Test the workloads on the new EKS cluster

Step 3 - Unpublish the workloads on the old EKS cluster

Step 4 - Deprovision the old EKS Cluster


In Place Upgrades

This approach is well suited for scenarios where "it is not possible" or "it is impractical" to pursue a migration based upgrade strategy.

Note that unlike actual in-place, zero downtime upgrades for on-prem or VM based MKS (Upstream k8s) clusters, upgrades of Amazon EKS cluster worker nodes result in replacement of the instances backing the worker nodes. As a result, administrators need to allow for substantial time (~40 minutes) for upgrades to complete.


Notification

The administrator is shown a notification when an upgrade is available for a cluster. Clicking on the notification will provide the administrator with additional information on what is available.

  • A red banner indicates that the cluster is multiple versions behind the latest version.
  • A blue banner indicates that the cluster is one version behind the latest version.

Upgrade Notification


Scope of Upgrade

There are three components that need to be upgraded during an EKS Cluster upgrade.

  1. EKS Control Plane
  2. Attached Node Groups (both Managed and Self Managed)
  3. Critical Add Ons (coredns, kube-proxy and aws-node)

The Control plane always needs to be upgraded first before the node groups and the critical addons. Note that the Controller automatically upgrades the critical cluster addons as part of the upgrade process reducing the manual, operational burden for administrators.

Important

EKS Cluster upgrades performed by the controller are always performed in a manner where the cluster and resident workloads never encounter any downtime.


Upgrade Everything

This is the default option. When selected, it upgrades everything to the new version of k8s i.e.

  • EKS Control Plane
  • All Node Groups (Both Managed and Self Managed)
  • Critical Cluster Addons

Upgrade Default

Tip

Upgrading the entire EKS cluster can take ~40 minutes.

During the upgrade, an indication is provided to administrators on the main cluster list page. An illustrative example is shown below.

Cluster Upgrade Indication

Preflight Checks

A number of preflight checks are first performed before the upgrade is attempted.

Preflight Checks

EKS Control Plane

The EKS control plane is upgraded next. This can be a time-consuming step.

Upgrade In Process

Critical Cluster Addons

Critical addons for the cluster such as "core-dns", "kube-proxy" and "aws-node" addons are then upgraded to the required versions.

Node Groups

Node Groups are then upgraded to the selected version of k8s. Shown below is an example of an EKS Cluster's node group that was upgraded from v1.16.x to v1.17.x

Upgraded Node Group

Post Upgrade Checks

Once upgrades is complete, a round of post upgrade tests are performed to ensure that there are no loose ends and everything is as expected.


Control Plane Only Upgrade

If selected, only the EKS Control Plane is upgraded. The attached node groups are left untouched. The administrator can then upgrade the node groups individually one by one when needed.

Upgrade Control Plane Only

Tip

Upgrading the EKS Control Plane can take ~25-30 minutes.

Control Plane Only Upgrade


Node Group Upgrade

This option is displayed to the administrator only if the controller detects that the node group is behind the EKS Control Plane from a Kubernetes version perspective.

  • Click on cluster name
  • Select Node Groups
  • Click on upgrade notification for available node groups.

Proceed with the instructions presented. An illustrative screenshot of the administrative experience is shown below.

Node Group Upgrade


Upgrade History

The Controller maintains the upgrade history associated with every upgrade action whether it was successful or not. Administrators can view the entire history by

  • Clicking on the cluster name
  • Click on the Upgrade History tab

This will display the entire history of upgrades for the cluster.

Upgrade History

Upgrade Details

Clicking on a specific row will provide detailed information about the specific upgrade. An illustrative example is shown below.

Upgrade History Detail