Skip to content

Overview

Introduction

This system template enables the configuration, templating, and provisioning of a Rafay Managed Kubernetes Cluster (Rafay MKS) on a Nutanix environment. It simplifies the deployment process by automating the provisioning of nodes on Nutanix and setting up the Kubernetes cluster with essential networking, storage, and add-on capabilities. The template provisions a fully managed Kubernetes cluster based on the Rafay distribution, along with integrated Container Network Interface (CNI), Container Storage Interface (CSI), and the add-ons defined in the Cluster Blueprint. Once the deployment is complete, the template outputs a kubeconfig file, providing end users with secure, cluster-wide privileged access to the Kubernetes cluster.

For detailed steps to get started, refer to the MKS Cluster on Nutanix System Template Get Started Guide

Resources

This system template will deploy the following resources:

  • Nutanix Virtual Machines configured as Kubernetes nodes.
  • Rafay Managed Kubernetes Cluster running on the provisioned Nutanix VMs.

Initial Setup

The platform team is responsible for performing the initial configuration and setup of the Nutanix infrastructure template. The sequence diagram below outlines the high-level steps. In this process, the platform team selects the Nutanix template from the system catalog, configures it, and shares it with the project. The template can then be reused by end users for consistent provisioning.

sequenceDiagram
    participant Admin as Platform Admin
    participant Catalog as System Catalog
    participant Project as End User Project

    Admin->>Catalog: Selects Nutanix Infra Template from System Catalog
    Admin->>Project: Shares Template with Predefined Controls
    Project-->>Admin: Template Available in End User's Project

End User Flow

The end user launches the shared Nutanix infrastructure template, supplies required values such as API Key, REST endpoint, and image name, and then deploys the cluster on Nutanix.

sequenceDiagram
    participant User as End User
    participant Project as Rafay Project
    participant Cluster as Kubernetes Cluster on Nutanix

    User->>Project: Launches Shared Template for Nutanix
    User->>Project: Provides Required Input Values (API Key, REST Endpoint, Image Name)
    User->>Project: Clicks "Deploy"
    Project->>Cluster: Provisions a Kubernetes Cluster on Nutanix
    Cluster-->>User: Cluster Deployed Successfully
    Cluster-->>User: Provides Kubeconfig File as Output

This system template allows the configuration, templating, and provisioning of a Kubernetes MKS cluster on Nutanix infrastructure, using Rafay's Kubernetes distribution.

The templates are designed to support both:

  • Day 0 operations: Initial setup
  • Day 2 operations: Ongoing management

Infrastructure Type

This system template enables the provisioning of Rafay Managed Kubernetes Clusters (MKS) on Nutanix infrastructure using pre-configured parameters to streamline cluster setup and deployment.

The Nutanix-based provisioning supports:

  • Virtual Machines: Clusters are deployed on VMs managed by Nutanix, with configurable CPU, memory, and disk settings
  • Predefined Node Pools: Users can define control plane and worker node pools based on Nutanix VM specifications
  • Custom OS Images: Supports selection of Nutanix-compatible images for node provisioning

The template abstracts the complexity of the underlying Nutanix environment, providing users with a simplified form-based interface to launch and manage clusters efficiently.

Pre-Requisites

  1. API Key: Provide the API key for the Rafay controller as input for authentication
  2. Private SSH Key: Supply the SSH private key to access Nutanix VMs for cluster installation and configuration
  3. Nutanix Infrastructure Access: Ensure that access to the Nutanix infrastructure is available to perform provisioning and management operations
  4. Nutanix Infrastructure Details: Users must provide the Nutanix Endpoint, Nutanix Port, Nutanix Username, Nutanix Password, Nutanix Cluster Name, Nutanix Subnet Name, and optionally Private Key Path and Public Key Path; additionally, ensure that the images for both Control Plane and Worker nodes are available in the Prism image list
  5. Node Configuration: Define the number and details for:
    • Control Plane Node(s): Specify the image name to be used for Control Plane VM deployment. This image must exist in the Nutanix Infrastructure and is essential for launching the VM
    • Worker Node(s): Specify the image name to be used for Worker VM deployment. This image must exist in the Nutanix Infrastructure and is required for provisioning and scaling the worker nodes
  6. The agent that drives this automation must be present in the Nutanix infrastructure. This agent communicates with the Nutanix endpoint to initiate and manage the automation process. To deploy the agent, refer to Agents

Input Variables

General

Name Default Value Value Type Description
Project $(environment.project.name)$ Expression Enter the project for the Upstream cluster
Cluster Name $(environment.name)$ Expression Name of the cluster
Cluster Location sanjose-us Text Location label where the cluster will be deployed
Cluster Kubernetes Version v1.31.4 Text Kubernetes version to use
Cluster Labels { "env": "dev", "release": "stable" } JSON Labels for the cluster
Installer Certificate TTL 365 Text Validity period for the Conjurer Installer certificate
Kubelet Args { "max-pods": "200", "cpu-manager-reconcile-period": "20s" } JSON Arguments to fine tune node-level kubelet configuration
Cluster Blueprint minimal Text Blueprint to be added to the cluster
Cluster Blueprint Version latest Text Version of the cluster blueprint
System Components Placement { "node_selector": {}, "tolerations": [] } JSON Placement settings for system components
Kubernetes Upgrade { "params": { "worker_concurrency": "50%" }, "strategy": "sequential" } JSON Kubernetes upgrade strategy and parameters
Cluster HA false Text Enable high availability (true or false)
Cluster Dedicated Controlplanes false Text Enable dedicated control planes (true or false)
VM Prefix $(environment.name)$ Expression Prefix for virtual machine names

Network, Controlplane, and Worker Parameters

Name Default Value Value Type Description
Network { "cni": { "name": "Calico", "version": "3.26.1" }, "pod_subnet": "10.244.0.0/16", "service_subnet": "10.96.0.0/12" } JSON Enter the network information
Proxy Config {} JSON Configure Proxy if infrastructure uses an Outbound Proxy
Controlplane VM Count 1 Text Number of controlplane VMs to create
Controlplane VM Type { "image_name": "ubuntu24.04-lts-noble-server-cloudimg-amd64.img", "vm_os": "Ubuntu24.04" } JSON Control plane VM specs
Worker VM Count 1 Text Number of worker VMs to create
Worker VM Type { "image_name": "ubuntu24.04-lts-noble-server-cloudimg-amd64.img", "vm_os": "Ubuntu24.04" } JSON Worker VM specs

Nutanix Configuration Parameters

Name Default Value Value Type Description
Nutanix Endpoint <IP> Text Endpoint for Nutanix Prism Element (IP or FQDN)
Nutanix Port 80 Text Port for Nutanix Prism Element
Nutanix Username demo@anycloud.vn Text Username for Nutanix Prism Element
Nutanix Password ******** Password Password for Nutanix Prism Element
Nutanix Cluster Name Demo-NTNX Text Name of the Nutanix cluster
Nutanix Subnet Name NTNX-VLAN-<ID> Text Subnet to attach the VM
Private Key Path (empty) Text Enter the private key path if preconfigured in the deployment environment
Public Key Path (empty) Text Enter the public key path if preconfigured in the deployment environment

Additional Configuration Parameters

Name Default Value Value Type Description
Enable Kata Deployment false Boolean Enables deployment of the Kata admission controller to the cluster
Enable Opa-gatekeeper Deployment false Boolean Enables deployment of the OPA Gatekeeper to the cluster
Opa Excluded Namespaces [] List List of namespaces to exclude from the OPA Gatekeeper effect
Opa Constraint Template YAML <<YAML ... >> YAML YAML content defining OPA constraint templates
Opa Constraints YAML <<YAML ... >> YAML YAML content specifying OPA constraints

Other Configuration Parameters

Name Default Value Value Type Description
Controller Endpoint https://console.rafay.dev/ URL Endpoint of the controller
API Key ra2.5c19897f46a1a... Token API key used to authenticate with the controller
private-key ******** SSH Key SSH private key content for secure authentication
authorized-key ******** SSH Key SSH public key to be added to the remote machine's authorized_keys file

Note: Users who prefer not to provide an authorized-key and private-key directly can instead specify the Public Key Path and Private Key Path (under Nutanix Configuration Parameters) associated with the image. If a driver is part of the resource template, the path can point to a location where the key is already available within the driver.

Launch Time

The estimated time to launch an MKS cluster on Nutanix using this template is approximately 20 minutes.