Kubernetes Component Configuration

Kubelet Arguments

Kubelet settings can be customized to optimize configurations based on workload and operational requirements. Kubelet parameters are exposed in both cluster and node settings, allowing configuration at a global or node-specific level. In addition to key-value Kubelet arguments, structured KubeletConfiguration YAML is supported to configure any valid upstream Kubelet parameter.

Cluster-level settings are applied to all nodes. When a node defines its own configuration, the node-level settings take precedence for matching fields. For fields that do not exist at the node level, the cluster-level configuration continues to apply. Each node receives a merged configuration using this priority model.

Types of Kubelet Configuration

Kubelet configuration is supported through two mechanisms, depending on the level of control required:

1. Kubelet Extra Args (Key–Value Arguments)

Kubelet Extra Args allow configuring command-line style Kubelet parameters using simple key-value pairs. These fields are suitable for flat arguments such as --max-pods, timeouts, feature gates, and other non-nested flags. Multiple arguments can be added in the Add/Edit Kubelet Extra Args dialog, and the settings are applied at the cluster or node level.

This method is intended for straightforward tuning where parameters can be expressed as single key-value entries.

2. KubeletConfiguration Overrides (YAML-Based)

KubeletConfiguration Overrides provide support for the full upstream KubeletConfiguration API and allow defining structured YAML, including nested fields, boolean policies, durations, and advanced configuration blocks. This method enables configuring any valid Kubelet parameter, including complex fields that cannot be expressed through simple key-value arguments.

Overrides can be applied at both the cluster and node level, with node-level settings taking precedence when both levels define the same fields.

YAML Reference Example

A built-in reference panel is available to help construct valid KubeletConfiguration YAML. Selecting the help icon in the editor opens a read-only example based on the upstream KubeletConfiguration API. This example includes commonly used fields such as authentication, authorization, cgroup settings, cluster DNS configuration, and other standard parameters defined by Kubernetes.

The reference panel is intended as a guide for understanding the structure and formatting of the KubeletConfiguration specification. Values can then be added or modified in the editable YAML editor on the right, where cluster-level or node-level overrides are entered.

This reference is provided to ensure that KubeletConfiguration Overrides follow the correct structure, field types, and formatting as defined by Kubernetes.

The following are the default Kubelet arguments configured by Rafay. These arguments cannot be overridden, and attempting to override them will result in deployment failure. However, additional arguments can be added if needed.

• feature-gates=ExecProbeTimeout=false
• register-with-taints=rafay.dev/Provisioning:NoSchedule
• node-ip=10.0.0.229
• runtime-request-timeout=15m
• container-runtime-endpoint=unix:///run/containerd/containerd.sock

KubeletConfiguration Examples

The following examples illustrate commonly used KubeletConfiguration fields that can be applied at either the cluster level or node level.

Example 1: Enable singleProcessOOMKill for selected nodes

singleProcessOOMKill: true

This configuration can be added as a node-level override so that the setting applies only to the selected nodes.

Example 2: Configure imageMaximumGCAge

imageMaximumGCAge: "1h"

Example 3: Combined advanced parameters

maxPods: 500
imageMinimumGCAge: "2m2s"
imageGCHighThresholdPercent: 80

Click Save after updating the configuration.

Kubelet Arguments can be added, edited, or deleted on Day 2. Kubelet Arguments cannot be modified at both the cluster level and node level simultaneously during Day 2 updates; changes can be applied either at the cluster level or in bulk for selected nodes. KubeletConfiguration Overrides are also supported on Day 2 and can be updated at the cluster level or node level as required.

---

Control Plane Overrides

Control Plane Overrides enable advanced customization of Kubernetes control plane components in upstream clusters. This feature allows modification of control plane arguments, volumes, and volume mounts without exposing or editing the full static pod manifest files.

This capability is supported only for control plane nodes and does not apply to worker nodes. There is no node-level configuration for this feature. All overrides are applied at the cluster level.

Purpose

Control Plane Overrides provide a controlled mechanism to customize Kubernetes control plane behavior while preserving required defaults and overall cluster stability.

This feature can be used to:

• Enable or disable feature gates
• Modify admission plugins
• Adjust logging verbosity
• Configure audit logging parameters
• Add or override volume definitions
• Update volume mount paths
• Apply organization-specific configuration requirements

The complete YAML manifest for control plane components is not exposed. Only specific override sections are configurable.

What Can Be Overridden

Only the following sections of the control plane manifests can be modified:

1. Extra args (key-value pairs)
2. Extra volumes
3. Extra volume mounts

Other sections such as container image, probes, resource settings, security context, networking configuration, and priority settings are not configurable.

Only the fields that require customization need to be provided. Existing values remain unchanged unless explicitly overridden.

Scope and Application

• Control Plane Overrides are cluster-level settings.
• Overrides apply uniformly to all control plane nodes.
• If multiple control plane nodes exist, the same configuration is applied to each API server, controller manager, and scheduler instance.
• Supported during cluster creation (Day 0).
• Supported during post-creation updates (Day 2).

Editing Control Plane Overrides

Selecting the Edit option under Control Plane Overrides opens a tabbed configuration panel.

Separate tabs are available for:

• Kube API Server
• Kube Controller Manager
• Kube Scheduler

Each tab provides:

• Extra args
• Extra volumes
• Extra volume mounts

Configuration entered in one tab applies only to that component.

Extra Args

Extra args allow configuration of control plane arguments using key-value pairs.

Each entry consists of:

• Key – Argument name
• Value – Argument value

Behavior

• If the key already exists, its value is overridden.
• If the key does not exist, it is added.
• Existing values are replaced, not merged.
• Only the fields that need to be customized must be provided.

Example 1: Simple Argument

UI entry:

Key	Value
profiling	false

Result

The existing profiling setting is updated to false for the selected component across all control plane nodes.

Example 2: Comma-Separated Argument

Certain arguments accept comma-separated values, such as:

• enable-admission-plugins
• feature-gates
• tls-cipher-suites

When modifying such arguments:

• The complete existing value must be copied.
• Any new value must be appended using a comma.
• Providing only a new value replaces the entire list.

Example UI entry:

Key	Value
enable-admission-plugins	NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,Priority,ResourceQuota,NodeRestriction,EventRateLimit,PodSecurity

Result

The existing list is fully replaced with the provided value, preserving required entries and appending the new plugin.

If only PodSecurity is provided, all previously configured plugins are removed and cluster functionality may break.

This rule applies to all comma-separated arguments.

For supported parameters and default values, refer to the official Kubernetes documentation.

Extra Volumes

The Extra volumes section accepts YAML input in string format.

Example:

- name: safe-dir
  emptyDir: {}

Mandatory Rules

• Each entry must begin with a hyphen (-).
• YAML must be correctly formatted.
• The name field is the matching key.

If formatting is incorrect or the hyphen is omitted, the control plane manifest may become corrupted and functionality may be affected.

Matching Behavior

• If the name matches an existing volume, that volume definition is overridden.
• If the name does not exist, a new volume is added.

The name field determines whether the configuration overrides or adds a volume.

Extra Volume Mounts

The Extra volume mounts section also accepts YAML input in string format.

Example:

- name: safe-dir
  mountPath: /tmp/safe-apiserver
  readOnly: false

Rules

• Each entry must begin with a hyphen (-).
• The name field determines matching behavior.
• Matching name overrides the existing mount configuration.
• A new name adds a new mount entry.

Volumes and volume mounts must use consistent name values.

---

Cluster Specification Example

Control plane overrides are defined in the cluster specification under controlPlaneOverrides

Below is an example spec:

apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
  name: test-pfug-2
  project: defaultproject
spec:
  blueprint:
    name: minimal
  config:
    autoApproveNodes: true
    kubernetesVersion: v1.33.5
    installerTtl: 365
    platformVersion: v0.1.0
    kubeletExtraArgs:
      max-pods: "600"
    controlPlaneOverrides:
      kubeApiserver:
        extraArgs:
          profiling: "false"
          v: "4"
        extraVolumeMounts: |
          - name: safe-dir
            mountPath: /tmp/safe-apiserver
            readOnly: false
        extraVolumes: |
          - name: safe-dir
            emptyDir: {}
      kubeControllerManager:
        extraArgs:
          profiling: "false"
          v: "2"
        extraVolumeMounts: |
          - name: safe-dir
            mountPath: /tmp/safe-controller
            readOnly: false
        extraVolumes: |
          - name: safe-dir
            emptyDir: {}
      kubeScheduler:
        extraArgs:
          profiling: "false"
          v: "2"
        extraVolumeMounts: |
          - name: safe-dir
            mountPath: /tmp/safe-scheduler
            readOnly: false
        extraVolumes: |
          - name: safe-dir
            emptyDir: {}
    network:
      cni:
        name: Calico
        version: 3.28.1
      podSubnet: 10.244.0.0/16
      serviceSubnet: 10.96.0.0/12
    nodes:
      - arch: amd64
        hostname: vih-12
        operatingSystem: Ubuntu22.04
        privateip: 10.0.0.128
        roles:
          - Master
          - Worker
          - Storage
        ssh:
          ipAddress: 129.146.39.135
          port: "22"
          privateKeyPath: /Users/vihari/.ssh/vihari_oci_ssh
          username: ubuntu

  type: mks

Day 2 Operations

Control Plane Overrides can be updated after cluster provisioning.

Go to Clusters → Select Cluster → Configuration → Control Plane Overrides → Edit.

Viewing Applied Configuration

The applied control plane configuration can be viewed from the Nodes tab.

Go to Clusters → Select Cluster → Nodes → Node Actions → View Control Plane Config.

This opens a read-only view of the Kube API Server, Controller Manager, and Scheduler configuration for that node. Editing is not supported from this screen.

A read-only view of the full control plane YAML is available for each control plane node. This allows verification that overrides have been applied correctly without accessing the node directly.

Important Considerations

• Only arguments, volumes, and volume mounts are configurable.
• Comma-separated arguments must include the complete existing value.
• The name field determines override versus addition for volumes and mounts.
• Overrides apply uniformly to all control plane nodes.
• Incorrect configuration may impact cluster functionality.

---