Skip to content

preBootstrapCommands

preBootstrapCommands are used to prepare nodes and run before a cluster is brought up. An example are any OS level configuration changes that have to be done for the cluster to come up successfully, like inserting CA certs or changes to containerd.

preBootstrapCommands is a list of commands (one or more) to run on a target cluster. These commands are input using the cluster configuration.

Notes

  • This setting is optional.
  • preBootstrapCommands will run every time a node comes up, during cluster creation and cluster or node pool scaling. Examples: Node restart and node creation.
  • preBootstrapCommands are only executed on Linux-based cluster nodes.

Example Use Case

If a cluster and its nodes need to be behind a proxy because of enterprise security guidelines. The nodes need to have a CA cert injected for the nodes to be able to communicate outside. In this use case, if the user configures the preBootstrapCommands at the time of configuring their cluster, the provisioning process will read the user's configuration and inject the CA cert while the node is coming up.

Configure preBootstrapCommands

Using the Console

When provisioning a GKE cluster from the Console, add the preBootstrapCommands under Advanced Settings.

preBootstrapCommands Advanced Settings

Using the CLI

When provisioning a GKE cluster using the CLI, add the preBootstrapCommands in the configuration file. See GKE Config Schema for more information about the configuration file.

preBootstrapCommands Guidelines

  • To refer to the root directory for a node, use the ${ROOT_DIR} variable. For example, /etc/ssl/certs/ should be ${ROOT_DIR}/etc/ssl/certs for the preBootstrapCommands.
  • To detect the container runtime, use the IS_CONTAINERD and IS_DOCKER variables. Both are boolean variables. The IS_CONTAINERD variable is set to true when Kubelet is using containerd as a container runtime.
  • To change the root directory for a specific command, prefix the command with chroot "${ROOT_DIR}". For example, chroot "${ROOT_DIR}" systemctl restart containerd.
  • base64 encoding the command is not required. Users can use base64 encoding to avoid any issues with any special characters that might be in the command. Since the commands are run as-is, the commands must be written in a way to be decoded upon execution.

preBootstrapCommands Examples

Most Linux commands are valid for preBootstrapCommands.

Download Kubectl 

curl -L "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" -o /tmp/kubectl
mv /tmp/kubectl /usr/local/bin/ && chmod +x /usr/local/bin/kubectl

Enable IP Forwarding 

sysctl -w net.ipv4.ip_forward=1
sed -i 's/net.ipv4.ip_forward=0/net.ipv4.ip_forward=1/' /etc/sysctl.conf
sysctl --quiet --system

preBootstrapCommands cannot pass bash-script. A work-around is to base64 encode the script, then convert it to a command that will decode and execute. 

echo "<base64-encoded-script>" | base64 -d | bash

Invalid Example 

<something with /mnt/ >

preBootstrapCommands Execution Flow

The preBootstrapCommands run before applying Cluster Operator Spec.

sequenceDiagram
    Rafay Core->>Bootstrap VM: Send kubectl commands to the infra-agent
    Note over Bootstrap VM: Infra-Agent
    Bootstrap VM->>Target GKE Cluster: Creates the DaemonSet and ConfigMap
    Note over Target GKE Cluster: DaemonSet
    Note over Target GKE Cluster: ConfigMap

DaemonSet is used to execute the preBootstrapCommands. This happens in two steps:

  • Setup - This sets up and runs the preBootstrapCommands on the target cluster. The preBootstrapCommands come from the user's configuration input.
  • Validation - After the setup is complete, the controller looks for any failures in the preBootstrapCommands execution on the nodes. Any errors should appear in the console.

Troubleshooting

The preBootstrapCommands are executed on all nodes spun up at the time of cluster provisioning.

Errors Display in Console

If any of the preBootstrapCommands fail, it is automatically reflected on the cluster provisioning page, with the pod name and error line.

preBootstrapCommands Example Error

Errors in Pod Logs

If all preBootstrapCommands are executed successfully on all nodes, then the cluster provisioning process proceeds to the next step.

Once the cluster is provisioned, you can track the preBootstrapCommands output using the kubectl command kubectl get pods -o wide.

preBootstrapCommands Kubectl Success

If CrashLoopBackoff displays for the pod status, check the pod log file for more information.

preBootstrapCommands Kubectl CrashLoopBackoff

Example Pod Log File Error

preBootstrapCommands Kubectl CrashLoopBackoff Log File Error