Considerations for Windows Containers on Kubernetes¶

With increasing adoption of Kubernetes in organizations, we are seeing interest from a number of customers that would like to deploy and operate their "legacy Windows applications" on Kubernetes as well.

In this blog, we have attempted to capture our learnings from working with customers that use the Rafay Kubernetes Operations Platform to deploy and operate Kubernetes clusters with Windows based containerized applications.

Why Windows Containers ?¶

Everytime a customer asks us about Windows support, we typically ask them "Why Windows Containers?". Their responses typically fall into the following categories:

Cost Reduction¶

By bringing Windows, .NET, IIS and other Windows programming frameworks to Kubernetes, they can enable their Windows developers to leverage modern cloud native frameworks. This enables them to dramatically reduce the cost associated with supporting "existing applications".

Operational Efficiency¶

Instead of having to operate and maintain multiple orchestrators, they plan to achieve operational efficiency (i.e. higher density relative to virtual machines) by deploying both modern and legacy applications on the same cluster/orchestration framework.

Avoid Big Bang Modernization¶

By implementing a "Crawl-Walk-Run" approach towards app modernization, they can reduce the business risk associated with "big bang" rewrites of their existing Windows based applications.

Options with Rafay¶

Customers can deploy and operate Windows containers based applications on Rafay provisioned and managed Kubernetes clusters in both "public clouds" (EKS, AKS and GKE) and "Upstream Kubernetes in data center environments". In this blog, we will specifically describe the options for Upstream Kubernetes clusters managed by the Rafay platform on bare metal and VM based environments.

flowchart LR
  subgraph DATA[Worker Nodes]
    direction LR
    subgraph NG1[Linux Nodes]
        i1[Linux Containers]
    end
    subgraph NG2[Windows Nodes]
        f1[Windows Containers]
    end
  end
  CP[Control Plane-Linux] <-.-> DATA

Click the button below to review the requirements to deploy and operate Upstream Kubernetes clusters in bare metal and VM based environments.

Requirements

Scheduling¶

Scheduling Windows containers in Pods on Kubernetes is similar to Linux-based containers. GA support for Windows on upstream Kubernetes has been available since v1.14 (March 2019).

Windows containers need to be scheduled on worker nodes based on Windows using a combination of "taints" and "node selectors". An illustrative example is shown below.

nodeSelector:
    kubernetes.io/os: windows
    node.kubernetes.io/windows-build: '10.0.17763'
tolerations:
    - key: "os"
      operator: "Equal"
      value: "windows"
      effect: "NoSchedule"

Isolation & Compatibility¶

Windows containers need to implement process isolation (i.e. they cannot depend on Hyper-V for isolation).

Important

With process isolation, containers share the kernel with other containers on the same host as well as with the host itself. This means the kernel versions of the container image have to match.

Strict compatibility rules apply where the host OS version must match the container base image OS version. For example, only Windows containers with a container operating system of Windows Server 2019 are fully supported on Windows worker nodes based on Windows Server 2019.

Important

The "must-match-operating-system" requirement is primarily due to licensing considerations. Because the child operating system is the same as the parent, Microsoft does not require a second license. With process isolation, containers share the kernel with other containers on the same host as well as with the host itself. This means the kernel versions of the container image have to match.

Other Considerations¶

Container Size¶

Windows containers are generally significantly larger in size compared to Linux containers. For example, do not be surprised to see container images that are 5-10 GB in size.

Note

This will impact how much time is required for containers to become live/active since downloading images over a slow network can take substantial time.

Resource Requirements¶

Plan for significant increase in resource allocation. Although Windows containers do not require significant memory when in active use (because read-only memory pages are shared between multiple containers), Windows Server containers tend to require more memory to start up successfully.

Note

Plan for slow startup time for the first container.

Root File System¶

Users cannot make the root file system read only (like with Linux containers) because Windows registry and system processes need write access.

Container Identity¶

Users need to specify “runAsUserName” (instead of runAsUser on Linux) to pick what user a container runs as or to restrict the container administrator account rather than the root user. Windows Server containers have two user accounts by default. But, neither of these are visible to the container host.

Type 1: Container User¶

Container user is for running workloads that don’t need extra privileges. It is the ideal choice if you plan to deploy containers in a multi-tenant environment.

Type 2: Container Administrator.¶

This lets you install user mode services and software that persist (e.g. IIS), create new user accounts and main configuration changes to the container OS.

Workload Identity¶

Users may need to support Active Directory to ensure applications have access to resources. For example, A Windows application talking to an external database server or file share will likely require the use a Windows identity for authorization.

However, since containers cannot be domain joined, workloads can use Group Managed Server Accounts (GMSA). This assigns an AD identity to the container that handles password management, service principal name management and delegation to other administrators in a way that can be orchestrated across the cluster.

Note

If a Windows node fails and the pod (Windows container) gets scheduled to another Windows node in the cluster, as long as all the Windows hosts in the cluster are domain-joined, the identity goes with it.

Try It Out¶

If you are interested in trying out Windows Containers on Upstream Kubernetes in bare metal and VM based environments, sign up for a Free Org and use our Getting Started guides.

Sign Up for a Free Org

Blog Ideas¶

Sincere thanks to those who spend time reading our product blogs and provide us with feedback and ideas. Please Contact the Rafay Product Team if you would like us to write about specific topics.