Overview

Container registries can be secured to prevent unauthorized entities from accessing images.

Kubernetes provides a feature called imagePullSecrets that allows pods to pull private docker images. To connect to container registry, you have to add an ImagePullSecrets field to the configuration for a Kubernetes service account.

This is a type of Kubernetes secret that contains credential information. An imagePullSecrets is an authorization token, also known as a secret, that stores credentials for accessing a container registry.

Challenges¶

• It is operationally cumbersome to MANUALLY provision and manage Image Pull Secrets on a fleet of Kubernetes clusters
• It is an INSECURE practice to manually handle secrets
• It is BAD HYGIENE to have unused, dangling imagePullSecrets on clusters where the workload is no longer operational.

Rafay's Solution¶

Rafay has invested in developing deep integrations with Container Registry providers such as DockerHub (public, private), Amazon ECR, Google GCR, JFrog, Nexus etc.

• Project Admins can securely create, store and manage imagePullSecrets for their private container registry repos.
• Developers can use these as "references/pointers" in their workloads
• The Rafay Controller will securely deliver and automatically provision/deprovision the imagePullSecrets on the clusters where the workloads need to be deployed.

Benefits¶

• No manual handling of imagePullSecrets by developers or operations
• No need to embed imagePullSecrets in Kubernetes yaml files
• Secure (encrypted) delivery of imagePullSecrets to managed clusters
• Automated provisioning and deprovisioning of imagePullSecrets on clusters where workloads are deployed
• No dangling or orphaned imagePullSecrets on clusters
• Can rotate/refresh imagePullSecrets centrally and have it transparently apply on all managed clusters

Tested Integrations¶

Any Docker Compatible Registry should work. In addition, Rafay tests compatibility with the following providers with every release.

• Docker Hub (Public)
• Docker Hub (Private)
• AWS ECR
• GCP GCR
• Quay by RedHat
• Nexus by Sonatype
• JFrog Artifactory
• Microsoft MCR
• Rafay Container Registry

Amazon ECR¶

Amazon's Elastic Container Registry is a fully-managed Docker container registry that allows developers to store, manage, and deploy Docker container images. Amazon ECR hosts container images in a highly available and scalable architecture.

Access to Amazon ECR requires client authentication. Successful authentication to an ECR registry provides an authorization token that is valid ONLY for 12 hours. Requiring authenticating every 12 hours ensures appropriate token rotation to protect against misuse.

Customer Challenge¶

Users that use Amazon ECR as their container registry but wish to pull container images on "Non Amazon" Kubernetes infrastructure will have to deal with constantly expiring authorization tokens

Rafay's ECR Credential Helper¶

Rafay has developed a Kubernetes CRD that will ensure that ECR authentication tokens are AUTOMATICALLY refreshed before they expire. This ensures that applications operating on non Amazon Kubernetes infrastructure can seamlessly access container images from Amazon ECR on an ongoing basis.

Manage Registry Integration¶

Project Admins can configure Registries in the Rafay Console

• Under Integrations, Select Registry
• Create Registry
• Select Registry type

• Enter access credentials

The credentials are encrypted before they are stored. See Security section for additional details on Key Management.

Use Registry Integration¶

Once a registry integration has been created by a Project Admin, it can be referenced and used in workloads operating in the Project.

Workload Wizard¶

For the wizard based workloads,

• Select the registry integration from the dropdown
• Enter the repository and tag

During deployment of the workload, the Rafay controller will automatically provision the imagePullSecrets to the targeted clusters.

Annotations for Yaml or Helm¶

For Kubernetes YAML and Helm based workloads, Rafay provides "annotations" that can be quickly added to an existing k8s yaml or Helm values.yaml

  annotations:
    rafay.dev/registry-key1:value1
    rafay.dev/registry-key2:value2

The reason for different keys is to ensure support for situations with pod specs with multiple containers where the registries from which images needs to be pulled are different.

As a result, the Rafay annotations need to contain references for multiple registry instances.

Here is an example of a k8s Deployment YAML that will pull the images from Amazon ECR.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: km-nginx2
  annotations:
   rafay.dev/registry-ecr: km-ecr
spec:
  selector:
    matchLabels:
      app: km-nginx2
  replicas: 1
  template:
    metadata:
      labels:
        app: km-nginx2
    spec:
      containers:
      - name: km-nginx2
        image: 679196758854.dkr.ecr.us-east-1.amazonaws.com/km-nginx:1.17
        ports:
        - containerPort: 80