Overview

Container registries can be secured to prevent unauthorized entities from accessing images.

Kubernetes provides a feature called imagePullSecrets that allows pods to pull private docker images. To connect to container registry, you have to add an ImagePullSecrets field to the configuration for a Kubernetes service account.

This is a type of Kubernetes secret that contains credential information. An imagePullSecrets is an authorization token, also known as a secret, that stores credentials for accessing a container registry.

Challenges¶

It is operationally cumbersome to MANUALLY provision and manage Image Pull Secrets on a fleet of Kubernetes clusters
It is an INSECURE practice to manually handle secrets
It is BAD HYGIENE to have unused, dangling imagePullSecrets on clusters where the workload is no longer operational.

Our Solution¶

We have invested in developing deep integrations with Container Registry providers such as DockerHub (public, private), Amazon ECR, Google GCR, JFrog, Nexus etc.

Project Admins can securely create, store and manage imagePullSecrets for their private container registry repos.
Developers can use these as "references/pointers" in their workloads
The Controller will securely deliver and automatically provision/deprovision the imagePullSecrets on the clusters where the workloads need to be deployed.

Benefits¶

No manual handling of imagePullSecrets by developers or operations
No need to embed imagePullSecrets in Kubernetes yaml files
Secure (encrypted) delivery of imagePullSecrets to managed clusters
Automated provisioning and deprovisioning of imagePullSecrets on clusters where workloads are deployed
No dangling or orphaned imagePullSecrets on clusters
Can rotate/refresh imagePullSecrets centrally and have it transparently apply on all managed clusters

Tested Integrations¶

Any Docker Compatible Registry should work. In addition, we test compatibility with the following providers with every release.

System (Integrated Registry)
Docker Hub (Public)
Docker Hub (Private)
AWS ECR
Azure ACR
GCP GCR
Quay by RedHat
Nexus by Sonatype
JFrog Artifactory
Microsoft MCR

Amazon ECR¶

Amazon's Elastic Container Registry is a fully-managed Docker container registry that allows developers to store, manage, and deploy Docker container images. Amazon ECR hosts container images in a highly available and scalable architecture.

Access to Amazon ECR requires client authentication. Successful authentication to an ECR registry provides an authorization token that is valid ONLY for 12 hours. Requiring authenticating every 12 hours ensures appropriate token rotation to protect against misuse.

Customer Challenge¶

Users that use Amazon ECR as their container registry but wish to pull container images on "Non Amazon" Kubernetes infrastructure will have to deal with constantly expiring authorization tokens

The ECR Credential Helper¶

We have developed a Kubernetes CRD that will ensure that ECR authentication tokens are AUTOMATICALLY refreshed before they expire. This ensures that applications operating on non Amazon Kubernetes infrastructure can seamlessly access container images from Amazon ECR on an ongoing basis.

Manage Registry Integration¶

Project Admins can configure Registries in the Web Console

Under Integrations, Select Registry
Create Registry
Select Registry type

Create Registry

Enter access credentials

The credentials are encrypted before they are stored. See Security section for additional details on Key Management.

Provide Registry Credentials

Registry Creation Success¶

On successful registry creation, user can view the list in the Container Registries page and perform a few actions if required

Display Annotation icon to view the annotation attached to this container registry. Use the copy icon to use the annotation for deploying a workload.
Validate icon to check the registry valid/invalid status
Edit icon to make any changed to the existing registries
Delete icon to remove the unnecessary registries from the system

Registry Action Icons

Use Registry Integration¶

Once a registry integration has been created by a Project Admin, it can be referenced and used in workloads operating in the Project.

Workload Wizard¶

For the wizard based workloads,

Select the registry integration from the dropdown
Enter the repository and tag

Wizard Registry

During deployment of the workload, the controller will automatically provision the imagePullSecrets to the targeted clusters.

Annotations for Yaml or Helm¶

For Kubernetes YAML and Helm based workloads, we provide "annotations" that can be quickly added to an existing k8s yaml or Helm values.yaml Annotation that has to be specified in the yaml file can be retrieved as below.

Under Integrations, Select Registries
Select the Registry that you have configured previously
Under Actions, Select Display Annotation to copy the annotation.

Registry Action Icons

Copy Registry Annotations

  annotations:
    rafay.dev/registry-key1:value1
    rafay.dev/registry-key2:value2

The reason for different keys is to ensure support for situations with pod specs with multiple containers where the registries from which images needs to be pulled are different.

As a result, the registry annotations need to contain references for multiple registry instances.

Here is an example of a k8s Deployment YAML that will pull the images from Amazon ECR.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: km-nginx2
  annotations:
   rafay.dev/registry-ecr: km-ecr
spec:
  selector:
    matchLabels:
      app: km-nginx2
  replicas: 1
  template:
    metadata:
      labels:
        app: km-nginx2
    spec:
      containers:
      - name: km-nginx2
        image: 679196758854.dkr.ecr.us-east-1.amazonaws.com/km-nginx:1.17
        ports:
        - containerPort: 80

System Registry¶

An integrated system container registry is available for non-production use (e.g. testing, demos etc.). This registry is based on Docker Registry v2 and also hosts community template container images.

Follow the instructions on how to authenticate and use the system registry.