Skip to content

Requirements

Here are the pre-requisites for installation of self hosted controller in AWS EC2 instances.


Infrastructure

Requirement Description
Operating System CentOS 7.9 OR Ubuntu 20.04 LTS
# Instances One (1)
System Specs 16 CPUs, 64 GB RAM or higher
Root Disk 100 GB or higher
/tmp >30 GB, if not part of root disk
Data Disk 500 GB formatted. Attached as /data
Networking Inbound 443/tcp allowed to all instances. All localhost ports reachable
DNS If no DNS, ensure 300053/UDP is reachable
Firewall Disabled in all nodes

DNS Records

Installation of the self hosted controller requires wildcard records as mentioned below. In the below examples, replace controller.example.com with the desired domain.

*.controller.example.com

In case, wildcard DNS is not available, individual records as below are needed.

  1. *api.
  2. console.
  3. fluentd-aggr.
  4. ops-console.
  5. rcr.
  6. regauth.
  7. *.core.
  8. *.core-connector.
  9. *.kubeapi-proxy.
  10. *.user.
  11. .cdrelay.

DNS records for the wildcard FQDN should point to the controller nodes’ IP addresses. If external LB is used with AWS, refer Step 6.


Logo (Optional)

Company logo of size less than 200KB in png format for white labeling and branding purposes.


X509 Certificates (Optional)

The controller uses TLS for secure communication. As a result, x509 certificates are required to secure all endpoints. Customers are expected to provide a trusted CA signed wildcard certificate for the target DNS (e.g. *.controller.example.com)

For non-prod/internal to org scenarios, If signed certificates are not available then the controller can generate self-signed certificates automatically. This can be achieved by setting the “generate-self-signed-certs” key to “True” in config.yaml during installation.


Email Addresses

The installation also requires below email addresses.

  • An email address for super user authentication to the controller’s admin
  • An email address for receiving support emails from the controller
  • An email address for receiving alerts and notifications(Optional)

External LB setup (Optional)

The self hosted controller supports AWS Classic LB with NLB (optional) which can be used to offload UI domain’s SSL certificates and redirect traffic to hosts. This LB is used for frontend including console UI, ops console, registry, repo endpoints of the air gapped controller, which terminates TLS traffic and requires the valid SSL certificate. To use LB, enable the below override-config in config.yaml.

override-config.global.external_lb: true

Requirements

  • CA signed wildcard certificate created as ACM
  • Ports 80/TCP and 443/TCP inbound to Load Balancer

LB Setup Procedure

  • Create classic load balancer with port and protocols as mentioned

External LB Setup

  • Setup security group to allow inbound access to LB from the end user

External LB Setup

  • Add the CA signed wildcard certificate generated for the controller FQDN *.<controller.example.com> or choose a certificate from ACM

External LB Setup

  • Setup health check for backends with 30326/HTTP on /healthz/ready path followed by adding the controller instance to the classic load balancer. Now the LB is configured.

External LB Setup


Creation of Network Load balancer

For mTLS traffic (Optional)

The self hosted controller uses a network load balancer to handle the mTLS traffic (mutual authenticated TLS connection) from the clusters. Before setting up the network load balancer, we need to set up Target groups for traffic diversion.

  • Create a Target Group and select the IP address as target type (not instances), protocol set to TCP and port 443

External LB Setup

  • Set health check for Target group on protocol HTTP, port 30326 with /healthz/ready path

External LB Setup

  • Register the controller instance internal IP on port 30526 and create a target group

External LB Setup

  • Once the Target Group is created, create a Network Load Balancer

External LB Setup

  • Select listener protocol TCP and port 443. Then map the created target group above with the NLB and create it.

External LB Setup


Update Security Group Inbound Rules

Next step is to update Security Group inbound rules for the controller instances.

With external load balancer support, the controller instance needs the below mentioned inbound ports to be open in the security group of the instance.

  • 30326/TCP connecting to Classic LB CIDR
  • 30426/TCP connecting to Classic LB CIDR
  • 30526/TCP connecting to Classic LB CIDR
  • 30726/TCP connecting to Classic LB CIDR
  • 443/TCP connecting to user address.(for e.x. 0.0.0.0/0)
  • 22/TCP connecting to user address.(for e.x. 0.0.0.0/0)

DNS settings

DNS settings when External Load Balancer is in use

For extended security, most of the controller's endpoints are mTLS and they do not support offloading SSL except the frontend UI endpoints. As a result, the following frontend FQDNs should be pointed to the Classic LB for SSL offloading:

  1. api.<controller.example.com>
  2. console.<controller.example.com>
  3. fluentd-aggr.<controller.example.com>
  4. ops-console.<controller.example.com>

FQDNs pointing to the NLB.[mTLS]

  1. rcr.<controller.example.com>
  2. regauth.<controller.example.com>
  3. *.core-connector.<controller.example.com>
  4. *.core.<controller.example.com>
  5. *.kubeapi-proxy.<controller.example.com>
  6. *.user.<controller.example.com>
  7. *.cdrelay.<controller.example.com>