Skip to content

Requirements

For organizations who need to maintain control over their environment, they can run the Self Hosted Controller in an Amazon Web Service Elastic Kubernetes Service (AWS EKS). While organizations might use cloud services, like AWS, their organization administrators will be the only ones with access to their Self Hosted Controller.

With the Self Hosted Controller, the organization is responsible for all aspects of the controller. This includes deployment, maintenance, backup procedures, and troubleshooting issues. For organizations that would prefer a SaaS model, there is a service available for that.

Here are the pre-requisites for installation of the self hosted Controller in Amazon EKS environments.

Note

If you use Terraform and need reference examples, contact Support at support@rafay.co.


Infrastructure

The AWS EKS requirements for installing the self hosted controller are as follows.

Requirement Description
Operating System CentOS 7
# Instances One (1) for Non-High Availability or three (3) for High Availability
System Specs 16 CPU threads, 64 GB RAM or higher
Root Disk 100 GB or higher
/tmp >30 GB, if not part of root disk
Data Disk 500 GB formatted. Attached as /data
Networking Inbound 443/tcp allowed to all instances. All localhost ports reachable
DNS If no DNS, ensure 300053/UDP is reachable
Firewall Disabled in all nodes

DNS Records

Installation of the self hosted controller requires DNS records as mentioned below. In the below examples, replace company.example.com with the desired domain. DNS records for the wildcard FQDN should point to the controller nodes’ IP addresses.

The following is an example of a wildcard FQDN.

*.company.example.com

The following individual records should be allowed. For AWS Cloud DNS, add these as Records.

  • Create the following DNS records with an "A" record and a "TXT" record.
ui.<company.example.com>
backend.<company.example.com>
  • The above DNS records should have their "A" record pointing to any sample address (example: 0.0.0.0).
  • The above DNS records should have their "TXT" record with the following value:
heritage=external-dns,external-dns/owner=default,external-dns/resource=service/istio-system/istio-ingressgateway-https
  • Create the following DNS records with a CNAME value of ui.<company.example.com>.
api.<company.example.com>
console.<company.example.com>
fluentd-aggr.<company.example.com>
ops-console.<company.example.com>
  • Create the following DNS records with a CNAME value of backend.<company.example.com>.
*.cdrelay.<company.example.com>
*.core-connector.<company.example.com>
*.core.<company.example.com>
*.kubeapi-proxy.<company.example.com>
peering.<company.example.com>
rcr.<company.example.com>
regauth.<company.example.com>
*.user.<company.example.com>

Logo (optional)

To change the logo displayed in the console, the company logo must be less than 600 KB and in the PNG format. Use this for white labeling and branding purposes.

X.509 Certificates (Optional)

The controller uses TLS for secure communication. As a result, X.509 certificates are required to secure all endpoints. Customers are expected to provide a trusted CA signed wildcard certificate for the target DNS (e.g. *.company.example.com)

For non-production or internal to organization scenarios, if signed certificates are not available, the controller can generate self-signed certificates automatically. This can be achieved by setting the generate-self-signed-certs key to true in config.yaml during installation.

generate-self-signed-certs: true

Email Addresses

The installation also requires below email addresses.

  • An email address for super user authentication to the controller’s admin
  • An email address for receiving support emails from the controller
  • An email address for receiving alerts and notifications (Optional)

External LB Setup (Optional)

The controller supports the AWS Classic Load Balancer with the Network Load Balancer (optional). This is used to offload the user-interface domain's SSL certificates as well as redirect traffic to hosts. The classic load balancer is used for the frontend, which includes the console user-interface, OPS console, registry, and repository endpoints of the airgapped controller (which terminates TLS traffic). This requires a valid SSL certificate.

To use a load balancer, set the override configuration in the config.yaml file.

override-config:
  global.external_lb: "true"
  • Create a classic load balancer

EKS

  • Set up a security group to allow inbound access to the load balancer from the end-user

EKS

  • Add the Certificate Authority (CA) signed wildcard certificate generated for the controller fully qualified domain name (FQDN). Or choose a certificate from Amazon Certificate Manager (ACM).

EKS

  • Set up a health check for backends with 30326/HTTP on /healthz/ready path followed by adding the controller instance to the classic load balancer.

EKS

Network Load Balancer for mTLS Traffic (Optional)

The controller uses a network load balancer to handle the mutual authenticated TLS connection (mTLS) from the clusters.

  • Set up target groups for traffic diversion.
    • Create a target group and select the IP address as target type (not instances), protocol set to TCP and port 443.

EKS

  • Set the health check for the target group on protocol HTTP and port 30326 with the path set to /healthz/ready.

EKS

  • Register all controller instances internal IP on port 30526 and create a target group

EKS

  • After creating the target group, create a network load balancer

EKS

  • Set the listener protocol to TCP and port 443. Then map the created target group above with the network load balancer and create it.

EKS

Update Security Group Inbound Rules for Controller Instances

With external load balancer support, the controller instance needs the below mentioned inbound ports to be open in the security group of the instance.

ALL traffic should be allowed to the VPC CIDR.

Port/Protocol Connection
30326/TCP Classic LB CIDR
30426/TCP Classic LB CIDR
30526/TCP Classic LB CIDR
30726/TCP Classic LB CIDR
443/TCP User address.(for e.x. 0.0.0.0/0)
22/TCP User address.(for e.x. 0.0.0.0/0)

DNS Settings (when External Load Balancer is used)

For extended security, most of the endpoints are mTLS and they do not support offloading SSL except the frontend user-interface endpoints.

The frontend FQDNs should be pointed to the classic load balancer for SSL offloading:

api.<company.example.com>
console.<company.example.com>
fluentd-aggr.<company.example.com>
ops-console.<company.example.com>

FQDNs pointing to the network load balancer (mTLS):

*.<company.example.com>