Overview
This is a reference design and implementation of an environment template to provision self service developer environments for Generative AI running on Amazon EKS with a LLM from Amazon Bedrock. This reference design is based on Rafay's Kubernetes Manager and Environment Manager.
Source Code for Template¶
The Terraform source code and related assets are available in a public Git repo as shown below
Design & Architecture¶
This design assumes a single, shared EKS cluster with Kubernetes namespace(s) that will be created and assigned to the developer for their Generative AI applications. The available options are:
- Namespace(s) per developer
- Workspace
- Project per application team
In this design, we use a "namespace per Gen AI developer" on the shared EKS Cluster.
Initial Setup¶
The platform team is expected to provision an EKS cluster in a Rafay project e.g. "genai". The EKS cluster will be shared with developers either via "Kubernetes Namespaces" or "Rafay Workspaces".
Request for Environment¶
When a developer requests for a Gen AI environment (based on Kubernetes), a series of automated workflows need to be performed to fulfil this request. The sequence diagram below describes the high level steps.
Important
Although the recommended workflow assumes and recommends using an Integration with an Identity Provider (IdP) to provide a Single Sign On (SSO) experience, organizations can also use locally managed users.
Assumptions¶
- You have access to a AWS Account
- You have access to a Mac or Linux machine
- You have a Git client on your machine that is setup for push/pull
- You have Docker installed on your machine