Overview
This is a reference design and implementation of an environment template to provision a self-service environment with a Generative AI application running on Amazon ECS with a LLM from Amazon Bedrock. This reference design is based on Rafay's Environment Manager.
The ECS cluster is configured to use the "Fargate Launch Type Model" which makes the environment extremely simple and cost effective. AWS Fargate allows Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances. With Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers. This removes the need to choose server types, decide when to scale your clusters, or optimize cluster packing.
Organizations also want developer environments to be extremely cost effective. With Fargate and ECS,
- You only pay for the amount of vCPU and memory resources that your containerized application requests.
- vCPU and memory resources are calculated from the time your container images are pulled until the Amazon ECS Task terminates, rounded up to the nearest second.
Source Code for Template¶
The Terraform source code and related assets are available in a public Git repo as shown below
Initial Setup¶
The platform team is expected to perform an initial configuration and setup of the Amazon ECS based Environment Template. The sequence diagram describes the high level steps. In this step, the platform team will create the Environment Template in a root level, central project they control and will share the template downstream with every developer.
sequenceDiagram
participant plat as Platform Team
participant rafay as Rafay <br> Environment Manager
participant csp as AWS
plat->>rafay: Create Central "genai" Project
plat->>rafay: Import Gen AI <br>Environment Template for ECS
plat->>csp: Enable Models in Amazon Bedrock
plat-->>rafay: Validate Template by <br>creating sample environment (optional)Request for Environment¶
When a developer requests for a Gen AI environment (based on ECS), a series of automated workflows will need to be performed to fulfil this request. The sequence diagram below describes the high level steps.
Important
Although the recommended workflow assumes and recommends using an Integration with an Identity Provider (IdP) to provide a Single Sign On (SSO) experience, organizations can also use locally managed users.
sequenceDiagram
participant dev as Developer
participant plat as Platform Team
participant rafay as Rafay <br> Environment Manager
participant idp as Identity Provider <br> (IdP)
dev->>plat: Request Gen AI Environment <br>based on ECS
plat-->>rafay: Create New Project in Org <br> for Developer
rafay->>idp: Create Group for New Project
rafay->>idp: Add developer to Group
rafay-->>rafay: Create Group in Rafay <br>with Env Template User Role
rafay->>rafay: Share "genai" Env Template <br>with New Project
rafay-->>dev: Environment Ready for Use Developer Creates Environment¶
The developer will have the ability to deploy and deprovision environments based on the shared environment template. Note that the developer
- Does not need to have any knowledge of Terraform
- Does not need access to privileged credentials for AWS
- Does not any help from the Platform team to deploy their environment
sequenceDiagram
participant dev as Developer
participant rafay as Rafay <br> Environment Manager
participant csp as ECS Cluster
participant idp as Identity Provider
dev->>idp: Access Environment
idp-->>dev: Redirect to Rafay
dev-->>rafay: SSO to Rafay with <br> RBAC (Env Template User)
dev->>rafay: Create Environment <br>based on Env Template
rafay->>csp: Provision new ECS Cluster w/VPC, subnets and Gen AI App
rafay-->>dev: Environment Ready
dev->>csp: Uses GenAI Environment Assumptions¶
- You have access to a AWS Account
- You have access to a Mac or Linux machine
- You have a Git client on your machine that is setup for push/pull
- You have Docker installed on your machine