Skip to content

Product Blog

Vector Databases for Generative AI on Kubernetes

Many of our customers use Kubernetes extensively for AI/ML use cases. This is one of the reasons why we have turnkey support for Nvidia GPUs on EKS, AKS, Upstream Kubernetes in on-prem data centers. Recently, we have had several customers look at adding support for Generative AI to their applications. Doing so will require looking at a slightly different technology stack.

Traditional relational databases are adept at handling structured data. They do this by storing data in tables. However, AI use cases are focused on handling unstructured data (e.g. images, audio, and text). Data like this is not well suited for storage and retrieval in a tabular format. This critical technology gap with relational databases has opened the door for a new type of database called a Vector Database that can natively store and process vector embeddings. The rapid rise of large-scale generative AI models has further propelled the demand for vector databases.

In this blog, we will review why vector databases are well suited and critical for AI and Generative AI. We will then look at how you can deploy and operate vector databases on Kubernetes using the Rafay Kubernetes Operations Platform in just one step.

Least Privilege Developer Access for Kubernetes Clusters

In our recent release in July, we enhanced our zero trust kubectl service to address the needs to two completely different cohorts of customers. At a high level, users of the Rafay Platform fall into two ends of the access spectrum when it comes to answering the following question.

Should developers be allowed to view Kubernetes Secrets?. Should they be allowed to exec into running containers/pods for troubleshooting purposes?

Opposite Ends of the Spectrum

In-Place Upgrades to Amazon EKS v1.26 using Rafay

In our recent release in July, we added support for in-place upgrades of existing EKS clusters to Kubernetes v1.26. Customers tell us that they wish to be extremely careful with in-place upgrades of their existing EKS clusters because there is no benefit in rushing this and impacting mission critical applications etc. Provisioning of new EKS v1.26 clusters using Rafay has been supported for a while now.

Upgrade to EKS v1.26

Note

Organizations that wish to perform sophisticated checks for API deprecation etc are strongly recommended to use Rafay's Fleet Operations for Amazon EKS.

Declarative Cluster Lifecycle Management

Every Kubernetes user is familiar with the kubectl apply command. This is used to create or modify Kubernetes resources as defined in a manifest file.

This pattern is referred to as Declarative usage where the state of the resource is "declared" in the manifest file and the command is used to implement the declared state. Unlike an imperative approach where the user needs to specify both what and how to perform a task, with the declarative approach, the user just needs to specify what to do and not worry about how to do it.

The apply approach is preferred and recommended because it is well suited for version control. The kubectl apply command works extremely well for resources inside the Kubernetes cluster.

What if there was a way to bring this declarative approach for cluster lifecycle management as well?

Auto Inject Project Name in Cluster Labels

In our July 2023 release, we added support automatic injection of project name as a cluster label. In Kubernetes, labels are key/value pairs that are attached to objects such as Pods, nodes etc.

Support for labels in Kubernetes ends at a node level. We extended the "label" construct to clusters over two years back as cluster labels. Cluster labels are critical for multi cluster workflows and have been the backbone of several unique and differentiated features in the Rafay Kubernetes Operations Platform.

In this release, we have automated the injection of the "name of the project" as a cluster label to all clusters in the project. Read on to understand some of the use cases that are enabled by this feature.

Cluster Labels

Note

In a separate blog, we will describe how cluster labels powers various multi cluster workflows and operations.

Display Workload's Details for Enhanced Developer Experience

Many of our customers are platform teams that are focused on providing their end users (application developers) a great developer experience especially with ease of use with deploying and managing their applications on remote clusters. In our July 2023 release, we have enhanced the existing developer experience further by providing developers with a facility to quickly verify details about the workload under management.

Developer Experience

Important

This enhancement was requested by multiple enterprise customers.

AWS Cross Account Support for EKS LCM in Rafay

Our recent release update in July to our Preview environment adds support for a number of new features and enhancements. This blog is focused on Cross Account Role ARN Support for Amazon EKS.

In July 2023, Rafay introduced a new feature to its Kubernetes Operations Platform: Cross Account Role ARN support for Amazon Elastic Kubernetes Service (EKS). This feature is designed to cater organizations that operate multiple AWS accounts, providing a seamless and efficient way to manage EKS clusters across these accounts. In this blog post, we'll delve into the significance of this enhancement, explore its use cases, and understand how it simplifies EKS cluster management across multiple AWS accounts.

Cross Account Support

Integrated FinOps Role for Cost Management

In our June 2023 release, we added support for a new turnkey role in the Rafay Kubernetes Operations Platform specifically targeted at users in a FinOps function.

This new role allows the FinOps team to access and view cost and usage data in the Rafay Kubernetes Operations Platform. Users with this role do not have the ability to impact infrastructure or applications.

FinOps

Kubernetes v1.27 for Rafay MKS

Our recent release update in June adds support for a number of new features and enhancements. This blog is focused on support for Kubernetes v1.27 with Rafay MKS (i.e. upstream Kubernetes for bare metal and VM based environments).

Both new cluster provisioning and in-place upgrades of existing clusters are supported. As with most Kubernetes releases, this version also deprecates and removes a number of features. To ensure there is zero impact to our customers, we have made sure that every feature in the Rafay Kubernetes Operations Platform has been validated on this Kubernetes version.

Kubernetes v1.27 Release