Skip to content

Requirements

In order to set this up, please make sure you have access to the following:

  • Administrative access to a project in Google Cloud Platform (GCP) with the ability to create a service account with the following roles:

    • Cloud MemoryStore Redis Admin - Creation and management of Redis MemoryStore
    • Cloud SQL Admin - Creation and management of MLOps SQL databases
    • Compute Admin - Creation and management of GKE Clusters
    • Kubernetes Engine Admin - Creation and management of GKE Clusters
    • Project IAM Admin - Provide Service accounts access to MLops resources in GCP
    • Service Account Admin - Creation and management of Service accounts for MLOps applications
    • Service Account User - Management of GKE Clusters
    • Storage Admin - Creation and management of Cloud storage for MLOPs
    • Storage HMAC Key Admin - Creation and management of HMAC keys for cloud storage access by MLOPs applications

  • Or Administrative access to a project in Google Cloud Platform (GCP) with the ability to create a service account with the following roles and permissions for least privilege:

    Roles

    • Compute Admin - Creation and management of GKE Clusters
    • Kubernetes Engine Admin - Creation and management of GKE Clusters
    • Service Account User - Management of GKE Clusters

    Permissions

    • cloudsql.databases.create
    • cloudsql.databases.delete
    • cloudsql.databases.get
    • cloudsql.databases.list
    • cloudsql.databases.update
    • cloudsql.instances.create
    • cloudsql.instances.delete
    • cloudsql.instances.get
    • cloudsql.instances.list
    • cloudsql.users.create
    • cloudsql.users.delete
    • cloudsql.users.get
    • cloudsql.users.list
    • cloudsql.users.update
    • iam.serviceAccounts.create
    • iam.serviceAccounts.delete
    • iam.serviceAccounts.get
    • iam.serviceAccounts.getIamPolicy
    • iam.serviceAccounts.setIamPolicy
    • iam.serviceAccounts.update
    • monitoring.timeSeries.list
    • redis.instances.create
    • redis.instances.delete
    • redis.instances.get
    • redis.instances.getAuthString
    • redis.instances.update
    • redis.instances.updateAuth
    • redis.operations.get
    • resourcemanager.projects.get
    • resourcemanager.projects.getIamPolicy
    • resourcemanager.projects.setIamPolicy
    • storage.buckets.create
    • storage.buckets.delete
    • storage.buckets.get
    • storage.buckets.getIamPolicy
    • storage.buckets.setIamPolicy
    • storage.buckets.update
    • storage.hmacKeys.create
    • storage.hmacKeys.delete
    • storage.hmacKeys.get
    • storage.hmacKeys.update
    • storage.objects.list
    • storage.objects.delete

  • Administrative access to a project in Google Cloud Platform (GCP) with the ability to enable the following APIs:

    • Compute Engine API
    • Cloud Resource Manager API
    • Kubernetes Engine API
    • Cloud SQL Admin API
    • Cloud SQL
    • Google Cloud Memorystore for Redis API
    • Service Networking API
    • Service Usage API

  • Administrative access to setup applications in Okta

  • Administrative access with Organization Admin privileges in Rafay
  • Ability to configure DNS for a domain you will expose to data scientists
  • A Mac or Linux instance
  • A Git client on your machine that is setup for push/pull
  • Docker installed on your Mac/Linux machine

If using a shared VPC, be sure the following requirements are met:

  • The service account being used to create the cluster has the following roles:
    • Compute Network User
    • Compute Security Admin
  • The Compute Engine default service account for the project where the cluster is located has the following roles:
    • Compute Network User
    • Compute Security Admin
  • The Compute Engine default service account for the project where the cluster is located is added into IAM in the shared VPC host project with the following roles:
    • Compute Network User
    • Compute Security Admin
  • Enable the Service Networking API in both the GCP cluster project and the VPC host project
  • The shared VPC has Private Services Access configured. Instructions
  • The full path name of the shared network is known (ex. projects/dev-382813/global/networks/shared-net)
  • The full path name of the shared subnet is known (ex. projects/dev-382813/regions/us-west1/subnetworks/shared-subnet"
  • The names of the shared subnet's secondary IPv4 ranges for pods and services are known

Note

Below are the minimum required settings for a reliable, functioning environment. Users can update these settings to scale the environment based on the performance/capacity required.

Minimal Configuration

We have tested extensively with various types of VMs and resource types for SQL, Redis etc. We recommend using at least the minimal configuration described below for a reliable, stable system.

GKE

The MLOps platform is a Kubernetes native application.

Variable Value
gke_node_pool_machine_type e2-standard-4
gke_nodepool_size 4

SQL

The managed SQL database in GCP is used to persist data.

Variable Value
gcp_sql_instance_tier db-f1-micro

Redis

Variable Value
gcp_redis_instance_memory_size_gb 1
gcp_redis_instance_tier Basic