Skip to content

CLI

Users can use RCTL to create the Resources Quotas with a Yaml spec file.

The values ClusterResourceQuota and DefaultClusterNamespaceQuota are included in the project YAML spec file. These fields are not mandatory and contain optional subfields of various resources.

  • ClusterResourceQuota: Implies the quota available for each cluster present under a specific project

  • DefaultClusterNamespaceQuota: Implies the default quota that is available to a namespace under a project

The total namespace quota utilization for a project cannot exceed the available project quota.

Below is an illustrative example of the YAML spec file to configure the resource quotas for the project demo-project

apiVersion: system.k8smgmt.io/v3
kind: Project
metadata:
  name: demo-project
spec:
  clusterResourceQuota:
    configMaps: "10"
    cpuLimits: 2000m
    cpuRequests: 1000m
    memoryLimits: 2048Mi
    memoryRequests: 1024Mi
    persistentVolumeClaims: "10"
    pods: "10"
    replicationControllers: "10"
    secrets: "10"
    services: "10"
    servicesLoadBalancers: "10"
    servicesNodePorts: "10"
  defaultClusterNamespaceQuota:
    configMaps: "5"
    cpuLimits: 1000m
    cpuRequests: 500m
    memoryLimits: 1024Mi
    memoryRequests: 512Mi
    persistentVolumeClaims: "5"
    pods: "5"
    replicationControllers: "5"
    secrets: "5"
    services: "5"
    servicesLoadBalancers: "5"
    servicesNodePorts: "5"

Create Resource Quota

The below create command enables the users to create a project with the specified Resource Quotas configuration

.rctl create project -f <filename.yaml>

Example

.rctl create project -f project1_quotas.yaml

Validations

The below validations are mandatory to complete a Resource Quotas configuration on a project:

  • Configuration values of DefaultClusterNamespace is mandatory when Quota ClusterResourceQuota is specified and vice-versa
  • Number of fields defined in ClusterResourceQuota & DefaultClusterNamespaceQuota must be the same
  • Same number of fields with same resources specified/ configured in ClusterResourceQuota & DefaultClusterNamespaceQuota
  • Quota assigned per field in DefaultClusterNamespaceQuota must not exceed in value with the corresponding field’s value in ClusterResourceQuota

KubeCTL

View Quota Configuration Status

On successful project creation, the quota configurations are automatically applied to the clusters deployed within the specified project.

  • Login to the console and access your project
  • Select the required cluster and click Kubectl
  • Enter the command get crd in the Kubectl screen to know whether the Resource Quotas configuration is applied to the clusters within a project
  • Output with the resource projects.system.k8smgmt.io confirms the quota deployment
kubectl get crd
NAME                                             CREATED AT
bgpconfigurations.crd.projectcalico.org               2022-05-20T04:22:38Z
bgppeers.crd.projectcalico.org                        2022-05-20T04:22:38Z
blockaffinities.crd.projectcalico.org                 2022-05-20T04:22:38Z
blockdeviceclaims.openebs.io                          2022-05-20T04:27:00Z
blockdevices.openebs.io                               2022-05-20T04:27:00Z
clusterinformations.crd.projectcalico.org             2022-05-20T04:22:38Z
felixconfigurations.crd.projectcalico.org             2022-05-20T04:22:38Z
globalnetworkpolicies.crd.projectcalico.org           2022-05-20T04:22:39Z
globalnetworksets.crd.projectcalico.org               2022-05-20T04:22:39Z
hostendpoints.crd.projectcalico.org                   2022-05-20T04:22:39Z
ipamblocks.crd.projectcalico.org                      2022-05-20T04:22:39Z
ipamconfigs.crd.projectcalico.org                     2022-05-20T04:22:39Z
ipamhandles.crd.projectcalico.org                     2022-05-20T04:22:39Z
ippools.crd.projectcalico.org                         2022-05-20T04:22:39Z
kubecontrollersconfigurations.crd.projectcalico.org   2022-05-20T04:22:39Z
namespaces.cluster.rafay.dev                          2022-05-20T04:28:09Z
networkpolicies.crd.projectcalico.org                 2022-05-20T04:22:39Z
networksets.crd.projectcalico.org                     2022-05-20T04:22:39Z
projects.system.k8smgmt.io                            2022-05-20T18:13:25Z
tasklets.cluster.rafay.dev                            2022-05-20T04:28:09Z
tasks.cluster.rafay.dev                               2022-05-20T04:28:09Z

View Per Cluster Resource limits

To view the resource quota details deployed to a cluster, use the command get projects.system.k8smgmt.io -o yaml (or) get projects -o yaml. Below is an example of the output

Project Dashboard

Note: Users can make the changes to the Resources Quota through the controller and those changes will also get updated when viewing this output.

View Namespace Resource Quota details

To view the namespace and its allotted resourced limits, use the command get ns <namespace-name> -o yaml. Below is an illustrative example of a namespace and its configuration details

Project Dashboard

View Quota Request and limits

Use the command get resourcequotas --all-namespaces to view the list of namespaces, resource request and the Utilizations

kubectl get resourcequotas --all-namespaces
NAMESPACE   NAME                 AGE     REQUEST                                                                                                                                                                                                                                                LIMIT
ns2         ns2-resource-quota   3h10m   configmaps: 1/2, pods: 0/2, secrets: 2/2, services.nodeports: 0/2                                                                                                                                                                                      
ns3         ns3-resource-quota   3h9m    configmaps: 1/2, pods: 0/2, secrets: 2/2, services.nodeports: 0/2                                                                                                                                                                                      
ns4         ns4-resource-quota   3h8m    configmaps: 1/2, pods: 0/2, secrets: 2/2, services.nodeports: 0/2                                                                                                                                                                                      
ns5         ns5-resource-quota   3h7m    configmaps: 1/2, pods: 2/2, secrets: 2/2, services.nodeports: 0/2                                                                                                                                                                                      
ram-ns4     object-counts        114m    configmaps: 1/10, persistentvolumeclaims: 0/4, pods: 0/4, replicationcontrollers: 0/20, requests.cpu: 0/1, requests.memory: 0/1Gi, requests.nvidia.com/gpu: 0/4, secrets: 2/10, services: 0/10, services.loadbalancers: 0/2, services.nodeports: 0/2   limits.cpu: 0/2, limits.memory: 0/2Gi

View a namespace resource allocation

To view the resource allocation of a specific namespace, use the command describe resourcequotas -n <namespace>

The below example shows the resource details of a namespace ns2.

kubectl describe resourcequotas -n ns2
Name:               ns2-resource-quota
Namespace:          ns2
Resource            Used  Hard
--------            ----  ----
configmaps          1     2
pods                0     2
secrets             2     2
services.nodeports  0     2

Note: Only the configured resources appears in the output


Cluster Quota Override

Users can override specific resource quotas at the project level using the replace operation. To provide more flexibility, a YAML-based quota override specification allows configuring resource quotas on a per-cluster basis, enabling users to manage quotas for one or more required clusters as needed.

Below are examples for both V3 and V2 YAML specifications to illustrate how these overrides can be set up and applied.

Step 1: Prepare the Value File

  • Create a value file to define the required resources and quotas as shown below:
apiVersion: system.k8smgmt.io/v3
kind: Project
metadata:
  name: defaultproject
patch:
- op: replace
  path: /spec/clusterResourceQuota/pods
  value: 200
- op: replace
  path: /spec/clusterResourceQuota/cpuLimits
  value: 400m
- op: replace
  path: /spec/clusterResourceQuota/cpuRequests
  value: 200m

Step 2: Define the Quota Override Specification

  • Use the following YAML spec, referencing the value file
apiVersion: infra.k8smgmt.io/v3
kind: ClusterQuotaOverride
metadata:
  labels:
    rafay.dev/overrideScope: clusterLabels
    rafay.dev/overrideType: manifestsFile
  name: v3quotaoverridedemo
  project: defaultproject
spec:
  placement:
    labels:
    - key: rafay.dev/clusterName
      value: democluster
  resource:
    selector:
      selector: rafay.dev/system=true,rafay.dev/component=cluster-resource-quota
    type: ClusterQuota
  type: Yaml
  valuesPath:
    name: file://overrides/quota_override_values.yaml

Step 3: Apply the Configuration

  • Run the following command to create the cluster quota override:
./rctl apply -f <quotaoverride_filename.yaml>

V2 Configuration

Step 1: Define the Quota Override Specification

  • Below is the YAML specification to configure a Cluster Quota Override for v2:
kind: ClusterOverride
apiVersion: config.rafay.dev/v2
metadata:
  name: quotaoverridedemo
spec:
  resourceSelector: rafay.dev/system=true,rafay.dev/component=cluster-resource-quota
  type: ClusterOverrideTypeClusterQuota
  overrideValues: |
    apiVersion: system.k8smgmt.io/v3
    kind: Project
    metadata:
      name: defaultproject
    patch:
    - op: replace
      path: /spec/clusterResourceQuota/pods
      value: 200
    ---
  clusterPlacement:
    placementType: ClusterSpecific
    driftAction: DriftReconcillationActionNotSet
    clusterLabels:
    - key: rafay.dev/clusterName
      value: democluster
  artifactType: NativeYAML

Step 2: Apply the Configuration

  • Run the following command to apply the specification:
./rctl create co -f <quotaoverride_filename.yaml>