CLI
Users can use RCTL to create the Resources Quotas with a Yaml spec file.
The values ClusterResourceQuota and DefaultClusterNamespaceQuota are included in the project YAML spec file. These fields are not mandatory and contain optional subfields of various resources.
-
ClusterResourceQuota: Implies the quota available for each cluster present under a specific project
-
DefaultClusterNamespaceQuota: Implies the default quota that is available to a namespace under a project
The total namespace quota utilization for a project cannot exceed the available project quota.
Below is an illustrative example of the YAML spec file to configure the resource quotas for the project demo-project
apiVersion: system.k8smgmt.io/v3
kind: Project
metadata:
name: demo-project
spec:
clusterResourceQuota:
configMaps: "10"
cpuLimits: 2000m
cpuRequests: 1000m
memoryLimits: 2048Mi
memoryRequests: 1024Mi
persistentVolumeClaims: "10"
pods: "10"
replicationControllers: "10"
secrets: "10"
services: "10"
servicesLoadBalancers: "10"
servicesNodePorts: "10"
defaultClusterNamespaceQuota:
configMaps: "5"
cpuLimits: 1000m
cpuRequests: 500m
memoryLimits: 1024Mi
memoryRequests: 512Mi
persistentVolumeClaims: "5"
pods: "5"
replicationControllers: "5"
secrets: "5"
services: "5"
servicesLoadBalancers: "5"
servicesNodePorts: "5"
Create Resource Quota¶
The below create command enables the users to create a project with the specified Resource Quotas configuration
.rctl create project -f <filename.yaml>
Example
.rctl create project -f project1_quotas.yaml
Validations¶
The below validations are mandatory to complete a Resource Quotas configuration on a project:
- Configuration values of DefaultClusterNamespace is mandatory when Quota ClusterResourceQuota is specified and vice-versa
- Number of fields defined in ClusterResourceQuota & DefaultClusterNamespaceQuota must be the same
- Same number of fields with same resources specified/ configured in ClusterResourceQuota & DefaultClusterNamespaceQuota
- Quota assigned per field in DefaultClusterNamespaceQuota must not exceed in value with the corresponding field’s value in ClusterResourceQuota
KubeCTL¶
View Quota Configuration Status¶
On successful project creation, the quota configurations are automatically applied to the clusters deployed within the specified project.
- Login to the console and access your project
- Select the required cluster and click Kubectl
- Enter the command
get crdin the Kubectl screen to know whether the Resource Quotas configuration is applied to the clusters within a project - Output with the resource
projects.system.k8smgmt.ioconfirms the quota deployment
kubectl get crd
NAME CREATED AT
bgpconfigurations.crd.projectcalico.org 2022-05-20T04:22:38Z
bgppeers.crd.projectcalico.org 2022-05-20T04:22:38Z
blockaffinities.crd.projectcalico.org 2022-05-20T04:22:38Z
blockdeviceclaims.openebs.io 2022-05-20T04:27:00Z
blockdevices.openebs.io 2022-05-20T04:27:00Z
clusterinformations.crd.projectcalico.org 2022-05-20T04:22:38Z
felixconfigurations.crd.projectcalico.org 2022-05-20T04:22:38Z
globalnetworkpolicies.crd.projectcalico.org 2022-05-20T04:22:39Z
globalnetworksets.crd.projectcalico.org 2022-05-20T04:22:39Z
hostendpoints.crd.projectcalico.org 2022-05-20T04:22:39Z
ipamblocks.crd.projectcalico.org 2022-05-20T04:22:39Z
ipamconfigs.crd.projectcalico.org 2022-05-20T04:22:39Z
ipamhandles.crd.projectcalico.org 2022-05-20T04:22:39Z
ippools.crd.projectcalico.org 2022-05-20T04:22:39Z
kubecontrollersconfigurations.crd.projectcalico.org 2022-05-20T04:22:39Z
namespaces.cluster.rafay.dev 2022-05-20T04:28:09Z
networkpolicies.crd.projectcalico.org 2022-05-20T04:22:39Z
networksets.crd.projectcalico.org 2022-05-20T04:22:39Z
projects.system.k8smgmt.io 2022-05-20T18:13:25Z
tasklets.cluster.rafay.dev 2022-05-20T04:28:09Z
tasks.cluster.rafay.dev 2022-05-20T04:28:09Z
View Per Cluster Resource limits¶
To view the resource quota details deployed to a cluster, use the command get projects.system.k8smgmt.io -o yaml (or) get projects -o yaml. Below is an example of the output
Note: Users can make the changes to the Resources Quota through the controller and those changes will also get updated when viewing this output.
View Namespace Resource Quota details¶
To view the namespace and its allotted resourced limits, use the command get ns <namespace-name> -o yaml. Below is an illustrative example of a namespace and its configuration details
View Quota Request and limits¶
Use the command get resourcequotas --all-namespaces to view the list of namespaces, resource request and the Utilizations
kubectl get resourcequotas --all-namespaces
NAMESPACE NAME AGE REQUEST LIMIT
ns2 ns2-resource-quota 3h10m configmaps: 1/2, pods: 0/2, secrets: 2/2, services.nodeports: 0/2
ns3 ns3-resource-quota 3h9m configmaps: 1/2, pods: 0/2, secrets: 2/2, services.nodeports: 0/2
ns4 ns4-resource-quota 3h8m configmaps: 1/2, pods: 0/2, secrets: 2/2, services.nodeports: 0/2
ns5 ns5-resource-quota 3h7m configmaps: 1/2, pods: 2/2, secrets: 2/2, services.nodeports: 0/2
ram-ns4 object-counts 114m configmaps: 1/10, persistentvolumeclaims: 0/4, pods: 0/4, replicationcontrollers: 0/20, requests.cpu: 0/1, requests.memory: 0/1Gi, requests.nvidia.com/gpu: 0/4, secrets: 2/10, services: 0/10, services.loadbalancers: 0/2, services.nodeports: 0/2 limits.cpu: 0/2, limits.memory: 0/2Gi
View a namespace resource allocation¶
To view the resource allocation of a specific namespace, use the command describe resourcequotas -n <namespace>
The below example shows the resource details of a namespace ns2.
kubectl describe resourcequotas -n ns2
Name: ns2-resource-quota
Namespace: ns2
Resource Used Hard
-------- ---- ----
configmaps 1 2
pods 0 2
secrets 2 2
services.nodeports 0 2
Note: Only the configured resources appears in the output
Cluster Quota Override¶
Users can override specific resource quotas at the project level using the replace operation. To provide more flexibility, a YAML-based quota override specification allows configuring resource quotas on a per-cluster basis, enabling users to manage quotas for one or more required clusters as needed.
Below are examples for both V3 and V2 YAML specifications to illustrate how these overrides can be set up and applied.
V3 Configuration (Recommended)¶
Step 1: Prepare the Value File
- Create a value file to define the required resources and quotas as shown below:
apiVersion: system.k8smgmt.io/v3
kind: Project
metadata:
name: defaultproject
patch:
- op: replace
path: /spec/clusterResourceQuota/pods
value: 200
- op: replace
path: /spec/clusterResourceQuota/cpuLimits
value: 400m
- op: replace
path: /spec/clusterResourceQuota/cpuRequests
value: 200m
Step 2: Define the Quota Override Specification
- Use the following YAML spec, referencing the value file
apiVersion: infra.k8smgmt.io/v3
kind: ClusterQuotaOverride
metadata:
labels:
rafay.dev/overrideScope: clusterLabels
rafay.dev/overrideType: manifestsFile
name: v3quotaoverridedemo
project: defaultproject
spec:
placement:
labels:
- key: rafay.dev/clusterName
value: democluster
resource:
selector:
selector: rafay.dev/system=true,rafay.dev/component=cluster-resource-quota
type: ClusterQuota
type: Yaml
valuesPath:
name: file://overrides/quota_override_values.yaml
Step 3: Apply the Configuration
- Run the following command to create the cluster quota override:
./rctl apply -f <quotaoverride_filename.yaml>
V2 Configuration¶
Step 1: Define the Quota Override Specification
- Below is the YAML specification to configure a Cluster Quota Override for v2:
kind: ClusterOverride
apiVersion: config.rafay.dev/v2
metadata:
name: quotaoverridedemo
spec:
resourceSelector: rafay.dev/system=true,rafay.dev/component=cluster-resource-quota
type: ClusterOverrideTypeClusterQuota
overrideValues: |
apiVersion: system.k8smgmt.io/v3
kind: Project
metadata:
name: defaultproject
patch:
- op: replace
path: /spec/clusterResourceQuota/pods
value: 200
---
clusterPlacement:
placementType: ClusterSpecific
driftAction: DriftReconcillationActionNotSet
clusterLabels:
- key: rafay.dev/clusterName
value: democluster
artifactType: NativeYAML
Step 2: Apply the Configuration
- Run the following command to apply the specification:
./rctl create co -f <quotaoverride_filename.yaml>