Usage by GPU Type
Cloud Providers need a way to report aggregate GPU-hours used by customers in their datacenters. With Rafay GPU PaaS, cloud providers can retrieve this usage data by SKU so that they have visibility into which GPU (models/type) are used for how long across tenants.
Add GPU Type to SKU¶
Consider a scenario where a cloud provider offers three SKUs, each with with different GPU models/types.
| SKU | GPU Model |
|---|---|
| Small | Nvidia L40 |
| Medium | Nvidia A100 |
| Large | Nvidia H100 |
Info
Each SKU offered by the cloud provider will map to a compute profile in Rafay GPU PaaS.
For each profile (SKU), the cloud provider needs to append a label or annotation to the billable parameters section with the GPU model or type. This information can then be retrieved programmatically when the usage APIs are called.
In the example below, the GPU model Nvidia H100 has been added as an annotation to the compute profile backing the SKU.
The billable parameters config should always be defined in the extraConfig section of the compute profile. This ensures that the data is persisted with the profile and can be retrieved later when required. In the example below, the annotation "gpu_type" has been mapped to the "extra_config".
Important
billableParameters config should always be defined in extraConfig. The actual parameters that are added to billable parameters can be in labels OR annotations OR variables OR extraconfig.
Retrieve Usage by GPU Type¶
To retrieve the usage data by GPU Type, we will use the Usage API supported by Rafay GPU PaaS.
In the example below, we use Rafay's API to retrieve usage data from June 1 to July 1, 2025 (UTC) i.e. a 30-day usage window.
https://<controller_url>/apis/billing.envmgmt.io/v1/metrics/partner/profile/kind/compute/usage?range_from=2025-06-01T00%3A00%3A00Z&range_to=2025-07-01T00%3A00%3A00Z&offset=0&limit=10
This API call will return usage data per SKU (i.e. compute profile) organized by Org for all Orgs. In the table below, we can see that the Nvidia L40 is used for different hours by the three Orgs (A, B and C).
| SKU | GPU Model | Org | Usage |
|---|---|---|---|
| Small | Nvidia L40 | Org-A | 30 hours |
| Org-B | 60 hours | ||
| Org-C | 10 hours | ||
| Medium | Nvidia A100 | Org-A | 10 hours |
| Org-B | 20 hours | ||
| Org-C | 30 hours | ||
| Large | Nvidia H100 | Org-A | 100 hours |
| Org-B | 40 hours | ||
| Org-C | 33 hours |
To calculate the aggregate usage hours per GPU Model/type across all Orgs, the cloud provider will need to add the usage for all Orgs. In the example above, for the Nvidia L40 GPU type, the total usage in the time period would be “30+60+10” = 100 hours.