API Keys

Developers and users will use the Inference Endpoint via an API. This provides a simple interface to state-of-the-art AI models for text generation, natural language processing, computer vision, and more.

Rafay's Serverless Inference offering implements OpenAI's Completions API, Chat API etc. This allows applications to programmatically interact with the Inference endpoint using an HTTP client.

Users are required to authenticate with the Serverless Inference Endpoint using an API Key before they can use it.

from openai import OpenAI
client = OpenAI(
    base_url="http://<Rafay Inference Endpoint>",
    api_key="token-abc123",
)

completion = client.chat.completions.create(
    model="NousResearch/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "user", "content": "Hello!"}
    ]
)

print(completion.choices[0].message)

---

Create API Key

They can create "API Keys" from the self service portal for authentication.

• Click on API Keys
• Click on New API Key
• Provide a user friendly name and an optional description
• Click on Create

Once the user clicks on "Create", the API Key is displayed to the user.

Important

Please make sure you copy the API Key Details and store in a secure location. This information is not accessible later.

---

Export API Key

Once you've generated an API key, export it as an environment variable in your terminal so that you can use it programmatically.

macOS and Linux

Export the API Key as an environment variable on macOS or Linux.

export API_KEY="your_api_key_here"

Windows

Export an environment variable in PowerShell

setx API_KEY "your_api_key_here"

---

List API Keys

Users can have multiple API Keys at the same time. Users are shown the list of "active" API Keys that they created.

Info

The "Created on" column indicates when the API key was created by the user.

---

Delete API Key

Under Actions on the far right of an API Key, click on the "..." (Ellipses). Now click on Delete. Acknowledge the deletion by clicking on "Delete".

Important

This action is not reversible. You cannot recover the API Key once it has been deleted.