Skip to content

Unsloth

In this example, we will create an 3rd Party App Marketplace SKU for Unsloth Studio. This is distributed as a Docker Container via DockerHub.

Prerequisites

As a service provider, you already have completed the following:

  1. Provisioned a Kubernetes cluster (e.g. Rafay MKS) with a GPU
  2. Your Cluster has connectivity to the Internet to download container images from DockerHub
  3. Deployed a cluster blueprint with critical addons (i.e. Ingress Controller, GPU Operator, etc)
  4. Access to a domain (e.g. paas.demo.gorafay.net) with TLS certificates to support https based access via the user's web browser

Info

If you are operating in an air gapped or locked down environment, please download and host the container image in an internally accessible container registry.

Create Unsloth Studio App SKU

  • Login into the "Default Org" and navigate to the SKU Studio
  • Click on Service Profiles and click on "New Service Profile"

This will launch a wizard to create a new service profile.

  • Provide a unique name (e.g. unsloth)
  • Provide an optional Display Name (e.g. Unsloth Studio)
  • Provide an optional description (This will be presented to the end user when they select and launch the card for the SKU)

Create Service SKU

  • For Environment Template, select "system-serverless-pods"
  • For version, select the latest version (e.g. v7.1)
  • For Service Type, select from the dropdown list (managed by the super admin role)
  • Enable the toggle for auto creation of compute
  • Save & Continue

SKU Configuration

General

Update the fields in the "general" section if needed

Card Design

Card Design

Info

Click on "Preview" to view what the readme will look like for the end user.

Input Settings

In this section, you get to specify

  • Default values for required inputs
  • Which inputs can be overridden by end users
  • How the inputs can be selected by end users (e.g. dropdown list, text box etc)

Here is the documentation to run Unsloth

docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

Although the docker command lists multiple ports (webapp, Jupyter notebook, ssh), we will limit user access only via the Unsloth web application.

Info

Due to security considerations, service providers will want the web app be accessible only on "port 443" (https).

User Overrides

Disable "Allow Overrides for All" This disables all overrides for end users when they use it. We will selectively enable some fields in the subsequent steps.

Default Inputs

In this step, you will customize the standard profile with "default" values that are relevant for your deployment.

Name Value Override
Access Mode ReadWriteOnce NO
Container Port 8000 NO
CPU 2000 NO
Enable Storage true NO
Enable Web Access true NO
GPU Count Whole Number YES
Host Cluster Name Host Cluster NO
Hostname Suffix Hostname Suffix NO
Ingress Class Name Ingress Class Name NO
Kubeconfig Copy/Paste Kubeconfig NO
Memory 16000 NO
Node Type Value matching inventory label "node_type" NO
Pod Image unsloth/unsloth NO
Storage Class Storage Class Name NO
Storage Path /workspace/work NO
Storage Size Size of PVC (e.g., 10Gi) NO

Once configured, it should look like the image below.

Input

Hostname Suffix

Specify the DNS hostname that you intend to use to serve the web applications. In our example, we are using "paas.demo.gorafay.net". Every instance of the web app can be accessed on a dynamically generated URL pattern that will look like "https://random prefix.paas.demo.gorafay.net"

Kubeconfig

  • In the Infrastructure portal, click on the "gear" symbol on the far right of the target Kubernetes cluster
  • Select "Download ZTKA Kubeconfig"
  • Open the downloaded file with an IDE
  • Copy the content and paste it into the Kubeconfig field in the above step.

Download Kubeconfig

Output Settings

Once the web application has been successfully deployed, we need to help user quickly find the URL for the app.

  • Click on "Add Output"
  • Under name, enter "Host Name"
  • Under label, enter "Unsloth Studio URL"

Outputs

Info

The name "Host Name" will dynamically retrieve the https URL for the deployed web app.

Schedules & Actions

Skip these

Click on "Save" to save the profile. Now, we have a ready to use the Unsloth Studio App SKU. It should look like the following

Saved SKU

Test Unsloth Studio App SKU

Now, we are ready to test our newly created Unsloth Studio App SKU.

  • In the SKU Studio, click on "Service Instances"
  • Ensure the "system-catalog" project is selected
  • Click on "New Service Instance"
  • Click Select on the Unsloth Studio Card

New App

  • Enter a name for the instance
  • Click Save & Continue

New App

  • Click Deploy

New App

Within 1-2 minutes, you should see a successfully deployed instance. See example below.

Successful Instance

Now, click on the URL and you should be taken to your Unsloth Studio App. You should see something like the image below.

Successful Access

Share SKU with Customer Orgs

At this point, although the SKU works, it is not available to Customer Orgs automatically. Service Providers need to perform the following to make the SKU available to customer orgs

  • Login into the Rafay Ops Console
  • Click on System Resources
  • Click on Service Profiles

You should now see the newly created SKU (e.g. unsloth).

  • Click on Actions to initiate SKU sharing
  • You will be presented with the option to share the SKU with "Select Orgs" or "All Orgs"
  • Select one of these options and the SKU will now be immediately available and usable by the selected customer orgs.

Share SKU