Skip to content

Explainer

The code in this Jupyter notebook is implementing a simple Katib experiment for hyperparameter tuning. This notebook runs a simple Katib experiment that optimizes two hyperparameters (a and b) to maximize the result of an objective function. We will use Katib to perform hyperparameter tuning across 12 trials and use 2 CPU cores per trial. Once all trials are complete, it returns the best-performing hyperparameters.

Now, let's look at the code block and explain what is happening underneath the covers.

Step 1: Install Katib Packages

!pip install --user kubeflow-katib

This command installs the kubeflow-katib Python package, which allows the use of Katib functionalities in the notebook.

Note

After installing the Katib package in a Kubeflow notebook, you generally need to restart the kernel for the newly installed package to be recognized. This is because the package is not automatically available to the current session until the environment is reloaded.

Step 2: Define an Objective Function

In Katib, the objective function is the function that Katib tries to optimize during hyperparameter tuning. It evaluates the performance of a machine learning model based on a set of hyperparameters, and returns a performance metric (like accuracy, loss, or any other relevant measure). The goal of the optimization process is to either maximize or minimize this objective function, depending on the task.

def objective(parameters):
    import time
    time.sleep(5)
    # Calculate objective function.
    result = 4 * int(parameters["a"]) - float(parameters["b"]) ** 2
    # Katib parses metrics in this format: <metric-name>=<metric-value>.
    print(f"result={result}")

This function represents the objective function to be optimized. It takes hyperparameters a and b as input and the result is calculated using the expression 4 * a - b^2. The objective function’s goal is to maximize the result of this mathematical expression. The goal of the Katib experiment is to find the combination of hyperparameters a and b that maximize the value of this result metric. The optimization process will adjust a and b within the given search space to achieve the highest possible value of the result.

  • Hyperparameter a: This contributes positively to the result because it’s multiplied by 4. Increasing a within its range (10 to 20) will generally lead to a higher result.

  • Hyperparameter b: Subtracts from the result, as it is squared. Larger values of b will have a stronger negative impact on the result.

Note that Katib expects the result to be printed in the format "=".

Step 3: Define Hyperparameter Search Space

In Katib, the hyperparameter search space refers to the range or domain of possible values that Katib explores for each hyperparameter during the tuning process. This space is defined for each hyperparameter, and Katib uses optimization strategies (e.g., random search, Bayesian optimization, grid search) to intelligently search through this space to find the best-performing set of hyperparameters.

parameters = {
    "a": katib.search.int(min=10, max=20),
    "b": katib.search.double(min=0.1, max=0.2)
}

This code defines the hyperparameter search space:

  • a: An integer value between 10 and 20.
  • b: A floating-point (double) value between 0.1 and 0.2.

These are the hyperparameters Katib will search over to optimize the objective function.

Step 4: Create Katib Experiment

katib_client = katib.KatibClient(namespace="tim")

name = "tune-experiment"
katib_client.tune(
    name=name,
    objective=objective,
    parameters=parameters,
    objective_metric_name="result",
    max_trial_count=12,
    resources_per_trial={"cpu": "2"},
)

This section creates a Katib experiment:

  • katib_client: The Katib client is initialized in the namespace "tim".
  • The experiment name is "tune-experiment".
  • The tune() function launches the experiment:

It will attempt to maximize the “result” metric (calculated by the objective function). The experiment will run for a maximum of 12 trials, each trial using 2 CPU cores.

Step 5: Experiment Completion

katib_client.wait_for_experiment_condition(name=name)

This line waits for the Katib experiment to complete.

Info

It will block the execution until the experiment finishes running all trials or meets the defined criteria.

Step 6: Retrieve the Best Hyperparameters

print(katib_client.get_optimal_hyperparameters(name))

Once the experiment is complete, we will retrieves the best set of hyperparameters that Katib found during the search process.