Skip to content

Evidently

Evidently helps evaluate, test, and monitor data and ML-powered systems. Evidently is both a library of 100+ ready-made evaluations, and a framework to easily implement yours: from Python functions to LLM judges.

Evidently has a modular architecture, and you can start with ad hoc checks without complex installations. There are 3 interfaces with which you can:

  • Get a visual report with a summary of evaluation metrics
  • Run conditional checks with a TestSuite to get a pass/fail outcome, and
  • Plot the evaluation results over time on a Monitoring Dashboard.

Watch a video of this integration below.

Configuration

The below content provides a sample notebook which can be used to create and show both Test Suite results and a Data Drift Report within Evidently Cloud (i.e. their SaaS offering).

You can create a free account here

Within you Rafay Kubeflow based MLOps environment:

  • Navigate to Notebooks
  • Click New Notebook
  • Enter a name for the notebook
  • Select JupyterLab
  • Set the minimum CPU to 1
  • Set the minimum memory to 1
  • Click Launch

Launch

It will take 1-2 minutes to create the notebook.

Launch

  • Navigate to Notebooks
  • Click Connect on the previously created notebook

Download Notebook

  • In the left hand folder tree, click on the upload files icon
  • Upload the previously downloaded evidently.ipynb file
  • Double click the evidently.ipynb file in the folder tree to open the notebook

Credentials

You need to provide credentials for the notebook to connect to your Evidently Cloud account and aggregate model metrics etc.

  • Update the section in the notebook that contains ENTER API KEY with the value of your API key found here
  • Update the sections in the notebook that contains ENTER PROJECT UUID with the value of your project UUID found here and then be either selecting or creating a project
  • Click the Restart kernel and run all cells icon

Model Metrics

The Test suite dashboard will be sent to your Evidently Cloud account and can be accessed by going to your Evidently project and selecting Test Suites on the left hand tree

TestSuite

  • Click Explore on the test suite that was created

TestSuite

Drift Reports

The Data Drift Report will be sent to your Evidently Cloud account and can be accessed by going to your Evidently project and selecting Reports on the left hand tree

Data Drift

  • Click ** Explore** on the report that was created

Data Drift