Skip to content

Notifications

Customers can optionally configure the platform to automatically send "notifications" when Alerts are generated (i.e for both alerts being opened and closed).

Notifications can be configured to be sent to a list of "email addresses" that are specified and managed by the Org Admins. Alternately, customers can also customize the Alert Manager add-on which is part of the default cluster blueprint to send the notifications directly to 3rd party systems such as "Slack", "Pager Duty", "Ops Genie" etc.

Opt In Clusters

By default, email based notifications are disabled for all clusters under management. Administrators need to explicitly enable clusters for which they would like to receive email based notifications from the Controller.

  • Click on Notifications on the cluster card
  • Toggle notifications to enabled

Enable Notifications by Cluster

Email Notifications

It is strongly recommended that customers configure a monitored email address for notifications. It may also be useful to implement customized email handling logic and workflows to process the incoming notifications.

Configuration

To manage the notifications at Organization level, perform the below steps:

  • Navigate to System -> Alerts
  • Click on Manage Notifications

Manage Notifications

  • Enable/disable the required notification(s)
  • Click Add Email to add one or more users to receive the selected alert notification(s)
  • Click Save

Manage Notifications

Metadata

Email notifications are sent with meta-data in a specific format so that they can "programmatically parsed" by incident response systems as well as consumed by end users.

The meta-data in the email comprises "Key-Value" pairs that can be automatically parsed and used for downstream workflows. An illustrative example of a notification is shown below.

Notification Email

Customize Alert Manager

You can optionally customize and configure the Alert Manager "system addon" in the default cluster blueprint to send notifications to your existing incident response systems like Slack, Pager Duty, Ops Genie etc.

Step 1

  • Go to Infrastructure > Addons
  • Create a new addon and select the "Customize System Addon" option.

Custom AlertManager

Step 2

  • Create a new version for the custom system Alert Manager addon.

Custom AlertManager

Step 3

  • Download the default configuration manifests and customize them.

Custom AlertManager

Step 4

  • Upload the modified configuration to the addon
  • Save the custom Alert Manager addon version

Custom AlertManager

Step 5

  • Go to Infrastructure > Blueprints
  • Add this custom Alert Manager add-on to the custom blueprint for the cluster

Custom AlertManager

Step 6

  • Update the new blueprint version with the customized Alert Manager addon to the clusters
  • The new system Alert Manager addon will be deployed to the clusters

Custom AlertManager

After couple minutes, the blueprint will be published successfully and the customized Alert Manager addon will be deployed to the clusters.

Custom AlertManager

Integration Recipes

Step-by-step recipes to customize the Alert Manager configuration for proactive notifications are documented below. Also, additional information on Alert Manager configuration is available.

Integration Recipe
Slack
Pager Duty
Ops Genie
MSFT Teams
ServiceNow