Anomaly Detection

The Anomaly Detection feature uses your cloud’s billing data to detect and flag abnormal spend events down to an hourly granularity.

Overview

While you can preset thresholds to be alerted through Budgets and get notifications of trends on a weekly basis, the unplanned and sudden spikes in spend can become costly if they go unnoticed. Anomaly Detection will alert the channels and emails associated with a View about spikes in spend that have not been seen through historical trends. This can help reduce and prevent unplanned expenses on events such as bugs in new deployments, tests that have been forgotten, and other unplanned and accidental situations.

How It Works

Anomaly Detection is automatically enabled across your CloudZero account and all views using Real Cost data, see Real Cost. It checks globally across the Cloud Provider Dimensions of Accounts, Service, and Usage Family. In addition for each View that is created, Anomaly Detection is enabled for that subset of data.

Cost Impact

Cost impact for detected anomalies is the difference between what the model calculates the expected cost to be from what the cost actually is over the anomaly period.

Anomaly Detection Thresholds

An Anomaly Threshold is the minimum amount that the spend must exceed to be considered an anomaly.

By default, the automatic anomaly threshold is enabled to determine if anomalous spend is found globally or within any Views. To further refine the threshold, a manual threshold can be set as a percentage of the View's daily spend. Note: Setting a manual threshold is only available for View anomalies, global anomalies use the automatic anomaly threshold.

Automatic Anomaly Threshold

Automatic thresholds look at a sliding scale based on the previous 30 days of spend. The following table outlines the default thresholds for various levels of spend in the 30 day window. Note: For Global anomalies the 30 Day Spend is total cloud spend and for View anomalies it is total View spend.

Setting a Manual Threshold

Thresholds for each view can be set manually to override the default thresholds noted above. To edit thresholds, a user must have an Organizer permission set on their account.

1. Navigate to Views under the Settings tab in the left-hand navigation.
2. On the View you want to adjust the threshold, click the 3 dots then select “Edit”.
3. Scroll down to Threshold section and select Manual
   
4. Enter a percentage of the View’s spend. This percentage represents the percentage of the average daily spend over the last 30 days, and is applied to the individual elements of the principal dimension. An anomaly will be triggered on an individual element if the spend increase over 24 hours is equal to or greater than manual threshold. For example, a view with $1000 of daily spend set to 50% would have a threshold of $500 and would trigger an Anomaly if the spend for an individual element in the Views principal dimension increases by $500 over 24 hours. Another example, if the dimension is Teams, an anomaly would trigger if Team A increases from $300 to $800 (+$500 over a 24 hour period).
5. Click ‘Save’

Disabling Anomaly Detection

Anomaly Detection is enabled by default for all Views. You can disable Anomaly Detection for a specific View by editing the View. See editing Views.
Simply toggle the View Anomalies switch to disable Anomaly Detection.

Viewing Anomalies

Anomalies can be viewed under Insights as well as in the Homepage, Explorer, and Notifications

The Anomaly Details page is functionally similar to the Insights detail page and includes the following additional details:

• Description: includes Anomaly start time and whether that spend is ongoing or not.
  • Note: Anomalies that reoccur will create a new Anomaly.
• View Name, Principal Dimension and Element are listed under the description to help distinguish Anomalies.
• 90 day daily cost graph with Anomaly start time highlighted to help quickly identify if the anomaly is a historical trend.
• Link to Details: View in Explorer link takes you to the Explorer for more granular exploration. The anomaly will be expanded in the Anomalies and Events pop out and other anomaly and event lines will be grayed out.

The home page Main Dashboard provides an overview of the total number of Anomalies in the past 30 days and the total cost of detected anomalies in the last 30 days. By clicking on an anomaly next to the pie chart, you will be taken to the detail page for that Anomaly.

The Explorer view of anomalies will provide the most granular details about the anomaly. To access anomalies in the Explorer, select “Anomalies & Events”. When an anomaly is selected, the anomaly will expand with more details and other Anomaly and Events bars will be grayed out. Changing your time granularity to “Hourly” will provide a precise view for when the Anomaly was detected.

Anomaly alerts will also be visible in the Notifications tab along the top right of the navigation. Selecting an anomaly next will will take you the detail page in Insights similarly to selecting one on the Home Page.

API and Exports

Anomalies can be exported to CSV via the Export button on the Insights page or on the detail page for the anomaly. Anomalies are also available via API at the /v2/insights end point.

External Alerts

By default Anomaly Detection alerts are sent via email to all Admin users in your organization and can be updated in the View settings for ‘Global View’. Notifications can also be delivered directly to relevant teams by creating a View, see Creating a View. The notifications can be delivered to an email address or Slack channel. To learn more about enabling Slack notifications, see Enabling Slack Integration.

When an Anomaly is detected, a notification will be sent once. The system will not continue to send notifications on the specific anomaly so as to prevent noise and spamming users.