Efficient CI/CD Pipelines with Wavefront Multi-Threshold Alerts

May 22, 2019 Gaanesh Kapatralla

Today’s consumers expect continuous improvement in the apps they use. As a result, cloud-based application development teams need to innovate and release new app services and updates continually. An efficient alerting mechanism that augments the Continuous Integration and Continuous Delivery (CI/CD) pipeline is crucial to support this engineering and product velocity.

Wavefront is enhancing its alerts capability by introducing Threshold Mode, a new way to define and configure alerts. With Threshold Mode, you can define multiple severities and thresholds in a single alert, and you can notify a different alert target for each severity.

Classic Alerts

In Classic Mode, each Wavefront alert can be assigned to only one severity (Figure 1.). You define a condition on a metric and associate it with a single severity. When the condition is met, Wavefront notifies the target or targets. If you wanted to notify different people depending on the severity of the condition, you’d have to create 3 different alerts:

  • Alert 1: If CPU is over 95, send a SEVERE PagerDuty notification to the on-call SRE and the manager of the SRE group.
  • Alert 2: If CPU is over 90, send an INFO email to all SREs and a WARN email to the on-call SRE.
  • Alert 3: If CPU is over 80, send an INFO email to all SREs.

Figure 1. Classic Alerts: One Alert per Threshold

Multi-Threshold Alerts

In Threshold Mode, you can define different thresholds and assign a severity to each threshold (Figure 2.). For example:

  • If CPU is over 95, send a SEVERE PagerDuty notification to the on-call SRE and the manager of the SRE group.
  • If CPU is over 90, send a WARN email to the on-call SRE.
  • If CPU is over 80, send an INFO email to all SREs

Figure 2: Multi-Threshold Alerts: Single Alert with Multiple Severities, Thresholds, and Targets

Reduce the Number of Alerts in Your Environment

The benefits of introducing Threshold Mode alerts are:

  • Ease of Administration: Using Threshold Mode increases your overall efficiency by reducing the number of alerts that you need to set up and manage in your environment. You can have a single alert on a metric with multiple severities, as shown in Figure 2.
  • More flexibility: You can notify certain people, e.g., a manager, only when the alert becomes severe. At the same time, all users who receive notifications for lower-level targets continue to be notified as long as that target threshold is met. For example, if you send a SEVERE notification to the manager if CPU is over 95, then the condition for the other thresholds is still met and those notifications continue to be sent.

Start Benefiting From Enhanced Alerting

To experience Threshold Mode for alerts, create a new alert, and choose Threshold as the Type. Then, associate each severity of interest with a threshold value and notify different people depending on the severity of the condition. This threshold values will trigger the alerts at the specified severities.

With Wavefront multi-threshold alerts, you reduce the number of alerts you need to set up and manage. Also, with more flexible alert-tailoring to your DevOps team members, you speed up the customization of your CI/CD pipeline.

If you’re not yet an existing Wavefront customer, check out these new alerting features via a Wavefront free trial. If you have any questions, or you want to share your feedback, please slack us.

Get Started with Wavefront Follow @Gaanesh_K Follow @WavefrontHQ

Co-Author: Chung Kung Wang is a is a Member of Technical Staff at VMware. He is working as a full-stack engineer in the Wavefront team. Chung Kung Wang used to be a gamer, and now he’s trying to beat this game called Life. He loves to exercise in his spare time and has been hooked on climbing recently.

The post Efficient CI/CD Pipelines with Wavefront Multi-Threshold Alerts appeared first on Wavefront by VMware.

About the Author

Gaanesh Kapatralla

Gaanesh Kapatralla is a Senior Product Manager (SPM) at Wavefront, now VMware. He was previously a SPM at Dell Emc where he managed the conception and launch of VxBlock, a hyper converged infrastructure solution. Prior to that he built forecasting models using Machine Learning and automated the credit risk platform at Toyota Financial Services in UK. He holds a double Masters in Computer Science from University of Hertfordshire, and Harrisburg University, and an MBA from University of California in Davis.

Follow on Twitter More Content by Gaanesh Kapatralla
Previous
Wavefront Multicloud Observability Helps Bossa Studios Improve Its Gamers’ Experience
Wavefront Multicloud Observability Helps Bossa Studios Improve Its Gamers’ Experience

I recently had a chat with Nick Warr, Head of IT at Bossa Studios. We talked about how Bossa’s DevOps team ...

Next
How to Auto-Detect Cloud App Anomalies with Analytics: 10 Smart Alerting Examples  – Part 4
How to Auto-Detect Cloud App Anomalies with Analytics: 10 Smart Alerting Examples – Part 4

When supporting some of the world’s largest and most successful SaaS companies, we at Wavefront by VMware g...

SpringOne. Catch all the highlights

Watch now