New: Wavefront Distributed Tracing Enhanced with Span Logs, Alerting Context, and Unified Views

August 30, 2019 Rob Markovich

Earlier this week, we announced key enhancements to Wavefront’s application observability with support for span logs, context-enriched alerts, and unified status dashboards combining service, application, and the underlying system. In this blog, I will cover the details of Wavefront’s Integrated Application Observability and show you how it empowers SREs and developers to quickly troubleshoot any issue and find the root cause faster. To get more details, on how you can go from getting an alert to finding the relevant service, trace and log in seconds – check out the following demo.


With Wavefront, we provide a hosted Opentrace- and OpenCensus-compliant tracing solution. Having a hosted solution eliminates the headache of maintaining a tracing server which is already a huge benefit. However, the biggest benefit of Wavefront is in the tools that it provides on top of the traces to discover and troubleshoot issues. Furthermore, traces have been compared to logs by different industry experts. Logs are really revealing, however, navigation through vast amounts of logs is a very hard and expensive task. Tracing combined with metrics, histograms and span logs is a more cost-effective way of getting observability of your application.

Guided Troubleshooting using Metrics and Histograms

Wavefront has had Rate, Errors, and Duration (RED) metrics and histograms over tracing data for quite a while and provides a very easy to understand guided top down flow from applications to services dashboard that guides you to the right traces.

Observe how you can see the details of the RED metrics. If you see interesting information – like a spike in error rate or abnormal increase in request rate – you can quickly drill on the top requests below the chart and get directly to the related traces which will shed more light on the problem.

To make things easier Wavefront has enhanced its application observability with the following new capabilities:

  • Span Log Analysis
  • In Context Alerting
  • Unified Service and Application Status Pages

Investigate Faster with Span Logs

Wavefront now supports Span Logs for traces. These logs provide extra information about an error – such as the request timed-out or waiting on mutex lock – in a specific span. Instead of needing to go to an external tool to view application logs you can see them in one enterprise observability platform with no integration needed, and with no extra cost.

As you can see combining the depth of tracing with the strength of metrics and the details of logs provides a very clear and quick troubleshooting path. The main steps of this troubleshooting path are as follows:

  1. Start from the metrics and the red color of the services or an alert
  2. Drill down on the spikes in the charts
  3. Select the most outlier traces to investigate the issue
  4. Follow the orange-colored critical path in the trace
  5. Look at the span and the related error log
  6. Investigation done and it is time to fix the issue

In Context Alerts Enable Faster Troubleshooting

Using the Wavefront’s powerful query language, very simple query will generate an alert on a specific service if any of the RED metrics are deviating from normal behavior. While the query is very generic, the alert itself will be very specific with a link that sends you to a dashboard with the right context (application and service) which is the perfect place to do initial triage and drill down into traces.

The alert above is defined to alert on any service which has latency above a certain threshold. While this is a generic alert for any application and service, but when it is fired it provides details about the actual application and service and provides a link to the relevant service dashboard for quick troubleshooting and correlation.

Unified Service and Application Status Pages

Custom dashboards are great but having a landing page to start from without creating your own dashboard is even better. Wavefront now has a status page for applications and services which provides quick guidance to any services that may be misbehaving. As you can see in the screenshot below you can see the RED metrics for each of the application and see the most erroneous applications which need attention.

To summarize, in this blog you saw how Wavefront provides integrated application observability with metrics, histograms, traces and span logs. Furthermore, automatic context enriched alerts enable SREs and developers to quickly find the root cause of an issue. To learn more about Wavefront’s distributed tracing check out our free trial today.


Get Started with Wavefront Follow @eranc Follow @WavefrontHQ

The post New: Wavefront Distributed Tracing Enhanced with Span Logs, Alerting Context, and Unified Views appeared first on Wavefront by VMware.

About the Author

Rob Markovich leads Product and Technical Marketing for Wavefront by VMware, and was CMO for Wavefront before the VMware acquisition. He has over 15 years of experience with monitoring and analysis of applications and infrastructure, having started his career as a data center engineer at a Fortune 50 enterprise, and later as a co-founder/early employee for several technology startups that successfully exited via IPOs and M&A. His current interests revolve around the performance, reliability, and scale of cloud-native applications, services, and workloads.

More Content by Rob Markovich
Secureworks Surfing the Wavefront with Observability
Secureworks Surfing the Wavefront with Observability

Recently, we had the pleasure to talk to Manu Mathew, Senior Manager – Performance Engineering from Securew...

Wavefront’s Next-Gen Collector for Kubernetes: Now with Auto-Discovery and Expanded Deployment Options
Wavefront’s Next-Gen Collector for Kubernetes: Now with Auto-Discovery and Expanded Deployment Options

We are thrilled to announce the expansion of the Kubernetes observability experience in Wavefront with the ...