Stephen Belanger | Node.js observability with diagnostics_channel and AsyncLocalStorage

Summary

Stephen gives a technical talk on diagnostics and tracing in Node.js applications to monitor performance. He provides background on trace spans, the Diagnostics Channel for publishing metric events, and the Tracing Channel which aggregates diagnostic events into traces. He then demonstrates how to instrument Node.js code to utilize these APIs for diagnostics, including async context propagation with Async Local Storage for distributed tracing across async boundaries. The goal is to help developers understand these tools to incorporate diagnostics in their own modules and improve observability.

Takeaways

• 😀 Stephen works at DataDog focusing on observability, diagnostics and tracing for Node.js
• 📊 He explains how tracers work by aggregating contextual spans to measure executions
• 🔗 Spans link together via IDs to create traces that group related operations
• 📤 Diagnostic Channel provides high performance event pipeline for publishing telemetry
• 🚦 Tracing Channel offers start/end events to trace durations of sync and async actions
• ⏱ Async local storage propagates context across asynchronous callbacks
• 🔗 Diagnostics Channel can bind context to store updates for distributed tracing
• 😎 Creating tracers by wrapping code in tracing and using async local storage for context
• 💡 Ideas for using Diagnostics and Tracing Channels for instrumentation
• 👍 Stephen demystifies how these APIs enable building observability in Node.js

Q & A

What is a trace in the context of observability?

A trace is a collection of spans, where each span has a start time, end time, and contextual data describing what that span was for. For example, a span could represent a web request, including what route was requested.

How do spans represent relationships between requests?

Spans can contain child spans, which describes a relation where the child span was caused by the parent span. For example, a web request span could have a child span for the next API request that was triggered by the initial web request.

What is the purpose of using a diagnostics channel?

A diagnostics channel is a high performance event emitter focused on emitting generic diagnostics events. It allows tracing data to be published without needing a direct interface between libraries and tracers. This eliminates monkey patching and decouples the implementations.

How does tracing channel build on top of diagnostics channel?

Tracing channel represents a span using a group of 5 correlated diagnostics channels - start, end, async start, async end, and errors. This allows selectively listening to only the tracing events you need for a given tracer implementation.

What does async local storage provide?

Async local storage allows propagating data through calls without needing to explicitly pass anything into those calls. As long as there is a causal execution path, the data can flow through async local storage between the get and set calls.

How can lost context be recovered when tracing?

The tracing events share a message object between start/end etc. By storing span data on this message in the start event, it can be retrieved from future events to recover lost context if needed before continuing the trace.

How do tracing channel and async local storage integrate?

Diagnostics channel provides bind/run store APIs. Run store suggests a state change, while bind store accepts it and updates state. This allows tracing channel span events to update async local storage span state.

What are the options for interpreting span scope?

Some tracers consider the async start callback to end the span, while others consider it part of an overall span. The APIs are flexible enough to allow either interpretation.

What are some use cases for instrumentation?

You can use diagnostics channel and tracing channel APIs to instrument your own modules and publish tracing data. For example, tracking how often a function is called, durations, and results.

How can I learn more about implementing my own tracing?

See the video and demo code examples explaining tracing channel APIs for sync functions, async callbacks, and promises. Also reference the diagnostics channel and async local storage documentation.