Telemetry Over Events: Developer-Friendly Instrumentation at American... Ace Ellett & Kylan Johnson

Summary

In this insightful talk, Kylin Johnson, the Android Lead for the mobile team at AMX, discusses the company's journey with Telemetry, emphasizing the importance of observability in mobile app development. Starting from scratch, the team focused on understanding user experience through real user monitoring and manual instrumentation. They tackled challenges like app startup, server connections, and complex user journeys, highlighting the need for developers to be experts in their features while the embedded system team supports Telemetry principles. The talk concludes with lessons learned, emphasizing the value of custom events and testing in ensuring effective observability.

Takeaways

• 😀 The speaker, Kylin Johnson, is the Android Lead for the mobile team at AMX, discussing the company's journey with telemetry and observability in mobile apps.
• 🔧 AMX started with no telemetry in their mobile app and aimed to understand user experience and app performance from the customer's perspective.
• 📈 They focused on real user monitoring and observability to gauge how the app felt to use, especially during app startup which is critical for user experience.
• 📱 The mobile team, including both Android and iOS, worked on manual instrumentation to capture telemetry data for tens of millions of users across multiple countries.
• 🚀 Kylin emphasized the importance of making observability a first-class feature in the application architecture, integrating it into the build process and development workflow.
• 🛠 The team faced challenges in abstract modeling and the cognitive load of introducing new APIs to developers already dealing with a rapidly evolving Android landscape.
• 🔄 They built a protocol for logging in, which included sending a login request, decoding JSON, scanning biometrics, and handling multi-factor authentication (MFA).
• 📊 Kylin highlighted the complexity of what seems like a simple journey from login to landing page, with many potential 'bumps' in the process that need to be monitored.
• 👥 The speaker stressed that developers should be experts in their features, with embedded specialists like Kylin being the experts in telemetry and observability.
• 🔄 The team learned to simplify the telemetry process by using events and signposts that everyone can understand, making it easier to implement and maintain.
• 📝 Kylin advocated for constantly reviewing and refining metrics to ensure they provide value and lead to actionable decisions, removing any that do not.
• 🛡 The AMX team created a clear barrier between the app's functionality and the telemetry measurement, encapsulating the 'how' of measurement separately from the 'what' of the app's decisions.

Q & A

Who is the speaker in the provided transcript?

The speaker is Kylin Johnson, the Android Lead for the mobile team at AMX.

What is the main focus of Kylin Johnson's talk?

The main focus is on the telemetry journey of the mobile team at AMX, specifically discussing the implementation of observability and real user monitoring in their mobile application.

Why did AMX decide to start with app startup as the critical measurement point?

AMX decided to start with app startup because it is a critical point where the app is not in memory and the user experience can be significantly impacted by the time it takes to instantiate components and render the UI.

What is the significance of progressive rendering in the context of the talk?

Progressive rendering is significant as it aligns with user profiles and device usage patterns, allowing for a more efficient and user-friendly experience by loading and displaying content in parts rather than waiting for everything to load at once.

How does the mobile team at AMX approach server connection issues?

They initially assumed server connection would be the most critical and error-prone aspect to measure. However, with a coexistent server team and a tight API contract, they found that server issues were less of a concern than initially thought.

What is the importance of observability as a first-class feature in the development process?

Observability as a first-class feature ensures that it is integrated into the architecture and the development process from the start, making it easier to maintain, understand, and improve the application based on real user data and feedback.

How does Kylin Johnson describe the complexity of the login journey in the mobile app?

Kylin describes the login journey as very complex, involving multiple steps such as biometric scanning, multi-factor authentication, and rendering the final view, which can vary depending on the user.

What is the role of the login Observer in the Android application architecture?

The login Observer is used to monitor and record the login process, sending the result of the login attempt to an Observer, which is part of the dependency injection setup in the Android application.

Why is it important to separate the 'what' from the 'how' in the context of telemetry?

Separating the 'what' from the 'how' allows developers to focus on the app-level decisions without being distracted by the specifics of how the telemetry is measured, making the system more adaptable and easier to maintain.

How does the mobile team at AMX ensure the reliability of their telemetry data?

They ensure reliability by testing everything at scale, including the observability features. They assert the occurrence of traces, the presence of child spans, and various tags and metadata to validate the telemetry data.

What is the final takeaway from Kylin Johnson's talk regarding telemetry and observability?

The final takeaway is the importance of customizing and adapting telemetry to fit the specific needs of the application and the team, ensuring that all developers can participate in and benefit from the observability features.