GAAP for data and the SOMA standard ft. Abhi Sivasailam

Summary

Obby, the head of Lovers Labs, introduces the Soma standard, a project aimed at standardizing operating metrics and analytics for companies. He explains that Soma, which stands for Standard Operating Metrics and Analytics, seeks to make companies more efficient by providing a unified approach to metrics, analytics, and the upstream pipelines that support them. With a deep personal commitment, Obby has dedicated hundreds of hours to this passion project, aiming to create an open, community-driven framework. He covers the importance of having the right metrics, the non-innovative nature of company-specific metrics, and how Soma addresses these issues by documenting, cataloging, and helping produce essential business metrics. The project is ambitious, offering tools and guidelines for adopting standardized metrics, and is currently seeking contributors and testers to further refine the approach.

Takeaways

• 📝 Soma stands for Standard Operating Metrics and Analytics, aiming to standardize operating metrics, analytics, and upstream pipelines for companies.
• 📊 The project is community-driven, non-commercial, and focuses on providing companies with the necessary metrics and analytics to support data-driven decisions.
• 🚀 The foundation of Soma is built on three core beliefs: metrics matter most, companies need the right metrics, and innovation shouldn't focus on creating new metrics but on leveraging standard ones.
• 📖 Soma covers everything after raw data, including modeling, expressing, consuming data, and leveraging it for Last Mile use, with the flexibility to adopt parts of the framework as needed.
• 🔨 The initiative categorizes, documents, and catalogues essential business metrics, drawing parallels to the financial standards board's work with GAAP for accounting principles.
• 🛠️ Soma provides two options for data modeling: activities (business events) and entities (wide facts), allowing businesses to choose the best fit for their needs.
• 📈 Activities are defined as atomic business events that capture the essence of business operations and are logged into immutable ledger tables in a data warehouse.
• 💻 For expressing metrics, Soma offers semantic layer options and NETs (flattened caches of metrics), supporting flexible and efficient data analysis and reporting.
• 📗 Metric trees are a novel concept within Soma, enabling organizations to visualize and analyze the interconnectedness of various metrics to facilitate comprehensive analytics.
• 🛡️ Engagement with Soma can be through joining private review, contributing to committees for domain-specific standards, or submitting issues and PRs to help refine the standards.

Q & A

What does SOMA stand for and what is its primary goal?

SOMA stands for Standard Operating Metrics and Analytics. Its primary goal is to standardize operating metrics for companies, the analytics that support those metrics, and the upstream pipelines and telemetry that enable those metrics to exist.

What are the three founding beliefs that animate SOMA?

The three founding beliefs are: 1) Metrics are what matters for most companies; 2) Companies need enough of the right metrics to effectively proxy reality; 3) Companies shouldn't be innovating on metrics, as many operating metrics can and should be standardized across competitors.

How does SOMA propose to resolve the challenges identified in its founding beliefs?

SOMA resolves these challenges through a community-driven, non-commercial approach by identifying, documenting, and cataloging essential metrics; working backwards to help produce those metrics through upstream data processing; and moving forwards from metrics to standardize analytics and last mile use cases.

What does the SOMA footprint encompass?

The SOMA footprint encompasses everything after raw data, including data modeling, expressing data, consuming it or preparing it for consumption, and getting last mile use out of it. It's designed to work as a whole but can also be used in parts.

What are the 'activities' and 'entities' in the context of SOMA?

In SOMA, 'activities' are business events or sets of events considered essential for computing metrics for a business. 'Entities' are wide facts and their associated dimension tables that are necessary for analysis. SOMA provides specifications for both, allowing for a structured approach to model data.

What are Nets in SOMA, and why are they recommended?

Nets in SOMA are flattened caches of metrics, essentially a metric table or a 2D projection of a cube. They are recommended because they allow for quick access to metrics across different dimensions and time, support the bitemporality of metrics, and are portable across different BI tools.

How does SOMA approach the standardization of Last Mile use cases and artifacts?

SOMA standardizes Last Mile use cases and artifacts by specifying the kind of dashboards for consumption and supporting the creation of metric trees. These trees plot an organization's metrics in a web of relationships to answer core analytics questions through visualization of metric interdependencies.

What domains is SOMA currently working on, and what does this work include?

SOMA is currently working on domains such as B2B SaaS, B2C SaaS, e-commerce, marketplace, and logistics. This work includes developing standards for metrics, schemas for activities, specifications for entities, semantic layer files, and supporting scripts across these domains.

How can one contribute to or engage with the SOMA project?

To engage with the SOMA project, one can join private review to implement and provide feedback on SOMA, join a committee of subject matter experts for specific domains, or submit issues and pull requests on GitHub to suggest edits, new metrics, or discuss existing ones.

What is the significance of the metric trees concept within SOMA?

Metric trees in SOMA represent a web of relationships among an organization's metrics, enabling users to perform root cause analysis, forecasting, and prescriptive analytics by navigating through the interconnected metrics. This concept allows for a comprehensive understanding of how different metrics influence each other.