The Power of Naming Things: Why Metric Stores Matter
A conversation with Amit Pahwa, Data Platform Engineer at Databricks
For those who love to read, this blog post summarizes an episode of the Data Culture Podcast; that being said, we definitely recommend listening to the episode!
Five People, Five Answers
When Airbnb’s CEO Brian Chesky was asked in a board meeting how many visitors come to the website, he didn’t have the number handy. So he went back to his data leads. Five different people gave him five different answers, none of which were necessarily wrong, rather they were using different terminology, different definitions, different slices of the same underlying reality.
That story captures the problem Amit Pahwa has spent over a decade trying to solve. Amit is currently on the internal data platform team at Databricks and previously spent six years at Airbnb, where he helped build Minerva, the company’s open-source metric store. His career evolved from BI consultant to data engineer to software engineer building internal data tooling, which gives him a unique perspective: he’s both built the tools and lived with the pain they’re meant to address.
Amit’s insight is simple, and in it’s simplicity, underlines challenging organizational pain and choices: when an organization can’t agree on what its own numbers mean, decision-making breaks down at the highest levels.
What a Metric Store Actually Is
A metric store is, at its simplest, a single source of truth repository for business metrics. Implementations range from pre-aggregated core tables that users can query without doing joins themselves, to key-value stores like Uber’s UMetrics, to complex ETL layers like Airbnb’s Minerva that handle table manipulation and provide APIs for direct metric access.
The real differentiator isn’t technical complexity, it’s how global and consistent you want the metrics to be. Is this a source of truth for a product domain, an organization, or the entire company? Are dimensions part of the metric definition with enforced semantic consistency, or are developers trusted to manage that themselves?
Amit leans toward maximum consistency with enforced definitions and centralized ETL, with this reasoning becoming more clear as the conversation unfolds.
Tension as a Feature
One key value proposition in a metric store is that it surfaces disagreement and subsequently channels it in a productive way. When Airbnb’s metric definitions lived in GitHub, people opened PRs to propose changes, challenge existing logic, and negotiate toward consensus. The disagreement itself became productive because it happened in a structured, visible space rather than in scattered dashboards and one-off queries.
Definitional conflicts often reveal genuine business insights. Different groups may be measuring legitimately different things, like whether “average handle time” in a call center includes after-call note-taking or just the live conversation. Surfacing those gaps creates an opportunity to decide intentionally how to slice a metric rather than discovering discrepancies during a board presentation.
The shift from Airbnb’s experimentation-only metric store to the company-wide Minerva system happened precisely because of this tension. Teams noticed that experimentation metrics didn’t align with their team dashboards. Amit and his manager realized they were each spending more than half their week helping people debug why numbers didn’t match.
Democratization Requires Centralization
This sounds paradoxical, but it’s the core architectural insight.
A single centralized team modeling everything can’t keep pace. By the time you’ve reconciled every team’s terminology and built the canonical tables, the product has moved on. People start to DIY. The source of truth stops being the source of truth. As Sid puts it, borrowing from Jurassic Park: life finds a way.
Minerva’s approach threads the needle. The centralized platform standardizes the ETL process and ops model, handling the pain of managing and operating tables. But individual teams own the business logic for their product area, the people closest to the domain are the ones who define it. A star schema approach guides contributors through proper modeling without requiring a centralized team to enforce every decision. Teams can move almost as fast as the product itself rather than waiting in a queue.
It’s both data mesh and data fabric, federated ownership combined with automated combination of those layers. Though as both Amit and Sid acknowledge, the industry can’t quite agree on what either of those terms mean.
The Flywheel: Start From the Top
Implementing a metric store is high-friction at the start and gets easier over time. Amit’s recommended approach: start with leadership. Get buy-in for a dashboard that tracks company priorities on a regular cadence. That gives you a base set of top-line metrics and points of contact across every organization. The centralized team eats the initial friction by onboarding these metrics themselves.
From there, the incentive cascades naturally. Once leadership is tracking top-line metrics, individual orgs want to know how they’re contributing. They want their dashboards to show the same numbers. Contributing teams don’t define new metrics from scratch, they add dimensions. Everyone keeps building on each other’s predefined definitions, and eventually the only new work is when the product evolves and something genuinely new shows up.
The key enforcement: one name, one meaning, one semantic, one definition. Conflicts get resolved as things are added. If someone disagrees later, the code is the source of truth and they can contribute a change through the normal process.
Data Integrity Score
Beyond metric definitions, Amit is tackling a broader problem: incentivizing good behavior for owning data. Good platforms reduce friction for producing data, but that ease leads to tables everywhere with unknown quality and no contracts. People search, find something that looks about right, and start building on it.
The solution is a data integrity score, a composite measure covering documentation standards, reliability through centralized SLA and quality check tracking, and trust through heuristics like query frequency and dashboard usage levels. You plaster that score everywhere, on tables, in dashboards, visible at every layer. Then you train the organization to respect it. If a leader looks at a dashboard and sees a low score, or no score at all, they should ask why. The dashboard is telling you not to trust it, so why would you?
Why This Matters for LLMs
One of the biggest barriers to effective LLM-powered data tools is the absence of a clean, consistent terminology catalog. Without one, LLMs can’t disambiguate terms that mean different things across teams, and automated answers risk reproducing the five-different-answers problem, just faster.
A metric store serves as that glossary. Every metric defined, every dimension documented, ownership tracked, usage heuristics collected. The LLM gets one consistent set of golden tables with enforced naming and semantics rather than five competing data marts.
Amit frames it simply: we are the eyes, the ears, all the senses for the LLM. They don’t observe. We do. They’re only going to understand it as much as we allow them to, and we can allow them to understand it better if things are named well.
Final Thoughts
The naming problem is deceptively old and persistently unsolved. What’s changed is scale: more people producing and consuming data, more teams, more tools, more surface area for inconsistency. Databricks itself grew from one thousand to four thousand employees in two years. When you’re onboarding that fast, alignment doesn’t happen by accident.
Amit’s approach, centralized platform, federated logic, enforced naming, scored data quality, doesn’t eliminate the hard conversations about what things mean. It makes those conversations visible, structured, and productive. And increasingly, it makes them machine-readable too.
This blog post is based on a conversation between Sid Atkinson and Amit Pahwa on the Data Culture Podcast. Amit Pahwa is a Data Platform Engineer at Databricks, previously at Airbnb, where he helped build the Minerva metric store. Connect with him on LinkedIn.