The Semantic Layer Solves the Wrong Half of Your Problem

Answering a data question is two jobs. First decide which metric the person meant. Then compute it correctly. A semantic layer only does the second job, and mixing up the two is why smart teams keep disagreeing about whether you need one.

The disagreement is real. An independent benchmark ran one model up six levels of grounding and found the semantic layer was the biggest single jump on the ladder, twenty points. Another study measured a well-built layer as slightly worse than no layer at all, at twice the cost. Anthropic keeps one and tells their agent to check it first, but writes every definition by hand. All three are right.

Eleven things are called revenue

Someone asks what revenue was last quarter. Before anything can be computed, something has to decide which of the eleven things called revenue they meant. Bookings or recognised. Gross or net of refunds. The finance number or the dashboard number that has been three percent off since the reorg. That is orientation, and it is a judgment about what the person meant.

Then the agreed definition has to be applied with the right filters, grain and joins. That is computation, and it is mechanical.

A semantic layer makes computation deterministic, which is exactly what you want. It says almost nothing about orientation. When a question does not map onto a metric in the layer, the layer cannot help, and the agent guesses anyway. Give it clean questions that land inside the layer and the layer carries the whole answer. Give it real questions from a real company, most of which land outside, and it carries very little. The studies measured different halves.

Ramp built both halves and drew the line: domain documents are the map to the metrics, the semantic layer is the rule to compute each one. One disambiguates. One governs. Most teams buy the second and expect it to fix the first. Then the agent returns the wrong revenue number and the conclusion is that the semantic layer did not work. It worked. It was answering a different question.

Write the map first

Which metrics exist, what each one means, which table is the source of truth, what the common confusions are. That is prose, it is cheap, and it decides whether the right metric gets picked at all.

Then make the important metrics deterministic. A semantic layer earns its keep when a definition has to change in one place and take effect everywhere.

And do not let a model write your definitions unattended. Anthropic tried and got "plausible-looking definitions that encoded the very ambiguities we were trying to eliminate." The point of writing a definition down was never the writing. It was the deciding.

We are updating our own position

We argued last year that you never strictly need a semantic layer, only meaning recorded where a machine can read it. We got our agent from 10% to 90% on Adyen's DABstep benchmark without one. That still holds. What we would add now: the half a semantic layer solves is the half more prose cannot fix. If your agent picks the wrong number, more semantic layer will not save you. If it picks the right number and computes it three different ways, that is exactly what a semantic layer is for.

If this excites you, we'd love to hear from you. Get in touch.