How 14 Companies Built AI Data Analysts
Fourteen companies have published how they built an internal AI data analyst. They share no code, no stack and no vendor. They converged anyway.
The sharpest example: Anthropic gave their agent raw access to thousands of past queries, and accuracy moved less than a point. Eighty percent of the wrong answers had the right answer sitting somewhere in that pile. The same knowledge, rebuilt as small routed documents, took them from 21% to over 95%. Same words, different structure, seventy points.
That is the shape of almost every lesson here. Below is every published build we know of, and the five things that repeat.
The census
| Company | What they built | Published results | The lesson in one line |
|---|---|---|---|
| OpenAI | Internal data agent, six context layers | 3,500 users, 600 PB, 70k datasets | Shrink what the agent decides at runtime. Meaning lives in code |
| Anthropic | Claude + data foundations, sources of truth, skills | 95% of analytics queries automated at ~95% accuracy; 21% → 95%+ from skills | Procedural knowledge beats raw access. Maintenance is engineering |
| Ramp | Ramp Research, in Slack on dbt + Snowflake | 10-20× more questions than the legacy help channel; 300+ users | The repo is the brain: owned, versioned domain docs |
| Gorgias | Cortex, a company-wide AI workspace | Company-wide; ~72% of employees use it weekly | Progressive disclosure was their biggest single improvement |
| Lyft | Aria, on a governed metric semantic layer | Prototype to production web app in 3 weeks | Every golden metric gets a business owner and an operational owner |
| Vercel | d0, filesystem + bash over a YAML semantic layer | 11 tools → 2; success 80% → 100%; 3.5× faster | Rich context lets the architecture almost disappear |
| Uber | QueryGPT (and Finch for finance) | ~140,000 hours saved per month; queries from 10 minutes to 3 | Decompose: intent → tables → columns → SQL |
| SQL Bot inside their analytics platform | 53% technical accuracy, 95% rate it passing or better | Accuracy and trust are different variables | |
| Stripe | Hubert on the Hubble catalog | ~900 weekly users | "The biggest failure mode has been data discovery" |
| Text-to-SQL inside Querybook | Table-search hit rate 40% → 90% from documentation alone | Solve table discovery before query generation | |
| Netflix | LORE, on a governed semantic layer | Not published | They bet on explainability. Trust before capability |
| Salesforce | Horizon Agent in Slack | 50% → 80% efficacy; knowledge fixes ship in ~15 minutes | Transparency made users better askers |
| ClickHouse | DWAINE on LibreChat + MCP | ~70% of internal analytics; 250+ users on a 3-person data team | "LLMs magnify data quality issues dramatically" |
| DoorDash | Agentic platform with a data-explorer agent | Support chatbot: 90% fewer hallucinations via layered guardrails | Example values fix WHERE clauses. A portfolio, not a silver bullet |
Two adjacent builds teach the same lessons from other fields. Airbnb's support AI found that restructuring knowledge for the model beat upgrading the model. Instacart's search team found that a generic model is a commodity and your business context is the moat. One independent benchmark belongs next to the census too: Decision Spine ran a single model up six levels of context, from 41% correct on messy raw data to 92% with a full metric tree.
Here is what repeats.
The model was never the bottleneck
Almost every team started out believing the hard part was writing SQL. Almost every team learned the hard part is finding what things mean before any SQL gets written.
Stripe says it flatly: "The biggest failure mode has been data discovery." Their fix was not a better model. They deprecated low-quality tables and wrote documentation for the canonical ones. Pinterest measured the same thing: table search found the right table 40% of the time without documentation in the index, and 90% with it. Uber spends two of its four agents on picking the domain and the tables before any SQL is attempted.
An agent that picks the wrong table writes clean SQL and a confidently wrong answer. The mature builds spend most of their machinery upstream of the query.
Dumped context is worthless. Routed context wins.
The evidence here points both ways until you see the split.
Anthropic's query corpus is one half of it. Decision Spine ran the other: they added a free-text knowledge base as one of six grounding levels, and it never improved accuracy once.
Yet the same kind of text, organized differently, is the strongest lever anyone found. Anthropic rebuilt that knowledge as small routed documents and went from 21% to over 95%. Gorgias calls progressive disclosure, loading only the piece of context a question needs, their single biggest improvement.
So free text is not the problem. Dumped free text is. Notes that cover one topic, load at the right moment, and have an owner stay valuable. A wiki export poured in whole is noise the agent pays to ignore.
The evidence also splits on semantic layers. Decision Spine measured its biggest single jump from one, 20 points. Anthropic consults theirs first, but found auto-generating it fails and writes it by hand. Finding the right metric and computing it correctly are two different problems, and a semantic layer only solves the second.
When context stops helping, fix the data model
Vercel says it directly. Their minimal agent works because the semantic layer under it is clean and consistently named. "If your data layer is a mess of legacy naming conventions and undocumented joins, giving Claude raw file access won't save you. You'll just get faster bad queries." ClickHouse, which runs about 70% of internal analytics through its agent, puts it in one line: "LLMs magnify data quality issues dramatically."
We reached the same conclusion on Adyen's DABstep benchmark: a configured agent went from 10% to 90%, and the work was a clean data model plus written-down meaning, not a bigger model. We argued the general form last year: your data model is the prompt.
Accuracy and trust are different variables
LinkedIn published the most uncomfortable pair of numbers in the census. 53% of SQL Bot's answers score as technically correct. 95% of users rate its answers as passing or better. Not because users are naive. The tool shows its work, helps them find tables, and its most-used feature is a "Fix with AI" button on failed queries. That button is 80% of all sessions. People do not need an oracle. They need a competent collaborator whose work they can check.
Salesforce learned it in reverse. At launch their agent was right half the time and said "I don't know" without explanation. When they made it explain its interpretation and ask clarifying questions, users learned how to ask better, and efficacy climbed to 80%.
Delivery matters just as much. Salesforce's web prototype worked and nobody used it. The Slack version, with worse answers, took off. LinkedIn's adoption grew 5 to 10 times when the bot moved inside the platform analysts already used. Meet people where they already work.
The chapter nobody writes: month two
Every write-up describes reaching reliability. Almost none describes keeping it.
The exception is Anthropic, and their number should hang on every data team's wall. After reaching about 95% accuracy, one month of ordinary changes to schemas and definitions dropped them to 65%. Nothing broke. The business moved and the context did not. Their fix: treat context as code. About 90% of their data-model pull requests now ship with a matching context change, enforced in CI.
The fragments elsewhere point the same way. Salesforce ships a knowledge-base fix in about 15 minutes, with regression tests so it breaks nothing else. Lyft gives every golden metric two named owners, one for the definition and one for the data health. The systems that stay trusted have someone who owns a boring weekly routine: review failures, prune stale context, realign definitions.
It compresses to five sentences
The census compresses to five sentences. Fix discovery before generation. Curate and route your context, never dump it. When context stops helping, fix the data model. Ship it where people already work. Plan for month two before you celebrate month one.
We folded all of it, plus what we see across our own deployments, into The AI Data Analyst Playbook.
If this excites you, we'd love to hear from you. Get in touch.
Rick Radewagen
Rick is a co-founder of Dot, on a mission to make data accessible to everyone. When he's not building AI-powered analytics, you'll find him obsessing over well-arranged pixels and surprising himself by learning new languages.