Why Your AI Assistant and Your Dashboard Give Different Numbers

If three teams define revenue three ways, a natural-language assistant will answer the same question three ways, faster and with more confidence than the dashboards did. Why a semantic layer is the prerequisite nobody sells, and why it is the part of an AI programme that keeps its value.

If three teams calculate revenue three ways, a natural-language assistant sitting on top of that data will answer the same question three ways, faster and with more confidence than the dashboards did. Retrieval does not fix a definition problem. Nothing fixes a definition problem except deciding the definition.

The situation, as it actually arrives

What the dashboard had that the assistant removed

• The dashboard. A titled artefact. Slower about the disagreement, and each one carried an owner and a title that told you whose version of the number you were looking at.
• The assistant. One confident sentence. The sentence does not say which of the four revenue tables it came from, two phrasings can join differently, and a contested figure arrives in the tone of an uncontested one.

Both numbers are defensible and both are drawn from real tables. The disagreement does not get resolved, it gets distributed, at speed, to more people, with the evidence stripped off.

Source: Thinklytics governance practice, 2026.

A finance director asks the new assistant what last quarter's revenue was and gets a number. A sales director asks the same assistant the same question a day later, from a different starting context, and gets a different number. Both numbers are defensible. Both are drawn from real tables. Neither is wrong in the sense of being miscalculated.

What has happened is that the organisation has more than one definition of revenue, one that recognises on invoice, one on cash, one that treats an intercompany transfer differently, and the assistant has quietly chosen between them based on which table its query landed in.

The dashboards had the same problem. The dashboards were slower about it, and each one had a title on it that told you whose version you were looking at. The assistant removed both of those protections.

Why this gets worse rather than better with AI

Three things change when you put a language interface over the data.

The provenance disappears from the answer. A dashboard is a fixed artefact with an owner and a title. An assistant returns a sentence. The sentence does not say which of the four revenue tables it came from, and the person reading it has no reason to ask.

Query generation is non-deterministic. Ask the same question with slightly different wording and the generated query may join differently. Two people, two phrasings, two numbers, no way to see why.

Confidence is uniform. The assistant delivers a contested figure in exactly the tone it uses for an uncontested one. Ambiguity that used to be visible as two dashboards disagreeing becomes invisible as one confident sentence.

The net effect is that the disagreement does not get resolved. It gets distributed, at speed, to more people, with the evidence stripped off.

The thing that fixes it

What a semantic layer gives you

One place where each business term is defined once, in a form a machine executes rather than a document a human is supposed to read.

Source: Thinklytics governance practice, 2026.

A semantic layer. It is unglamorous infrastructure and it is the single highest leverage thing most organisations can build before they put AI in front of anyone.

A semantic layer is one place where each business term is defined once, in a form a machine executes rather than a document a human is supposed to read. Revenue is defined once, with its edge cases written down: what happens to credits, to intercompany, to partial shipments, to the deal that closed on the 31st at 11pm. Every query for revenue, from a dashboard or an assistant or a spreadsheet, resolves through that definition.

What it gives you:

• One answer to one question, regardless of who asks or which tool they ask through.
• A place to put the edge cases, which is where all the actual disagreement lives. Nobody argues about the simple invoices.
• A named owner per metric, so the next dispute has an address rather than a meeting.
• Lineage, so the answer can be traced to source when somebody outside asks.
• A governable surface for AI. The assistant queries certified metrics rather than raw tables, which is also the only version of this that survives an audit.

Why this is the wedge, not a prerequisite to get past

The common framing is that the semantic layer is boring plumbing you have to finish before the interesting AI work starts. That framing causes organisations to rush it or skip it.

It is better understood the other way round. The semantic layer is the part of an AI programme that keeps its value regardless of what happens to the AI. Models will be replaced. Vendors will be replaced. Interfaces will change three times in five years. An agreed, owned, machine-executed definition of what a customer is does not depreciate, and every subsequent system inherits it.

It is also the part nobody else will do for you. A model vendor cannot decide when your revenue recognises. That is a decision your organisation has been avoiding, and the AI project is merely the thing that made the avoidance expensive.

The sequence

The sequence, and the step no technology shortens

Step one takes a week and is usually the most persuasive document produced all year.

• Ask four teams for last quarter's number, separately, and compare
• Decide, in front of everyone, which one is right
• Write the edge cases down
• Implement it once, where every tool reads it
• Then point the assistant at certified metrics

Step two is a decision, not an analysis, and it needs someone with the authority to overrule a team that loses. The definition is not the sentence, it is the sentence plus the fourteen exceptions.

Source: Thinklytics governance practice, 2026.

1. Find out how many definitions you actually have. Ask four teams for last quarter's number separately and compare. This takes a week and it is usually the most persuasive document produced all year. 2. Decide, in front of everyone, which one is right. This is a decision, not an analysis. It needs someone with the authority to overrule a team that loses. 3. Write the edge cases down. The definition is not the sentence. The definition is the sentence plus the fourteen exceptions. 4. Implement it once, where every tool reads it. 5. Then put the assistant on top of it, pointed at certified metrics and refusing to answer from uncertified tables.

Step two is the hard one and no technology shortens it. Every organisation that has tried to route around it, by building the assistant first and hoping consensus would follow the tool, has ended up back at step two with an additional system to reconcile.

What this looks like when it has gone right

Somebody asks the assistant a question about margin. They get a number, and under it, the definition that produced it and the date it was last agreed. If they disagree, they disagree with a written definition and a named owner, not with a black box.

That is a smaller and less impressive claim than most AI material makes. It is also the condition under which people start trusting the system enough to act on it, which is the only outcome that was ever worth paying for.

Frequently asked questions

Why does our AI assistant give a different number than our dashboard?

Because the organisation has more than one definition of the metric, and the assistant quietly chose between them based on which table its query landed in. The dashboards had the same problem, but each one carried a title telling you whose version you were looking at. The assistant removed that protection.

Does retrieval fix a metric definition problem?

No. Retrieval fixes a knowledge problem, where the model does not have your content. A definition problem is a disagreement between people about what a number means, and no technology settles it. Someone with authority has to decide, and then the decision has to be implemented once where every tool reads it.

What is a semantic layer?

One place where each business term is defined once, in a form a machine executes rather than a document a human is supposed to read. Revenue is defined once, with its edge cases written down, and every query for revenue from a dashboard, an assistant or a spreadsheet resolves through that definition.

Why does a language interface make definition problems worse?

Three reasons. Provenance disappears from the answer, because a sentence does not say which of four revenue tables it came from. Query generation is non-deterministic, so two phrasings can join differently. And confidence is uniform, so a contested figure is delivered in the same tone as an uncontested one.

Should we build the semantic layer before the AI assistant?

Yes. Every organisation that has tried to route around it, by building the assistant first and hoping consensus would follow the tool, has ended up back at the definition decision with an additional system to reconcile. Certified metrics first, then the assistant pointed at them and refusing to answer from uncertified tables.

Is a semantic layer just plumbing you have to finish first?

It is better understood the other way round. It is the part of an AI programme that keeps its value regardless of what happens to the AI. Models will be replaced, vendors will be replaced, interfaces will change three times in five years. An agreed, owned, machine-executed definition of what a customer is does not depreciate.

How do we start?

Ask four teams for last quarter's number separately and compare. That takes a week and it is usually the most persuasive document produced all year. Then decide which one is right, in front of everyone, write the edge cases down, implement it once, and only then put the assistant on top.