For buyers
Twelve questions for every vendor in this category.
You will evaluate more than one product in agentic analytics over the next eighteen months, and the demos will look alike. These twelve separate architecture from presentation. Ask them of every vendor. We would be pleased if you asked them of us.
On evidence
- Does the language model produce the business numbers, or does a query engine? Ask to see the boundary in the architecture, not in the pitch.
- From any number on any screen, how many clicks to the query that produced it? Count them. If explanation is displayed but verification is expensive, the research says you will get acceptance, not accuracy.
- What happens to a number the system cannot prove? Silently dropped, silently shipped, or held with a recorded reason you can inspect?
On persistence
- Is the output of a run an object with an identity, or a message? Can you cite it in six months?
- Can anything be physically deleted? If yes, your audit trail has a hole in it by design.
- When an object is revised, is the previous version recoverable?
On governance
- Is authorisation enforced on the server and inside the query, or is the interface hiding rows the backend returned?
- Is the producing identity and row-level scope frozen onto the object at creation, and re-checked at read time?
- Can the model serving a run change automatically, through a fallback or a quality escalation? If it can, you cannot reproduce last quarter’s output.
On honesty
- Ask which capabilities in the demo are shipping and which are roadmap. Ask for it in writing. The answer, and the willingness to give one, tells you more than the demo did.
- Does the product represent absence — no data, stale data, a truncated result — or does absence look identical to a clean negative finding?
- Which records and AI-governance standards do you position against, and what in the architecture maps to them?
Copy this list freely; no attribution required. It is Appendix A of From Answers to Records.