What a good AI visibility report looks like (and what a junk one looks like)
A good AI visibility report lets you check it: every claim traces to an actual answer you could re-read, dated, with the question shown. A junk report asks you to trust it: a score, some alarming red, no questions disclosed, no answers shown. That’s the whole test, and it applies to anything that claims to measure you, whether a vendor PDF, an agency deck, or the dashboard you’re paying for, ours included.
The five-second tell
Look for the answers. A trustworthy AI visibility report shows the questions it asked, the dates it asked them, and what the AI actually answered, so you can verify any line by asking the same question yourself. A junk report shows conclusions without the material behind them, usually led by an invented score. If you can’t check it, you’re not reading measurement, you’re reading marketing.
Anatomy of a junk report
A composite illustration, assembled from patterns across the industry rather than any one vendor’s real document. It opens with a score: “Your AI Visibility: 31/100,” red, no formula shown. It never discloses which questions were asked, so nothing is reproducible. Its findings are fear-shaped (“your business is INVISIBLE to AI”) built on a single screenshot, which by itself proves almost nothing. There are no dates, so you can’t know if any of it is still true. And it ends with urgency: lock in the retainer this week.
The score deserves its own sentence: a composite number with no published formula is the junk report’s load-bearing wall, because it converts unverifiable inputs into false precision. It’s the reason our own method reports graded findings instead of a score.
Anatomy of a useful one
The useful version is almost boring. The questions asked, listed. The dates. What each AI answered, stored word for word, with your business named or not named, and the competitors who were named instead. Findings graded with the supporting answer visible under each grade. Claims about what influences answers either tied to something observable or labeled as inference. And the limits stated: what the report can’t conclude, said in the report itself. This structure is what we build, and describing it here isn’t a secret we’re giving away, because the structure was never the hard part. Doing it honestly every time is.
The questions to ask whoever handed you one
Five, pointed, whatever the source: Which exact questions did you ask, and can I see them? Can I reproduce any finding by asking the same question myself today? What’s the date on every answer shown? Which claims here are observed and which are inferred? What does this report refuse to conclude? A good vendor answers all five easily, and the longer interrogation kit extends the list to the whole engagement.
The uncomfortable note
Our opinion, and it cuts against our interest in a tidy story: some junk reports contain a true finding. A fear-formatted PDF with an invented score might still be right that your profile is half-empty. The problem with junk isn’t that it’s always wrong; it’s that you can’t tell which parts are right, because nothing is checkable. That’s why the test is verifiability, not tone, and why the fix for a junk report isn’t a nicer one. It’s one with the answers attached.