How to Measure AI Search Visibility (Without Fooling Yourself)

“Are we showing up in AI answers?” is a reasonable question with an unreasonable number of bad answers attached to it. Most of what gets sold as AI visibility measurement fails one of three tests: the prompt set is not fixed, the engine coverage is not stated, or the scoring collapses several different things into one number. Here is a method that passes all three, written so you can run it yourself.

Scope of this article

The method below is engine-agnostic, but every figure we quote from our own runs comes from a generative engine API, measured on a stated date. It is not a measurement of Google’s AI Overviews, which requires a different collection method entirely. Any provider who shows you one number covering “AI search” without naming the engines behind it is showing you an average of things that do not average.

Step 1 — Build a prompt set that does not cheat

The single most common error is putting the brand name in the prompt. Asking a model “what do you think of Acme Corp?” measures whether the model has heard of Acme. It tells you nothing about whether a buyer who has never heard of Acme would be shown it.

A usable prompt set has these properties:

• No brand names. Not yours, not competitors’. The whole question is who the model volunteers.
• 50–100 prompts. Fewer than about 50 and single-answer noise dominates the result.
• Spread across buying stages. Informational, comparative, and buying-stage prompts, tagged so you can score each tier separately.
• Written the way buyers type. Not keyword strings. Real sentences, in the language your buyers use.
• Frozen. Once you run a baseline, the prompt set does not change. Changing it is how you accidentally manufacture improvement.

Step 2 — Run and store the raw answers

Run every prompt, store the full text of every answer, and record the date and engine version. Storing raw text is not bureaucratic overhead — it is the only thing that lets you or a third party re-score the run later under a different definition, or audit a number you find surprising.

Two collection details that change results more than people expect: temperature, and whether the engine’s own web-browsing or grounding is enabled. A grounded run and an ungrounded run answer meaningfully different questions. Fix both settings and record them.

Step 3 — Score three metrics, separately

Keep them apart. A brand can sit at a healthy Mention Rate and a Citation Rate of zero — that combination is common, and it prescribes completely different work from the reverse.

Step 4 — Match strings carefully

This is where automated scoring quietly breaks. Naive substring matching produces false positives at an embarrassing rate: a two- or three-letter brand token will match inside longer unrelated words, and in Vietnamese it will also match across diacritics. Match on word boundaries, normalise accents on both sides of the comparison, and hand-check a sample of at least twenty hits before you trust any aggregate. We have seen a careless matcher inflate a mention count by a factor of three.

Step 5 — Build the competitor board

The same run that scores you also tells you who the model currently treats as the category. Count every vendor named, rank them, and keep the ranking. This is usually the most commercially useful artefact of the whole exercise, and it costs nothing extra to produce.

Step 6 — Re-measure on the same terms

Re-run at a fixed interval — 90 days is a sensible default — with the identical prompt set, identical engine settings, and identical scoring code. Movement is then attributable to your work rather than to a changed test. If you must extend the prompt set, keep the original as a frozen core and report it separately.

What this method will not tell you

It will not tell you traffic, and it should not pretend to. Mention Rate is a visibility measure, not a demand measure. It also cannot separate a genuine ranking improvement from a model update that happened to reshuffle its associations — which is exactly why the competitor board matters: if every vendor moved, the model changed, not your site.

More from Insights

• How to measure AI search visibility — the full scoring method — Mention Rate, Citation Rate, Share of Model
• GEO vs SEO: what actually differs — where the two overlap and where they genuinely diverge
• How brands get cited by ChatGPT — what the citation surface rewards, and what it ignores
• llms.txt: useful, overrated, or both? — an honest read on a file a lot of agencies oversell
• What AI search optimisation costs — real ranges, and what drives the number up or down

Frequently asked questions

How many prompts is enough?

Fifty is the practical floor. Below that, one unusual answer moves the headline percentage by two points and you will over-read noise. One hundred is comfortable.

Should the prompt set be in English or the local language?

Whichever your buyers type. If they type both, run both and score them as separate sets — results routinely differ, sometimes substantially.

Is a 0% Mention Rate bad?

It is normal for a brand with little third-party coverage, and it is useful, because it is unambiguous. Our own first baseline was 0% on both mention and citation.

Can I check an agency using this?

Yes, and that is the point of publishing it. Ask any provider for their prompt set, their engine list, their dates and their raw answers. All four, or the number means nothing.