Questions to ask before you buy an AI visibility tool

Before buying an AI visibility tool, ask the vendor about six things: how answers are collected, whether every number traces to a saved answer, how repeated runs and uncertainty are handled, what any headline score is made of, whether inclusion or ranking can be influenced by payment, and what the method cannot see. Clear, specific answers are a good sign. Vague ones are your answer too.

Use the list below as a script. You can ask it of any vendor, including us.

1How do you collect the answers?

You are looking for plain detail.

• Official APIs, automation of the consumer apps, or both?
• Is live web search on for every assistant?
• What exact request is sent? Is anything added to my question?
• Which assistants, and how are versions recorded?

The trade-offs are explained in API or scraping: how AI visibility tools collect answers. A good vendor can tell you the route without hedging.

2Can I see the answers behind a number?

Pick any figure on the dashboard and ask to click through.

• Do I see the full text of each answer?
• Are citations shown as the assistant gave them?
• Can I see the question, assistant and date?
• Could I recompute the number from what you show me?

Traceability is the foundation. Without it you are trusting a figure you can never check. It is also the standard Citewake sets for itself, described in the methodology.

3How do you handle variation?

Answers change between runs, so a tool must account for it.

• Are questions repeated, or asked once?
• Are results shown with an uncertainty range or margin?
• How do you decide that a change is real, not noise?
• Are daily moves reported as trends?

The reasoning is in why AI answers change every time you ask and how many times to ask. A vendor who treats every movement as meaningful is overselling the data.

4What exactly do your metrics count?

Ask for definitions.

• What is "visibility": mention, recommendation or citation?
• What is the denominator of each rate?
• Do you separate a positive recommendation from a neutral mention or a warning?
• Are failed, refused or empty answers counted and disclosed?
• If there is a headline score, what is it made of, and how were its weights chosen and tested?

AI visibility metrics that actually mean something is a useful reference for what to expect.

5Whose questions do you ask?

• Can I supply and fix my own question set?
• Where do suggested questions come from?
• Are brand names kept out of discovery questions?
• If the set changes, is the change recorded and are comparisons restricted to like with like?

A moving question set breaks comparison over time. See how to find the questions your customers ask AI.

6Can anyone buy a better result?

This matters most for tools that publish rankings or indexes.

• Can a brand pay for inclusion, placement or a better presentation?
• Can results be edited by hand?
• If you own or partner with brands in the category, how is that handled and disclosed?
• How are errors corrected, and is the original kept visible?

A trustworthy index cannot be bought into. You want a clear statement that nobody can purchase a place, a number or a correction.

7What can't the method see?

Every method has limits. Honest vendors name them specifically.

• Personalised answers inside signed-in apps.
• All real conversations, as opposed to a designed set of questions.
• Behaviour in locations or languages the tool does not cover.
• Evidence that some assistants report and others do not.

Be sceptical of any tool that claims to show "everything".

8What happens to my data and history?

• Can I export everything, including raw answers?
• How long is history kept?
• Who else sees my questions and competitors?
• What happens to my data if I cancel?

9What does it cost, and what is capped?

Search costs money on the provider's side, so someone is paying for each answer.

• How many questions, brands, assistants and refreshes are included?
• What happens when I go over?
• Are "unlimited" claims backed by a fair-use limit?

Is a free trial or free check enough?

A trial is useful for testing the interface and the questions. It is less useful for judging the method, because a few days of data cannot show how the tool handles variation. Ask about method in writing, and judge the tool again after a full cycle of reporting, when you can see how stable the numbers are.

How do you judge the answers?

You are not hoping for perfection. You are looking for specificity, willingness to show evidence and honesty about limits. If a vendor will let you test their tool against your own manual checks, do it. The five-step comparison is in how to choose an AI search visibility tool. For what the category does at a basic level, see what AI search visibility tools do.

What should you do with the answers?

Write them down and compare vendors side by side. Note where a vendor was specific, where it was evasive and where it offered to show you evidence. Then run the same short test on each: ten of your questions, checked by hand and in the tool. The vendor whose numbers you can explain to someone else is usually the right one.

Disclosure: Citewake is in this category, so test us against this list as you would anyone else. For a free first look at your site, there is the Citewake checker.