API or scraping: how AI visibility tools collect answers

AI visibility tools collect answers in two main ways: through the AI providers' official APIs, or by automating the consumer apps and websites and reading what appears. APIs are stable, documented and repeatable, but can differ from what a person sees in the app. Automation can mirror the consumer view but is more fragile and may conflict with providers' terms. Ask any vendor which it uses, and what that does to the numbers.

Neither approach is automatically right. What matters is that the vendor is clear about the trade-offs.

What does collecting an answer involve?

At its simplest, a tool sends a question to an assistant and saves the response. The detail determines whether the result is meaningful:

• the exact question sent;
• which assistant and which version;
• whether web search was on;
• any instructions added;
• whether the cited sources were captured;
• the date and time.

Without these, a saved answer is hard to interpret or reproduce.

How does API collection work?

An API is the official route a developer uses to talk to an assistant. The tool sends a request and receives the answer, usually with structured data about citations when the provider supplies it.

Strengths:

• Documented and stable. Behaviour and settings are specified.
• Repeatable. The same request can be sent again on a schedule.
• Clean. No memory, personalisation or custom instructions unless added deliberately.
• Structured. Citations and sources may come back in a machine-readable form.
• Permitted. It is the provider's intended way for software to use the assistant.

Limits:

• Not identical to the consumer app. The app may add features, personalisation, memory or a different interface. API answers describe a clean, signed-out style of request.
• Providers report different things. Some return the sources they consulted, others only what they cite. A fair tool does not blend them without saying so.
• Cost. Web search through an API is charged, which affects how many questions a tool can run.

How does scraping work?

Here the tool drives the consumer app or website like a person would, often with automated browsers, and reads what appears on screen or in the page.

Strengths:

• Closer to what a person sees, including interface features.
• Can capture elements an API does not expose, such as particular displays of sources.

Limits:

• Fragile. Interface changes can break collection without warning.
• Terms of service. Providers may prohibit automated use of their consumer products. Ask the vendor how it handles this.
• Bot detection. Automated access may be blocked or may trigger different behaviour.
• Harder to keep clean. Account state, location and history can affect results in ways that are hard to control.
• Less structured. Answers and citations must be parsed from page layout.

Which one should you prefer?

It depends on the question you are asking.

Many tools combine approaches. What matters is that you know which numbers come from which route.

Citewake's method uses the providers' official APIs with live web search switched on, no memory and no custom instructions, and one neutral search instruction. It also states the limit: the results describe answers to a designed set of questions through the APIs, not all conversations or personalised answers inside signed-in apps. That is spelled out in the methodology. Using the API route does not make a tool better; it makes certain properties easier to document.

What should a vendor tell you?

Ask these in plain words.

1. How do you collect answers: API, automation or both?
2. Is web search on? Is it the same for every assistant?
3. What exact request is sent? Is anything added to my question?
4. Are full answers and cited sources stored?
5. How do you handle variation between runs?
6. How do you compare with what a person sees in the app?
7. What can your method not see?

A vendor who answers clearly is usually one whose numbers you can trust. One who avoids the question is giving you a different kind of answer.

How can you check for yourself?

Run a handful of questions by hand, signed out, and compare with what the tool reports for the same questions. Differences are not necessarily errors; the tool may use settings that differ from your app experience. But the vendor should be able to explain them. How to test AI answers properly describes a clean manual setup.

For the wider buying picture, read how to choose an AI search visibility tool and questions to ask before you buy.

Does the method change what you can claim?

Yes. If answers come from APIs, you can say what a clean, signed-out style request returns, and nothing about personalised use. If they come from automating an app, you can say what that session displayed, with its account state. Match the claim to the method. A report that says "ChatGPT recommends us" without saying how the answer was collected claims more than it can support.

Disclosure: Citewake is itself in this category, so weigh our description of the trade-offs accordingly. For a first free look at how a site appears to assistants, you can use the Citewake checker.