What an AI research lab is, and why AI recommendations need one

An AI research lab studies what AI assistants say, as an outside observer. It asks the same questions of ChatGPT, Claude, Gemini and Perplexity on a schedule, keeps every answer word for word, and measures what the answers show: which brands get recommended, which sources get used, and how all of it moves over time. AI recommendations need one because a single answer proves almost nothing, and nobody else is keeping the record.

What problem is a research lab solving?

People now ask AI assistants what to buy, whom to hire and which tool to use. The answer they get shapes a real decision. Yet the answer is not published anywhere. It is generated on the spot, for one person, and it is gone when the chat closes.

That creates three gaps.

• No record. Nobody can say what an assistant recommended last month, because nobody kept it.
• No baseline. Without a fixed set of questions, every test is a new test. Results cannot be compared.
• No outside view. Brands, tool vendors and the assistants themselves all have a stake in the story. Few parties are positioned to simply report what was said.

A research lab fills those gaps. It turns a stream of private, changing answers into a dataset that can be checked, cited and compared.

What does an AI research lab actually do?

The work is plain, and the discipline is in doing it the same way every time.

1. Choose categories and questions. A category is a market defined by a job a buyer wants done. The questions are written the way people type them, with no brand names in them.
2. Ask every assistant the same questions. The same wording, the same settings, in one pass, so that the answers describe one moment.
3. Keep everything. The full answer, the sources the assistant cited, the model that answered and the time. See why every AI answer should be kept.
4. Read the answers consistently. One reader applies one standard to all four assistants, so a "recommendation" means the same thing everywhere. The reasoning is in one reader for four AIs.
5. Measure with honest uncertainty. Report rates with intervals, not single scores.
6. Publish, including the method. Anyone should be able to trace a number back to the words an assistant said.

Citewake works this way. The full method is on the methodology page, and the first published results are in the editions.

How is that different from a dashboard or a monitoring tool?

The difference is in who the work is for and what it must withstand.

A monitoring tool serves one customer. It tracks the questions that customer chose, about the customer's brand, and its output is a private report. That is useful and legitimate.

A research lab serves anyone who wants to check its work. It chooses its questions in advance, applies the same rules to every brand, publishes the method, and keeps the evidence. A researcher, a journalist or a competitor should be able to read the same data and reach the same numbers.

Neither is better in the abstract. They answer different questions. The lab answers "what do the assistants say about this category, and how sure are we?" The tool answers "how is my brand doing against the questions I care about?"

Why does independence matter so much?

Because the output is a measurement, and a measurement is only worth anything if it was not shaped by who paid for it.

If a lab let brands buy a place in its results, the results would describe the lab's customers, not the assistants. So a credible lab lets brands in through evidence alone: an assistant named them, or a cited page listed them. Nobody pays to be included. We set out why in why an index of AI answers must never sell inclusion.

Independence also means saying plainly what the data cannot show. The answers a lab collects describe the questions it asked. They do not describe every conversation anyone has had.

What can a lab not do?

Being straight about limits is part of the job.

• It cannot see private conversations or personalised answers in signed-in apps.
• It cannot tell you why an assistant chose a brand. It can show what was said and what was cited.
• It cannot promise what an assistant will say next month. Assistants change.
• It cannot make a small sample into a large one. Rates come with intervals for that reason; see margins of error in AI measurement, explained simply.

Who uses this kind of research?

The readers are wider than brands.

• Marketers and agency owners who need a defensible picture before advising a client.
• Founders deciding whether a category is worth entering.
• Journalists who want to quote what assistants recommend without relying on a screenshot.
• Researchers and students who need citable, reproducible data about how AI search behaves. See AI answers as a dataset.

What does good practice look like in a lab?

A few habits separate research from casual testing. Questions are written and published before results come in. Methods carry version numbers, and any change to them is recorded and dated. Raw answers are never edited, only read. Results are reported with their uncertainty, and surprises are checked before they are announced. When a lab finds it made a mistake, it says so and corrects the record. None of this is glamorous, but it is what lets a stranger trust a number they did not produce. It is also what separates a lab from a marketing exercise that happens to use data: the first is willing to publish results that embarrass its own assumptions, and the second usually is not.

How do you get started yourself?

You do not need a lab to begin thinking like one. Pick one category, write ten questions without brand names, ask each in a fresh chat several times, and write down what you see. Then notice how much it varies. That experience is the case for the whole discipline. If you want a quick, free look at how assistants describe a site, the Citewake checker is a place to start.