How to compare answers across ChatGPT, Claude, Gemini and Perplexity

To compare ChatGPT, Claude, Gemini and Perplexity fairly, ask each the same questions in the same words, under the same conditions, save every answer, and then compare the brands named, the role each brand plays and the sources cited. Compare patterns across many questions, not single answers.

The four assistants overlap, but they do not agree. Knowing where they differ is more useful than knowing your average.

Why bother comparing them at all?

Because buyers do not all use the same assistant. Some use one by habit, some switch depending on the task, and some use whichever is built into their phone or browser. If you only check one, you may be strong where you looked and invisible where you did not.

The assistants also behave differently. Some show citations prominently. Some search the web on every question, others only when they judge it necessary. Which pages each one finds and trusts is a big part of why recommendations differ. For a closer look at each, see the four AI assistants that matter when people buy.

How do you set up a fair comparison?

Fairness comes from holding everything constant except the assistant. Use this checklist.

• Identical wording. Copy and paste the same question into each. Do not adapt it to the assistant.
• No brand names in the question. If you ask "is Acme any good?", you have asked a different question from "which tool should I use for X?". Discovery questions should not name anyone.
• Same conditions. Signed out where you can, no custom instructions or memory, and web search on in all four. If an assistant answers from memory alone, it is a different kind of answer.
• Same time window. Run all four on the same day, so the answers describe one moment.
• Repeat each question. One run is one draw. See how many times to ask.

The signed-out and private-browsing details are in how to test AI answers properly.

What should you record for each answer?

Save the whole thing, not a summary. For each answer, capture:

1. the assistant and the date;
2. the exact question;
3. every brand named, in order;
4. each brand's role: recommended, neutral or negative;
5. any expressed first choice;
6. every cited URL.

Roles matter because a brand named as a cautionary example is not a win. If you are unsure how to assign them, how to read an AI answer like an analyst walks through it.

How do you lay the results side by side?

A simple grid does most of the work. Put your questions in rows and the four assistants in columns, and in each cell write the role your brand played: recommended, mentioned, cited, or absent.

The table above is an illustration of the layout, not data. When you fill it in for real, three views are worth taking:

• Down a column: how you do in one assistant across all questions.
• Across a row: which questions you win everywhere and which nowhere.
• The gaps: a question where one assistant recommends you and three do not tells you something specific to investigate.

Why might the four disagree about you?

There are a few common reasons, and none require guessing at internals.

• Different sources found. Each assistant runs its own search and may reach different pages. A roundup that one cites may never appear for another.
• Different citation habits. Some attach many links, some few. That changes what you can observe, not only what they use.
• Different tone. Some give a short list of picks, others a long balanced overview with caveats. Longer answers name more brands, mostly neutral.
• Plain variation. Any assistant can give a different answer on a second run. Differences between assistants must be larger than differences between runs of the same assistant before you read anything into them.

That last point is the one people forget. If ChatGPT recommends you in two of five runs and Gemini in one of five, you have not shown a difference.

Should you combine the four into one number?

Be careful. A single combined figure hides the cases that matter most: strong in two assistants, absent in two. Show each assistant's result next to the overall one.

Also be careful with evidence that some assistants report and others do not. If one assistant exposes which pages it consulted and another does not, you cannot compare that measure across them. Citewake keeps such measures separate per assistant for that reason, and says where a measure is unavailable rather than filling the gap. The reasoning is in the methodology.

What do you do with the comparison?

Use it to choose where to act.

1. Find the assistant where you are weakest on questions that matter, then look at which pages it cites for those questions.
2. Find the pages that recur across assistants. A page that several of them cite is a strong lead.
3. Re-run after changes. Repeat the same questions later, on the same terms, to see whether anything moved. Tracking over time covers the routine.

What if the results are mixed?

Mixed results are the usual case. You may be recommended by two assistants, named neutrally by one and absent from another. Do not average that into a single verdict. Write it as it is: where you win, where you are present but not recommended, and where you are missing. Each pattern points to a different task, from fixing a page an assistant reads to earning a place on a third-party list.

If you want a first look before building a grid, the Citewake checker shows how a site appears to AI assistants.