# Citewake research record Started on 29 Oct 2026 as the lab's running scientific record. This is its public text, published at citewake.com/research with its SHA-256 fingerprint. The record has four parts: hypotheses (written down before the data that tests them is analysed), protocols (exactly how each edition was run), calibration (how the instruments were checked), and open questions. **Rules** 1. A hypothesis is registered, with its test and decision rule, before the data that tests it is analysed. The date it is added is its registration date. Each entry says honestly whether its test data had already been collected when it was registered. 2. An entry is never edited after registration. A change becomes a new dated entry that names the one it replaces. 3. Exploratory evidence (what made us suspect something) is kept apart from confirmatory tests (data analysed after registration). 4. Negative and null results are recorded as carefully as positive ones. 5. Published editions never change (fix forward only). This record cites them by edition and DOI. --- ## 1. Hypotheses Statuses: **registered** (test pending), **exploratory support** (suggested by earlier data; awaiting the registered test), **supported**, **not supported**, **inconclusive**. ### H-001 · AIs favour their own maker - **Statement:** each AI recommends its own maker's products more often than the other three AIs recommend those products. - **Exploratory evidence (Edition 1, AI writing assistants, 10.5281/zenodo.23763253):** ChatGPT recommended ChatGPT 14 times, and OpenAI's site was among its sources 79% of the time. Claude recommended Claude 25 times (and OpenAI's products 24 times), with Anthropic's site among its sources 4% of the time. Gemini recommended Anthropic's products 26 times, OpenAI's 23 and Google's 8. So the evidence is mixed: Claude favoured its maker, Gemini did not. - **Registered test:** in each category where one of the four makers sells a product (AI writing assistants; cloud computing platforms for Google), compare the maker's Recommendation Rate from its own AI against the mean of the other three AIs, over decision questions only. Supported if the 90% bootstrap interval of the difference is above zero, and not supported if it includes zero. - **Test data:** Edition 2 cloud computing platforms (collected 29 Oct 2026, not yet analysed at registration). AI writing assistants from its next edition. - **Status:** exploratory support for Claude; registered. ### H-002 · First recommendations are unstable - **Statement:** asked the same question a day apart on the same model, each AI keeps the same first recommendation on fewer than half of the questions. - **Exploratory evidence (Edition 0 to Edition 1, same models, a day apart; citewake.com/models):** the first recommendation held 35.7% (ChatGPT), 32.3% (Claude), 29.1% (Gemini) and 35.7% (Perplexity), with 90% intervals. Brands in common were about 23–31%. - **Registered test:** the next same-model repeat of an anchor set. Supported if each AI's upper 90% bound is below 50%. - **Status:** exploratory support; registered. ### H-003 · The four AIs rarely agree - **Statement:** among questions where all four AIs give a first-choice recommendation, they name the same first choice in fewer than a quarter. - **Exploratory evidence (Edition 1, email marketing software, 10.5281/zenodo.23763258):** an independent reader counted 8 agreements in 44 such questions (18%). - **Registered test:** every Edition 2 category, per category and per panel. Supported if the pooled agreement rate's upper 90% bound is below 25%. - **Test data:** Edition 2 (collected, not yet analysed at registration). - **Status:** exploratory support; registered. ### H-004 · Claude names far more brands than Perplexity - **Statement:** Claude names at least twice as many brands per answer as Perplexity. - **Exploratory evidence (Edition 1, CRM software):** 429 brand readings from Claude's 60 answers (7.2 per answer) against 147 from Perplexity's (2.5). Caveat: Edition 1's extraction dropped about 8% of brand readings (Gemini about 14%) through a quote-check fault, fixed from Edition 2 (section 2). - **Registered test:** Edition 2, the mean brands per valid answer by AI, per category. Supported if the ratio exceeds 2 in at least five of the seven categories. - **Status:** exploratory support; registered. ### H-005 · Gemini often answers without searching - **Statement:** with web search available, Gemini answers without searching more often than the other three AIs. - **Exploratory evidence:** Gemini answered 41% of Edition 0 questions without searching; in Edition 1 email marketing, 16 of 60. - **Registered test:** Edition 2, the no-search share by AI, per category. Supported if Gemini's share is the highest in at least six of the seven categories. - **Status:** exploratory support; registered. ### H-006 · Searching improves qualification on regulated money (pilot) - **Statement:** on the financial ambiguity tests, answers that searched pass more rubric v1 criteria than answers that did not. - **Registered test:** once the judges are calibrated against the human reference set (section 3), compare the mean criteria passed by searched versus unsearched answers across the financial ambiguity tests. No result before calibration. - **Test data:** Edition 2's ambiguity tests (collected, not yet judged at registration). - **Status:** registered. ### H-007 · AIs cite third parties, not the brands they recommend - **Statement:** most citations in a category point to third-party pages, not to the recommended brands' own sites. - **Exploratory evidence (Edition 1, live race tracking):** YouTube was cited in 13 of 240 answers and never recommended. ChatGPT cited specific how-to videos, while Gemini cited the bare domain. - **Registered test:** Edition 2, the share of cited URLs on recommended brands' own domains, per category. Supported if it is below 25% in at least five of the seven categories. - **Status:** registered. ### H-008 · AI concentrates charity recommendations on a few large organisations - **Registered:** 29 Oct 2026, **before any charity data was collected** (fully prospective). - **Statement:** when people ask AI where to give, recommendations concentrate on a small number of large, well-known charities. Smaller and local charities rarely appear, even when the question names a city or region. - **Why we suspect it:** reports of an emerging "AI visibility gap" for nonprofits (October 2026), and Citewake's finding in other categories that the four AIs mostly name the biggest brands. No Citewake charity data exists yet. - **Registered tests:** the five charity categories (animal, environmental, children's, humanitarian aid, health), per market panel (US; Europe composite), decision questions only. - **H-008a (concentration):** supported if the five most-recommended organisations receive more than 50% of all recommendations in at least four of the five categories, in each panel (90% bootstrap interval above 50%). - **H-008b (small charities):** supported if organisations with annual income under US$1 million, or the local-currency equivalent, by the official charity register of their country, receive under 10% of recommendations in each category and panel, including on questions that name a city or region (90% interval upper bound below 10%). Organisations whose income can't be found in an official register are reported separately and never counted as small. - **Primary test data:** the first charity edition (quarterly from February 2027, or earlier if run ad hoc). - **Status:** registered. --- ## 2. Protocols The exact request sent to each AI is stored with every observation (method 4.1). This table is the summary. | | Edition 0 | Edition 1 | Edition 2 | |---|---|---|---| | Collected | 28 Oct 2026 | 29 Oct 2026 | 29 Oct 2026 | | Categories | 3 | 8 | 7 (batch 3: money, payments, banking, cards, accounting, cloud, endpoint protection) | | Benchmark | v1 (60 scored + 15 private per category) | v1 | v1, batch 3 (frozen Draft 3, 29 Oct) | | Answering AIs | gpt-6-luna, claude-haiku-5-5, gemini-3.1-flash-lite, perplexity/sonar | same | same | | Search | on, no location sent | same | same; per-market questions name the US or a European country | | Answer reader (Grand Master) | claude-opus-5-5 | same | same, with the fixed quote check | | Page reader | claude-haiku-5-5 | same | same | | Method | v0.8 | v0.8 | v0.8 addenda (panels, ambiguity tests, extraction fix) | | DOIs | 10.5281/zenodo.23755361, …363, …365 | 10.5281/zenodo.23763251 to …275 (8) | reserved at release | **Changes that affect comparison between editions** (from the method changelog): - **Extraction quote check (from Edition 2):** the check that a brand's quote appears in the answer now reads the answer as rendered (without markdown) and accepts a brand's short name under strict rules. In Editions 0 and 1 it dropped 1,264 + 104 readings, about 8% (Gemini about 14%); in Edition 2's first category it dropped 1 of 1,099. Editions 0 and 1 stay as published, so comparisons that cross this change say so. - **Per-market panels (from Edition 2):** the US and Europe (a composite of 13 countries) are reported apart and never pooled. - **Ambiguity tests (from Edition 2):** 14 questions with rubric v1. Their answers are published verbatim, ungraded until calibration. --- ## 3. Calibration | Instrument | Check | Status | |---|---|---| | Answer reader (brands, roles, top pick) | Known-answer gate: 14 hand-built answers must all pass; quote drops ≤ 2% | In force from Edition 2 | | Answer reader recall (brands never returned) | Stratified audit of 320 Edition 1 answers | Planned after 17 Nov 2026 | | Answer-quality judges | Two blind judges from other providers, against a human-reviewed reference set (every disagreement plus a stratified 25% of agreements) | Pilot on Edition 2; reference set pending human review | | Stability baseline | Same model, same questions, a day apart, 90% intervals | Edition 0 to 1 measured; next repeat to be scheduled | --- ## 4. Open questions - **Q-001 · Self-knowledge or retrieval?** Claude recommends itself while rarely citing Anthropic (4%), and ChatGPT cites OpenAI 79% of the time. Does Claude recommend from what it knows, and ChatGPT from what it finds? - **Q-002 · Imprecise citations:** why does Gemini so often cite a bare domain (youtube.com) instead of a page, and how much does that weaken its citation evidence? - **Q-003 · Recall:** how many brands does the reader never return at all? The Edition 2 fix covers dropped readings; the audit after 17 Nov measures the rest. - **Q-004 · Video as evidence:** why do how-to videos appear as evidence for software choices (YouTube in live race tracking)? - **Q-005 · Panel breadth:** do answers in the US and Europe panels differ in how many brands they name? This is descriptive only, never a claim about either market. ## 6. Amendments ### A-001 · 29 Oct 2026 · Confirmatory data for H-001 to H-007 - **Replaces nothing; adds to H-001 to H-007.** These hypotheses were registered after Edition 2's answers were collected, though before any were analysed for them. Edition 2 therefore counts as a **secondary test, registered after collection**, never as a prospectively registered one. - **Primary confirmatory test:** the first edition collected entirely after registration (Edition 3 or later) that includes the relevant categories. For H-002, that is the first same-model repeat collected after registration. - **Publication (29 Oct 2026):** the hypotheses, their tests and decision rules, evidence summaries, statuses, outcomes and amendments are public on citewake.com/research, with this file's SHA-256 fingerprint. The private validation questions are never published. Every outcome is shown, whether supported, not supported or inconclusive. ### A-002 · 29 Oct 2026 · H-008 measurement contract (before any charity data) - **Adds to H-008; replaces nothing.** Registered after the independent critique of the batch 4 questions and before any charity data was collected. H-008's original text stands as registered. - **Questions are donor scenarios.** The charity questions are editorially built donor scenarios, not observed donor prompts. Citewake never describes them as what donors actually type unless independent demand evidence shows it. - **Which organisation was recommended.** Each recommendation is resolved to the registered entity named in the official register of its country, using the answer's own wording and cited pages. A shared or ambiguous name is never assigned to a national organisation by default. If the entity can't be resolved, it's marked **unresolved**, the AI's wording is kept, and it counts in neither the large nor the small group. - **What counts as a small charity.** - **Measure:** total income (gross revenue) in the organisation's most recent annual return filed with its official register before the collection date. Sources: IRS Form 990 data for the US; the Charity Commission (England and Wales), OSCR (Scotland) and CCNI (Northern Ireland) for the UK; each European country's official register where it publishes income. - **Threshold:** small means income below US$1 million, with other currencies converted at the European Central Bank reference rate on the first collection day. - **Unknown income:** if no income can be found in an official register, the organisation is **income unknown**. It is reported separately and never counted as small. - **Branches:** a separately registered local affiliate is its own entity and is sized on its own return. - **Geographic relevance, measured apart from size.** For questions that name a place, each recommended organisation is coded **operates there** (an office, programme or service in that city or region, from its own site or register), **national/international only**, or **unknown**. Size and geographic relevance are reported separately. Concentration on large organisations is reported as concentration, never as unfairness: large organisations may genuinely serve many places. - **Statistical contract.** - **Unit:** the decision question within a category and panel. Each answer's recommendations are weighted 1/n, where n is the number of organisations it recommends, so an AI that names more organisations doesn't dominate. Unweighted results are reported too. - **H-008a (concentration):** the weighted share of recommendations going to the five most-recommended organisations, per category and panel, with a 90% bootstrap interval resampling questions. - **H-008b (small charities):** the weighted share going to resolved small charities, with income-unknown and unresolved shares shown beside it. The baseline compares questions that name a place with questions that don't. - **Outcomes, per H-008's thresholds:** **supported** if the interval is entirely on the hypothesised side; **contradicted** if it is entirely on the other side; **inconclusive** otherwise, including when unresolved plus income-unknown exceed 25% of weighted recommendations. - **Sample size:** 10–14 named-place questions per category and panel give descriptive coverage, not a decisive per-category test. Conclusions are drawn pooled across the five categories within each panel. Per-category results are descriptive. ### A-003 · 29 Oct 2026 · What the charity editions report - **Adds to H-008 and A-002; replaces nothing.** 29 Oct 2026: the charity special editions report what every Citewake category reports, which is whether each charity is named, recommended and cited (rank, Recommendation Rate and citations, per panel). Charity size isn't part of the edition reports. - **H-008 stays registered as written.** Its size and concentration analysis (A-002) is deferred and isn't run with the editions. The stored answers and citations keep everything needed to run it later exactly as A-002 specifies. If it is run, it is reported as a separate analysis, with the date it was run. ### A-004 · 29 Oct 2026 · Where each question came from - **Adds to H-001 to H-008; replaces nothing.** 29 Oct 2026: these hypotheses began as questions the lab set out to answer, not as patterns mined from the editions. The "exploratory evidence" in each entry is what we had seen when we registered it, not where the question came from. The origin of each question: - **H-001 (self-preference):** the lab's question, "Do the AIs search, or just recommend themselves?", raised while reviewing Edition 1's AI writing assistants. - **H-002 (stability):** the lab's question, "Same question, same model, a day apart: does the answer hold?", which led to the baseline on the models page. - **H-003 (agreement):** the independent reviewer's question while auditing Edition 1: how often do the four AIs agree on a winner? - **H-004 (breadth):** the lab's question, raised while building the reports: do some AIs name far more brands than others? - **H-005 (searching):** the lab's question, "Does it do the search, or answer from memory?" - **H-006 (qualification on regulated money):** the lab's question, raised while designing the batch 3 ambiguity tests: does financial advice get treated as financial advice? - **H-007 (sources versus recommendations):** the lab's question about YouTube being cited in live race tracking: is being cited the same as being recommended? - **H-008 (charities):** the lab's question, prompted by the October 2026 reports of an AI visibility gap for nonprofits. - **Exploratory evidence stays labelled as such.** It describes what earlier editions showed, never a test. The registered tests and their data are unchanged. --- ## 5. Log - **29 Oct 2026:** record started. H-001 to H-007 registered, with Edition 2 collected but not analysed (except money transfer's extraction, read only to check the extractor, not tested against any hypothesis). Protocols for Editions 0–2 recorded. Calibration plan recorded. - **29 Oct 2026:** A-001 added: Edition 2 is a secondary test (registered after collection); the primary confirmatory test is the first edition collected after registration. The register goes public on citewake.com/research. - **29 Oct 2026:** extraction check, `name_not_in_text` drops (a brand the reader returned whose name the word-for-word check couldn't find in the answer), read from stored readings with no provider call. Edition 0: 42 as published (18 under the current check). Edition 1: 68 as published (45 under the current check). Editions 0 and 1 stay exactly as published; these counts are for the record only. Edition 2, cloud computing platforms: verification drops 2.15% (35 of 1,631) under the check it was read with, 1.29% after the name-check fix, 0.43% (7 of 1,631) once the standard abbreviations (GCP, GKE, EKS, AKS, Azure ML, ExpressRoute, CycleCloud) were added as aliases on its brand list (v1). The extraction gate allows 2%. - **29 Oct 2026:** H-008 registered before any charity data was collected (the charity research programme, quarterly). - **29 Oct 2026:** A-002 registered: the H-008 measurement contract (entity resolution, small-charity definition, geographic relevance, weighting, outcomes), after the batch 4 critique and before any charity data. - **29 Oct 2026:** A-003: the charity editions report recommendation and citation like every category. The H-008 size analysis is deferred, and the data is kept to run it later. - **29 Oct 2026:** A-004: the origin of each hypothesis recorded as the question the lab set out to answer. Exploratory evidence stays separate.