# Citewake research record Started on 29 Oct 2026 as the lab's running scientific record. This is its public text, published at citewake.com/research with its SHA-256 fingerprint. The record has four parts: hypotheses (written down before the data that tests them is analysed), protocols (exactly how each edition was run), calibration (how the instruments were checked), and open questions. **Rules** 1. A hypothesis is registered, with its test and decision rule, before the data that tests it is analysed. The date it is added is its registration date. Each entry says honestly whether its test data had already been collected when it was registered. 2. An entry is never edited after registration. A change becomes a new dated entry that names the one it replaces. 3. Exploratory evidence (what made us suspect something) is kept apart from confirmatory tests (data analysed after registration). 4. Negative and null results are recorded as carefully as positive ones. 5. Published editions never change (fix forward only). This record cites them by edition and DOI. --- ## 1. Hypotheses Statuses: **registered** (test pending), **exploratory support** (suggested by earlier data; awaiting the registered test), **supported**, **not supported**, **inconclusive**. ### H-001 · AIs favour their own maker - **Statement:** each AI recommends its own maker's products more often than the other three AIs recommend those products. - **Exploratory evidence (Edition 1, AI writing assistants, 10.5281/zenodo.23763253):** ChatGPT recommended ChatGPT 14 times, and OpenAI's site was among its sources 79% of the time. Claude recommended Claude 25 times (and OpenAI's products 24 times), with Anthropic's site among its sources 4% of the time. Gemini recommended Anthropic's products 26 times, OpenAI's 23 and Google's 8. So the evidence is mixed: Claude favoured its maker, Gemini did not. - **Registered test:** in each category where one of the four makers sells a product (AI writing assistants; cloud computing platforms for Google), compare the maker's Recommendation Rate from its own AI against the mean of the other three AIs, over decision questions only. Supported if the 90% bootstrap interval of the difference is above zero, and not supported if it includes zero. - **Test data:** Edition 2 cloud computing platforms (collected 29 Oct 2026, not yet analysed at registration). AI writing assistants from its next edition. - **Status:** exploratory support for Claude; registered. ### H-002 · First recommendations are unstable - **Statement:** asked the same question a day apart on the same model, each AI keeps the same first recommendation on fewer than half of the questions. - **Exploratory evidence (Edition 0 to Edition 1, same models, a day apart; citewake.com/models):** the first recommendation held 35.7% (ChatGPT), 32.3% (Claude), 29.1% (Gemini) and 35.7% (Perplexity), with 90% intervals. Brands in common were about 23–31%. - **Registered test:** the next same-model repeat of an anchor set. Supported if each AI's upper 90% bound is below 50%. - **Status:** exploratory support; registered. ### H-003 · The four AIs rarely agree - **Statement:** among questions where all four AIs give a first-choice recommendation, they name the same first choice in fewer than a quarter. - **Exploratory evidence (Edition 1, email marketing software, 10.5281/zenodo.23763258):** an independent reader counted 8 agreements in 44 such questions (18%). - **Registered test:** every Edition 2 category, per category and per panel. Supported if the pooled agreement rate's upper 90% bound is below 25%. - **Test data:** Edition 2 (collected, not yet analysed at registration). - **Status:** exploratory support; registered. ### H-004 · Claude names far more brands than Perplexity - **Statement:** Claude names at least twice as many brands per answer as Perplexity. - **Exploratory evidence (Edition 1, CRM software):** 429 brand readings from Claude's 60 answers (7.2 per answer) against 147 from Perplexity's (2.5). Caveat: Edition 1's extraction dropped about 8% of brand readings (Gemini about 14%) through a quote-check fault, fixed from Edition 2 (section 2). - **Registered test:** Edition 2, the mean brands per valid answer by AI, per category. Supported if the ratio exceeds 2 in at least five of the seven categories. - **Status:** exploratory support; registered. ### H-005 · Gemini often answers without searching - **Statement:** with web search available, Gemini answers without searching more often than the other three AIs. - **Exploratory evidence:** Gemini answered 41% of Edition 0 questions without searching; in Edition 1 email marketing, 16 of 60. - **Registered test:** Edition 2, the no-search share by AI, per category. Supported if Gemini's share is the highest in at least six of the seven categories. - **Status:** exploratory support; registered. ### H-006 · Searching improves qualification on regulated money (pilot) - **Statement:** on the financial ambiguity tests, answers that searched pass more rubric v1 criteria than answers that did not. - **Registered test:** once the judges are calibrated against the human reference set (section 3), compare the mean criteria passed by searched versus unsearched answers across the financial ambiguity tests. No result before calibration. - **Test data:** Edition 2's ambiguity tests (collected, not yet judged at registration). - **Status:** registered. ### H-007 · AIs cite third parties, not the brands they recommend - **Statement:** most citations in a category point to third-party pages, not to the recommended brands' own sites. - **Exploratory evidence (Edition 1, live race tracking):** YouTube was cited in 13 of 240 answers and never recommended. ChatGPT cited specific how-to videos, while Gemini cited the bare domain. - **Registered test:** Edition 2, the share of cited URLs on recommended brands' own domains, per category. Supported if it is below 25% in at least five of the seven categories. - **Status:** registered. --- ## 2. Protocols The exact request sent to each AI is stored with every observation (method 4.1). This table is the summary. | | Edition 0 | Edition 1 | Edition 2 | |---|---|---|---| | Collected | 28 Oct 2026 | 29 Oct 2026 | 29 Oct 2026 | | Categories | 3 | 8 | 7 (batch 3: money, payments, banking, cards, accounting, cloud, endpoint protection) | | Benchmark | v1 (60 scored + 15 private per category) | v1 | v1, batch 3 (frozen Draft 3, 29 Oct) | | Answering AIs | gpt-6-luna, claude-haiku-5-5, gemini-3.1-flash-lite, perplexity/sonar | same | same | | Search | on, no location sent | same | same; per-market questions name the US or a European country | | Answer reader (Grand Master) | claude-opus-5-5 | same | same, with the fixed quote check | | Page reader | claude-haiku-5-5 | same | same | | Method | v0.8 | v0.8 | v0.8 addenda (panels, ambiguity tests, extraction fix) | | DOIs | 10.5281/zenodo.23755361, …363, …365 | 10.5281/zenodo.23763251 to …275 (8) | reserved at release | **Changes that affect comparison between editions** (from the method changelog): - **Extraction quote check (from Edition 2):** the check that a brand's quote appears in the answer now reads the answer as rendered (without markdown) and accepts a brand's short name under strict rules. In Editions 0 and 1 it dropped 1,264 + 104 readings, about 8% (Gemini about 14%); in Edition 2's first category it dropped 1 of 1,099. Editions 0 and 1 stay as published, so comparisons that cross this change say so. - **Per-market panels (from Edition 2):** the US and Europe (a composite of 13 countries) are reported apart and never pooled. - **Ambiguity tests (from Edition 2):** 14 questions with rubric v1. Their answers are published verbatim, ungraded until calibration. --- ## 3. Calibration | Instrument | Check | Status | |---|---|---| | Answer reader (brands, roles, top pick) | Known-answer gate: 14 hand-built answers must all pass; quote drops ≤ 2% | In force from Edition 2 | | Answer reader recall (brands never returned) | Stratified audit of 320 Edition 1 answers | Planned after 17 Nov 2026 | | Answer-quality judges | Two blind judges from other providers, against a human-reviewed reference set (every disagreement plus a stratified 25% of agreements) | Pilot on Edition 2; reference set pending human review | | Stability baseline | Same model, same questions, a day apart, 90% intervals | Edition 0 to 1 measured; next repeat to be scheduled | --- ## 4. Open questions - **Q-001 · Self-knowledge or retrieval?** Claude recommends itself while rarely citing Anthropic (4%), and ChatGPT cites OpenAI 79% of the time. Does Claude recommend from what it knows, and ChatGPT from what it finds? - **Q-002 · Imprecise citations:** why does Gemini so often cite a bare domain (youtube.com) instead of a page, and how much does that weaken its citation evidence? - **Q-003 · Recall:** how many brands does the reader never return at all? The Edition 2 fix covers dropped readings; the audit after 17 Nov measures the rest. - **Q-004 · Video as evidence:** why do how-to videos appear as evidence for software choices (YouTube in live race tracking)? - **Q-005 · Panel breadth:** do answers in the US and Europe panels differ in how many brands they name? This is descriptive only, never a claim about either market. ## 6. Amendments ### A-001 · 29 Oct 2026 · Confirmatory data for H-001 to H-007 - **Replaces nothing; adds to H-001 to H-007.** These hypotheses were registered after Edition 2's answers were collected, though before any were analysed for them. Edition 2 therefore counts as a **secondary test, registered after collection**, never as a prospectively registered one. - **Primary confirmatory test:** the first edition collected entirely after registration (Edition 3 or later) that includes the relevant categories. For H-002, that is the first same-model repeat collected after registration. - **Publication (29 Oct 2026):** the hypotheses, their tests and decision rules, evidence summaries, statuses, outcomes and amendments are public on citewake.com/research, with this file's SHA-256 fingerprint. The private validation questions are never published. Every outcome is shown, whether supported, not supported or inconclusive. --- ## 5. Log - **29 Oct 2026:** record started. H-001 to H-007 registered, with Edition 2 collected but not analysed (except money transfer's extraction, read only to check the extractor, not tested against any hypothesis). Protocols for Editions 0–2 recorded. Calibration plan recorded. - **29 Oct 2026:** A-001 added: Edition 2 is a secondary test (registered after collection); the primary confirmatory test is the first edition collected after registration. The register goes public on citewake.com/research.