Key point. Not supported for the part that matters. Asked to write expert copy about facts that existed only in our unpublished data, two AI models got none of them right, and most stated a specific wrong value anyway. Original information has to come from people or from measurement.
Verdict: not supported for facts only you hold. A model can write fluent copy about what is already public. Asked to write about what only your company knows, it invents the facts, in the same confident voice.
What did we measure?
We gave two AI writer models questions whose answers exist only in our own unpublished measurements, and asked for short, publishable expert copy. Then we checked every fact.
"We asked two OpenAI writer models to write short, publishable expert copy answering 38 questions whose answers exist only in our own unpublished measurements. Across 71 answers, none got the fact right (0 of 71; 95% interval 0% to 5%)."
The checker was not broken: "On ordinary facts the same models do know, the same checker scored 140 of 140 correct, so the zero is not a broken ruler."
The wrong answers did not look wrong. "In 52 of the 71 answers (73%; 95% interval 62% to 82%) the model committed to a specific value, and every one of those 52 values was wrong."
The writing request changed how the models behaved: "Asked the same 38 questions directly, the same models were also never right (0 of 76), but they said they were unsure: the median confidence they reported was about 15%. Told to write, they stopped saying so."
Method
The table gives the models, the samples, the intervals and the dates behind each number above.
| Item | Detail |
|---|---|
| Models | OpenAI gpt-5.4 and gpt-5.5, as writers; one vendor, so not a cross-vendor result |
| Questions | 38 questions whose answers exist only in our unpublished measurements, chosen to be absent from the web |
| Samples | 71 graded written answers; 140 known facts (instrument check); 76 direct answers (asking test) |
| 95% intervals | 0% to 5% (0 of 71); 62% to 82% (52 of 71) |
| Dates | September 2026 |
| Design | The models were asked to write, not to answer a quiz; a separate arm asked the same questions directly |
Each number has a methodology card. The full study, with the hedging and fact-checking results, is in can AI write original content?
Where does the claim come from, and what do the docs say now?
The claim stretches Google's 2023 statement about AI content into a promise about expertise. Google's current guidance draws the line more carefully.
| Source | What it says | Published | Read |
|---|---|---|---|
| Earlier guidance: Google Search Central Blog, Google Search's guidance about AI-generated content | "Our focus on the quality of content, rather than how content is produced", with E-E-A-T (experience, expertise, authoritativeness and trustworthiness) as the guide. | February 8, 2023 | October 22, 2026 |
| Current documentation: Google Search Central, Google Search's guidance on using generative AI content | Generative AI can help research a topic and add structure to original content; generating many pages without adding value for users may violate the spam policy on scaled content abuse. | current page | October 22, 2026 |
| The common claim | "AI can write your thought leadership for you." | repeated widely | October 22, 2026 |
What the common claim gets wrong. Google's guidance accepts AI as a tool for original content. It does not say a model can supply the experience or the facts that make content original, and our measurement shows it invents them when asked.
How far does the result reach?
The test is about facts a model cannot know: your own data, your clients' results, your team's experience. It does not say AI-written copy is wrong in general, and both models came from one vendor.
That is also why it matters for thought leadership. The part of a piece that makes it worth citing is the part nobody else has published, and that is exactly the part a model has to invent.
What should you do instead?
Use AI for drafting and editing, and source every fact from people or measurement.
- Before writing, list the facts only your company holds: numbers, cases, decisions and lessons.
- Get them from the people who know them, in an interview or a short written brief.
- Let a model help with structure and wording, never with the facts.
- Check every number in the final copy against its source before it is published.
Work with us
If you want a team to measure and improve how AI search treats your company, our managed AEO services do that work with you: Book a call. If you would rather see where you stand first, Lenz gives you a free analysis of how ChatGPT and Gemini answer your buyers' questions: Get your free AI visibility report.
More claims we tested.