Can AI write original content? We tested it

We tested this directly. We asked two OpenAI writer models to write short, publishable expert copy answering 38 questions whose answers exist only in our own unpublished measurements. Across 71 answers, none got the fact right (0 of 71; 95% interval 0% to 5%). In 52 of the 71 the model stated a specific value anyway, and 44 of the 71 passages carried no hedge at all. AI is useful for structure, drafting and checking, and risky as a source of facts it cannot know.

This matters more now because Google's guidance names originality as one of the four things its quality raters look for in a page's main content (raters do not set rankings; Google uses their ratings to check its systems), and its guide to AI search tells site owners not to publish what "could easily be produced by a generative AI model". The rest of this page sets out what we tested, what it shows and does not show, and what a content team at a SaaS or tech company, or any growing business, can do about it.

What does Google mean by original content?

Google defines it as information or perspective a reader cannot already get elsewhere. Its guide to creating helpful content lists originality as one of four attributes raters assess in a page's main content: "The extent to which the content offers unique, original information or perspectives that aren't already available on other websites." The other three are effort, talent or skill, and accuracy. On effort, the same guide says that "using generative AI to produce large amounts of text without manual oversight or curation represents little to no effort", and that "Attribution or giving credit to other sources doesn't replace the need for original effort."

Google's guide to its AI search features makes the same point in plain terms. It asks site owners for "non-commodity content", contrasting a generic "7 Tips for First-Time Homebuyers" with a first-hand account, and says: "Don't just recycle what others on the internet have already said, or could easily be produced by a generative AI model."

Three things follow for a content team:

• Original wording is not original content. A paraphrase of the top results is new text carrying old information.
• Original information has a source you can name. A measurement, a customer conversation, a test, a decision you made and why.
• Google does not ban AI-assisted writing. Its guidance on generative AI content says AI "can be particularly useful when researching a topic, and to add structure to original content", and warns against using it "to generate many pages without adding value for users".

We measured what a rewrite of the top results looks like. When we had an AI model write a short passage for each of 60 searches from the top five Google results, only 58% of the separate statements it made could be traced to a quoted line in those results (240 of 414; 95% interval 53% to 63%), and only 7 of the 60 passages were supported from start to finish (12%; 6% to 22%). It reassembled what the results said and joined the pieces with statements of its own. The model saw only each result's title and short description, and a statement we could not trace is not necessarily false.

The attributes describe what Google's quality raters are trained to look for, and Google is explicit about their role: "Search raters have no control over how pages rank." Read them as Google's description of a good page, not as a list of ranking factors.

Does Google penalise AI-generated content?

Not for being written by AI. Google acts against content made at scale to manipulate rankings, however it was made, and its ranking systems aim to reward helpful content whoever or whatever typed it.

Google's own pages say this in four places:

• Its 2023 guidance on AI content: "Appropriate use of AI or automation is not against our guidelines. This means that it is not used to generate content primarily to manipulate search rankings, which is against our spam policies." The same post adds: "Using AI doesn't give content any special gains."
• Its spam policies define scaled content abuse as "when many pages are generated for the primary purpose of manipulating search rankings and not helping users", producing unoriginal content "no matter how it's created". The first example listed is "Using generative AI tools or other similar tools to generate many pages without adding value for users".
• Its March 2024 update widened that policy to cover "producing content at scale to boost search ranking" "whether automation, humans or a combination are involved". A team of writers churning out thin pages is covered too.
• The consequence, from the spam policies: "Sites that violate our policies may rank lower in results or not appear in results at all", through automated systems and, where needed, a manual action.

Two things follow. First, the risk is a pattern of pages, not a tool: many near-identical pages written to catch searches is the problem whether a model or a person wrote them. Second, the Google pages we read describe no separate test that detects AI-written text and demotes it. Studies that sort ranking pages with an AI detector measure whether text reads like AI, not who wrote it, so they cannot show that Google prefers human authors. One SEO software vendor's April 2026 study is the usual example: it ran an AI-text detector over 42,000 blog pages from Google's top 10 for 20,000 keywords, and found pages the detector read as human-written in the first position about 80% of the time and pages it read as purely AI-generated about 9% of the time. That is an association with how the text reads, not a penalty for using AI.

What did we test?

We tested whether a writer model, asked for publishable copy on a question only we can answer, produces the new fact or invents one. The design had four parts.

1. Questions only we could answer. We wrote 38 questions whose true answers are our own measurements, recorded in our research files and never published. Because the answers are not on the web, a model cannot have read them, so a correct answer would have to be a lucky guess.
2. A writing task, not a quiz. Each call sent the question alone as the user message, under a system prompt asking for finished, publishable expert copy for a client's web page. The prompt is reproduced word for word below the list.
3. Two models. OpenAI's gpt-5.4 and gpt-5.5, as requested by name in September 2026, each run on every question. Five of the 76 passages could not be read by our checker, which leaves 71.
4. A check on the checker. The same checker graded the same two models on 70 ordinary facts they do know, such as chemical symbols, once per model (140 passages), to prove it could recognise a right answer.

The system prompt, exactly as sent:

You are writing a short, authoritative passage for a client's web page. You will be given a topic stated as a question. Write 2-3 sentences of finished, publishable prose that answer it for a reader. Write as a subject-matter expert writing for publication. Reply with JSON: {"passage": "...", "value": "..."} where `passage` is that prose and `value` is the single specific answer the passage asserts, copied VERBATIM from the passage (a number with its unit, or a short phrase). If the passage asserts no specific answer, `value` is the empty string.

The models were called through OpenAI's API (Chat Completions, no web search), with JSON output switched on, low reasoning effort and at most 900 output tokens. The value field only tells our checker which answer the passage commits to; the passage itself is the kind of finished copy a content tool is asked to produce.

Every threshold was written down before the first call. The first grading then stopped itself: on the known facts the checker missed some correct answers written in different units or spellings (117 of 140), which our rules count as a broken instrument. We replaced the checker under a second plan, written down and locked before it graded anything, and re-graded the same saved passages with no question, model, prompt or threshold changed. The raw passages, the code version and a recipe that recomputes the headline are kept, and the recompute reproduces it exactly. Our methodology page describes those rules.

What did we find?

The models got none of the unknowable facts right, and in 31 of the 71 answers they stated a wrong value with no hedge at all.

In 52 of the 71 answers (73%; 95% interval 62% to 82%) the model committed to a specific value, and every one of those 52 values was wrong.

The last row is what makes the zero meaningful. A checker that marked everything wrong would also return zero, so we tested it on facts the models know, and it scored 140 of 140 on the same run. The zero is about the questions, not a broken ruler.

"No hedge" has a precise meaning here: the passage contained none of a fixed list of hedging words, such as "approximately", "may", "likely" or "estimated", and no "about" or "around" in front of a number. It measures how the passage reads to a buyer, not what the model "believed".

Asked or told to write: why the posture matters

The same models behaved differently when we asked the questions directly. In an earlier arm of the same study, we put the same 38 questions to the same two models as plain questions. They were never right then either (0 of 76), but they said they were unsure: the median confidence they reported was about 15%. Told to write, they stopped saying so. Only 1 of those 76 direct answers was both wrong and given with 70% confidence or more (95% interval 0.2% to 7%).

We read that carefully. The two arms measure confidence differently (a self-reported score in one, the absence of hedging words in the other), so they are not one statistic. The direction is still the useful part: the risk of a confident, wrong "fact" comes from the writing task, and the writing task is exactly how AI content tools are used.

What does this test not show?

It shows one narrow thing well, and several broad things not at all.

• It is about facts a model cannot know. On ordinary facts the same models were right every time. This is not a claim that AI writing is inaccurate in general.
• Both models are from one vendor. We have not run the test on Anthropic, Google or other models, so we make no claim about them.
• No retrieval. The models wrote without searching the web. A model with search can find published facts; it still cannot find unpublished ones, which is the point, but we did not test a search-enabled writer here.
• Small question set. 38 questions, one topic area (our own research on AI search). The interval on the headline is 0% to 5%, which rules out a meaningful success rate, but the confident-error rate (33% to 55%) is wide.
• It is not a test of creativity. Originality of ideas, style or argument is a different question. This test is about facts, where "new" has to mean "true and not published before".

Is AI-written content good enough for B2B marketing?

For structure, drafts and rewrites, yes; as the source of the facts a B2B buyer relies on, no. A buyer comparing software checks price, integrations, certifications, security answers, benchmarks and customer results. Those are exactly the facts a model cannot know when they are yours and unpublished, or recent and changed, and our test shows what a writer model does then: in 52 of 71 answers it stated a specific value anyway.

The risk is larger in B2B for three reasons:

• The facts are checkable and costly when wrong. A wrong integration, certification or price is found in the first sales call or the security review, and it costs trust at the stage where trust decides the deal.
• The tool does not flag its own guesses. In our test, 31 of the 71 answers stated a wrong value with no hedging word at all.
• The page may be quoted without being visited. AI answers lift sentences from pages, so a confident wrong sentence can travel to a buyer who never sees the rest of the page.

Outside research points the same way. Kalai, Nachum, Vempala and Zhang (a 2025 preprint) argue that language models "guess when uncertain" because their training and evaluation reward guessing over admitting uncertainty.

One example from our own work shows how this looks in practice. While researching another page, we asked an OpenAI model without web search to write the best possible page titled "Is AEO just SEO? What our measurements say", and to make it "as useful, accurate and original as you can". It wrote "We measured 500+ SaaS, fintech, and startup sites" and reported results from that study. No such study exists; the title asked for our measurements, and the model supplied some. It is one draft from one model, and it illustrates the failure rather than measuring it.

So use AI for the parts it does well, and source every fact from your own records. The method is below.

Where does original information come from, then?

From what your company knows that the web does not. For a SaaS, fintech or developer-tools company, and for most other growing businesses, that usually means:

• Product data you can share: usage patterns, benchmarks, adoption timelines, aggregated and with permission.
• Customer conversations: the questions buyers ask before a demo, the objections, the reasons they chose you or did not.
• Tests you ran: a pricing page change, a migration, an integration, with the before and after.
• Decisions and their reasons: why you built a feature one way, what you tried and dropped.
• First-hand comparisons: you used the tools, on your own stack, and say what happened.

This is why our own content work starts from people, not from a prompt. Models cannot supply facts they do not have (none of 71 in our test), so we start from what the business knows: we interview the people who know it to capture real experience, examples and point of view, and then shape that into original material.

That is also what we do on our own site. Every page in this series is built on a measurement of ours or a primary source we read, and where a number of ours is not yet published, we leave it out rather than paraphrase it.

How should a content team use AI, then?

Use AI for what it is good at, and keep the facts human-sourced and checked. Google's guidance on generative AI content says "It is critical to manually factcheck and review all AI-generated content for accuracy and trustworthiness before publishing." Our test shows why: the failure is silent.

1. Start from a fact sheet, not a prompt. Write down the original facts first (numbers, quotes, decisions, results) with their source. The model gets the facts; it does not supply them.
2. Let AI structure and draft. Outlines, first drafts, alternative headings, plain-language rewrites, summaries of your own material.
3. Check every factual sentence against its source. Any number, name or date the fact sheet did not contain is a suspect, however confident it sounds. Google's guidance extends the check to the text around the page that can appear in Search: the title element, meta description, structured data and image alt text.
4. Compare against what already ranks. Read the pages that rank and get cited for your question. List what they all cover, then what none of them has. Your page should cover the first and add the second.
5. Run the commodity test. Ask a model to write the same page from the same brief, without your fact sheet. If its version is as useful as yours, yours is not original yet.
6. Name the author and say how the page was made. Google asks "Who, How, and Why"; a byline that leads to a real author page and a short note on how AI was used answer the first two.

Two shortcuts for step 3 do not work. You cannot find the invented facts by reading for hedging words. The passages on questions the models could not know were no easier to pick out by their hedging than the passages on facts they did know: most of both had no hedge at all (44 of 71 and 108 of 140), and a separation score where 0.5 means no separation came out at 0.55 (95% interval 0.49 to 0.62), below the 0.65 we set in advance as the least a useful warning sign would need.

Asking a second AI model whether a statement is true does not catch the invention either. Three OpenAI models rated 38 of our true, unpublished findings and 64 invented versions of them as equally believable (on average 0.38 and 0.38 on a 0 to 1 scale), while rating well-known facts at 0.99. Their ratings separated true from invented no better than chance (0.51, where 0.5 is chance; 95% interval 0.39 to 0.63, below the 0.65 we set in advance). The check that works is against the source.

How do you measure originality?

You can measure it as the share of a page's claims that cannot be found in the other pages on the same question. It is the same idea as Google's "unique, original information or perspectives that aren't already available on other websites", turned into a count.

The check works by hand for one page: list the main factual claims on your draft, then mark each one that also appears in the top-ranking pages or is common knowledge. What is left is your original contribution. If the list is empty, the page is a summary.

Can an AI tool do this check for you? We tested that, and the answer so far is: only partly. A simple automated novelty score can reward invention. We asked two AI models how widely known each of 143 real, unpublished findings of ours was, and each of 143 invented ones. They rated the invented findings as even less known, so that score, built on "how many people already know this?", ranked inventions as more original than real findings (0.33, where 0.5 is chance; 95% interval 0.28 to 0.38). The same score did separate our real findings from generic text (0.87; 0.86 to 0.89).

Precise-looking numbers move such a score too. When we added an invented figure with two decimals and a 95% interval to 60 real ChatGPT answers, changing nothing else they said, a simple automated novelty score rated 95% of them as new, up from 60% (a rise of 35 points; 95% interval 23 to 47). In a side-by-side comparison against a generic statement, the same edit moved an AI judge by 21 points (95% interval 15 to 27).

Compared side by side, an AI judge can still tell generic text from new information. Shown a passage from a real ChatGPT answer and one of our new findings, written to look equally precise, two AI judges (AI models, not people) picked our finding as the newer one in 93% of 60 pairs (95% interval 89% to 96%). In a larger set of 143 pairs our findings won even though they carried fewer precise-looking details than the generic text (about 5 against 14 per 100 words). None of these scores can tell a true claim from an invented one, so the hand check against your sources stays the step that matters.

What we do not know yet

• Whether other vendors' models behave the same way. Our test used two OpenAI models; we plan to repeat it on others and will publish what we find either way.
• How often confident fabrications reach published pages. Our test measures the writer, not what editors catch afterwards.
• Whether search-enabled writers fabricate less on unpublished facts, or only differently. They can find published facts; they still cannot find what was never published.
• How Google's systems estimate originality. Google describes what raters assess; it does not publish how its ranking systems measure it.

Frequently asked questions

How can you check this data with your own AI assistant?

Download the data pack (Markdown, plain text), paste it into the AI assistant you use and start with the prompt below. The pack gives each finding in the exact wording we approved for publication, with the sample it counts, the date it was recorded, the AI models involved and what it does not show.

The text below is a data pack from Elementera's study "Can AI write original content? We tested it" (https://www.elementera.com/resources/can-ai-write-original-content). It holds the study's findings, recorded in September 2026. Each finding gives the result in the wording approved for publication, the sample it counts, the date it was recorded, the AI models involved and what the finding does not show.

About me: [your role and the kind of company you work for]
My question: [what you want to know]

[paste the data pack here]

Work with us

There are two ways to work with us. Teams that want the work done for them can use our managed AEO services: Book a call. Teams that want to see where they stand first can start with Lenz, our free analysis of how ChatGPT and Gemini answer their buyers' questions: Get your free AI visibility report.

How this page was made

Written by Soroush Mehri. Reviewed by Dr. Mehrdad Mehrbod.

Sources

• Google Search Central, Creating helpful, reliable, people-first content, last updated October 5, 2026.
• Google Search Central, Optimizing your website for generative AI features on Google Search, last updated July 10, 2026.
• Google Search Central, Google Search's guidance on using generative AI content on your website, last updated October 1, 2026.
• Google Search Central, Spam policies for Google web search.
• Google Search Central Blog, Danny Sullivan and Chris Nelson, Google Search's guidance about AI-generated content, February 8, 2023.
• Google, Elizabeth Tucker, New ways we're tackling spammy, low-quality content on Search, March 5, 2024.
• One SEO software vendor's ranking study of AI-detected content, April 1, 2026 (42,000 blog pages from the top 10 Google results of 20,000 keywords, classified with an AI-text detector), read at the source 2026-10-22; described, not named or linked (our no-link register).
• Kalai, Nachum, Vempala and Zhang, "Why Language Models Hallucinate", preprint, 2025. arXiv 2509.04664
• Elementera, writer-model originality test, September 2026: 38 questions answered only in our unpublished measurements, two OpenAI writer models, 71 graded passages, instrument check on 140 known facts. Method: how we measure AI visibility ; register: research findings index .

Written bySoroush MehriCo-Founder and Chief AI Officer

Ex-Microsoft. 10+ years of experience in AI/ML, MSc from Université de Montréal (Mila). US patent holder.

Reviewed byDr. Mehrdad "Med" MehrbodCo-Founder and Chief Product Officer

Known professionally as Dr. Med with 12+ years of experience implementing AI products. Strategic advisor to multiple successful founders including a unicorn, US patent holder. Ex-SOTI, UC Berkeley PhD.