If your writing got flagged as AI, the score is a hint, not a verdict. Even the best detectors make mistakes, and the company behind the most talked-about one says its confidence labels are not true probabilities.
NPR ran a piece on October 31 asking what an AI flag really means, after a wave of writers said polished human prose was being marked as machine-made. Here’s what’s known, and what to do if it happens to you.
Here are the quick facts before we get into the details:
- Pangram, the detector at the center of the debate, launched as Checkfor.ai in October 2023.
- Its latest version, Pangram 4, shipped on July 29, 2026.
- The company claims a false positive rate of about 1 in 24,000, but independent tests are mixed.
- It reads prose of at least 50 words, so short texts, code and math are a weak fit.
The takeaway: a flag is a reason to look closer, never proof on its own.
Why polished writing gets flagged as AI
Detectors like Pangram are trained on pairs of human and AI texts matched by topic, length and tone. They learn the statistical fingerprints of machine output, then score new text against them.
The trouble is that clean, well-edited, structured writing can look a lot like model output. Tidy paragraphs, even rhythm and common phrasing all push the score up.
The vendor itself says the tool is built for prose. Templated writing, short factual answers, code and math are more error-prone or outside its intended use, according to Wikipedia’s summary of the company’s own documentation.
What the accuracy numbers actually show
Pangram says Pangram 4 wrongly flagged 41 of 1,000,000 human samples, which is where the 0.0041% figure comes from. That’s a company number tested on web text, so treat it as a best case.
Independent results paint a more mixed picture. This table pulls together the main figures reported so far.
| What the claim or test says | Result | Source |
|---|---|---|
| Pangram 4 false positive rate on human text | About 0.0041%, roughly 1 in 24,000 (company figure) | Pangram, July 2026 |
| Epoch AI test of imitated author styles | Missed 30 of 297 passages (10.1%) | Epoch AI, July 2026 |
| Notre Dame preprint on humanized abstracts | Pangram 3.2 missed over 96% at the study threshold | Preprint, August 2026 |
| Saha and Feizi study | Often flagged human text that had only light AI edits | Study, 2025 |
Notice the pattern. Errors run both ways: real AI text slips through, and some human text gets caught. That’s why one researcher team in June 2026 recommended investigating flagged work instead of trusting the detector alone.
Real cases where a flag caused trouble
The stakes are not just academic. In March 2026, the publisher Hachette dropped the planned editions of a novel after Pangram’s CEO said the tool classified 78% of it as AI-generated. The author denied using AI herself and blamed an editor.
In April, a Wall Street Journal editor called the tool a “defamation machine” after it flagged three writers, though two later admitted using AI for revision. And in May, The Atlantic pointed to a false positive involving the writer Taylor Lorenz, which the company acknowledged.
The lesson is not that detectors are useless. It’s that a public accusation built on a score alone has already gone wrong more than once.
What to do if your writing is flagged as AI
Stay calm and gather evidence. These steps help most, roughly in this order:
- Open your version history in Google Docs, Word or Notion. A timeline of edits is the strongest proof you wrote it.
- Save notes, outlines, earlier drafts and research links.
- Ask which exact passages were flagged, and which tool and version produced the score.
- Request a human review of your past work, not just this one piece.
Evidence of process beats arguments about style. If you want to understand the other side of this problem, our guide on how Claude’s AI text watermark works and our look at the ChatGPT watermark for EU text explain why labeling AI output is so hard.
How to lower the odds of a false flag
You can’t game a detector, and you shouldn’t try. But a few habits make your authorship easy to prove and your voice harder to confuse.
Write in a tool that keeps history, and don’t paste in big blocks from other sources. Add specifics only you would know: names, places, numbers from your own work. Generic filler is the thing detectors catch most.
If you use AI for light editing, say so where rules require it. A disclosed tweak is far safer than a hidden one that a detector later finds.
Frequently Asked Questions
Can an AI detector be wrong?
Yes. Every detector has false positives and false negatives. Studies show errors in both directions, especially with edited, imitated or humanized text.
Is a high AI score proof I used ChatGPT?
No. The score is a statistical estimate. Pangram’s own confidence labels are not calibrated probabilities, so a high number does not mean a high chance of guilt.
Does Pangram work on short text?
It analyzes prose of at least 50 words. Very short answers, templated text, code and math are more error-prone or outside its design.
How can I prove I wrote something myself?
Show your version history, drafts and notes. A timestamped edit trail in a document editor is the clearest evidence.
My take: use detectors as a smoke alarm, not a judge. If you’re a reader, editor or teacher, ask for process evidence before you accuse anyone. If you’re a writer, keep your drafts and version history from day one, because that habit will protect you far better than any rewrite to “sound more human.”
Leave a Reply