On July 21, 2026, Substack added an AI detection feature built on a partnership with Pangram, a company that specializes in identifying machine-generated text. The tool scans posts, notes, replies and comments longer than 100 words and shows readers an estimated split between human-written, AI-assisted and AI-generated content. It launched on web and iOS, with Android to follow.
The move is a small product change with a large question behind it. As generative text becomes indistinguishable from human writing at a glance, platforms are reaching for detection as a way to preserve trust. Whether that works depends entirely on how reliable the detection is, and the honest answer is that it is very good in some conditions and quietly unreliable in others.
What Substack actually shipped
Substack's implementation is deliberately modest in its claims. It runs Pangram's classifier on text over a 100-word threshold, below which detection is unreliable, and returns an estimated three-way breakdown rather than a binary verdict. Substack has been explicit that the score is a judgment aid, not a final ruling, and that it is meant to inform readers rather than to police writers.
That three-way framing matters. A simple AI-or-not label collapses the most common real-world case, a human writer who used AI for part of the work, into a misleading binary. By separating "AI-assisted" from "AI-generated," Substack acknowledges that authorship in 2026 is a spectrum, and that the interesting signal is degree of involvement rather than a yes-or-no stamp.
How AI text detection works, in brief
Modern detectors like Pangram are themselves machine-learning classifiers, trained on large collections of human-written and AI-generated text to learn the statistical fingerprints that separate the two. Machine-generated writing tends to be more predictable token by token, sitting closer to the model's most probable next word, and detectors learn to pick up on that and many subtler patterns. Independent evaluations have reported that Pangram achieves near-zero false-positive and false-negative rates on standard machine output, and that it is the rare detector able to meet a stringent policy cap on false positives, reported at 0.005 or below, without giving up detection power.
A low false-positive rate is the number that matters most for fairness. When a detector is used to make consequential decisions, wrongly flagging genuine human writing as machine-made is the failure that does real harm, to a student, a journalist, or a writer accused on the strength of a bad score. Pangram's reported ability to hold false positives near half a percent is what makes it defensible as a labeling aid rather than an accusation engine.
Where detection quietly fails
The reliability that holds for raw machine output degrades sharply once a human edits the text. Research from Epoch AI has found that detectors can miss up to 18% of stylized AI text, meaning nearly one in five machine-written passages slips through once it has been lightly rewritten, reformatted, or run through a humanizing pass. The more a person edits an AI draft, the more the statistical fingerprints blur, which is precisely the workflow most people actually use.
Detection holds up on raw output, and slips on edited text
The share of AI-written text that evades detectors rises sharply once the output is stylized or lightly rewritten by a human.
Illustrative. The 18% figure is Epoch AI's reported upper bound for stylized AI text; the "standard" bar reflects the near-zero false-negative rates independent evaluations report for tools like Pangram on unedited machine output.
Detection works best on exactly the content that matters least: raw, unedited machine output that few people publish unchanged.
Metir AI analysis
This is the structural weakness of any detection-based approach. The easiest text to catch, an unedited model dump, is also the text least likely to cause harm, because it is low effort and often low quality. The hardest text to catch, a carefully edited hybrid, is the one where the authorship question is genuinely murky. Detection is strongest where the stakes are lowest and weakest where they are highest, which is why every serious deployment, Substack's included, frames the output as a signal rather than a verdict.
There is a deeper limit that no accuracy number captures. As Pangram itself acknowledges, a detector can estimate whether AI shaped a piece of text, but it cannot measure care, originality of thought, or whether AI was used only as a research aid. A thoughtful essay drafted with AI assistance and a lazy one written entirely by hand can receive scores that invert their actual quality. Detection measures process, not worth, and conflating the two is the most common misuse.
The arms race, and a saner frame
Detection and evasion improve together. Better detectors train better humanizers, which force better detectors, a cycle familiar from spam filtering and plagiarism checking. There is no stable end state where detection simply wins, which is why the more durable responses are shifting from detection toward disclosure: provenance signals, cryptographic content credentials, and platform norms that ask creators to label their own use rather than relying on a classifier to catch them.
Substack's launch sits sensibly inside that reality. It uses a strong detector, sets a conservative threshold, presents a spectrum instead of a binary, and calls the result a judgment aid. That is roughly the right posture for a tool that is useful but fallible. The mistake to avoid is treating any detection score as proof, in either direction. For anyone building products on generative models, the same lesson applies to the output side: the value is not in whether text can be labeled machine-made, but in the workflow, editing, and human judgment wrapped around it, which is the logic behind treating models as interchangeable components rather than the product itself, as model-agnostic platforms like Metir AI do.
Looking ahead
Substack's AI labels will make the scale of AI-assisted writing more visible, which is valuable on its own even if the labels are imperfect. The open questions are whether readers interpret a probabilistic score as the signal it is rather than the verdict it is not, and whether the industry moves toward disclosure standards that do not depend on winning an unwinnable detection race. In the meantime, the most honest way to read any AI detection label is as one input among several, useful for context and dangerous as a conclusion.
Work with AI on your terms, across every model
The authenticity debate is really about keeping human judgment in the loop as AI output becomes ubiquitous. With Metir AI you get unified access to leading models from OpenAI, Google, Anthropic and others in one workspace, so you can draft, compare and refine with the right model for each task while keeping full control of the result. Try Metir AI free.
Sources:
- Substack is adding an AI detection feature | Engadget
- Substack Introduces New AI Transparency Features | Dataconomy
- Substack AI detection: Scan posts with Pangram | TechMyMoney
- How can I detect AI on Substack? | Substack Support
- Substack launches Pangram AI detector to label AI-generated content | AINave
Image credits
Header image: hands typing on a laptop, by Simon Hattinga Verschure via Wikimedia Commons, released under CC0 1.0 (public domain dedication). In-body photograph of hands typing on a keyboard with digital overlays, by ResDigital18 via Wikimedia Commons, licensed under CC BY-SA 4.0. Both images are generic illustrations of writing and are not screenshots of any detection tool.