Substack Built an AI Detector Because Its CEO Coined a Word for the Problem: Claudefishing

Substack shipped a scan feature on July 21 that estimates how much of a post, note, reply, or comment was written by AI. CEO Chris Best calls the problem it targets Claudefishing: letting a reader assume a human wrote something an AI actually wrote.

What the scan does

The scan runs on any piece of writing over 100 words, whether that's a post, a note, a reply, or a comment, and returns an estimate of the human-to-AI proportion of the text. Writers can add a disclosure statement titled How I make this, dispute a scan they think got it wrong, and run a scan on their own draft before publishing to see what a reader would see. It launched on web and iOS, with Android support coming later. The detection itself runs on Pangram, a third-party AI-detection model Substack licensed rather than building in house, and the score sits next to the piece rather than gating whether it can be published at all.

Disclosure, not a ban

This is good use of AI. The problem is when there is a mismatch between a reader's expectation and reality.

Writing assisted by AI, drafting help or an editing pass, is not the target, and Substack isn't banning it. The target is content presented as a personal, human account when it wasn't written by the person whose name is on it. That's a narrower and more defensible claim than most AI-detection tools make, and it explains why the feature ships as a disclosure prompt and a disputable score rather than a takedown mechanism or an outright ban.

The accuracy caveat

Pangram has published independent research claiming high accuracy for its detection model, but Substack's own rollout is careful to say a scan result isn't guaranteed to be right. That caveat matters more than usual here. A false positive on a human writer's scan is a reputational problem for a platform built on writer trust, not a minor interface glitch to shrug off, and it's likely why Substack built a dispute path into the feature from day one instead of treating the score as final.

Why a build studio cares

We ship content and community tooling for clients, and we write our own editorial for this blog and news section, which puts us on the writer's side of exactly the question Substack is now answering with a shipped product. A disclosure norm that separates AI-assisted writing from misrepresented authorship is one we'd rather see standardized early, with a clear scoring method and a clear appeal path, than fought over post by post once every platform has shipped its own incompatible version of it. Whatever a client's platform ends up doing here, the design choice worth copying is the dispute path, not just the score.

Next step: read Substack's own announcement and TechCrunch's coverage. If you're building disclosure or moderation tooling for a content platform, write to hello@gattyworks.com.