An institution recently emailed and asked:
“Are you doing some AI experiment on my collection without telling me? I see this one transcriber who really feels like they are using AI.”
I started typing a standard response: Of course we’re not testing AI on your collection without telling you. Have you checked the suspicious behaviors dashboard? Have you considered turning on AI Drafts?
And then I stopped and went to investigate.
The user’s behavior wasn’t flagged in the suspicious behaviors dashboard, which immediately worried me. I thought: Oh no. Our carefully thought-out catchers aren’t working. I turned to Ben, my partner, who built that dashboard, and said, “We’re not catching this user’s use of AI. How could that be?”
So we started digging.
I looked at the pages the institution had sent over, along with other projects this transcriber had worked on—a surprisingly diverse set of materials, from the Jane Addams Papers to Italian estate inventories. The transcriptions were strange. They were almost robotically focused on individual letterforms. You could see how the transcriber was making decisions, and letter by letter, they were plausible decisions—but when you stepped back and read the sentences as a whole, they often didn’t make much sense.
I was convinced this had to be some kind of bot. Maybe not a large language model like Gemini, but something more transcription-engine-like.
Meanwhile, Ben approached the problem from a completely different angle, looking at the account activity itself: where the user was coming from, whether the behavior looked scripted, how long they were spending on each page, and whether they were typing steadily or pasting in chunks. But none of the evidence pointed to automation. No bot spends thirty-five minutes transcribing a page, and no human secretly copy-pasting from another tool behaves like this either.
So Ben’s conclusion was simple: this had to be a human.
We emailed the transcriber and asked about her process. She wrote back with a thoughtful explanation: she carefully studies the documents, identifies each letter she can make out, and works her way through unfamiliar handwriting one character at a time.
In other words: not a bot at all.
This wasn’t malicious AI use. This was a diligent, conscientious human volunteer who was taking a very different approach from most other volunteers, and probably was motivated by different things. To make her work match the needs of the reporting project, she needed more training, better judgment, or stronger review.
And honestly, that’s a harder problem.
Bad actors are easier to reason about. But sincere contributors producing low-quality work? That’s a quality control challenge every crowdsourcing project faces, whether AI is involved or not.
That experience got us thinking differently about the role AI might play in transcription workflows. Our work using AI in transcription focuses on draft generation—using AI to create a first pass for humans to review and correct. That’s one useful approach. But there’s another possibility that may be just as interesting: using AI not to generate content, but to help identify work that deserves closer human attention.
FromThePage already helps project owners prioritize review by surfacing work from first-time contributors, one-off volunteers, or pages that seem to need substantial correction. But we also have tools for comparing transcriptions—including comparing human work against AI-generated drafts. What if we treated disagreement itself as a signal? A page where human and AI transcriptions diverge significantly may simply be the page that needs another pair of human eyes.
We’ve been hearing interest in exactly this kind of workflow. The Library of Virginia told us they aren’t especially comfortable offering AI drafts directly to volunteers. But they were intrigued by the idea of using AI after human transcription is complete, as a way to flag pages that may need additional review. It’s the same technology—but in a completely different role.
We’re exploring a similar concept in a project with Industrial Archives, where we’re running two different AI models against the same materials and comparing their outputs. When the models substantially disagree, that disagreement becomes a useful quality signal, helping surface records that deserve expert attention.
That’s the direction that excites me most: not replacing human expertise, but helping direct human attention where it matters most.
Because sometimes the hardest transcription problems aren’t the ones involving bots.
They’re the humans trying their best.