From FromThePage
Don't ask whether AI can read handwriting. Find out whether it can help with one of your collections.
AI can produce an impressive transcription of a carefully chosen page. But that doesn’t tell you whether it will work on your material—or whether using it will actually save you any work.
This experiment is designed to help you find out.
Choose a small sample from a real collection. Run AI transcription on it. Correct the results. Look closely at what the AI got right, what it got wrong, and how much human work was still required.
By the end, you should have a much better idea of whether AI is worth exploring for this collection—and some actual evidence to bring back to your colleagues.
Time: About 30–60 minutes
Cost: Free
You'll need: 10–20 pages from a collection you'd like to experiment with
Created by FromThePage for Fantastic Futures 2026.
1. Choose something difficult
Don't make this easy on the AI.
Choose at least 10 pages from a collection that presents real transcription challenges. You might try:
- difficult handwriting
- faded ink or bleed-through
- cross-writing
- ledgers and tables
- scrapbooks or complex layouts
- annotations and marginalia
- inconsistent spelling
- emotionally difficult or offensive material
If possible, choose pages that represent the range of the collection rather than ten very similar pages.
And don't worry if the material isn't formally digitized yet. For an experiment, photos from your phone are fine.
Before you continue, record:
What collection are you testing?
Why did you choose it?
What makes this material difficult?
Download the Experiment Worksheet
2. Decide what success would look like
Before you see what the AI can do, decide what you're hoping it will accomplish.
Are you trying to:
- make handwritten material searchable?
- improve accessibility?
- reduce the amount of staff transcription required?
- tackle a backlog that otherwise wouldn't be transcribed?
- create draft transcriptions for students or volunteers to review?
- extract names, dates, amounts, or other structured information?
- make material more discoverable to researchers?
Now ask the harder question:
How good would the AI need to be for that use to be worthwhile?
The answer doesn't have to be a percentage. You may care more about whether names are accurate, whether errors are easy to spot, or whether reviewing a draft is substantially faster than transcribing from scratch.
Write down your hypothesis before running the experiment.
3. Run the AI
We'll use FromThePage for the experiment so that you can compare the original AI output with your corrected transcription.
Create a Free FromThePage Trial
Upload your test pages as a collection and click the checkbox "Generate AI Draft text using Gemini AI."
Open your first page in the transcription interface and select AI Draft.
Now you have your first result.
Resist the temptation to judge it immediately as "good" or "bad." You're going to investigate that.
4. Correct it
Carefully compare the AI draft against the original image and correct every error you find.
Be exacting. Your corrected version will become the ground truth against which you evaluate the AI.
As you work, pay attention to something that's easy to overlook:
How much work are you doing?
Record approximately how long it takes you to review and correct each page.
Notice the kind of work you're doing, too.
Were mistakes obvious? Were there plausible errors you almost missed? Did the AI struggle with names but handle ordinary prose well? Did tables or formatting require substantial intervention? Did reviewing the draft feel easier than transcribing the page yourself?
And definitely note any hallucinations.
If you have time, transcribe one comparable page from scratch without an AI draft. Compare the time and effort.
AI didn't necessarily eliminate the work. Where did the work move?
5. Measure what happened
Once you've corrected a page, switch to the AI tab in FromThePage.
You can compare the original AI transcription against your corrected version and see the differences between them.
Look at the Character Error Rate (CER).
Don't worry too much about whether a particular CER is objectively "good." Instead, compare pages.
Find:
Your best page: Where did AI perform particularly well?
Your worst page: What caused it trouble?
Now look beyond the number.
What kinds of errors did the AI make?
Did it correctly transcribe the words that matter most to researchers? What about names, places, dates, amounts, or specialized terminology?
A transcription can have a very good overall error rate while getting exactly the information you care about wrong.
6. Look under the hood
You've looked closely at what the AI produced. Now take a look at what produced it.
From the AI tab, record the:
- model
- prompt
- original AI output
Then look at the token usage for your experiment.
These details matter. AI models change quickly, and different models and prompts can produce different results on exactly the same material. If you want an experiment to be useful—or repeatable—you need to know what you actually tested.
Think about transparency
If you used AI on a larger project, what would you want to document?
Would researchers need to know which model was used?
Would you want to preserve the prompt and original AI output?
If you tested a newer model six months from now, how would you compare its performance with what you found today?
And what does the token usage tell you about the resources required to process this material at a much larger scale?
Record the model, prompt, and token usage on your Experiment Worksheet. Together with the accuracy and review effort you've already measured, they give you a much more complete picture of what this experiment actually involved.
7. Make something useful
So far, you've been evaluating the transcription. Now let's do something with it.
FromThePage can turn your collection into an accessible PDF, creating a reading copy that can be read by assistive technology and searched by researchers.
Download an accessible PDF of your test collection by going to the Download tab of the document and clicking Configure next to Accessible PDF.
When you create the PDF, you can choose what transcription to include:
- Human-reviewed text only — include transcription only for pages that a person has reviewed and saved.
- AI Drafts when human-reviewed text isn't available — use human-reviewed transcription where it exists and fill in the remaining pages with AI-generated drafts.
Try the second option if you haven't reviewed every page in your test set.
Then open the PDF and look through it.
You started with handwritten material. You now have something searchable and accessible—and you can see exactly where human review fits into that process.
Think about your real collection
Would this output be useful to researchers?
Could it help you make digitized material more accessible while more extensive transcription or review is still underway?
Which material would you be comfortable publishing with AI-generated text? Which would you insist on having a human review first?
And if this were 10,000 pages rather than 10, how would you decide where human attention matters most?
Record your thoughts on the Experiment Worksheet.
8. Decide whether this scales
Now imagine that instead of 10 pages, you have 10,000.
Look back at your worksheet.
Quality
Was the transcription good enough for what you're trying to accomplish?
Were the important words correct?
Were errors predictable—or dangerously plausible?
Human work
How much review did each page require?
Who would do that work at collection scale: staff, students, volunteers, or someone else?
Was reviewing AI genuinely faster or easier than starting from scratch?
Workflow
What happens after transcription?
Does someone need to review the results?
What output do you actually need: searchable text, an accessible PDF, structured data, publication on the web, or something else?
Where will the finished transcription be published?
How will researchers get to it?
Transparency
Would you want to document which model and prompt produced the transcription?
Do you need to preserve the original AI output?
How will you distinguish AI-generated work from human-reviewed work?
These questions are often more important than:
"Can AI read the handwriting?"
So, should you use AI on this collection?
You may have discovered that AI isn't good enough for this material.
That's a useful result. You now have evidence rather than assumptions—and you may have identified exactly what makes this collection difficult.
You may have found that AI is useful, but substantial human review is still necessary.
Now the problem is less about AI itself and more about designing the right human-and-AI workflow.
Or you may have discovered that AI is remarkably good.
Great. Your next questions are about scale: processing thousands of pages, quality assurance, collaboration, structured data, accessibility, and publishing the results.
Those are exactly the kinds of problems FromThePage is designed to help cultural heritage organizations solve.
Want a second opinion?
If you've run the experiment on your collection, we'd love to see what you found.
Send us your results—or just show us the pages that surprised you—and we'll talk through what we'd try next.
You've already done the interesting part: you have a real collection and real evidence.
Bonus: Take this back to your colleagues
Don't just tell your colleagues that AI "worked" or "didn't work." Show them what you learned.
We've created a simple five-slide template:
1. The Collection
What we tested and why
2. The Experiment
What material, model, prompt, and workflow we used
3. The Results
Best result, worst result, accuracy, and review effort
4. What We Learned
Where AI helped, where it struggled, and where the human work went
5. What We Recommend
Stop, experiment further, run a larger pilot, or scale
Download the Presentation Template
About FromThePage
FromThePage helps libraries, archives, museums, and other cultural heritage organizations turn handwritten collections into searchable, accessible, structured text.
Use staff, students, volunteers, responsible AI—or a combination of all four.