HTR Β· 02
Practical guide: how to train your first HTR model
Creating your own Handwritten Text Recognition (HTR) model does not require programming or machine learning knowledge. The basic principle is that we present the system with a sufficient number of images of lines of handwriting together with their correct transcription. This pair β the image of the document and its verified transcription β forms the so-called Ground Truth (GT), i.e. the reference data from which the model learns. [1]
The following procedure is primarily focused on Transkribus, as it allows the entire process to be performed in a graphical environment. However, the same basic principle is also used by open source tools, such as eScriptorium with the Kraken recognition system. [2]
1. Define the material first
Before you start creating training data, it's a good idea to determine what the future model should actually be able to read. The model intended, for example, for Czech administrative registers from the 18th century solves a different problem than the model for personal correspondence from the beginning of the 20th century.
For the first experiment, it is advantageous to choose a relatively homogeneous set:
- documents from around the same period,
- same language,
- similar handwriting,
- preferably one scribe or a small number of scribes,
- qualitatively similar images.
The more diverse material we present to the model, the more training data we usually need. Therefore, Transkribus recommends that the training set represent different types of handwriting, languages, and other features that the model should later recognize. [3]
2. Select representative pages
It is not advisable to simply take the first few dozen pages of the book. It is better to choose pages that represent the material that we will work on later.
For example, if an archival volume contains common entries, tables, headings, marginal notes, and several different scribes, we should decide in advance which of these elements the model should handle.
It is not necessary to prepare hundreds of pages for the first model. The current Transkribus documentation lists approximately 5,000 to 15,000 words depending on the nature of the material, which corresponds to roughly 25-75 pages. [4] In the case of manuscripts, the amount of necessary data depends mainly on their variability. The more recent instructions of Transkribus indicate as an indicative minimum around 10,000 words per scribal hand ("hand"). [1]
These numbers should be understood as indicative, not as a fixed limit.
3. Try an existing model first
Before creating your own model, it is advisable to check if there is already a model suitable for the given material. Transkribus offers public models created by both its developers and the user community. They can be searched, for example, by language, period or script type. [5]
The existing model may not provide a perfect transcription. However, if it creates a usable basis, it can significantly speed up the preparation of the Ground Truth: instead of manually transcribing the entire page, we only correct the automatically generated transcription.
4. Analyze the page layout
The HTR system needs to know where each line of text is on the page. Therefore, text recognition is preceded by segmentation or layout analysis.
For a regular page, the system usually automatically creates text areas and so-called baselines β reference lines along individual text lines. These need to be checked.
Misplaced, merged, or missing lines can negatively affect training data. Transkribus explicitly points out that the correctness of baselines is important when creating a quality model. [6]
5. Make an accurate transcription
Now comes the most laborious part of the whole process.
Each line must have the correct text transcription. We can create it manually or first use an existing HTR model and then correct its result.
More important than speed is accuracy and consistency. The model does not recognize that an error in Ground Truth is a human typo. A faulty transcription is simply training information for it. [1]
Before starting a more extensive transcription, it is therefore advisable to create simple transcription rules. For example, it is necessary to decide:
- whether we keep the original spelling,
- how we write capital and small letters,
- how we handle abbreviations,
- whether we expand abbreviated forms,
- how we write punctuation,
- how we handle crossed out or added text,
- whether we preserve historical characters.
A consistent way of transcribing tends to be more important for training a model than text modernization.
6. Mark verified pages as Ground Truth
We consider a page as Ground Truth only after we have checked both its transcription and its segmentation.
In Transkribus, pages checked in this way can be assigned Ground Truth status. We will subsequently create a training and validation set from this verified data. [1]
It's helpful not to consider automatic transcription to be Ground Truth just because it "looks pretty good". Even a relatively small amount of systematic errors can teach the model an undesirable transcription pattern.
7. Divide the data into training and validation
A model should not be evaluated only on the pages it learned from. We will therefore set aside part of the Ground Truth as a validation set.
Transkribus recommends approximately 90% data for training and 10% for validation. [1] [3]
So if we have, for example, 50 high-quality transcribed pages, we can roughly use:
45 pages β training data
5 pages β validation data
Validation pages should be representative. For example, if we have several scribes in our documents, we should not build the validation set from only the most legible of them. [3]
8. Consider using the base model
We don't always have to train the model from scratch.
If there is a model built on a similar language, period and type of manuscript, it can be used as a base model and adapted to your own data. This procedure, generally referred to as fine-tuning, can reduce the amount of data needed and improve results. [6]
A similar principle is also offered by eScriptorium/Kraken, where the existing model can be further trained on own data. [2]
For a beginner, it is therefore reasonable to first try a suitable base model and only if necessary experiment with training from the very beginning.
9. Start training
After data preparation, you can create a new Text Recognition Model in Transkribus, select training and validation data and, optionally, a base model, and start training. [3]
The calculation itself is already running automatically. So a beginner does not need to set up a neural network or program his own algorithm.
The result is a model specialized for the handwriting represented in Ground Truth.
10. Check the CER
One of the basic indicators of the quality of the HTR model is the Character Error Rate (CER), i.e. the error rate at the character level.
In simple terms, CER expresses how many character operations are needed for the automatic transcription to match the correct reference transcription. It therefore applies:
lower CER = more accurate recognition.
CER 10%, for example, does not simply mean "90% correct words". It is a metric based on the differences between the characters of the reference and recognized text.
Transkribus lists CER below 10% as a generally applicable result for search and further analysis; values ββof around 5% or less can be achieved for a well-modelled, coherent body of handwriting. [1] [6]
However, the number itself is not the only criterion. For archival work it is important to also look at the nature of errors. For example, the model can reliably read ordinary text, but systematically make mistakes precisely in personal and place names, numbers or abbreviations, which are essential for historical research.
11. Try the model on previously unused pages
After training the model, select a few pages that were not part of the training or validation set and perform their automatic recognition.
This gives you a more practical answer to the question:
"How well will this model perform on other documents?"
A test on previously unseen material is also recommended by the Transkribus guide. [1]
If the model performs significantly worse on the new pages than on the validation data, the problem may be, for example, an unrepresentative selection of Ground Truth or too much diversity of manuscripts.
12. Correct errors and create more Ground Truth
The first model may not be final.
The practical procedure may look like this:
Ground Truth β first model β automatic transcription of other pages β manual correction β new Ground Truth β new training β better model
The iterative expansion of high-quality training data is one of the basic ways of improving your own HTR model. [7]
At the same time, it is important to point out a common mistake: correcting the automatically recognized text alone does not automatically retrain the model. The corrected pages must be included in the new training data and another training run must be started. [8]
Simple first experiment
A beginner does not need to digitize the entire archival collection right away. It is better to start with a small controlled experiment.
Let's take, for example, one manuscript book with mostly one scribe. We will prepare approximately 25-50 representative pages from it. We perform segmentation, correct baselines and create an accurate transcription.
Then we use most of the data for training and about a tenth for validation. If there is a suitable public model for a similar language, period and handwriting, we will use it as a base.
After training, we will try our own model on a few other pages that have not been used yet.
We don't just fix errors mechanically. We trace their cause:
Does the model make errors across all characters?
We probably need more or better Ground Truth data.
Does it perform particularly poorly on one scribe?
This handwriting may be underrepresented in the training data.
Does it make errors at line boundaries or merge text?
The problem may be related to segmentation and baselines.
Does it systematically misrecognize a particular character?
It is advisable to check the consistency of its transcription in Ground Truth.
Is CER good but the model fails on new documents?
The training and validation data are probably not sufficiently representative of the real material.
Only after this evaluation does it make sense to add more corrected pages and train the model again.
Such a procedure has one significant advantage: the Ground Truth created in the first experiment is not wasted work. It is a long-term usable database that can be gradually expanded and used when creating further versions of the model.
Resources used
[1] Transkribus: How to train a model in Transkribus, 22/10/2025.
https://www.transkribus.org/blog/how-to-train-a-text-recognition-model-in-transkribus
[2] eScriptorium Documentation: Train models.
https://escriptorium-tutorial.readthedocs.io/en/latest/train/
[3] Transkribus Help Center: Model Setup and Training.
https://help.transkribus.org/model-setup-and-training
[4] Transkribus Help Center: How do I train a model?
https://help.transkribus.org/how-do-i-train-a-model
[5] Transkribus Help Center: Public Models.
https://help.transkribus.org/public-models
[6] READ-COOP: How to improve the CER of your model, 16/04/2024.
https://blog.transkribus.org/en/how-to-improve-the-cer-of-your-model
[7] READ-COOP: How to retrain a model in Transkribus, 8/11/2024.
https://blog.transkribus.org/en/how-to-retrain-a-model-in-transkribus
[8] Transkribus Help Center: Training Text Recognition Models.
https://help.transkribus.org/training-text-recognition-models