{'fraction_non_alphanumeric': 0.019628567114600634, 'fraction_numerical': 0.004831647289747849, 'mean_word_length': 5.190186915887851, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
OCR Challenges for a Latvian Pronunciation Dictionary

This paper covers the devlopment of a custom OCR solution based on the Tesseract open source engine developed for digitization of a Latvian pronunciation dictionary where the pronunciation data is described using a large variety of diacritic markings not supported by standard OCR solutions. We describe our efforts in training a model for these symbols without the additional support of preexisting dictionaries and illustrate how word error rate (WER) and character error rate (CER) are affected by changes in the dataset content and size. We also provide an error analysis and postulate possible causes for common pitfalls. The resulting model achieved a CER of 2.07%, making it suitable for digitization of the whole dictionary in combination with heuristic post-processing and proofreading, resulting in a useful resource for further development of speech technology for Latvian.


Problem Description
Accurate speech recognition and speech synthesis are greatly helped by a large database of words and their phonetic notations. For Latvian the most comprehensive resource of phonetic information currently available is "Latviešu valodas pareizrakstības un pareizrunas vārdnīca" (LVPPV) [1], "Latvian spelling and pronunciation dictionary", which contains over 80000 words with full pronunciation transcription. Unfortunately, a machine-readable version of this dictionary is not available because of historical reasons of how this dictionary was originally developed.
During earlier digitization efforts, the LVPPV was scanned. As seen in Figure 1, each dictionary entry contains a pronunciation section enclosed in square brackets. However, in the earlier digitization only the spelling part of the entries was suitable for OCR technologies available at that time, as the pronunciation information is encoded in a custom set of symbols not expected by the available OCR models.
The availability of this scanned data and the requirement for a large machinereadable resource of Latvian pronunciation motivates our research to develop an accurate OCR model for this custom phonetic alphabet to assist a full digitization of this dictionary. Although OCR results are rarely perfect and would be expected to contain mistakes that need human review, an effective custom OCR model would ensure that this data can be reviewed and corrected with a reasonable amount of manual work.
In LVPPV the pronunciation is denoted by using symbols that extend the standard Latvian alphabet ( Figure 2) with additional symbols with diacritic modifiers illustrated in Figure 3. Accurate recognition of this phonetic alphabet is challenging because meaningful differences in pronunciation are expressed by minor variations of diacritic marks -for example, there is a common need to distinguish the pronunciation of the letter 'a' as 'ã' or 'â'. Standard OCR systems distinguish visually similar characters through assistance of dictionaries and statistical n-gram frequency models, which is not possible in this scenario as we need to digitize the only dictionary which has this data.

Related Work
There has been no known work in digitizing this style of printed phonetic transcription of Latvian. There is phonetic transcription available for a limited number of words in digital resources "Mūsdienu latviešu valodas vārdnīca" Dictionary of contemporary Latvian (MLVV) and tezaurs.lv online dictionary [7]. These resources are much smaller than LVPPV, but they can be used as a test set for verifying the OCR accuracy on the subset of words contained in both resources.
There is earlier work on rule-based approaches to derive phonetic transcriptions of Latvian [5]. However, these methods are limited and would also directly benefit from a larger digital database of phonetic transcriptions such as the final result of digitizing LVPPV.
In reviewing research on OCR technology applications, we were not able to identify any useful publications on digitizing phonetic transcription of other languages. However, there is extensive literature on the more general task of developing OCR solutions for new scripts. The prevailing paradigm for such OCR systems relies on retraining existing general-purpose OCR systems on a targeted set of training data examples for the new script using supervised machine learning and deep neural networks. While there are also examples using custom systems implemented from scratch, especially for commercial solutions, we consider that it is reasonable to adapt an existing system in order to reuse existing functionality of recognizing the latin alphabet and only adjust the specific characters (diacritic combinations) that are used in this pronunciation dictionary.
The two leading OCR tools that support training additional languages are Abby Finereader 1 and Tesseract [6]. For this research we have chosen Tesseract as it is a free open-source solution and has been the basis for multiple successful implementations of OCR for a new script [2,4].

Method
Since version 4 Tesseract uses a LSTM [3] based neural network architecture which significantly outperforms previous versions. Tesseract 4 provides three training methods[8]: 1. fine-tuning for impact (adding a few extra characters to an existing model); 2. training just a few layers (removing the top layers from an existing model and replacing with new ones); 3. training from scratch. These approaches differ substantially by how much training data is required. As the pronunciation data introduces a significant number of new characters and we want to limit training data size, the second training method is considered the most appropriate. LVPPV largely contains characters and letter patterns characteristic to the Latvian language, thus Tesseract's pre-trained Latvian language model was chosen as the base model.

Data Preparation
The Tesseract training process requires training data that consists of scanned images annotated with character bounding boxes aligned with appropriate characters. In this work, whole pages of the scanned LVPPV dictionary were used as the basic units of data. The input data was selected, so that the chosen images contain all the new phonetic characters frequently enough to minimize errors in the final model.
OCR requires good quality images and LVPPV was scanned in 600 dpi resolution. Additionally, as the goal of this research is to recognize text from this particular dictionary, we know that all data will be a single font, thus we did need to adapt it for font variation as is commonly required for general purpose OCR solutions.

Post-processing Heuristics
We developed a method to determine a particular character's error rate. It uses Levenshtein distance, also known as edit distance, that measures the difference between two sequences, in this case, the expected character string and the OCR generated character string.
This allowed to both strategically choose pages from LVPPV for training data and detect instances where rule-based post-processing could be applied using known phonetic transcription characteristics and gained insights.
Analysis of OCR errors during development revealed that a common error pattern involves character duplicates -for example, 'ã' which was mistaken for 'ãâ'. Some examination revealed that this is an unresolved issue in Tesseract implementation wherein if two characters have a similar probability of being the correct then both are output 2 . Some proposed solutions suggested looking at the character bounding boxes but because of the way Tesseract is implemented this still would not be a complete solution. However, as the model improves, the frequency of these errors should decrease. A large part of such errors can be automatically corrected with heuristic post-processing methods because these mistakes generate character sequences that are not plausible in Latvian words.

Training
We performed experiments to analyze the effect of training set size on OCR accuracy to determine when the effort put into preparing training examples exceeds the effort needed to check for errors in the processed pages manually.
Multiple models were trained with various limitations on training data size. Data was increased gradually, with one LVPPV page as the step size. The final model was trained on a dataset of 2949 lines (dictionary entries with a word spelling, its pronunciation, and auxiliary comments); Table 1 illustrates how the LVPPV page count corresponds to the line count in the training data. The dataset was split into three parts; separate training and test sets were used during development to train the models and choose when to stop the training  It is important to note that we must evaluate not only the OCR accuracy of the pronunciation OCR. LVPPV entries contain both regular Latvian words and phonetic transcriptions and both categories need to have a high accuracy -it is not sufficient to obtain a correct pronunciation if it can not be automatically mapped to the proper dictionary word because of an OCR mistake in the spelling part.

Result Assessments
It was expected that (a) larger training size would correspond to a smaller error rate, and (b) increased character frequency would decrease the character's error rate.
Initially we trained some models for exploratory research to eliminate and fix any persistent errors in character set and input data. For the experimental data included in this paper, 16 models were trained with different amounts of training data, from 10 pages (1175 lines) to with 25 pages (2949 lines).
The analysis uses character error rate CER and word error rate WER to measure the quality of the trained models. Figures 5, 7 show that both rates fell with the increase in the training set. However, the trend is not consistent and has some significant outliers. Overall, error rate patterns in training and verification sets are similar, although, as expected, the training set has higher accuracy.

Character Level Analysis
On average, the total error rates fell which support expectation (a), however, character level errors exhibit a different pattern. An important hypothesis was the question of whether a particular character's error rate would decrease with the increase of its frequency in the training set. Table 2 shows the character frequencies in the training data in the various experiments as more pages of training data were added. Additionally, by examining the verification set character errors (insertions, deletions, and substitutions) in the table 3 a few things can be noted:   (a) The general trend is towards fewer errors as training data increases, particularly for characters with a smaller diversity of diacritic marks like "r" (b) Larger frequency does not consistently correspond to fewer errors. For instance, the frequency of letter l with tilde almost doubled when page 22 was added, however, its error actually increased and the letter was consistently not being recognized. (c) An inappropriately large frequency can cause 'overconfidence'. This can be seen in the last models for the letter "ì" where it went from being missed to being overused in place of "i" As exemplified, the data illustrates an instability of the models towards different specific errors, as training a new model on a different, larger set of training data may result in a very different pattern of errors, possibly because of random initialization effects on the neural network model.
In this particular application, a significant subset of errors could be easily corrected in heuristic post-processing as some OCR output character combina-tions are not used in Latvian pronunciation. It might also be plausible to use a small statistical language model for this disambiguation, but due to technical difficulties (limited quantity of available data, and this would need to be a subword model for short character n-grams inside a single word) of integrating this modification, it was not attempted at this time.

Conclusion
Our research demonstrates that freely available OCR models can be adapted for a new, specific set of diacritic markings with reasonable accuracy (2.07% character error rate) even with no availability of preexisting dictionaries or language models and only a small quantity of training data, 25 pages in our experiments. However, our error analysis indicates that the training of Tesseract OCR models is not clear cut: more data might not necessarily mean better accuracy for any specific erroneous characters. Care should be given to the frequencies of characters in the training data to avoid detrimental effects, and a specific character's accuracy can fluctuate widely.
Whether these effects can be diminished by using other training methods could be explored by testing different OCR engines and their versions or experimenting with character frequencies in the training data.
The resulting OCR models will be applied to the full LVPPV data and, as the manual post-processing and proofreading work is finalized, result in up to 80000 phonetic transcriptions added to the publicly available lexical resources for Latvian. This data will facilitate the development of accurate speech synthesis and speech-to-text solutions for Latvian.