# Towards Robust Handwritten Text Recognition with On-the-fly User Participation

**Authors:** Ajoy Mondal, Rohit saluja, C. V. Jawahar  
**arXiv:** [2212.08834](https://arxiv.org/abs/2212.08834)  
**Date:** 2022-12-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.08834v1/

## Key points

- Handwritten text recognition is challenging for Indic scripts with over 100 unique basic Unicode characters.
- Existing OCR systems perform well when the test set's distribution is similar to the data used to train them, but performance drastically reduces in real-world scenarios.
- A controlled framework for long-term OCR services has been proposed to consistently upgrade an existing Handwritten Hindi OCR model three times on the dataset of 15 users.
- The proposed strategy involves fixing the budget of four users for each iteration and using Curriculum Learning (CL) on data available in current and previous iterations.
- Insights into investigations on CL, user selection, and data from unseen writing styles have been provided along with testing of upgraded models on held-out sets of one page each from all 23 users.
- This work can be used for long-term OCR services in crowd-sourcing scenarios for service providers and end-users alike.
- Most Indic languages derived from Indic script are at risk of vanishing due to lack of efforts to preserve them.
- Several methods offer solutions to Indic handwritten text recognition tasks categorized into segmentation free but lexicon dependent methods; segmentation dependent methods; and sequence-to-sequence i.e., cnn-rnn methods.

## Summary

Handwritten text recognition is a challenging task, especially for Indic scripts that have over 100 unique basic Unicode characters. The existing OCR systems perform well when the test set's distribution is similar to the data used to train them. However, in real-world scenarios, the performance of these systems drastically reduces when the test set's distribution deviates from the training set. This often leads to dissatisfied user experience. To address this issue, a controlled framework for long-term OCR services has been proposed that aims to provide high-quality output at competitive costs by consistently upgrading an existing Handwritten Hindi OCR model three times on the dataset of 15 users. The proposed strategy involves fixing the budget of four users for each iteration. For the first iteration, the model directly trains on the dataset from the first four users. For subsequent iterations, all remaining users write a page each which service providers later analyze to select four new best users based on quality predictions on human-readable words. Selected users then write 23 more pages for upgrading the model using Curriculum Learning (CL) on data available in current and previous iterations. Insights into investigations on CL, user selection and data from unseen writing styles have been provided along with testing of upgraded models on held-out sets of one page each from all 23 users. This work can be used for long-term OCR services in crowd-sourcing scenarios for service providers and end-users alike. Furthermore, it has been observed that most Indic languages derived from Indic script are at risk of vanishing due to lack of efforts to preserve them. In many Indic scripts two or more characters often combine to form conjuncts considerably increasing vocabulary tackled by OCR systems. Several methods offer solutions to Indic handwritten text recognition tasks categorized into segmentation free but lexicon dependent methods; segmentation dependent methods; and sequence-to-sequence i.e., cnn-rnn methods.