Handwriting to Text Conversion: A Robust

Approach Using KNIME and AWS

Rekognition
Shreeya Kosireddy
Department of Electronics and Communication Engineering,
National Institute of Technology, Warangal,
Warangal, Telangana, India
Email: sk23ecb0f24@student.nitw.ac.in

Ravi Kishore Kodali

Department of Electronics and Communication Engineering,
National Institute of Technology, Warangal,
Warangal, Telangana, India
Email: kishore@nitw.ac.in

Abstract—The digitization of handwritten docu- I. I NTRODUCTION

ments plays a crucial role in modern document man-
Handwritten documents are integral to many
agement, from educational assessments to archival
preservation. This paper presents a solution for fields, including education, administration, and his-
handwriting-to-text conversion, aimed at automating torical record-keeping. Despite the growing adop-
time-consuming processes such as examination paper tion of digital tools, many processes continue to
grading and document analysis. The system, imple- rely on handwritten records, such as examination
mented on the KNIME analytics platform, integrates
grading, administrative file management, and pre-
with AWS Rekognition, a cloud-based Optical Char-
acter Recognition (OCR) service, to extract text from serving historical documents. These processes are
images of handwritten documents stored in an S3 often manual and require significant time and ef-
bucket. A custom Python script, utilizing the boto3 fort, leading to errors and inefficiencies, particularly
library, automates interaction with AWS Rekognition, when dealing with large volumes of data.
ensuring efficient text extraction by filtering out low- Existing OCR technologies primarily excel at rec-
confidence outputs. KNIME handles preprocessing
tasks such as path manipulation and data integration, ognizing printed text, but they face significant chal-
while join nodes combine the extracted text with rele- lenges when applied to handwritten content. Factors
vant metadata, producing a comprehensive, structured like varying handwriting styles, image quality, and
output. inconsistent spacing between characters often result
The proposed system reduces manual effort, min- in incomplete or inaccurate text extraction. While
imizes errors, and saves time, demonstrating strong
accuracy in processing diverse handwriting styles. recent advancements in OCR technology have im-
Key applications include digitizing archival records, proved accuracy, there is still a gap in handling com-
automating educational assessments, and improving plex handwritten documents efficiently and reliably.
administrative document handling. Future enhance- This paper proposes a scalable system for
ments will focus on improving OCR accuracy for com- handwriting-to-text conversion that integrates AWS
plex handwriting, expanding multilingual support, and
integrating advanced analytics. By leveraging cloud- Rekognition, a cloud-based OCR tool, with the KN-
based technologies, this scalable solution highlights the IME analytics platform. AWS Rekognition extracts
potential of modern tools in transforming document text from handwritten document images stored in an
analysis workflows. S3 bucket, while KNIME processes and organizes
the data into a structured format. A Python script is strengths in feature extraction and classification.
used to filter out low-confidence OCR results, ensur- These models, while accurate, demand extensive
ing high-quality outputs. This combination of tools training and fine-tuning for handwriting-specific
addresses the challenges of handwriting digitization tasks [3], [4]. Additionally, a literature review on
and provides a reliable, automated solution. handwriting recognition systems highlighted the ef-
The proposed system is designed to reduce man- fectiveness of combining CNNs with OCR frame-
ual effort and improve efficiency in document pro- works for classification and feature extraction [3].
cessing. Its applications span across various do-
C. Cloud-Based and Hybrid Systems
mains, such as digitizing archives, automating grad-
ing systems, and streamlining administrative work- Cloud-based OCR solutions, such as AWS
flows. The modular nature of the workflow allows Rekognition and EasyOCR, are increasingly popular
for easy adaptation and future enhancements, such due to their scalability and ease of integration. A no-
as support for multiple languages and advanced data table example demonstrated converting handwritten
analytics. trip sheets to CSV using EasyOCR combined with
By leveraging the strengths of cloud-based OCR regular expressions [5]. These tools offer practical
and data analytics platforms, this solution demon- alternatives to custom-built systems by leveraging
strates the potential to transform handwritten doc- pre-trained models for rapid deployment and mini-
ument management. It not only addresses current mal maintenance.
limitations but also opens new possibilities for au- D. Applications of Handwriting Recognition
tomating and improving document workflows in
Applications of handwriting recognition include
diverse fields.
digitizing archival records, real-time grading sys-
II. R ELATED W ORK tems, and improving accessibility for visually im-
paired users. For instance, a system combining
The digitization of handwritten text has been
text-to-speech (TTS) capabilities with handwriting
extensively studied, with diverse methodologies re-
recognition was developed to provide audio outputs
flecting the evolution of technology in handwriting
for handwritten content [4]. Additionally, handwrit-
recognition systems. The following research efforts
ing detection frameworks have been employed for
provide critical insights into the current state of the
diverse languages using hybrid CNN-RNN models,
field, laying the foundation for the design and im-
showcasing versatility [1], [2].
plementation of the system described in this paper.
E. Key Differentiators of the Proposed System
A. Deep Learning and Handwriting Recognition
While many studies focus on resource-intensive
Several studies have demonstrated the effective- deep learning models, the proposed system adopts a
ness of combining Convolutional Neural Networks practical approach using AWS Rekognition for OCR
(CNNs), Long Short-Term Memory (LSTM) net- and KNIME for data preprocessing. This hybrid
works, and Connectionist Temporal Classification solution minimizes the need for extensive training
(CTC) layers in handwriting recognition tasks. Sys- and model development, ensuring adaptability to di-
tems trained on various databases, such as IAM verse handwriting styles [5]. By integrating Python-
Handwriting, have shown high accuracy due to ad- based automation, the system addresses challenges
vanced preprocessing techniques like segmentation. such as low-confidence outputs and ensures efficient
For example, a CNN-based deep learning model us- processing.
ing LSTM and CTC loss was proposed for handwrit-
ten text recognition, achieving reliable performance F. Insights and Opportunities for Enhancement
across diverse datasets [1], [2]. The review of related work highlights areas for
future development, including multilingual support
B. Architectural Comparisons in Neural Networks and real-time analytics. Hybrid approaches combin-
Comparative studies on architectures such as ing cloud-based tools with localized models may
AlexNet, ResNet-50, and VGG-16 emphasize their balance accuracy and scalability [4], [5].
 R EFERENCES accessing the dataset, performing basic preprocess-
ing, and removing unsuitable files such as corrupted
[1] J. Jebadurai, I. J. Jebadurai, G. J. L. Paulraj, and S. V.
Vangeepuram, “Handwritten text recognition and conversion or empty ones.
using convolutional neural network (cnn) based deep learning Metadata is crucial for tracking and indexing the
model,” in 2021 Third International Conference on Inventive dataset. Each image is assigned a unique identifier
Research in Computing Applications (ICIRCA), 2021, pp.
1037–1042. along with metadata, including upload date, source,
[2] P. Mishra, P. Pai, M. Patel, and R. Sonkusare, “Extraction and document type. This structure improves data
of information from handwriting using optical character management and traceability, making downstream
recognition and neural networks,” in 2020 4th International
Conference on Electronics, Communication and Aerospace processing more efficient. KNIME nodes are used
Technology (ICECA), 2020, pp. 1328–1333. to automate this preprocessing, ensuring that only
[3] J. Memon, M. Sami, R. A. Khan, and M. Uddin, “Hand- clean, structured data is passed to the OCR stage,
written optical character recognition (ocr): A comprehensive
systematic literature review (slr),” IEEE Access, vol. 8, pp.
reducing the potential for errors.
142 642–142 668, 2020.
[4] V. R. Vaddadi, C. Bharathi, A. K. Rout, and A. K. Tirunagari,
C. Optical Character Recognition (OCR)
“A handwriting recognition system that outputs editable text The core of the workflow is OCR, where hand-
and audio,” in 2024 International Conference on Advances
in Modern Age Technologies for Health and Engineering
written images are analyzed, and text is extracted
Science (AMATHE), 2024, pp. 1–7. using AWS Rekognition. Each detected word or
[5] S. N, K. S, V. C, D. N, and S. G, “Handwritten trip line is assigned a confidence score. A Python script
sheet to digital conversion using ocr,” in 2023 International in KNIME filters out text with a confidence score
Conference on Advances in Electronics, Communication,
Computing and Intelligent Information Systems (ICAECIS), below 50
2023, pp. 7–10. This threshold strikes a balance between retaining
useful data and avoiding low-quality results. AWS
III. M ETHODOLOGY Rekognition’s parallel processing capabilities make
it efficient for large datasets, reducing processing
A. Introduction to Workflow
time. The extracted text is then formatted into a
This project outlines a process to transform hand- structured output for the next stage of the workflow.
written text into structured digital text by combining
D. Text Vectorization
AWS Rekognition, a cloud-based Optical Character
Recognition (OCR) tool, with KNIME, a platform After text extraction, the next step is text vec-
for data preprocessing, analysis, and visualization. torization, where the text is transformed into a
The modular workflow allows customization at each numerical format suitable for analysis or machine
step to meet specific requirements. learning models. Techniques like one-hot encoding
As shown in Figure 1, the process consists of or word embeddings are used, depending on the
several stages: dataset loading, OCR processing, fil- application.
tering outputs by confidence levels, text conversion Vectorization prepares the data for advanced tasks
into numerical form, and result generation. Each such as clustering, summarization, or sentiment
stage ensures the system’s accuracy and reliability. analysis. By converting text into numerical vectors,
The modularity of the workflow allows for easy the system leverages mathematical models to gain
substitution of components. For instance, AWS insights and make predictions.
Rekognition can be replaced with a more advanced E. High Confidence Table Visualization
OCR tool, or additional preprocessing steps can be
To ensure OCR result quality, KNIME’s Table
incorporated based on the dataset. This flexibility
View node displays detected text alongside its confi-
makes the system adaptable and scalable for various
dence score, categorizing it into ”High Confidence”
applications.
(50
The table also serves as a diagnostic tool dur-
B. Accessing the Dataset and Preprocessing
ing testing and development, providing immediate
The input consists of images of handwritten text feedback on how the system handles different input
stored in an Amazon S3 bucket. This step involves types and facilitating iterative improvements.
 Fig. 1. Complete Workflow in KNIME

F. Final Results data processing environment, the system can han-

The final output pairs the original handwritten im- dle large volumes of handwritten data, making it
ages with the extracted text, showcasing the system’s suitable for various applications, such as digitizing
ability to accurately digitize handwritten documents. historical documents, automating exam paper cor-
The modular design allows the system to handle rections, and analyzing handwritten records.
a range of use cases, including digitizing archives,
The system’s flexibility allows easy adjustments
processing exam papers, or analyzing handwritten
to components like OCR tools and preprocessing
records.
methods, making it adaptable to diverse use cases.
The results can be further enhanced by incorpo-
The final results demonstrate the system’s ability to
rating Natural Language Processing (NLP) tools for
convert handwritten text into digital format, ready
tasks like summarization or grammatical correction.
for further analysis or integration into other appli-
Machine learning can also be added to provide
cations. Confidence-based filtering further enhances
additional insights, such as sentiment analysis or
OCR accuracy by ensuring only high-confidence
topic modeling, directly from the handwritten data.
results are considered for analysis.

Future work could expand the system by inte-

IV. C ONCLUSION
grating advanced NLP techniques for tasks like au-
This project presents an efficient and scalable so- tomatic summarization, grammatical correction, or
lution for converting handwritten text into structured sentiment analysis. Additionally, machine learning
digital format using AWS Rekognition and KNIME. models could be incorporated to further improve
The system’s modular workflow, which includes OCR accuracy, particularly for challenging hand-
preprocessing, OCR extraction, text vectorization, writing styles. Overall, this project lays a robust
and result generation, ensures accurate and efficient foundation for building intelligent systems capable
data processing. By leveraging AWS Rekognition’s of processing handwritten text for a wide range of
powerful OCR capabilities and KNIME’s robust practical applications.
 Fig. 2. Basic workflow of an OCR engine

V. ACKNOWLEDGMENT [4] P. Mishra, P. Pai, M. Patel, and R. Sonkusare, ”Extraction

of Information from Handwriting using Optical Character
I would like to thank the Department of Electron- recognition and Neural Networks,” 2020 4th International
ics and Communication Engineering, NIT Warangal, Conference on Electronics, Communication and Aerospace
Technology (ICECA), Coimbatore, India, 2020, pp. 1328-
for providing resources and support for this project. 1333, doi: 10.1109/ICECA49313.2020.9297418.
[5] S. N, K. S, V. C, D. N, and S. G, ”Handwritten trip
R EFERENCES sheet to digital conversion using OCR,” 2023 International
Conference on Advances in Electronics, Communication,
[1] N. Teslya and S. Mohammed, ”Deep Learning for Hand- Computing and Intelligent Information Systems (ICAE-
writing Text Recognition: Existing Approaches and Chal- CIS), Bangalore, India, 2023, pp. 7-10, doi: 10.1109/ICAE-
lenges,” 2022 31st Conference of Open Innovations Associ- CIS58353.2023.10170030.
ation (FRUCT), Helsinki, Finland, 2022, pp. 339-346, doi: [6] V. R. Vaddadi, C. Bharathi, A. K. Rout, and A. K. Tiruna-
10.23919/FRUCT54823.2022.9770912. gari, ”A Handwriting Recognition System That Outputs
[2] B. Vidhale, G. Khekare, C. Dhule, P. Chandankhede, A. Editable Text And Audio,” 2024 International Conference
Titarmare, and M. Tayade, ”Multilingual Text Handwritten on Advances in Modern Age Technologies for Health and
Digit Recognition and Conversion of Regional languages Engineering Science (AMATHE), Shivamogga, India, 2024,
into Universal Language Using Neural Networks,” 2021 pp. 1-7, doi: 10.1109/AMATHE61652.2024.10582076.
6th International Conference for Convergence in Tech- [7] J. Memon, M. Sami, R. A. Khan, and M. Uddin, ”Hand-
nology (I2CT), Maharashtra, India, 2021, pp. 1-5, doi: written Optical Character Recognition (OCR): A Com-
10.1109/I2CT51068.2021.9418106. prehensive Systematic Literature Review (SLR),” in IEEE
[3] B. Debnath et al., ”Automatic Handwritten words on Touch- Access, vol. 8, pp. 142642-142668, 2020, doi: 10.1109/AC-
screen to Text file converter,” TENCON 2018 - 2018 IEEE CESS.2020.3012542.
Region 10 Conference, Jeju, Korea (South), 2018, pp. 0219- [8] K. Vijay, A. Mukundh, S. Pooja, T. N. Aravind,
0223, doi: 10.1109/TENCON.2018.8650269. and R. Manoj Adhithya, ”Hand Written Text Recogni-
 Fig. 3. High confidence table

tion and Global Translation,” 2023 International Confer-

ence on Computer Communication and Informatics (IC-
CCI), Coimbatore, India, 2023, pp. 1-5, doi: 10.1109/IC-
CCI56745.2023.10128327.
[9] J. Jebadurai, I. J. Jebadurai, G. J. L. Paulraj, and S. V.
Vangeepuram, ”Handwritten Text Recognition and Conver-
sion Using Convolutional Neural Network (CNN) Based
Deep Learning Model,” 2021 Third International Confer-
ence on Inventive Research in Computing Applications
(ICIRCA), Coimbatore, India, 2021, pp. 1037-1042, doi:
10.1109/ICIRCA51532.2021.9544513.
[10] Z. Zhou, F. Shao, and H. Xu, ”A Survey on Hand-
written Text Recognition Techniques Using Deep Learn-
ing,” 2018 International Symposium on Signal Processing
Systems (SSPS), Beijing, China, 2018, pp. 154-159, doi:
10.1109/SSPS.2018.8476140.
[11] K. Patil, A. Shinde, and S. Khan, ”Handwritten Script
Recognition Using Deep Neural Networks,” 2019 In-
ternational Conference on Recent Trends in Electron-
ics, Information Communication Technology (RTEICT),
Bangalore, India, 2019, pp. 165-170, doi: 10.1109/RTE-
ICT.2019.8749985.
Fig. 4. Output handwritten images and detected text