International Journal of Scientific Research in Engineering and Management (IJSREM)

Volume: 08 Issue: 12 | Dec - 2024 SJIF Rating: 8.448 ISSN: 2582-3930

Virtual Drawing Board Using Hand Tracking

1st SINDHU M P 2nd AKHILESH A 3rd AFSAN M 4thAKSHAY K SUNIL
assoc professor,cse dept 4th year CSE student 4th year CSE student 4th year CSE student
SNGCE ,KOLENCHERY SNGCE ,KOLENCHERY SNGCE ,,KOLENCHERY SNGCE ,,KOLENCHERY
KERELA,INDIA KERELA,INDIA KERELA,INDIA KERELA,INDIA

Abstract—This project presents the development of a virtual It have almost 3,000 functions and 14 million downloads. I can
drawing board that leverages hand tracking technology for an explain this in a easy language or in a proper way ,it is a library
intuitive and immersive drawing experience. The system uses which is used for processing the image properly. It is used to
computer vision algorithms to detect and track hand movements
in real time, allowing users to draw in a virtual environment do all the types of operations which is exactly to the occupied
without the need for physical input devices like a mouse or stylus. images . To create a digital canvas for drawing on. To recog-
The key objectives of the project include achieving high accuracy nize the color marker left by the human finger. To performing
in hand gesture recognition, minimizing latency, and creating a the phonological operations. To design a user interface that
user-friendly interface. The proposed solution is implemented connects the human hand to the system. OpenCV is a library
using [Technology/Tools], and experimental results show that
the system is effective in accurately capturing hand movements for images and drawing a figure.. It approximately supports
for drawing. This project contributes to the field of human- all the important programming languages. Mainly opencv is
computer interaction by providing a novel method for digital used in python and C++. OpenCv is used to read or write an
content creation, with potential applications in art, design, and image in a manner or in a general way and for modification of
education. images. It changes color to gray binary and Hue , Saturation
Index Terms—component, formatting, style, styling, insert
and value etc. It is also open source. MediaPipe-enables the
development of cross-platform pipelines for the development
I. INTRODUCTION
of unique machine learning techniques for streaming and live
Around 2000 BC, the practice of writing was first discov- media. The graph-based an open source programme for media
ered. The cave wall was first written on by Neolithic people. processing, which Google also made available for There are
Soon after, stone replaced these walls, then fabric, and i now some of it’s important applications that are follows:
present a written report. In a slow manner transitioning to a . 1. Palm detection
form that is primarily digital, based on ability and adapt the 2. Multi-hand tracking
technology to exploring it in the serval ways and organizing 3. Hand landmarks identification
the electronic text papers created with modern technology. 4. Detecting an object
using the QWERTY keyboard, These digital text documents 5. Tracking an object
are editable and programmable. electronic text papers created 6. Auto flip : Auto flip is a pipeline which is used to crop
with modern technology. using the QWERTY keyboard, These the videos naturally. The two most fundamental morphological
digital text documents are editable and programmable.. Elec- processes are erosion and dilation
tronic text and keyboards are gradually displacing traditional
writing with pens and paper. With the help of our technology, II. LITERATURE REVIEW
also known as open cv and mediapipe, it is possible to paint 1) Hand Gesture Recognition for Virtual Drawing
in the air. VirtualBoard using OpenCV and Mediapipe is an Authors: Chen et al. (2015)
application which prints the motion of an object. By using the Methodology: This study utilizes a depth camera, specif-
great feature to track , the person can draw on the screen by ically the Kinect, to detect hand gestures and translate
moving the hand by the human in this project in front of the them into drawing strokes within a virtual space. The
webcam or camera . download ,was in the alpha stages at the authors emphasize the importance of gesture recognition
time.. Media pipe aims to make media processing much easier accuracy for enabling intuitive virtual drawing experi-
for our by preparing the machine learning features and some ences. They propose a robust framework that integrates
joined computer vision object and helps the person to make a real-time gesture detection with graphical output, al-
things easy which are both fascinating and testing. OpenCV lowing users to create virtual art through natural hand
Stands for Open Source Computer Vision was launched in movements.
August 1999 at the Computer Vision and Pattern Recognition 2) Vision-Based Hand Tracking for Human Computer
conference which is a programming language which contains Interaction
library of different types of functions only for computer vision. Author: Johnson, M.

© 2024, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM39912 | Page 1

 International Journal of Scientific Research in Engineering and Management (IJSREM)
Volume: 08 Issue: 12 | Dec - 2024 SJIF Rating: 8.448 ISSN: 2582-3930

Methodology: This research employs advanced com- speed. The findings suggest that machine learning can
puter vision techniques to detect and track hand move- significantly enhance the robustness of hand gesture
ments in real time. Johnson develops an algorithm recognition systems.
that processes video frames to identify hand positions 8) Hand Gesture Recognition System for Human-
and movements, thereby facilitating seamless interac- Computer Interaction
tion with computer applications. The study discusses Author: Rekik, I.
challenges related to occlusion and lighting conditions, Methodology: This research utilizes a vision-based ap-
proposing solutions that enhance tracking reliability. proach to achieve real-time hand gesture recognition.
3) Hand Gesture Interaction in Virtual Reality Envi- Rekik combines skin color detection with contour anal-
ronments ysis techniques to identify and classify gestures effec-
Authors: Garcia, S. tively. The study addresses challenges related to varia-
Methodology: This paper integrates hand tracking tech- tions in skin tone and environmental lighting, presenting
nology with VR systems to enhance user interaction. solutions that improve the system’s overall performance
Garcia explores various interaction paradigms, focusing in real-world scenarios.
on how users can manipulate virtual objects using natu- 9) Robust Hand Detection via Convolutional Neural
ral gestures. The study provides experimental results that Networks
demonstrate improved user engagement and satisfaction Author: Juefei-Xu.
when using hand gestures in VR, compared to traditional Methodology: Juefei-Xu explores the application of
input methods. Convolutional Neural Networks (CNNs) for hand de-
4) A Comparative Study of Hand Tracking Technologies tection in real-time environments. The paper details
Authors: Patel, N. the architecture of the CNN model and the training
Methodology: This study analyzes multiple hand track- process, highlighting its effectiveness in accurately de-
ing technologies, including depth sensing, infrared track- tecting hands in diverse conditions. Experimental results
ing, and optical tracking, assessing their effectiveness demonstrate the model’s robustness against occlusions
across various applications. Patel conducts a series of and background clutter.
experiments to compare the accuracy, latency, and us- 10) Real-Time Hand Gesture Recognition for HCI Using
ability of different systems, providing a comprehensive Machine Learning
overview that helps researchers and practitioners choose Author: Rafique, M.A.
appropriate technologies for their specific needs. Methodology: This research applies machine learning al-
5) Real-time Hand Tracking for Gesture Recognition gorithms, particularly Support Vector Machines (SVM)
Author: Smith, J. and K Nearest Neighbors (KNN), for recognizing hand
Methodology: Smith employs deep learning techniques gestures from depth sensor data. Rafique discusses
to classify hand gestures in real time using video data. the advantages of using depth information for gesture
The paper outlines the architecture of a convolutional recognition, leading to improved accuracy compared to
neural network (CNN) designed for gesture recognition traditional RGB image processing methods. The paper
and discusses the dataset used for training. Results provides a detailed analysis of the system’s performance
demonstrate high accuracy rates in recognizing a range in various interaction scenarios.
of gestures, highlighting the potential of deep learning 11) Hand Tracking with Augmented Reality for Virtual
for improving gesture-based interfaces. Drawing
6) Augmented Reality Drawing Applications Authors: Chen, Q., & Li, Y. (2021)
Author: Patel, R. Methodology: This study focuses on the application
Methodology: This study combines AR frameworks with of hand tracking technology within augmented reality
hand tracking technology to create interactive drawing environments for virtual drawing. The authors utilize a
applications. Patel explores how users can draw in combination of optical hand tracking and sensor-based
augmented environments using hand gestures, providing inputs to create an intuitive AR drawing interface. The
a novel approach to digital art creation. The paper dis- paper emphasizes user experience, presenting feedback
cusses user feedback on the intuitiveness and enjoyment from user studies that indicate a high level of satisfaction
of the AR drawing experience, indicating its potential with the AR drawing capabilities.
for creative applications. 12) Hand Gesture-Based Virtual Interface for Interactive
7) Hand Gesture Recognition Using Machine Learning Applications
Author: Chen, L. Author: Al Rashed, S.
Methodology: Chen implements various machine learn- Methodology: Al Rashed explores a gesture-based vir-
ing algorithms, including decision trees and neural tual interface system that utilizes hand tracking for
networks, for gesture detection. The study presents a interaction with digital applications. The study presents a
comparative analysis of different algorithms based on framework for recognizing a variety of gestures, allow-
performance metrics such as accuracy and processing ing users to navigate and control applications without

© 2024, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM39912 | Page 2

 International Journal of Scientific Research in Engineering and Management (IJSREM)
Volume: 08 Issue: 12 | Dec - 2024 SJIF Rating: 8.448 ISSN: 2582-3930

traditional input devices. User evaluations highlight the

system’s effectiveness in enhancing interaction and user
engagement.

PROPOSED METHODOLOGY

The proposed methodology aims to develop an advanced

hand gesture recognition and tracking system to enhance user
interaction in virtual and augmented reality environments.
The system will utilize a depth camera, such as a Kinect
or Intel RealSense, to capture hand movements in a three-
dimensional space. This captured data will be processed using
computer vision techniques and machine learning algorithms
to recognize hand gestures and translate them into commands
for virtual applications. Data acquisition will involve employ-
ing depth sensing to capture 3D images of the user’s hand
movements while simultaneously recording RGB video to Fig. 1. architecture
provide additional contextual information. The preprocessing
phase will include noise reduction through filtering techniques
augmented reality (AR), virtual reality (VR), and various
and segmentation via skin color detection and contour anal-
computer vision applications.
ysis to isolate the hand from the background. For gesture
NUMPY
recognition, key features such as hand shape, position, and
NumPy is a powerful, open-source Python library used for
movement trajectory will be extracted from the segmented
numerical computing. It provides support for large, multi-
data. Several machine learning algorithms, including Support
dimensional arrays and matrices, along with a collection of
Vector Machines (SVM), K-Nearest Neighbors (KNN), and
mathematical functions to operate on these arrays efficiently.
Convolutional Neural Networks (CNN), will be implemented
It’s widely used in scientific computing, data analysis, machine
to classify gestures based on the extracted features, with
learning, and more.
a diverse dataset compiled for training the models. Cross-
TENSOR FLOW
validation techniques will be employed to optimize model
TensorFlow is an open-source machine learning library de-
performance and prevent overfitting. The trained models will
veloped by Google, primarily used for training and deploying
then be integrated into a real-time processing pipeline, allow-
machine learning models, particularly deep learning models.
ing the system to recognize gestures as they are performed. A
It provides a flexible, high-performance platform for building
feedback mechanism will provide users with immediate visual
machine learning applications, ranging from simple linear
or auditory feedback upon successful gesture recognition,
models to large-scale deep neural networks. TensorFlow is
thereby enhancing user engagement. To evaluate the system’s
widely used in academia and industry for tasks like image
effectiveness and usability, a series of user studies will be
recognition, natural language processing, and more,
conducted, where participants will perform various gestures
TKINTER/PYQT
to assess recognition accuracy and user satisfaction. The
Tkinter and PyQt are two popular Python libraries for build-
quantitative and qualitative data collected will be analyzed to
ing Graphical User Interfaces (GUIs). While they both allow
identify the strengths and weaknesses of the proposed system,
developers to create desktop applications with a graphical
with metrics such as recognition accuracy, response time, and
interface, they have different features, use cases, and styles.
user satisfaction being assessed. Overall, the methodology
Mediapipe
aims to achieve a robust hand gesture recognition system
Mediapipe is a powerful framework developed by Google
that can accurately identify a variety of gestures in real
for building multimodal (e.g., video, audio) machine learning
time, enhancing user interaction experiences in virtual and
pipelines. One of its standout features is the Hand Tracking
augmented reality applications and providing comprehensive
solution, which enables real-time detection and tracking of
evaluation results to inform future improvements in gesture
hand landmarks. This capability is invaluable for applications
recognition technologies.
in augmented reality (AR), virtual reality (VR), gesture recog-
ARCHITECTURE nition, interactive installations, and more.

OPEN CV FUTURE SCOPE

OpenCV (Open Source Computer Vision Library) is an If i have more time to devote to this endeavor, i would
open-source library designed for computer vision and machine enhance hand contour recognition, investigate our initial Air
learning. It provides tools to help developers and researchers Canvas objectives, and make an effort to comprehend the
with tasks such as image processing, object detection, and multicore module. I would need to go further into OpenCV
video analysis. OpenCV is widely used in fields like robotics, in order to improve hand gesture tracking. There are other

© 2024, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM39912 | Page 3

 International Journal of Scientific Research in Engineering and Management (IJSREM)
Volume: 08 Issue: 12 | Dec - 2024 SJIF Rating: 8.448 ISSN: 2582-3930

ways to analyse contours, but for this particular procedure, of Innovative Technology and Exploring Engineering (IJITEE)
it would be beneficial to look at the color histogram that ISSN: 2278-3075 (Online), Volume-10 Issue-8, June 2021.
was used to draw the contours in question. Additionally, I 4. “ AIR CANVAS APPLICATION USING OPENCV AND
can test out various interpolation techniques. PyGame has a NUMPY IN PYTHON ”International Research Journal of
line drawing technique (pygame.draw.line ()) that might be Engineering and Technology (IRJET) Volume: 08 Issue: 08
helpful for creating lines that are smoother and cleaner. In the — Aug 2021.
same line, adding different brush types, textures, and perhaps 5. Alper Yilmaz, Omar Javed, Mubarak Shah, ”Object
a rubber to Air Canvas will strengthen its artistic capabilities. Tracking: A Survey”, ACM Computer Survey. Vol. 38, Issue.
Unique features that imitate actual creativity software could 4, Article 3, Pp. 1-45, 2006.
also include letting the user save their finished product or 6. H.M. Cooper, ”Sign Language Recognition: Generalising
watching their drawing process as an animation. There might to More Complex Corpora”, Ph.D. Thesis, Centre for Vision,
even be a way to link Air Canvas with real digital drawing Speech and Signal Processing Faculty of Engineering and
applications like Adobe Photoshop, Clip Studio Paint, or Physical Sciences, University of Surrey, UK, 2012.
GIMP! Finally, by understanding how multicore processing 7. Yusuke Araga, Makoto Shirabayashi, Keishi Kaida, Hi-
interacts with in-order information processing, i can make roomi Hikawa, ”Real Time Gesture Recognition System Using
significant progress. Posture Classifier and Jordan Recurrent Neural Network”,
1 Voice Assistant : Making use of Voice Assistant to IEEE World Congress on Compu- tational Intelligence, Bris-
navigate the website and identify photos. bane, Australia, 2012.
2 The need for image processing applications increased as 8. EshedOhn-Bar, Mohan ManubhaiTrivedi, ”Hand Gesture
a result of the inclusion of cameras in mobile devices such Recognition In Real-Time For Automotive Interfaces,” IEEE
smartphones, iPad, and tablets. The fact that the mobile device Transactions on Intelligent Transportation Systems, VOL. 15,
is solely powered by a battery means that these applications NO. 6, December 2014, pp 2368-2377.
must be quicker and use less power.
3 Robot Control : A system that uses numbering to count
the five fingers for controlling a robot via hand position signs
has been proposed as one of the fascinating applications in
this subject.
4 Online Teaching:- This method also supports and encour-
ages online teaching which involves HCI.

CONCLUSION
This AI Based virtual painter is capable of employing com-
plex conventional writing techniques. It provides a simple way
to take notes, eliminating the need to hold a smart phone in one
hand. The ultimate goal is to develop a computer vision device
learning application that supports human-computer interaction
(HCI), also known as human-laptop interaction which is the
relationship between people and computers in general and the
device in particular. With the help of this project, the client
can create an interactive environment in which he or she can
draw whatever they desire by selecting their chosen colours
from the palette.

REFERENCE
1. Vladimir I. Pavlovic, Student Member, IEEE, Rajeev
Sharma, Member, IEEE and Thomas S. Huang, Fellow, IEEE
“Visual Interpretation of Hand Gestures for Human Computer
Interaction: A Review” VOL. 19, NO. 7, JULY 1997.
2. Gangadhara Rao Kommu, Assistant ProfessorDepartment
Of Information technology, Chaitanya Bharati Institute of
Technology, Hyderabad, India “AN EFFICIENT TOOL FOR
ONLINE TEACHING USING OPENCV”.
3. PranaviSrungavarapu, Eswar Pavan Maganti,
SrilekkhaSakhamuri, Sai Pavan Kalyan Veerada, Anuradha
Chinta “Virtual Sketch using Open CV” International Journal

© 2024, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM39912 | Page 4