mVizn Pte. Ltd.
Deep Learning and Computer
Vision
for Video Analytics
Dr. Sambit Pal
Director & CTO
mVizn Pte. Ltd.
Block 81, JTC Launchpad
81 Ayer Rajah Crescent,
#03-61, Singapore 139967
Computer Vision:
Indispensable in a human-centric environment
Vision is our
predominant “sensor”
Machines need the
same!
mVizn Pte. Ltd.
Video Analytics
● Video recording is
all pervasive in
modern world
● Making machines
understand what’s
in the video
mVizn Pte. Ltd.
Video Analytics: Key Objectives
1. Identify patterns, features, objects in a snapshot/
frame
2. Identify how these patterns, features, objects
change over time
3. Implement rules to check how they are supposed to
change/behave over time and flag out exceptions
mVizn Pte. Ltd.
How it was “traditionally” done:
Detecting edges, corners, thresholding
colors/intensity, matching a bunch of
edges and corners (so called features)
mVizn Pte. Ltd.
How it was “traditionally” done:
Subtracting background, tracking “flow
of features” across frames.
mVizn Pte. Ltd.
Traditional/Rule-based Computer Vision
Works great for tasks involving regular shaped objects and tasks where
rules can be defined reasonably.
mVizn Pte. Ltd.
Traditional/Rule-based Computer Vision
Not so great to tell what’s going on here:
mVizn Pte. Ltd.
Describing “irregular” objects with rule-based
Computer Vision
Computer, what is this?
Just less than 10
years ago this was
pretty much
impossible to solve!
mVizn Pte. Ltd.
In past few years something radically changed
Computers have become almost as good as humans in
understanding images.
mVizn Pte. Ltd.
In past few years something radically changed
mVizn Pte. Ltd.
DEEP NEURAL NETWORKS
What changed?
POWERFUL ENOUGH HARDWARE
mVizn Pte. Ltd.
Recognize objects: Teach computers by
showing examples!
Neural Network
Image Source:
https://becominghuman.ai/@venkateshtata9
mVizn Pte. Ltd.
Is it enough for practical applications?
Instead of two separate images on
the left, what if we feed in, an
image like this one?
mVizn Pte. Ltd.
mVizn Pte. Ltd.
Why didn’t it work?
Neural Network
Image Source:
https://becominghuman.ai/@venkateshtata9
mVizn Pte. Ltd.
Locate Objects in a scene: Teach computers by
showing examples, once again!
Image Source:
FastRCNN
mVizn Pte. Ltd.
mVizn Pte. Ltd.
So far, we saw …
1. Recognizing Objects à Image Classification
2. Locating Individual Objects à Object Detection
mVizn Pte. Ltd.
Can we make a useful VA application already?
Yes, because …
mVizn Pte. Ltd.
Can be trained on any kind of images
mVizn Pte. Ltd.
A big change!
No need for rewriting of detection rules for each use
case, as was necessary for traditional computer vision!
mVizn Pte. Ltd.
Let’s look at a practical problem within PSA
mVizn Pte. Ltd.
mVizn Pte. Ltd.
How to solve this problem?
Train to identify and extract
corners of containers:
Object Detection
mVizn Pte. Ltd.
How to solve this problem?
Train a classifier for twist-lock
cone / no-twist-lock cone:
Image Classification
TLC: Yes
TLC: No mVizn Pte. Ltd.
Recognize objects: Teach computers by
showing examples!
Neural Network
TLC:
Yes
TLC: Yes
TLC:
No
Image Source:
https://becominghuman.ai/@venkateshtata9
mVizn Pte. Ltd.
… and let’s see the result
mVizn Pte. Ltd.
mVizn Pte. Ltd.
How we typically start a new VA project?
1. Identify a “problem” in a job/process flow
• “Problem” needs to be visible to human eye
2. Collect sample footages
3. Train deep-neural-network model
4. Test model on a new unseen footage to verify
performance
mVizn Pte. Ltd.
What mVizn specializes in!
1. Identify a “problem” in a job/process flow
• “Problem” needs to be visible to human eye
2. Collect sample footages
3. Train deep-neural-network model for customized
use cases
4. Test model on a new unseen footage to verify
performance
mVizn Pte. Ltd.
Let’s look at some other examples of what
deep-learning-based computer vision can do…
mVizn Pte. Ltd.
Video prepared by PSA EARTG TEAM
mVizn Pte. Ltd.
https://www.youtube.com/watch?
v=8jfscFuP_9k
mVizn Pte. Ltd.
https://www.youtube.com/watch?
v=yeS8TJwBAFs&t=71s
mVizn Pte. Ltd.
A new industrial revolution
mVizn Pte. Ltd.
But there are still many unsolved problems in
computer vision…
Thank you!
Reach us at: contact@mvizn.com mVizn Pte. Ltd.
