ICS 2207 Multimedia

Applications
Lecture 2d: Multimedia Data
Processing(Sound, Image/Video)
 Areas of Application of Digital
 Computers
Technology
 New telephone networks and capabilities
e.g video conferencing etc
 Digital television technology -e.g Interactive
TV, Video o
Digital Technology Concepts
 Digitization of information is an
important step in processing data
 Basics
– Digital signals that could be assigned
digital values
 Digital computer technology
– Digital signals
– Binary representation
 Encoded into ones and zeros
 Advantages Digital Signals
 Native language of the computer-Information
can be processed by the computer
 Control programmable services
 Better quality due to being able to reconstruct
exact digital patterns at the receiving end
 Faster communication speeds are possible
 Easy transmission of information over the
Internet and other computer networks
 Minimize loss of quality during transmission
 Digital Signal

1 0 1 1 0 1 0 0
Signal Strength

Pulse

Time
Pulse Duration
 Digitization of Information
 Information need to be digitized for
computer processing and the transmission
of information
 Convert into digital form for processing
storage and transmission
 Components of Information
 Alphanumeric data
 Image
 Audio
 Video
 Digital Information Processing

Data
Digitized and
Audio
Encoded
Digital
Image Transmission

Video
 Digitization Of Alphanumeric
Data
 Alphanumeric data is digitized using well
established coding systems
– Unicode
– EBCDIC
– ASCII
– Etc
– See notes on encoding schemes
Digitization Of Audio
 Digitization Of Audio: Overview
 Take samples of audio at pre-determined
time intervals known as the sampling rate
 Represent the sampled audio with digital
signals
– Pulse Amplitude Modulation (PAM)
 Encode signals into binary code
– Pulse Code Modulation (PCM) that incorporates
PAM as well
– Required for computer processing
 Digitization of Audio: Pulse
Amplitude Modulation (PAM)
Audio

9 8 7 6 7
9
Digital Signals must further
be encoded into binary
signals for computer
processing and
Sampling Interval transmission.
 Digitization and Encoding of
Audio: Pulse Code Modulation

(PCM)
PCM is a two step process
1. First the audio is sampled and represented
by digital signals
2. The digital signals are then encoded in
binary form
 Assign 1 Q1. Define signal sampling and
show how it is used to in processing audio
data
Binary Encoding of Signals in
Pulse Code Modulation (PCM)
PCM

9 8 7 6 5 6

1001 1000 0111 0110 0101 0110

The integer numbers have effectively been coded into zeros

and ones. The ones and zeros now contain the audio
information encoded in a form that could be processed by a
computer.
Salient Points on the Digitization
Of Audio
 Sampling rate? and the number of bits?
used for representing the samples will
determine the quality of the audio
 Quality is retained in transmission because
only codes are transmitted
 Audio can be recreated to the original
quality by extracting the pattern from the
digital code
 Sampling Factors
 Sampling interval determined by sampling
frequency
– Measured in Hz
 Sampling depth
– Measured in number of bits used to encode the
sample
 Sampling channels
– Mono or stereo, for example 1 for mono and 2
for stereo
– Assign: Distinguish between the two and
illustrate it use and advantages
 Sampling Example
 CD quality audio
– 44 KHz
– 16 Bits
– Stereo
Audio Quality, Bandwidth and
Streaming
Factors Affecting Quality

Number of bits used

for binary encoding.
9 8 7 6 7 Example: 4 bits allow
16 amplitude
9
variations to be
represented. 24=16

Sampling Interval
 Effect of Sampling Frequency
 Higher sampling frequency
– Smaller sampling intervals, time between
samples
– Frequent sampling
– Better quality because the audio pattern is
captured better
– Higher bandwidth required for transmission
– Higher disk space required for storage
 Computation of Bandwidth
Requirement for Transmission
 Problem:
– Compute the audio streaming rate for a
voice grade circuit given that the number of
bits used in the sampling is 8
 Background information
– A voice grade circuit has a bandwidth of
approximately 4000 Hz
 General rule
– For acceptable quality, the audio must be
sampled at twice the frequency of the voice
grade bandwidth
 Reason for Sampling at Twice the
Frequency
 According to Nynguist Theorem
 Two peaks in each cycle
– Half of a cycle is above the datum line
– The other half of the cycle is below the datum
line
 Therefore, sample the audio at twice the
frequency rate
 Example CD Sampling
 Sampling in this case is done for higher
quality
– 44 KHz
– 16-bits
– Stereo
Problem Representation

8 bits are used enabling

256 amplitudes to
represent the human
voice which is considered
to be adequate.
79 68 57 46 57
79

1/8000 Seconds (8000 HZ twice the frequency of the voice

grade circuit)
or 2X4000 samples per second
 Bandwidth Computation for Voice
 Number of samples
– 8000 per second
 Number of bits per sample
–8
 Bandwidth requirement
– 8X8000 bps = 64,000 bps
– Approximately 64K bps
 64K bps is the speed of a single ISDN (B)
channel designed to carry voice
 Bandwidth of Voice Circuits
 Generally speaking, the bandwidth
requirement for uncompressed voice circuit
is 64 Kbps
 An example is the ISDN – B channel that
was originally intended to carry voice
– Its bandwidth is 64 K bps
Examples in Audio Quality and
Bandwidth Requirement
 CD quality
– 44,100 Hz, 16 bit, Stereo
– 1376K bps
 Radio quality
– 22,050 Hz, 8 bit, mono
– 176K bps
 Telephone quality
– 11,025 Hz, 8bit, mono
– 88K bps
 Audio Transmission In WWW
Audio streaming requires compression.

Web
28-56K bps
Site

Client
Receive audio using
Real-time audio
Internet Explorer
broadcast support
and a plug-in to receive
using streaming
the audio stream.
server module.
 Delivery of Instruction Over the
WWW
Audio/Video streaming.

Web
28-56K bps
Site

Client

Store streamed audio/ Receive audio/video using

video using Windows Media. Internet Explorer and Media
Player.
 Types of Multimedia Transmission
 Unicasting
 Multicasting
 Broadcasting
 Sampling Considerations In
Communications

Digital audio transmission

Sender Receiver

Adjust quality (sampling interval and bit

representation) to suit bandwidth availability.
 Audio Files
 Audio can be stored in different formats
– Uncompressed or raw file format (wav)
– Compressed format
– Streaming format
 Streamed audio is also compressed
 It is also designed for real-time delivery of audio
 Audio File Format
 wav file format
– Basic file format in audio storage or raw file
 rm file format
– Real audio’s streamed file format
– Streamed file
 wma file format
– Microsoft’s audio streamed file format
– Streamed file
 mp3 file format
– Compressed file
 aac file format
–
Quality Retention in Digital
Transmission
 Quality Retention
 Quality is retained in digital transmission
because only the codes are transmitted
 Quality is subject to some deterioration in
analog transmission because the wave
pattern is transmitted
Analog Audio Transmission
Audio Prior
Audio with
to Transmission
Interference

Transmission

Audio After Filtering

Passage of Analog Audio Over
Analog Lines

Analog Analog
Audio Signals
Telephone

Analog Analog
Audio Signals
Telephone
Recreation of Audio from Analog
Signals
 A difficult task
 Complex algorithms are used to filter noise
etc. for better audio transmission
Signal Passage in Digital Audio
Transmission
Encode

Audio

Recreate Transmit

Decode Audio
 A Sample Digital Audio
Transmission Path

Analog Digital DSL

Audio Audio Modem
Sound
Card
Digital
Network

Analog Digital DSL

Audio Audio Modem
Sound
Card
 Sound Generation

 Sound is recreated at destination

– Using FM synthesis
– Using wave table generation
 Noise is not an issue in digital
communication although it is an issue in
digital transmission
– The reason, once again, is due to the fact
that only codes are transmitted in digital
transmission
 Better Sound Generation
 Wave table generation provides better
sound reproduction that FM synthesis
 Digital Advantage in Audio
Transmission
 Only codes are transmitted
 Original encoding is recreated
 Original audio is reproduced
 Again, sampling rate and number of bits
used in each sample will determine the
quality of audio transmitted
Digitized Signal Transmission
Over Analog Lines
Encode
Sampled Signals

Audio

Recreate Transmit

Decode Audio
Sample Digital Audio Transmission
Path Over Analog Lines

Analog Digital
Audio Audio Modem
Sound
Card
Analog
PSN

Analog Digital Modem

Audio Audio
Sound
Card
 Audio Transmission In WWW
Audio stream over analog/digital line.

Web
Site

Real-time audio Client

broadcast support Receive audio using
using Windows Media Internet Explorer
streaming server module. and Windows Media Player.
 Analog to Digital Converter
 A to D and D to A converter
 The chip that is responsible for this conversion is
known as the DSP (Digital Signal Processor) chip
 It is used in sound cards, modems etc. wherever
there is a need for A to D and D to A conversion
 The mass use of this chip in various devices has
led to a drastic drop in the price of the chip and the
devices
Digital Signal Processor (DSP)

DSP

Digital
Analog
Digitization Of Image
Image representation methods
 Bitmap Graphic
 A image is divided into a matrix of pixels,
where each pixel is a small dot.
– More pixels 
– Better representation of image 
– Better resolution 
– More memory
 Each pixel is assigned a bit pattern.

The size and the value of the pattern

depend on the image.
Figure 2-7
Bitmap graphic method of a
black-and-white image
 Color image
Each pixel is decomposed into 3 primary colors:
red, green and blue. (RGB)
The intensity of each color is measured, and a bit
pattern (8 bits) is assigned.

Figure 2-8 Representation of color pixels

 Resolution
 The term is most often used to describe monitors,
printers, and bit-mapped graphic images.
 Dot-matrix and laser printers,
the resolution indicates the number of dots per
inch. For example, a 300-dpi (dots per inch) printer is
one that is capable of printing 300 distinct dots in a
line 1 inch long. This means it can print 90,000 dots
per square inch.
 Graphics monitors,
the screen resolution signifies the number of dots
(pixels) on the entire screen.
For example, a 640-by-480 pixel screen is capable of
displaying 640 distinct dots on each of 480 lines, or
about 300,000 pixels.
 Vector Graphic

 A image is decomposed into a

combination of curves and lines.
 Each curve or line is represented by a
mathematical formula.
 Audio
 Audio is a representation of sound or music.
 Audio is by nature analog data.
 It is continuous, not discrete.
 Audio is converted to digital data and stored
in bit patterns.
1. Sampling
2. Quantization
3. Coding
4. Stored
Figure 2-9
Audio
representation
 Video
 Video is a representation of images (frames)
in time.
 A movie is a series of frames shown one after
another to create the illusion of motion.
 Today video is normally compressed.
 Image Digitization
 Image can be of the form black and white,
gray scales, color
 Factors that influence the digitization of
image are as follows
– Resolution measured in pixels
– Color depth expressed in number of color
variations
 Digitization Of Image: Overview
Pixel
Horizontal Resolution
Vertical Resolution
 Digitization of the Letter L

Number of bits
determine the
amount of
information that could
be stored.
 Digitization Of Image: The
Process
 Divide the image into a grid of pixels that
may be considered as the sampling points
of the image
 Digitize information on each pixel
 Store and transmit
 Resolution
 Horizontal resolution
– Number of horizontal pixels
 Vertical resolution
– Number of vertical pixels
 Image resolution
– Horizontal by vertical resolution
– Ex: 640 by 480
Digitization of Black and White
Image
 White
– A pixel lit represents a 1
 Black
– A pixel not lit represents a 0
 Storage required per pixel
– 1 bit
 Storage required for 640 by 480
resolution image
– 640 times 480 bits = 307,200 bits = 38.4K
Bytes
Digitization of Image Using Gray
Scales
 A pixel may take a value between 0 and 15
for 16 gray scales
 A gray scale of 3 can be coded as 0011 and
the others similarly using this 4 digit code
 The bandwidth requirement for the
transmission of a 640X480 image in this
case is as follows:
– 640X480X4 = 153.5K Bytes
 Digitization of Color Image
 Image coding
– Each pixel may take a value between 0 and 255 if 256
colors are to be represented
 Storage requirement
– Digitizing of images requires substantial number of
bytes and hence large storage space for processing
 Bandwidth requirement
– Higher bandwidths are required to transmit color images
 Bandwidth Computation for
Image with 256 Colors
 Resolution is 640X480
 8 bits are required to represent 256 colors
 bandwidth requirement for the transmission
of one image is as follows:
– 640X480X8 = 307.2K Bytes
The Effect of Color Depth and
Resolution
 Compare VGA, SVGA and XGA
– XGA provides the highest resolution
 Practical implication
– More colors less resolution if bandwidth or storage
is the limiting concern
– Example
 256 colors at lower resolution
 16 colors at higher resolution
 Rule
– Higher the resolution the lower the number of
colors available in general given the resource
constraints such as bandwidth constraints
 Factors Affecting Bandwidth
Requirement in Image Transmission
 The higher the resolution, the higher the
bandwidth requirement for transmission
 The higher the color representation, also known as
color depth, higher the bandwidth requirement
 For true color, 24 (32) bits are required to
represent each pixel
 The file sizes in raw image capture can thus
become very large
Compression of Digitized Images
Compression of Digitized Images
 Compression is required to reduce the size
of the image file
 Large blocks of unchanged data in an image
(background) offers an opportunity to
compress the image
 Image files are almost always compressed
 A Few Compression Formats
 GIF
 JPEG
 MIC (Microsoft Image Composer)
 PCD (KODAK) - Used by Corel
 Uncompressed file exist in the form of bit
mapped file with the extension of .BMP
 Image File Format Extensions
 File formats often represent the compression
procedure being used such as jpg representing
the jpeg compression technique
 Examples:
– Bmp – uncompressed file format
– Gif
– jpg
– pcd
– tiff
– pcx
 Loss-less Compression and
Others
 Some compression formats offer loss-free
compression of the image
 Others sacrifice minimal loss for the sake of
reduced storage and bandwidth
requirements
 Fortunately, the loss is not easily detected
by the naked eye
 Image Transmission
Considerations
Adjust image to suit available bandwidth.

Sender Receiver

Adjustable features are as follows.

- Resolution
- Color depth
Adjusting the size also reduces the bandwidth
requirement because of a corresponding reduction
in the number of pixels required to represent
the image.
 A Peek At Data Compression
 0 0 0 0 0 0 0 0 0 0 0 - - - - - -0 1 1 1 1 1 11
…... 0
 THE ABOVE CAN BE COMPRESSED
INTO = #9000$0#
– 9000 bits are compressed into 8 characters that
require approximately 64 bits for transmission
– 9000 #600$1#
ZEROS ARE CODED INTO #900$0# 600
NUMBER COUNT
INTERPRET WITHIN THE # SIGN 1

CHARACTER BEING
TRANSMITTED
 Compression Result
 In the previous example, 9000 bits are
compressed into 8 characters
 If 10 bits are used on the average for
transmitting each character, the 9000 bits of
information is now compressed into 80 bits
for transmission
 Modem Implication in Image
Transmission
 Modems also compress the data stream to
achieve higher transmission speeds
 Because of the fact that the images are already
compressed, the full speed benefit may not be
realized when images are transmitted over a
modem connection
 An already compressed image file does not, for
instance, offer itself well to further compression in
the modem
 Module

Digitization Of Video
 Digitization of Video
 Digitization of video is an extension of the
process of digitizing an image
 It amounts to the transmission of certain
number of still images known as frames per
second
 Obviously, digitized video requires higher
bandwidth for transmission and more space
for storage
 Frame Rate

 30 frames of images per second, in general,

defines continuos motion
 In communications, 25 frames per second is
considered to be continuous motion
 15 frames per second is currently used in
video conferencing over digital lines for
acceptable reception of video
 It is also possible to engage in video
conferencing at a frame rate of 5 frames per
second
 Computation of Bandwidth for
Raw Transmission of Video
 Image resolution is 640X480
 Number of colors is 256 (8 bit)
 Acceptable reception requires 15 frames per
second
 Therefore, the bandwidth for the raw
transmission is as follows:
– 640X480X8X15 = 36.86M bps = 4.6M Bps
Compression Standards Used in
the Digitization of Video
 MPEG 1, MPEG 2, MPEG 3 and MPEG 4
 Windows Media Video
 Real Media
 Indio
 QuickTime
 ActiveMovie
 AVI
 Streaming Formats for Video

 Various streaming formats are

supported by different vendors
– RealVideo
 Microsoft’s streaming format
– wma (Windows Media Audio)
– wmv (Windows Media Video)
– Active Streaming Format (ASF)
 Apple’s QuickTime format
 Etc.
Overview of Video Transmission
in Video Conferencing
 Minimum speed
– 3 to 5 frames per second
 Acceptable speed
– 15 frames per second
 Transmission techniques
– Data is compressed
– Only changes to the frame are transmitted
Bandwidth Optimization in Video
Conferencing
 Minimize Windows for maximum efficiency
– Transmit less number of pixels in minimized
form
 Decrease the resolution
– Has the same effect as above
 Decrease the number of colors displayed
 Communication Links for Video
Conferencing
 Possible on analog lines using 56,000 bps
transmission speed but not desirable
 Digital lines are preferred and the guidelines
are as follows:
– Possible at 128k bps using ISDN lines
– Acceptable at 384k bps
– 1M bps and above offer good quality video
transmission
 ISDN Line Suitability
 ISDN B channels can be assigned on a
dynamic basis depending on the bandwidth
requirement at any point in time during video
conferencing
 Video Conferencing Products
 Intel ProShare
 CU-See Me
 Picturetel
 C-phone
 etc.
End of Lecture