AUDIO AND VIDEO

CHAPTER 5
AUDIO
DIGITIZATION OF SOUND

• The signals we use in the real world, such as our voices, are called "analog"
signals.
• To process these signals in computers, we need to convert the signals to
"digital“ form.
• Analog signal is continuous in both
time
and amplitude, a digital signal is discrete
in both time and amplitude.
DIGITIZATION OF SOUND

• Digitization is a process of converting the analog signals to a digital

signal. There are three steps of digitization of sound.
▪ Sampling
▪ Quantization
▪ Encoding
SAMPLING

• In order for a computer to work with audio waves, they must be converted
from analog to digital form.
• This is done through a process called sampling, in which every fraction of a
second, a sample of the audio is recorded in digital bits.
• There are two factors that affect the quality of the digitized audio:
– Sample rate
– Sample size
SAMPLE RATE

• Sample rate is the number of times the sample is taken.

• The three most common sample rates are; 11.025 kHz, 22.05 kHz, and 44.1
kHz.
• The higher the sample rate, the more samples that are taken and, thus, the
better the quality of the digitized audio
SAMPLE RATE
SAMPLE SIZE

• Sample size is the amount of information stored about the sample.

• The two most common sample sizes are 8 bit and 16 bit.
• An 8-bit sample allows 256 values that are used to describe audio, whereas
a 16-bit sample provides 65, 536 values.
• The greater the sample size, the better the quality of the audio.
SAMPLING TOOL

• In practice, the continuous signal is sampled using analog-to-digital

converter (ADC), a device with various physical limitations.
• This results in deviations from the theoretically perfect reconstruction,
collectively referred to as distortion
IMPULSE SAMPLING
NYQUIST THEOREM

• For lossless digitization, the sampling rate should be at least twice the maximum
frequency response.
• In mathematical terms: f s > 2*fm
▪ where f s is sampling frequency and fm is the maximum frequency in the signal

As an example, humans can detect or hear frequencies in the range of 20 Hz to

20,000 Hz. If we were to store sound, like music, to a CD, the audio signal must be
sampled at a rate of at least 40,000 Hz to reproduce the 20,000 Hz signal. A standard
CD is sampled at 44,100 times per second, or 44.1 kHz.
 NYQUIST LIMIT
• max data rate = 2 H log2V bits/second,
• where
▪ H = bandwidth (in Hz)
▪ V = discrete levels (bits per signal change)
Shows the maximum number of bits that can be sent per second, on a noiseless channel with a bandwidth of H, if V bits
are sent per signal
Example:
1. what is the maximum data rate for a 3kHz channel that transmits data using 2 levels (binary) ?
(2x3,000xln2=6,000bits/second)
Exercise:
2. what is the bandwidth of data if maximum data rate for channel is 12000bits/Second that transmits data using 2 levels
(binary) ?
PCM(PULSE CODE MODULATION)

• Digital audio uses pulse-code modulation and digital signals for sound reproduction.
Includes analog-to-digital conversion (ADC), digital-to-analog conversion (DAC),
storage, and transmission.
• Pulse-code modulation (PCM) is a method used to digitally represent sampled
analog signals. It is the standard form of digital audio in computers, Compact Discs,
digital telephony and other digital audio applications. In a PCM stream, the
amplitude of the analog signal is sampled regularly at uniform intervals, and each
sample is quantized to the nearest value within a range of digital steps
QUANTIZATION

• Quantization is a process of representing the amplitude of each sample as integers or

numbers.
• Sampling in the amplitude or voltage dimension is called quantization
• How many numbers are used to represent the value of each sample known as sample
size or bit depth or resolution.
• Commonly used sample sizes are either 8 bits or 16 bits. The larger the sample size,
the more accurately the data will describe the recorded sound.
• The value of each sample is rounded off to the nearest integer (quantization) and if the
amplitude is greater than the intervals available, clipping of the top and bottom of the
wave occurs.
SAMPLING AND QUANTIZATION

Sampling and quantization: a) sampling the analog signal in the time dimension
b) quantization is sampling the analog signal in the amplitude dimension
ENCODING

• Encoding converts the integer base-10 number to a base-2 that is a

binary number. The output is a binary expression in which each bit is
either a 1(pulse) or a 0(no pulse).
SIGNAL SAMPLING REPRESENTATION

• The continuous signal S(t) is represented with a green colored line

while the discrete samples are indicated by the blue vertical lines.
HOW SAMPLING IS DONE?

• Sampling can be done for functions varying in space, time, or any other
dimension, and similar results are obtained in two or more dimensions.
• The sampling frequency or sampling rate, fs , is the average number of
samples obtained in one second (samples per second), thus
fs =1/T
HOW SAMPLING IS DONE?
• The following figure indicates a continuous-time signal x t and a sampled
signal xs t
• When x t is multiplied by a periodic impulse train, the sampled
signal xs t is obtained.
CONVERSION
 CHANNELS

Two types of channels

– Monophonic
– Stereophonic
MONOPHONIC CHANNEL
• Commonly called mono sound, mono, or non-stereo sound, this early sound
system used a single channel of audio for sound output.
• Monophonic sound is the most basic format of sound output.
• Mono (monophonic, or monaural) is sound from a single source.
• All speakers in a mono system (like an intercom) will carry the same signal.
 STEREOPHONIC CHANNEL
• Commonly called stereo sound or just
stereo, stereophonic sound divides sounds
across two channels (recorded on two
separate sources) then the recorded sounds
are mixed so that some elements are
channeled to the left and others to the right.
• Stereo (stereophonic) is sound from two
sources, ideally spaced apart, and
reproduces sound the way we hear it
naturally, with two ears
STORING SOUND DATA

• There are two means of storing sound data

▪Waveform Audio (or digital audio)
▪Musical Instrument Digital Interface (MIDI).
What is MIDI?
MUSICAL INSTRUMENT DIGITAL INTERFACE
MIDI is a communication standard that allows digital
(MIDI)
music gear to speak the same language. MIDI is short
for Musical Instrument Digital Interface. It’s a protocol
that allows computers, musical instruments and other
hardware to communicate.

MIDI never transmits an actual audio signal—it’s

information only. That means that if a MIDI keyboard
doesn’t have an onboard sound source like a synth or
sampler, it won’t make any sound!
MUSICAL INSTRUMENT DIGITAL INTERFACE
(MIDI)
MIDI (Musical Instrument Digital Interface) is an industry standard for passing musical
performance information among electronic musical instruments and computers. There are
very few details about MIDI that are neccessary for you to understand in order to use
Notation Composer.

Most audio equipment that we are familiar with-- such as CD players and radios-- receive
and send sound in the form of sound waves.
MIDI MESSAGES

• MIDI is a music description language in digital (binary) form which is made up of eight bit
status byte
• It was designed for use with keyboard-based musical instruments, so the message
structure is oriented to performance events, such as picking a note and then striking it, or
setting typical parameters available on electronic keyboards.
• For example, to sound a note in MIDI you send a "Note On" message, and then assign that
note a "velocity", which determines how loud it plays relative to other notes. You can also
adjust the overall loudness of all the notes with a Channel Volume" message. Other MIDI
messages include selecting which instrument sounds to use, stereo panning, and more.
DIFFERENCE BETWEEN MIDI AND DIGITAL AUDIO

MIDI Digital Audio

A MIDI file is a software for A digital audio refers to the
Definition representing musical information reproduction and transmission of
in a digital format. sound stored in a digital format.

Format type Compressed Compressed

•Files are tiny, often less than •They reproduce the exact sound
10K. files.
Advantages
•Fit easily on a floppy desk. •It reproduce better than CD
•The files are any time ideal. quality.

Do not contain a recording of

Contain Contain a recording of sound
sound

No actual sound stored in MIDI Actual sound stored in digital

Storage
file audio file
FILE FORMATS

▪ Wav audio (.wav)

▪ Sound (.snd)
▪ Real audio (.ra, rm)
▪ Audio File Format (.aif)
▪ MIDI (.mid)
▪ MP3 audio (.mp3)
▪ Windows Media (.wma)
HOW TO CALCULATE SOUND FILE SIZE

Sound File Size = Sample rate x sample size x channel x duration

Example:
Calculate how much storage space is needed to record a 16-
bit, 44.1khz, stereo music for a duration of 30 seconds.

Sound file size = 44100Hz x 2Bytes x 2channel x 30s =

5292000 bytes
EXERCISE
Qu 1)Calculate how much storage space is needed to record a 8-bit, 3khz,
mono sound for a duration of 30 seconds?
Qu 2) Moving Picture Experts Group (MPEG-1), was designed for a
1. PC
2. CD
3. DVD
4. Floppy

Qu 3) Which of the following is the best definition for Virtual Reality?

1. Any computer game involving graphics.
2.A 3D simulation of a real or imagined environment using computers.
3.A simulator which requires special eye glasses.
4.The process of coding in another dimension.
VIDEO
 VIDEO
• Video is the technology of electronically
capturing, recording, processing, storing,
transmitting, and reconstructing a sequence of
still images representing scenes in motion.
• Video makes use of all of the elements of
multimedia, bringing your products and
services alive, but at a high cost.
• The advantage of integrating video into a
multimedia presentation is the capacity to
effectively convey a great deal of information
in the least amount of time
 VIDEO

• Video is also very hardware-intensive (require the highest performance demand on

your computer)
– Storage issue: full-screen, uncompressed video uses over 20 megabytes per second
(MBps) of bandwidth and storage space.
– Processor capability in handling very huge data on real time delivery
DIGITIZING THE VIDEO SIGNAL

• Analogue video is essentially a product of the television industry and

therefore conforms to television standards.
• Digital video is a product of the computing industry and therefore conforms
to digital data standards
• Is usually recorded and played as an analog signal.
• It must therefore be digitized in order to be incorporated into a multimedia
title.
 DIGITIZING THE VIDEO SIGNAL

• Analog video digitization involves analyzing each

scan line of video, separating the color and intensity
levels and digitizing each component. For digital video
capturing from optical sensors (such as video
recorders with CCD sensors), each pixel element is
converted into a color type (red, green, and blue) which
has an intensity level (brightness).

• Converting video signals at 30 frames per second into

digital streams of data results in large amounts of data.

• The uncompressed data rate for standard definition

television (SDTV) is 270 Mbps (SDI format). The
uncompressed data rate for high definition television
(HDTV) is 1.5 Gbps.
DIGITIZING THE VIDEO SIGNAL

• This figure shows a basic process that may

be used to digitize images for pictures and
video. For color images, each line of image
is divided (filtered) into its color
components (red, green and blue
components). Each position on filtered
image is scanned or sampled and
converted to a level. Each sampled level is
converted into a digital signal.
ADVANTAGES OF DIGITAL VIDEO

• One of the advantages of digitized video is that it can be easily edited.

• The video is stored as a standard computer file.
• Software motion video does not require specialized hardware for playback.
DISADVANTAGES OF DIGITAL VIDEO

• Requires large storage capacity devices.

• Copies can be made illegally.
• Need fast computer system for playback and capture.
• Requires knowledge of digital compression technology
FILE SIZE CONSIDERATIONS

• Digitized video files can be extremely large.

• A single second of high-quality color video that takes up only one-quarter of
a computer screen can be as large as 1 MB.
• Several elements determine the file size:
▪ Frame Rate
▪ Image Size
▪ Color Depth
▪ the length of the video
DIGITAL VIDEO FILE FORMATS

• Motion Pictures Expert Group (.mpg)

• Quicktime (.mov)
• Audio Video Interleaved(.avi)
• Windows Media Video (.wmv)
• Adobe Flash video (.flv)
CALCULATE VIDEO FILE SIZE

• Video File Size = Frame size x Frame rate x color depth x duration Notes
(Color depth) :
B&W video = 1 byte
Color video = 3 bytes

Example 1: Calculate the file size for a video with 320 x 240 pixels, color video, 30 fps, and length 15
seconds.

Video file size = 320 x 240 x 3 x 30 x 15 = 103680000 bytes

CALCULATE VIDEO FILE SIZE

• Exercise 2:
Calculate the file size for a video with 320 x 240 pixels, color video, 30 fps, and
length 15 seconds.
Video file size = 320 x 240 x 3 x 30 x 15 = 103680000 bytes