Outline
 Introduction
 Audio and its digitization
 Video and its digitization
 Bandwidth requirements
 Multimedia transmission requirements
 SAS Factors for Audio andVideo
 Network Performance parameters
 Multimedia over Internet &Wireless
 Problems with Internet & solutions
2 Eng: Mohammed Hussein

Introduction
The continuous media are played during some
well-defined time interval with user interaction.
The most demanding are audio and video
Audio: what we hear through our ears
Video : what we see through our eyes

Demand for Multimedia Communication

What is Streaming???
 Streaming media refers to media types with time constraints
and a continuous Video and audio transmission
 Playback of streaming media starts while data is being
received, i.e. it is not necessary to download the entire file
before playback starts
 Advantage: doesn‘t require to download complete
content first (thus it avoids long delays)

Example 1
6
5
Excellent
4
Good
3
Fair
2
Poor
1
Bad
Pixilated Video
Distorted Speech
Eng: Mohammed Hussein

7
5
Excellent
3
Fair
2
Poor
1
Bad
4
Good Clear Video
Clear Speech
Example 2

MATLAB SIMULINK
 To start the Simulink software, you must first start the MATLAB®
technical computing environment.You can then start the Simulink
software in two ways:
 On the toolbar, click the Simulink icon.
 Enter the simulink command at the MATLAB prompt.

How we can digitize audio signal?
 Basic steps:
1. Conversion to electronic form using microphone (analog signal )
2. Sampling the analog signal based on PAM or PCM.
3. Quantization using Analog to Digital converter

Types of Waveforms (Aperiodic) Signals

Why we use these signals?
 Based on these Signals we can create complex Signals.
 Sinusoid, unit step and exponential are used to approximate
basically more complex signals.
 They help us to predict and analysis as we take the signals as input
to the system such as filters, high pass, low pass, band pass, etc
filters and see what the output is.

Digitizing audio signal
Analog signal
PAM signal (Sampling )
Quantized signal

Pulse Code Modulation (PCM)

Discrete-time and continuous-time signals
 A discrete-time signal is quantities that defined only on a discrete
set of times.A simple source for a discrete time signal is
the sampling of a continuous signal, approximating the signal by a
sequence of its values at particular time instants.
 A continuous-time real signal is any real-valued (or complex-
valued) function which is defined at every time t in an interval, most
commonly an infinite interval.

Analog and digital signals
 The figure shows a digital signal that results from approximating an
analog signal by its values at particular time instants.
 Digital signals are discrete and quantized, while analog signals
possess neither property.

Amplitude modulation

Carrier and Signal

Bandwidth requirements: Audio
 NC: Number of channels,
 UC: Uncompressed,
 C: Compressed

Video
 Best way to understand electronic video signal is understand how
we get a picture from a digital camera.
 Electronic sensors Camera Gain Calibration (CCD) convert
different levels of light to electronic signals of different amplitudes.
 It is useful to know this conversion for evaluating the performance of
the CCD camera.
 The light is passed through a RGB filter

Video
 The video consist of a sequence of frames
 The frames are displayed fast enough to get the impression of
motion.
 To get flicker free display, the frames are repeated 50 times per
second. Because the property of human eye at the rate of 50 frames
per second doesn’t identify the changes in the video
Frame 1 Frame 2

Digitizing video
 Each frame is divided in small grids, called pixels.
 For black-and-whiteTV, grey level of each pixel is represented by 8
bits.
 In case of color, each pixel is represented by 24 bit, 8-bit for each
primary colors (R G B ).
 Assuming that a frame is made of 640*480 pixels, the bandwidth
requirement is 2*25*640*480*24 =368.64 Mbps
 The 25 frames repeated 2 times with 24 bits color
 This is too high for transmission without compression through the
internet.

Resolution
 How many Megapixel of your camera?
 Example of resolution a 3.1 Megapixel digital camera
provides the following resolution options:
 The highest resolution 2048* 1536
 2048* 1536
 1600*1200
 1024*768
 640*480

Video and Audio Converter

Bandwidth Requirements : Video
 Now the requirements for applications such as : HDTV,TV,…The CIF
and QCIF used for conferencing applications.
 The number of pixels is horizontal (HR) 1920 and vertically (VR) 1080,
with each pixel 24 bits (CR) and 60 frames per second (FR) will
Required 2986 Mbps. But by using compression, it reduced.
25

Multimedia Transmission Requirements
Qualitative requirements are:
1. Response of the human ear (20 Hz to 20 KHz, sensitive to changes
to signal levels rather than absolute values, for animal can be more
then human ).
2. Response of the Human Eye (retains for few msec before decaying)
3. Tolerance to Error ( Higher error rate tolerance for uncompressed
signals), also as we communicate with network there became an
error after compressed.
4. Tolerance to delay and variation in delay (small for live applications)
5. Lip synchronization (most critical aspect) to avoid overlap.

Difference with classic applications
Highly delay-sensitive
Packets are useless if they arrive too late
Loss-tolerant (for the most part)
Packet loss can be concealed

Performance Parameters
 Synchronization Accuracy Specification (SAS) factors used to
specify the goodness of sync which are:
1. Delay:Acceptable time gap between transmission and
reception
2. Delay jitter: the variation between the desired presentation
times and actual presentation times of streamed multimedia
objects.
3. Delay skew:Average difference between the desired and
actual presentation times.
4. Error rate: Level of error is specified by bit-error rate (BER)

SAS Factors for Audio
 Delay : for conversation,one-way delay should be in 100-500
msec range, which requires echo cancellation.
 Delay Jitter: 10 times better than delay
 Lip synchronization : between audio and video should be
better than 80 ms
 BER: required less than 0.01 for telephones.
 Less than 0.001 for uncompressed CD quality audio
 less than 0.0001 for compressed CD quality audio

SAS Factors for Video
 Delay and jitter
Required less than 50 ms for HDTV
Less than 100 ms for broadcast qualityTV
Less than 500 ms for video conference
 Error rate
Required less than 0.000001 for HDTV
Less than 0.00001 for broadcastTV
Less than 0.0001 for video conference

Traffic Characterization Parameters
Variability of bit rates are :
1. Constant bit rate (CBR) applications:
 Example: uncompressed digitized voice
transmission required CBR.
2. Variable bit rate (VBR) applications:
 Example: video transmission using compression
 Most Multimedia applications generateVBR traffic

Network Performance Parameters
1. Throughput: Effective rate of transmission of information
bits.
2. Delay:Time required for a bit to traverse through a network
(end-to-end) (250 msec max).
3. Delay variance:Variation of delay as a packet traverses
though a network (10 msec max).
4. Error rate: Specified by bit-error rate (BER), Packet error
rate (PER), Packet Loss Rate (PLR) and Cell Loss Rate(CLR).
 The values of these NPPs, as compared to the bandwidth and SAS
factors of an application determines the QoS of a network.

Multimedia over Wireless and Internet
The Internet Protocol (IP)-
based Internet and
wireless systems are
converging into a
ubiquitous all-IP
information transport
infrastructure.
Thus, will allow mobile and
stationary users to access
the wireless Internet for
multimedia services
anywhere, and anytime.

Multimedia over LAN
 The packet may suffer of a larger number of collisions. So some
LAN type are not suitable for multimedia traffic.

Why Multimedia over Wireless?
 WLAN technologies are being increasingly used for
multimedia transmissions.
 IP multimedia is partly real-time, which needs low delay.
 There will be multiple of simultaneous services, with different
QoS requirements.
 All of these should be supported by cost-efficient manner by
using network resources efficiently.

QoS in Broadband wireless networks
New services based on multimedia applications
 VoIP
 Video conferencing
 Video on demand (VoD)
-They demand strict network guarantees such as reserved
bandwidth or bounded delays in BroadbandWireless Network
(BWN)
-The challenge for BWN is to provide QoS simultaneously to
services with different characteristics

Wireless Vs. Wired
 Multimedia supported in wireless networks is much
more difficult than wired networks.
 Wireless links are moving and unpredictable.
 Wired links are fixed position.
-That is why the QoS in wireless networks is managed at
a MAC layer (medium access control)

Multimedia over Cellular Networks
38 Eng: Mohammed Hussein Voice over IP

Multimedia over Internet
 The internet was not designed to carry multimedia
traffic.
 The existing protocols are:
1. TCP: Includes connection establishment, error
control, flow control and hence unsuitable for real-
time multimedia traffic.
2. UDP: connectionless protocol at the transport layer,
can deliver real-time data if error can be tolerated, as
with uncompressed audio and video.

Internet for Multimedia Traffic
To make the internet support multimedia we need some
enhancements:
 Multicasting:Used in audio and video conferencing (one-to-
many). Original IP is a “best-effort” is unicast approach.
 IP multicast: an extension to original IP protocol supports
 Dynamic and distributed group membership.
 Multiple group membership.
 Multiple send/receive nodes.
 Multicast Backbone (Mbone): Real-world implementation of
IP Multicast

Multicast Backbone
 It can be considered as internet radio orTV.
 User call up for movie and Mbone net view uncompressed
movies.
 It implement virtual overlay network on top of internet
 It consists of multicast islands connected by tunnels
Mbone
tunnel

Problems with Internet & solutions :
 TCP/UDP/IP suite provides best-effort, no guarantees on
expectation or variance of packet delay
 Performance deteriorate if links are congested (transoceanic)
 Most router implementations use only First-Come-First-Serve
(FCFS) packet processing and transmission scheduling

Problems and solutions
 Limited bandwidth
 Solution: Compression
 Packet Jitter
 Solution: Fixed/adaptive playout delay for Audio (example:
phone over IP)
 Packet loss
 Solution: FEC, Interleaving

Forward Error Control (FEC)
 FEC is a technique used for controlling
errors in data transmission over unreliable
or noisy communication channels.
 The central idea is the sender encodes their
message in a redundant way by using
an error-correcting code (ECC).
 Packet-level Forward Error Control (FEC)
for video streaming over a wireless
network.
 packet-level interleaving when combined
with FEC presents a remedy to time-
correlated error bursts, though it can
further increase delay.

Problem: Limited bandwidth Intro:
Digitalization
 Audio
x samples every second (x=frequency)
The value of each sample is rounded to a finite
number of values (for example 256).This is called
quantization.
 Video
Each pixel has a color
Each color has a value

Problem: Limited bandwidth
Need for compression
 Audio
 CD quality: 44100 samples per seconds with 16 bits per
sample, stereo sound
 44100*16*2 = 1.411 Mbps
 For a 3-minute song: 1.441 * 180 = 254 Mb = 31.75 MB
 Video
 For 320*240 images with 24-bit colors
 320*240*24 = 1843200/8000= 230KB/image
 15 frames/sec: 15*230KB = 3.456MB
 3 minutes of video: 3.456*180 = 622MB
254/8=31.75

Audio compression
 Several techniques
 GSM (13 kbps), G.729(8 kbps), G723.3(6.4 and 5.3kbps)
 MPEG 1 layer 3 (also known as MP3)
 Typical compress rates 96kbps, 128kbps, 160kbps
 Very little sound degradation
 If file is broken up, each piece is still playable
 Complex (psychoacoustic masking, redundancy reduction, and bit
reservoir buffering)
 3-minute song (128kbps) : 2.8MB

Discrete Cosine Transform (DCT)
 DCT brought blocks of 8x8 pixels information into frequency
domain.The low frequency means smoother image.
 Some data appeared more important to others. Because of some
properties of the human eye it isn't necessary give color
information for every pixel. So, to reduce image size by using
important data only. If the image does not change too rapidly in the
vertical direction, than apply DCT to the collection of eight values.
‫يالحظ‬ ‫االنسان‬ ‫نظر‬
‫اختالفات‬
low frequency
‫من‬ ‫اكثر‬
High frequency

Image compression: JPEG
 Divide digitized image in 8x8 pixel blocks
 Pixel blocks are transformed into frequency blocks using DCT
(Discrete CosineTransform).This is similar to FFT (Fast
FourierTransform).
 The quantization phase limits the precision of the frequency
coefficient.
 The encoding phase packs this information in a dense fashion
DCT

Square waves to Sine Waves
50
 Square sampled by 5 different coefficient frequency
High Frequency
Low Frequency

Square waves to Sine Waves
 If we take out number of high frequency to reduce amount of data
(Compression).

DCT with Quantization & Run length Encode

JPEG Compression

Video compression
 Popular techniques
 MPEG 1 for CD-ROM quality video (1.5Mbps)
 MPEG 2 for high quality DVD video (3-6 Mbps)
 MPEG 4 for object-oriented video compression

Video Compression: MPEG
 MPEG uses inter-frame encoding
 Exploits the similarity between consecutive frames
 Three frame types
 I frame: independent encoding of the frame (JPEG) :Coded without reference to
other frames.
 P frame: encodes difference relative to I-frame (predicted) : Coded with reference to
a previous reference frame (either I or P).
 B frame: encodes difference relative to interpolated frame: Coded with reference to
both previous and future reference frames (either I or P).
 Note that frames will have different sizes
 Complex encoding, e.g. motion of pixel blocks, scene changes, …
 Decoding is easier then encoding
 MPEG often uses fixed-rate encoding
I PB B BB B BP
55
Transmit Order
IBBPBBPBB

MPEG Compression (cont.)

MPEG System Streams
 Combine MPEG video and audio streams in a single
synchronized stream.
 Consists of a hierarchy with meta data at every level
describing the data:
 System level contains synchronization information
 Video level is organized as a stream of pictures group
 Pictures are organized in slices
 The motion vector isn't valid for the whole frame. Instead of this the
frame is divided into macro blocks of 16x16 pixels.

MPEG System Streams (cont.)

Problem: Packet Jitter
 Jitter:Variation in delay
Example
1
3
5 4 3 2Sender
No jitter
125 46
6
Receiver
Jitter
pkt 6
pkt 5

Dealing with packet jitter
 How does Phone over IP applications limit the effect of jitter?
 A sequence number is added to each packet
 A timestamp is added to each packet
 Playout is delayed
 Fixed playout delay

Dealing with packet jitter
Adaptive playout delay
 Objective is to use a value for p-r that tracks the network delay
performance as it varies during a transfer.
 The following formulas are used:
di = (1-u)di-1 + u(ri – ti) u=0.01 for example
i = (1-u)i-1 + u|ri-ti-di|
Where
ti is the timestamp of the ith packet (the time pkt i is sent)
ri is the time packet i is received
pi is the time packet i is played
di is an estimate of the average network delay
i is an estimate of the average deviation of the delay from the estimated average delay

Problem: Packet loss
 Loss is in a broader sense: packet never arrives or arrives later than
its scheduled playout time
 Since retransmission is inappropriate for RealTime applications,
FEC or Interleaving are used to reduce loss impact.

Recovering from packet loss
Forward Error Correction
 Send redundant encoded chunk every n chunks (XOR original
n chunks)
 If 1 packet in this group lost, can reconstruct
 If >1 packets lost, cannot recover
 Disadvantages
 The smaller the group size, the larger the overhead
 Playout delay increased

Recovering from packet loss Piggybacking
Lo-fi stream
 With one redundant low quality chunk per chunk, scheme can recover
from single packet losses

Recovering from packet loss Interleaving
 Divide 20 msec of audio data into smaller units of 5 msec each and
interleave.
 Upon loss, have a set of partially filled chunks
‫استرجاع‬
‫الباكتات‬
‫الخسرانه‬

Recovering from packet loss
Receiver-based Repair
 The simplest form: Packet repetition
 Replaces lost packets with copies of the packets that arrived
immediately before the loss.
 A more computationally intensive form: Interpolation
 UsesAudio before and after the loss to interpolate a suitable
packet to cover the loss .

Internet for Multimedia Traffic
 Another enhancements required.
 UDP is more suitable thanTCP for interactive traffic. But, it requires the
services of RTP.
 Real-timeTransport protocol (RTP):
 Application layer protocol designed to handle real time traffic
 RTP provides and-to-end transport services to real-time audio, video
and simulation data. So RTP used for data communication and RCTP
used for control signal communication.
 However, It does not provide QoS guarantees. RTP encapsulation is
only seen at the end systems: it is not seen by intermediate routers.

RTP runs on top of UDP
• RTP does not provide any mechanism to ensure timely delivery of data or
provide other quality of service guarantees.
• RTP encapsulation is only seen at the end systems: it is not seen by
intermediate routers.
• Routers providing best-effort service do not make any special effort to ensure that
RTP packets arrive at the destination in a timely matter.
69
RTP extend UDP and uses a even numbered
UDP port and the next number is used by a
companion protocol RealTimeTransport
Control protocol (RCTP).
• port numbers, IP addresses
• payload type identification
• packet sequence numbering
• time-stamping

RTP packet format
 PayloadType: 7 bits, providing 128 possible different types
of encoding; eg PCM, MPEG2 video, etc.
 Sequence Number: 16 bits; used to detect packet loss
 Timestamp: 32 bytes; gives the sampling instant of the first
audio/video byte in the packet; used to remove jitter
introduced by the network
 Synchronization Source identifier (SSRC): 32 bits; an id
for the source of a stream; assigned randomly by the source

Timestamp vs. Sequence No
 Timestamps relates packets to real time
 Timestamp value sampled from a media specific clock
 Sequence number relates packets to other packets
 Audio silence example :
 Consider audio data type
 What do you want to send during silence?
 Not sending anything
 Why might this cause problems?
 Other side needs to distinguish between loss and silence
 Receiver usesTimestamps and sequence No. to figure out what
happened .

RTP Control Protocol (RTCP)
 Used in conjunction with RTP. Used to exchange control information
between the sender and the receiver.
 Three reports are defined: Receiver reception, Sender, and Source
description.
 Reports contain statistics such as the number of packets sent, number
of packets lost, inter-arrival jitter
 Typically, limit the
RTCP bandwidth to 5%.
Approximately one
sender report for three
receiver reports

Feedback During Media Streaming
 Feedback can be used to control performance
 Sender may modify its transmissions based on feedback

Streaming Stored Multimedia Example
 Audio/Video file is segmented and sent over eitherTCP or
UDP, public segmentation protocol: Real-Time Protocol
(RTP)
 User interactive control is provided, e.g. the public protocol
RealTime Streaming Protocol (RTSP)

Streaming Stored Multimedia Example
 Helper Application: displays content, which is typically
requested via aWeb browser; e.g. RealPlayer; typical functions:
 Decompression
 Jitter removal
 Error correction: use redundant packets to be used for reconstruction
of original stream
 GUI for user control

Streaming from a Web Server (cont)
 Alternative: set up connection between server and player, then
download.
 Web browser requests and receives a Meta File.
(a file describing the object) instead of receiving the file itself;
 Browser launches the appropriate Player and passes it the Meta
File;
 Player sets up aTCP connection with a streaming server and
downloads the file.

Options when using a streaming server
 Use UDP, and Server sends at a rate (Compression andTransmission) appropriate for
client; to reduce jitter, Player buffers initially for 2-5 seconds, then starts display.
 UseTCP, and sender sends at maximum possible rate underTCP; retransmit when error
is encountered; Player uses a much large buffer to smooth delivery rate ofTCP

Real Time Streaming Protocol (RTSP)
 For user to control display: rewind, fast forward, pause, resume, etc…
 Out-of-band protocol (uses two connections, one for control messages
(Port 554) and one for media stream)
 RFC 2326 permits use of eitherTCP or UDP for the control messages
connection, sometimes called the RTSP Channel
 As before, meta file is communicated to web browser which then
launches the Player; Player sets up an RTSP connection for control
messages in addition to the connection for the streaming media

Meta File Example
<title>Twister</title>
<session>
<group language=en lipsync>
<switch>
<track type=audio
e="PCMU/8000/1"
src = "rtsp://audio.example.com/twister/audio.en/lofi">
<track type=audio
e="DVI4/16000/2" pt="90 DVI4/8000/1"
src="rtsp://audio.example.com/twister/audio.en/hifi">
</switch>
<track type="video/jpeg"
src="rtsp://video.example.com/twister/video">
</group>
</session>

RTSP Operations
C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0
Transport: rtp/udp; compression; port=3056; mode=PLAY
S: RTSP/1.0 200 1 OK
Session 4231
C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231 Range: npt=0-
S: RTSP/1.0 200 1 OK
Session 4231
C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231 Range: npt=37
S: RTSP/1.0 200 1 OK
Session 4231
C:TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session:4231
S: RTSP/1.0 200 1 OK
Session 4231

References
 Streaming in Mobile Networks
 Jun-Zhao Sun, Douglas Howie,Antti Koivisto and Jaakko Sauvola,“A
Hierarchical Framework Model of Mobile Security,” MediaTeam Oulu, infotech
Oulu, Finland, 2001, pp.A56-A59
 Claudio Cicconetti, Luciano Lenzini and Enzo Mingozzi,“Quality of Service
Support inWireless Networks,” University of Pisa, Nokia Research Center,
Italy,April 2006, pp. 50-55
 http://www.ece.umd.edu/~minwu/public_paper/Jnl/0603MuserVideo_IEEEfinal_
NetMag.pdf
 http://research.microsoft.com/china/papers/Scalable_Video_Coding_Transport.pdf
 http://islab.cis.nctu.edu.tw/seminar/doc/1.pdf
 http://www.tml.tkk.fi/Opinnot/T-
110.5120/2005/slides/06a.Streaming%20Multimedia%20Architecture,%20Codecs%
20and%20QoS2.pdf
 http://www.hindawi.com/journals/am/2009/982867/

Multimedia Network