LECTURE NOTES Multimedia Data Processing
LECTURE NOTES Multimedia Data Processing
Multimedia Introduction
Multimedia is an in teractive media and provides multiple ways to represent information to the user in
a powerful manner. It provides an interaction between users and digital information. It is a medium of
communication. Some of the sectors where multimedia is used extensively are education, training,
reference material, business presentations, advertising and documentaries.
Definition of Multimedia
By definition Multimedia is a representation of information in an attractive and interactive manner
with the use of a combination of text, audio, video, graphics and animation. In other words we can say
that Multimedia is a computerized method of presenting information combining textual data, audio,
visuals (video), graphics and animations. For examples: E-Mail, Yahoo Messenger, Video
Conferencing, and Multimedia Message Service (MMS).
Multimedia as name suggests is the combination of Multi and Media that is many types of media
(hardware/software) used for communication of information.
Components of Multimedia
Following are the common components of multimedia:
Text- All multimedia productions contain some amount of text. The text can have various types
of fonts and sizes to suit the profession presentation of the multimedia software.
Graphics- Graphics make the multimedia application attractive. In many cases people do not
like reading large amount of textual matter on the screen. Therefore, graphics are used more
often than text to explain a concept, present background information etc. There are two types of
Graphics:
o Bitmap images- Bitmap images are real images that can be captured from devices such
as digital cameras or scanners. Generally bitmap images are not editable. Bitmap
images require a large amount of memory.
o Vector Graphics- Vector graphics are drawn on the computer and only require a small
amount of memory. These graphics are editable.
Audio- A multimedia application may require the use of speech, music and sound effects.
These are called audio or sound element of multimedia.Speech is also a perfect way for
teaching. Audio are of analog and digital types. Analog audio or sound refers to the original
sound signal. Computer stores the sound in digital form. Therefore, the sound used in
multimedia application is digital audio.
Video- The term video refers to the moving picture, accompanied by sound such as a picture in
television. Video element of multimedia application gives a lot of information in small duration
of time. Digital video is useful in multimedia application for showing real life objects. Video
have highest performance demand on the computer memory and on the bandwidth if placed on
the internet. Digital video files can be stored like any other files in the computer and the quality
of the video can still be maintained. The digital video files can be transferred within a computer
network. The digital video clips can be edited easily.
Animation- Animation is a process of making a static image look like it is moving. An
animation is just a continuous series of still images that are displayed in a sequence. The
animation can be used effectively for attracting attention. Animation also makes a presentation
light and attractive. Animation is very popular in multimedia application
Applications of Multimedia
Following are the common areas of applications of multimedia.
Multimedia in Business- Multimedia can be used in many applications in a business. The
multimedia technology along with communication technology has opened the door for
information of global wok groups. Today the team members may be working anywhere and can
work for various companies. Thus the work place will become global. The multimedia network
should support the following facilities:
o Voice Mail
o Electronic Mail
o Multimedia based FAX
o Office Needs
o Employee Training
o Sales and Other types of Group Presentation
o Records Management
Multimedia in Marketing and Advertising- By using multimedia marketing of new products
can be greatly enhanced. Multimedia boost communication on an affordable cost opened the
way for the marketing and advertising personnel. Presentation that have flying banners, video
transitions, animations, and sound effects are some of the elements used in composing a
multimedia based advertisement to appeal to the consumer in a way never used before and
promote the sale of the products.
Multimedia in Entertainment- By using multimedia marketing of new products can be
greatly enhanced. Multimedia boost communication on an affordable cost opened the way for
the marketing and advertising personnel. Presentation that have flying banners, video
transitions, animations, and sound effects are some of the elements used in composing a
multimedia based advertisement to appeal to the consumer in a way never used before and
promote the sale of the products.
Multimedia in Education- Many computer games with focus on education are now available.
Consider an example of an educational game which plays various rhymes for kids. The child
can paint the pictures, increase reduce size of various objects etc apart from just playing the
rhymes.Several other multimedia packages are available in the market which provide a lot of
detailed information and playing capabilities to kids.
Multimedia in Bank- Bank is another public place where multimedia is finding more and
more application in recent times. People go to bank to open saving/current accounts, deposit
funds, withdraw money, know various financial schemes of the bank, obtain loans etc. Every
bank has a lot of information which it wants to impart to in customers. For this purpose, it can
use multimedia in many ways. Bank also displays information about its various schemes on a
PC monitor placed in the rest area for customers. Today on-line and internet banking have
become very popular. These use multimedia extensively. Multimedia is thus helping banks give
service to their customers and also in educating them about banks attractive finance schemes.
Multimedia in Hospital- Multimedia best use in hospitals is for real time monitoring of
conditions of patients in critical illness or accident. The conditions are displayed continuously
on a computer screen and can alert the doctor/nurse on duty if any changes are observed on the
screen. Multimedia makes it possible to consult a surgeon or an expert who can watch an
ongoing surgery line on his PC monitor and give online advice at any crucial juncture.
In hospitals multimedia can also be used to diagnose an illness with CD-ROMs/ Cassettes/ DVDs full
of multimedia based information about various diseases and their treatment.Some hospitals extensively
use multimedia presentations in training their junior staff of doctors and nurses. Multimedia displays
are now extensively used during critical surgeries.
Multimedia Pedagogues- Pedagogues are useful teaching aids only if they stimulate and
motivate the students. The audio-visual support to a pedagogue can actually help in doing so. A
multimedia tutor can provide multiple numbers of challenges to the student to stimulate his
interest in a topic. The instruction provided by pedagogue have moved beyond providing only
button level control to intelligent simulations, dynamic creation of links, composition and
collaboration and system testing of the user interactions.
Communication Technology and Multimedia Services- The advancement of high computing
abilities, communication ways and relevant standards has started the beginning of an era where
you will be provided with multimedia facilities at home. These services may include:
o Basic Television Services
o Interactive entertainment
o Digital Audio
o Video on demand
o Home shopping
o Financial Transactions
o Interactive multiplayer or single player games
o Digital multimedia libraries
o E-Newspapers, e-magazines
Multimedia Systems
Multimedia Hardware
Most of the computers now-a-days come equipped with the hardware components required to
develop/view multimedia applications. Following are the various categories in which we can define the
various types of hardwares required for multimedia applications.
ProcessorThe heart of any multimedia computer is its processor. Today Core 15 or higher
processor is recommended for a multimedia computer.
o CPU is considered as the brain of the computer.
o CPU performs all types of data processing operations.
o It stores data, intermediate result and instructions (program).
o It controls the operations of all parts of computer.
Memory and Storage Devices - You need memory for storing various files used during
production, original audio and video clips, edited pieces and final mined pieces. You also need
memory for backup of your project files.
o Primary Memory- Primary memory holds only those data and instructions on which
computer is currently working. It has limited capacity and data gets lost when power is
switched off. It is generally made up of semiconductor device. These memories are not
as fast as registers. The data and instructions required to be processed earlier reside in
main memory. It is divided into two subcategories RAM and ROM.
o Flash Memory- Cache memory is a very high speed semiconductor memory, which
can speed up CPU. It acts as a buffer between the CPU and main memory. It is used to
hold those parts of data and program which are most frequently used by CPU. The parts
of data and programs are transferred from disk to cache memory by operating system,
from where CPU can access them.
o Secondary Memory: This type of memory is also known as external memory or non-
volatile. It is slower than main memory. These are used for storing Data/Information
permanently. CPU directly does not access these memories; instead they are accessed
via input-output routines. Contents of secondary memories are first transferred to main
memory and then CPU can access it. For example, disk, CD-ROM, DVD, etc.
Input Devices - Following are the various types of input devices which are used in multimedia
systems.
o Keyboard- Most common and very popular input device is keyboard. The keyboard
helps in inputting the data to the computer. The layout of the keyboard is like that of
traditional typewriter, although there are some additional keys provided for performing
some additional functions. Keyboards are of two sizes 84 keys or 101/102 keys, but
now 104 keys or 108 keys keyboard is also available for Windows and Internet. The
keys are following:
Sr.
Keys Description
No.
Typing These keys include the letter keys (A-Z) and digits keys (0-9)
1
Keys which generally give same layout as that of typewriters.
The twelve functions keys are present on the keyboard. These are
Function
3 arranged in a row along the top of the keyboard. Each function
Keys
key has unique meaning and is used for some specific purpose.
Special
Keyboard also contains some special purpose keys such as Enter,
5 Purpose
Shift, Caps Lock, Num Lock, Space bar, Tab, and Print Screen.
Keys
o Joystick - Joystick is also a pointing device, which is used to move cursor position on a
monitor screen. It is a stick having a spherical ball at its both lower and upper ends. The
lower spherical ball moves in a socket. The joystick can be moved in all four directions.
The function of joystick is similar to that of a mouse. It is mainly used in Computer
Aided Designing (CAD) and playing computer games.
o Light Pen - Light pen is a pointing device, which is similar to a pen. It is used to select
a displayed menu item or draw pictures on the monitor screen. It consists of a photocell
and an optical system placed in a small tube. When light pen's tip is moved over the
monitor screen and pen button is pressed, its photocell sensing element detects the
screen location and sends the corresponding signal to the CPU.
o Track Ball - Track ball is an input device that is mostly used in notebook or laptop
computer, instead of a mouse. This is a ball, which is half inserted and by moving
fingers on ball, pointer can be moved.Since the whole device is not moved, a track ball
requires less space than a mouse. A track ball comes in various shapes like a ball, a
button and a square.
o Scanner - Scanner is an input device, which works more like a photocopy machine. It is
used when some information is available on a paper and it is to be transferred to the
hard disc of the computer for further manipulation. Scanner captures images from the
source which are then converted into the digital form that can be stored on the disc.
These images can be edited before they are printed.
Multimedia authoring
Definition
Multimedia authoring is a process of assembling different types of media contents like text, audio,
image, animations and video as a single stream of information with the help of various software tools
available in the market. Multimedia authoring tools give an integrated environment for joining together
the different elements of a multimedia production. It gives the framework for organizing and editing
the components of a multimedia project. It enables the developer to create interactive presentation by
combining text, audio, video, graphics and animation.
Features of Authoring Tools
Editing Features- Most authoring environment and packages exhibit capabilities to create edit
and transform different kinds of media that they support. For example, Macromedia Flash
comes bundled with its own sound editor. This eliminates the need for buying dedicated
software to edit sound data. So authoring systems include editing tools to create, edit and
convert multimedia components such as animation and video clips.
Organizing Features- The process of organization, design and production of multimedia
involve navigation diagrams or storyboarding and flowcharting. Some of the authoring tools
provide a system of visual flowcharting or overview facility to showcase your project's
structure at a macro level. Navigation diagrams help to organize a project. Many web-authoring
programs like Dreamweaver include tools that create helpful diagrams and links among the
pages of a website.
Visual programming with icons or objects- It is simplest and easiest authoring process. For
example, if you want to play a sound then just clicks on its icon.
Programming with a scripting language- Authoring software offers the ability to write
scripts for software to build features that are not supported by the software itself. With script
you can perform computational tasks - sense user input and respond, character creation,
animation, launching other application and to control external multimedia devices.
Document Development tools- Some authoring tools offers direct importing of pre-formatted
text, to index facilities, to use complex text search mechanism and to use hypertext link-ing
tools.
Interactivity Features- Interactivity empowers the end users to control the content and flow of
information of the project. Authoring tools may provide one or more levels of interactivity.
Simple branching- Offers the ability to go to another section of the multimedia production.
Conditional branching- Supports a go to base on the result of IF-THEN decision or events.
Playback Features- When you are developing multimedia project, you will continousally
assembling elements and testing to see how the assembly looks and performs. Therefore
authoring system should have playback facility.
Supporting CD-ROM or Laser Disc Sources- This software allows over all control of CD-
drives and Laser disc to integrate audio, video and computer files. CD-ROM drives, video and
laserdisc sources are directly controlled by authoring programs.
Supporting Video for Windows- Videos are the right media for your project which are stored
on the hard disk. Authoring software has the ability to support more multimedia elements like
video for windows.
Hypertext- Hypertext capabilities can be used to link graphics, some animation and other text.
The help system of window is an example of hypertext. Such systems are very useful when a
large amount of textual information is to be represented or referenced.
Cross-Platform Capability- Some authoring programs are available on several platforms and
provide tools for transforming and converting files and programs from one to the other.
Run-time Player for Distribution- Run time software is often included in authoring software
to explain the distribution of your final product by packaging playback software with content.
Some advanced authoring programs provide special packaging and run-time distribution for use
with devices such as CD-ROM.
Internet Playability- Due to Web has become a significant delivery medium for multimedia,
authoring systems typically provide a means to convert their output so that it can be delivered
within the context of HTML or DHTML.
Authoring Tools Classification
Card or Page based authoring tools
In these authoring systems, elements are organized as pages of a book or a stack of cards. In the book
or stack there are thousand of pages or cards available. These tools are best used when the bulk of your
content consists of elements that can be viewed individually, for example the pages of a book or file
cards in card file. You can jump from page to page because all pages can be interrelated. In the
authoring system you can organize pages or cards in the sequences manner. Every page of the book
may contain many media elements like sounds, videos and animations.
One page may have a hyperlink to another page that comes at a much later stage and by clicking on the
same you might have effectively skipped several pages in between. Some examples of card or page
tools are:
Hypercard (Mac)
Tool book (Windows)
PowerPoint (Windows)
Supercard (Mac)
Advantages
Following are the advantages of card based authoring tools.
Easy to understand.
One screen is equal to 1card or 1page.
Easy to use as these tools provide template.
Short development time.
Disadvantages
Following are the disadvantages of card based authoring tools.
Some run only on one platform.
Tools not as powerful as equivalent stand alones.
Icon based or Event driven authoring tools
Icon-based tools give a visual programming approach to organizing and presenting multimedia. First
you build a structure or flowchart of events, tasks and decisions by dragging appropriate icons from a
library. Each icon does a specific task, for example- plays a sound, open an image etc. The flowchart
graphically displays the project's logic. When the structure is built you can add your content text,
graphics, animation, video movies and sounds. A nontechnical multimedia author can also build
sophisticated applications without scripting using icon based authoring tools. Some examples of icon
based tools are:
Authorware Professional (Mac/Windows)
Icon Author (Windows)
Advantages:
Following are the advantages of icon/event based authoring tools.
Clear Structure.
Easy editing and updating
Disadvantages:
Following are the disadvantages of icon/event based authoring tools.
Difficult to learn.
Expensive.
Time based authoring tools
Time based authoring tools allow the designer to arrange various elements and events of the
multimedia project along a well defined time line. By time line, we simply mean the passage of time.
As the time advances from starting point of the project, the events begin to occur, one after another.
The events may include media files playback as well as transition from one portion of the project to
another. The speed at which these transitions occur can also be accurately controlled. These tools are
best to use for those projects, wherein the information flow can be directed from beginning to end
much like the movies. Some example of Time based tools are:
Macromedia's Director
Macromedia Flash
Advantages
Following are the advantages of time based authoring tools.
Good for creating animation.
Branching, user control, interactivity facilities.
Disadvantages
Following are the disadvantages of time based authoring tools.
Expensive
Large file size
Steep learning curve to understand various features.
Object-Oriented authoring tools:
Object oriented authoring tools support environment based on object. Each object has the following
two characteristics:
1. State or Attributes - The state or attributes refers to the built in characteristics of an object.
For example, a color T.V has the following attributes:
o Color receiver
o Volume control
o Picture control
o 128 channels
o Remote control unit
2. Behavior or Operations - The behavior or operations of an object refers to its action. For
example, a T.V can behave in any of the following manner at a given point of time:
o Switched on
o Switched off
o Displays picture and sound from
A TV cable connection
A TV transmitter
A DVD
A VCR
In these systems, multimedia elements events are often treated as objects that live in a hierarchical
order of parent and child relationships. These objects use messages passed among them to do things
according to the properties assigned to them. For example, a video object will likely have a duration
property i.e how long the video plays and a source property that is the location of the video file. This
video object will likely accept commands from the system such as play and stop. Some examples of
the object oriented tools are:
o mTropolis (Mac/Windows)
o Apple Media Tool (Mac/Windows)
o Media Forge (Windows)
Quantization of Audio
Quantization is a process to assign a discrete value from a range of possible values to each sample.
Number of samples or ranges of values are dependent on the number of bits used to represent each
sample. Quantization results in stepped waveform resembling the source signal.
Quantization Error/Noise - The difference between sample and the value assigned to it is
known as quantization error or noise.
Signal to Noise Ratio (SNR) - Signal to Ratio refers to signal quality versus quantization error.
Higher the Signal to Noise ratio, the better the voice quality. Working with very small levels
often introduces more error. So instead of uniform quantization, non-uniform quantization is
used as companding. Companding is a process of distorting the analog signal in controlled way
by compressing large values at the source and then expanding at receiving end before
quantization takes place.
Transmission of Audio
In order to send the sampled digital sound/ audio over the wire that it to transmit the digital audio, it is
first to be recovered as analog signal. This process is called de-modulation.
PCM Demodulation - PCM Demodulator reads each sampled value then apply the analog
filters to suppress energy outside the expected frequency range and outputs the analog signal as
output which can be used to transmit the digital signal over the network.
AUDIO CODING
Audio coding is used to obtain compact digital representations of high fidelity audio signals for the
purpose of efficient transmission of signal or storage of signal. Objective in audio coding is to
represent the signal with minimum number of bits while achieving transparent signal reproduction that
is generating output audio that cannot be distinguished from the original input.
Pulse code modulation (PCM)
PCM is a method or technique for transmitting analog data in a digital and binary way independent of
the complexity of the analog waveform.All types of analog data like video, voice; music etc. can be
transferred using PCM. It is the standard form for digital audio in computers. It is used by Blu-ray,
DVD and Compact Disc formats and other systemds such as digital telephone systems. PCM consists
of three steps to digitize an analog signal.
Sampling - Sampling is the process of reading the values of the filtered analog signal at regular
intervals. Suppose analog signal is sampled every Ts seconds. Ts are referred to as the sampling
interval.
Fs=1/Ts
It is called the sampling rate or sampling frequency.
Quantization - Quantization is the process of assigning a discrete value from a range of
possible values to each obtained sample. Possible values will depend on the number of bits
used to represent each sample. Quantization can be achieved by either rounding the signal up
and down to the nearest available value which is lower than the actual sample. The difference
between the sample and the value assigned to it is known as the quantization noise or
quantization error. Quantization noise can be reduced by increasing the number of quantization
intervals or levels because the difference between the input signal amplitude and the
quantization interval decreases as the number of quantization intervals increases.
Binary encoding - Binary encoding is the process of representing the sampled values as a
binary number in the range 0 to n. The value of n is choosen as a power of 2, considering the
accuracy.