1|Page
Decision Support Systems
Decision support systems (DSS) are interactive software-based systems
intended to help managers in decision-making by accessing large volumes of
information generated from various related information systems involved in
organizational business processes, such as office automation system,
transaction processing system, etc.
DSS uses the summary information, exceptions, patterns, and trends using the
analytical models. A decision support system helps in decision-making but
does not necessarily give a decision itself. The decision makers compile useful
information from raw data, documents, personal knowledge, and/or business
models to identify and solve problems and make decisions.
There are two types of decisions - programmed and non-programmed decisions.
Programmed decisions are basically automated processes, general routine
work, where −
These decisions have been taken several times.
These decisions follow some guidelines or rules.
For example, selecting a reorder level for inventories, is a programmed
decision.
Non-programmed decisions occur in unusual and non-addressed situations,
so −
It would be a new decision.
There will not be any rules to follow.
These decisions are made based on the available information.
These decisions are based on the manger's discretion, instinct,
perception and judgment.
For example, investing in a new technology is a non-programmed decision.
Decision support systems generally involve non-programmed decisions.
Therefore, there will be no exact report, content, or format for these systems.
Reports are generated on the fly.
2|Page
Decision-Making Process
Following are the important steps of the decision-making process. Each step
may be supported by different tools and techniques.
Step 1 − Identification of the Purpose of the Decision
In this step, the problem is thoroughly analyzed. There are a couple of
questions one should ask when it comes to identifying the purpose of the
decision.
What exactly is the problem?
Why the problem should be solved?
3|Page
Who are the affected parties of the problem?
Does the problem have a deadline or a specific time-line?
Step 2 − Information Gathering
A problem of an organization will have many stakeholders. In addition, there
can be dozens of factors involved and affected by the problem.
In the process of solving the problem, you will have to gather as much as
information related to the factors and stakeholders involved in the problem. For
the process of information gathering, tools such as 'Check Sheets' can be
effectively used.
Step 3 − Principles for Judging the Alternatives
In this step, the baseline criteria for judging the alternatives should be set up.
When it comes to defining the criteria, organizational goals as well as the
corporate culture should be taken into consideration.
As an example, profit is one of the main concerns in every decision making
process. Companies usually do not make decisions that reduce profits, unless
it is an exceptional case. Likewise, baseline principles should be identified
related to the problem in hand.
Step 4 − Brainstorm and Analyze the Choices
For this step, brainstorming to list down all the ideas is the best option. Before
the idea generation step, it is vital to understand the causes of the problem and
prioritization of causes.
For this, you can make use of Cause-and-Effect diagrams and Pareto Chart
tool. Cause-and-Effect diagram helps you to identify all possible causes of the
problem and Pareto chart helps you to prioritize and identify the causes with
the highest effect.
Then, you can move on generating all possible solutions (alternatives) for the
problem in hand.
Step 5 − Evaluation of Alternatives
Use your judgment principles and decision-making criteria to evaluate each
alternative. In this step, experience and effectiveness of the judgment principles
come into play. You need to compare each alternative for their positives and
negatives.
4|Page
Step 6 − Select the Best Alternative
Once you go through from Step 1 to Step 5, this step is easy. In addition, the
selection of the best alternative is an informed decision since you have already
followed a methodology to derive and select the best alternative.
Step 7 − Execute the decision
Convert your decision into a plan or a sequence of activities. Execute your plan
by yourself or with the help of subordinates.
Step 8 − Evaluate the Results
Evaluate the outcome of your decision. See whether there is anything you
should learn and then correct in future decision making. This is one of the best
practices that will improve your decision-making skills.
Attributes of a DSS
Adaptability and flexibility
High level of Interactivity
Ease of use
Efficiency and effectiveness
Complete control by decision-makers
Ease of development
Extendibility
Support for modeling and analysis
Support for data access
Standalone, integrated, and Web-based
Characteristics of a DSS
Support for decision-makers in semi-structured and unstructured
problems.
Support for managers at various managerial levels, ranging from top
executive to line managers.
5|Page
Support for individuals and groups. Less structured problems often
requires the involvement of several individuals from different
departments and organization level.
Support for interdependent or sequential decisions.
Support for intelligence, design, choice, and implementation.
Support for variety of decision processes and styles.
DSSs are adaptive over time.
Benefits of DSS
Improves efficiency and speed of decision-making activities.
Increases the control, competitiveness and capability of futuristic
decision-making of the organization.
Facilitates interpersonal communication.
Encourages learning or training.
Since it is mostly used in non-programmed decisions, it reveals new
approaches and sets up new evidences for an unusual decision.
Helps automate managerial processes.
Components of a DSS
Following are the components of the Decision Support System −
Database Management System (DBMS) − To solve a problem the
necessary data may come from internal or external database. In an
organization, internal data are generated by a system such as TPS and
MIS. External data come from a variety of sources such as newspapers,
online data services, databases (financial, marketing, human resources).
Model Management System − It stores and accesses models that
managers use to make decisions. Such models are used for designing
manufacturing facility, analyzing the financial health of an organization,
forecasting demand of a product or service, etc.
Support Tools − Support tools like online help; pulls down menus, user
interfaces, graphical analysis, error correction mechanism, facilitates the user
interactions with the system.
6|Page
Application of DSS:
For Inventory Management
To identify Customer buying patterns
Optimizing discounts/price markdowns.
Targeting direct- mail marketing customers
Store location
Evaluation of Potential drilling sites.
Flight Scheduling etc
Web-Based Decision Support System
The DSS based on Web & Internet can support decision making, by providing
online access to various databases and information pools along with the
software for data analysis.
Customer Decision Support Systems: In such DSS customers using web
interface to self-serve using the DSS tools of the sponsoring company.
General Architecture of Conventional Web-Based DSS Web technologies can be
used to implement any category or type of DSS. Web-based means the entire
application is implemented using Web technologies; Web-enabled means key
parts of an application like a database remain on a legacy system, but the
application can be accessed from a Web-based component and displayed in a
browser. Web-based DSS deliver decision support information or decision
support tools to a manager or business analyst using a “thin-client” Web
browser like Netscape Navigator or Internet Explorer that is accessing the
global Internet or a corporate intranet. The computer server that is hosting the
DSS application is linked to the user’s computer by a network with the TCP/IP
protocol.
Group Decision Support System
Group Decision Support System (GDSS) is a decision support system that
provides support in decision making by a group of people. It facilitates the free
flow and exchange of ideas and information among the group members.
Decisions are made with a higher degree of consensus and agreement resulting
in a dramatically higher likelihood of implementation.
7|Page
Following are the available types of computer based GDSSs −
Decision Network − this type helps the participants to communicate
with each other through a network or through a central database.
Application software may use commonly shared models to provide
support.
Decision Room − Participants are located at one place, i.e. the decision
room. The purpose of this is to enhance participant's interactions and
decision-making within a fixed period of time using a facilitator.
Teleconferencing − Groups are composed of members or sub groups
that are geographically dispersed; teleconferencing provides interactive
connection between two or more decision rooms. This interaction will
involve transmission of computerized and audio visual information.
Features of Group Decision Support System (GDSS)
Ease of Use: It consists of an interactive interface that makes working
with GDSS simple and easy.
8|Page
Better Decision Making: It provides the conference room setting and
various software tools that facilitate users at different locations to make
decisions as a group resulting in better decisions.
Emphasis on Semi-structured and Unstructured Decisions: It
provides important information that assists middle and higher level
management in making semi-structured and unstructured decisions.
Specific and General Support: The facilitator controls the different
phases of the group decision support system meeting (idea generation,
discussion, voting and vote counting etc.) what is displayed on the
central screen and the type of ranking and voting that takes place, etc. In
addition, the facilitator also provides general support to the group and
helps them to use the system.
Supports all Phases of the Decision Making: It can support all the four
phases of decision making, viz intelligence, design, choice and
implementation.
Supports Positive Group Behavior: In a group meeting, as participants
can share their ideas more openly without the fear of being criticized,
they display more positive group behavior towards the subject matter of
the meeting.
Group Decision Support System (GDSS) Software Tools
Group decision support system software tools helps the decision makers in
organizing their ideas, gathering required information and setting and ranking
priorities. Some of these tools are as follows:
Electronic Questionnaire: The information generated using the
questionnaires helps the organizers of the meeting to identify the issues
that need immediate attention, thereby enabling the organizers to create
a meeting plan in advance.
Electronic Brainstorming Tools: It allows the participants to
simultaneously contribute their ideas on the subject matter of the
meeting. As identity of each participant remains secret, individuals
participate in the meeting without the fear of criticism.
Idea Organizer: It helps in bringing together, evaluating and
categorizing the ideas that are produced during the brainstorming
activity.
9|Page
Tools for Setting Priority: It includes a collection of techniques, such
as simple voting, ranking in order and some weighted techniques that
are used for voting and setting priorities in a group meeting.
Policy Formation Tool: It provides necessary support for converting the
wordings of policy statements into an agreement.
Implementation of MIS
Implementation of a system is as much important as the creation of it.
Implementation can easily destroy the good work done in the earlier phases
and bring the system to a standstill. Implementation requires technical and
managerial skills as the implementers work as change agents. Implementation
is also a process that has a series of sequential steps which culminates in
making operational the new system.
Implementation as an activity has to be carefully managed. It requires client
interaction at every stage. The implementers need the full support and
cooperation of the client and the IS department functionaries to successfully
execute the implementation of information systems. In order to help them
perform this task of implementation smoothly, a series of predefined steps are
followed. These implementation tasks are as follows:
Implementation Plan
It is the series of action-oriented steps planned for making the implementation
smooth. It normally involves the following steps:
1. Creating a master schedule of the implementation activities
2. Setting timelines for critical and non-critical activities
3. Identifying major bottlenecks and their solutions
4. Communication of the plan.
This step is required to help the user community to understand the time frame
for installation of the new system. Communication plays a vital role in the
implementation and without proper communication especially, from the top
management on the installation and implementation of the new system the
change management will be difficult. Resistance to change related issues will
come to the fore making the difficult task of implementation more difficult.
Communication of the plan of implementation to the user community helps the
users to prepare for the change and makes them mentally prepared for it. The
10 | P a g e
communication is required to be formal so that rumors cannot be spread about
the system. The communication process may itself be in several phases. The
top level can communicate the general intent of the new system and then
detailed briefings to staff may be left to the divisional heads. The
communication process also indicates (indirectly), the role each employee is
required to play in the implementation process.
Organizing the MIS Department
The MIS department will be the custodian of the new system. Hence, they have
to be gear up to support the new system. Organization of the department is
therefore necessary before the new system becomes operational. The roles of
each member of the MIS department have to be clearly laid out before the new
system becomes operational. Effort is made to ensure that the role of the MIS
staff is understood by each member of the organization. Training is provided to
those who need training on the new system so that they in turn can help
others. This process of organizing the MIS department starts much before the
actual implementation process begins as it entails some hiring and training
which requires some lead time. The organization is done in such a timeframe
that staff is available when the actual implementation starts. This enables the
MIS staff to provide support to the implementation team, when the
implementation process starts. This will also help the MIS staff to understand
the nitty-gritty of the new system as they will be able to get a hands-on
experience in the implementation of the new system.
Selection and Procurement of Hardware
This step of the implementation process is an important step as it involves
huge investments. Proper care is taken to ensure that the organization gets the
best deal from such selection and procurement of the hardware. The process of
selection and procurement of hardware also varies greatly from firm to firm
depending on the size of the firm and the sector in which it operates the type of
management. However, the following procedure is followed:
1. Preparation of vendor list-a list of reliable vendors is prepared. This list
of vendors may be prepared after analyzing the vendor management
experience of the organization with different vendors or may be prepared
based on some accepted list of vendors in that business space prepared
by some organization of repute or some industry body/regulatory body.
The vendors that are selected to be part of the list are chosen carefully
after a thorough checking of their credentials and goodwill in the market.
11 | P a g e
This is essential as the vendor relationship is based on trust and
compromise and not only on the basis of strict commercial terms.
2. Preparation of RFP-the implementation team must prepare the request
for proposal document based on their understanding of the hardware
requirement of the new system. The RFP must have complete technical
details about the required hardware systems including specifications,
format, performance expectation, and warranty and service quality
requirements. This document is prepared by the implementers in
consultation with the development team, management of the
organization and the MIS team of the organization so that the need for
each specification is well established and there is no scope for any
difference of opinion. The consultative process results in the RFP which
is a technical document. The RFP also has commercial details which the
implementation team prepares in consultation with the management of
the organization. The RFP is a quasi-legal document in some countries
and proper legal opinion is normally sought before sending it to the
enlisted vendors.
3. Request for bids/proposal to select vendors-after the RFP is prepared
it is sent by some mode of communication to the enlisted set of vendors.
The communication medium can be an open advertisement in print or
electronic media or may be in the form of a letter to the vendors with a
deadline for submission of the proposal.
4. Evaluation of RFP-this is a difficult process. After bids are received
before the deadline, they are checked (preliminary check) for basic errors.
Those found to be prima facie proper are then evaluated. Several
methods of evaluation exist. The evaluation could be on the basis of cost
alone or quality alone or may be a mix of both cost and quality. Typically,
a score based system of evaluation is used to rank the vendors'
proposals. Scores are assigned to each attribute of a vendor's proposal
like cost, goodwill, track record and service quality guarantee. Based on
the weight age given to each attribute a composite score is prepared,
which is used to evaluate the proposals. Whatever tJ1emethodology for
evaluating the proposal, one must take care to apply the same evaluation
criteria to all proposals. Different yardsticks should not be applied to
different proposals.
5. Selection of vendor-based on the evaluation a single vendor or a select
set of vendors are chosen for delivery of hardware. Contract negotiations
12 | P a g e
and price negotiations are held with this select group of vendors and
following the successful completion of the negotiations the final contract
will be signed.
Procurement of Software
The new system being implemented will have been created based on
assumptions of operating environment of the organization. Procurement of
system software is done on similar lines as the procurement of hardware. The
only difference in the case of procurement of software is that the choice of what
software to purchase is already made at the design stage of the system
development and hence, the RFP preparation process is straightforward. The
implementation team need not prepare the specification for the system
software. They only need to procure the system software that the new system is
designed to run on. The rest of the process is almost similar to the hardware
procurement process.
Creating the Database
The new system to be implemented will have data stores. In modern systems,
data stores are databases. These databases are
relational database management systems, which is a separate application
software package. The database has to be created and structures inside the
database have to be created in order to enable it to store data. The
implementation team creates the database, its structures and rules so that the
application system being implemented can be plugged into the database and
start working.
Training of Users
Implementation is a larger issue than installation. The new system may get
installed but without proper training of users, it may not be of good use.
Implementation is a larger concept and focuses on the installation and hand-
holding part of the transition process. A training needs assessment is done to
understand the training needs of the users. A training programme is planned
and the required training given to users. This is an important part of the
implementation process and helps in reducing the resistance to change related
behavior among the user community. The training also helps users to
appreciate the new features of the new system and helps build trust and
appreciation for the new system.
13 | P a g e
Creating Physical Infrastructure
The new system being implemented may require a physical infrastructure. The
implementation team must ensure that the system performance must not
suffer due to infrastructure bottlenecks. The implementers will have to use
their persuasive skills and convince the management of the organization to
create the required physical infrastructure so that it does not affect the
performance of the new system.
Transition to the New System
This is the last step in the implementation process. The transition if done
wrongly leads to a lot of pain. Hence, it is necessary to move slowly on the
transition front. Normally, after the new system is installed and ready, the new
system and the old system are both used for a period to ensure that the
company performance does not suffer due to transition problems. Slowly when
the users gain more capability to handle the new system the old system is
phased out.
What is Expert System?
An expert system is an advanced computer application that is implemented for
the purpose of providing solutions to complex problems, or to clarify
uncertainties through the use of non-algorithmic programs where normally
human expertise will be needed. Expert systems are most common in complex
problem domain and are considered as widely used alternatives in searching
for solutions that requires the existence of specific human expertise. The expert
system is also able to justify its provided solutions based on the knowledge and
data from past users. Normally expert systems are used in making business
marketing strategic decisions, analyzing the performance of real time systems,
configuring computers and perform many other functions which normally
would require the existence of human expertise.
The difference between an expert system with a normal problem-solving system
is that the latter is a system where both programs and data structures are
encoded, while for expert system only the data structures are hard-coded and
no problem-specific information is encoded in the program structure. Instead,
the knowledge of a human expertise is captured and codified in a process
known as knowledge engineering. Hence, whenever a particular problem
requires the assistance of a certain human expertise to provide a solution, the
human expertise which has been codified will be used and processed in order
to provide a rational and logical solution. This knowledge-based expert system
14 | P a g e
enables the system to be frequently added with new knowledge and adapt
accordingly to meet new requirements from the ever-changing and
unpredictable environment.
Components of Expert System
An expert system has many core system components to function and interfaces
with individuals of various roles. The following diagram displaying expert
system components and human interfaces.
The major components of expert system are:
Knowledge base – a set of rules as representation of the expertise,
mostly in IF THEN statements.
Working storage – the data which is specific to a problem being solved.
Inference engine – the code at the core of the system which derives
recommendations from the knowledge base and problem-specific data in
working storage.
User interface – the code that controls the dialog between the user and
the system.
There are certain major roles of individuals who interact with the expert system
to fully exploit its functionality and capability. They are the:
15 | P a g e
Domain expert – the individual or individuals whose expertises are
solving the problems the system is intended to solve;
Knowledge engineer – the individual who encodes the expert’s
knowledge in a form that can be used by the expert system;
User – the individual who will be consulting with the system to get advice
which would have been provided by the expert.
Majority of the expert systems are built with expert system shells which
contains an inference engine and user interface. The shell will be used by a
knowledge engineer to build a system catered for specific problem domain.
Sometimes expert systems are also built with custom developed shells for
certain applications. In this scenario, there will be another additional
individual
System engineer – the individual who builds the user interface, designs
the declarative format of the knowledge base, and implements the
inference engine.
Depending on the size of the project, the knowledge engineer and the system
engineer might be the same person. For a custom built system, the design of
the format of the knowledge base and the coding of the domain knowledge are
closely related. The format has a significant effect on the coding of the
knowledge.
One of the major hurdles to overcome in building expert systems is the
knowledge engineering process. The process of the codifying the expertise into
a required rule format can be a challenging and tedious task. One major
advantage of a customized shell is that the format of the knowledge base can
be designed to facilitate the knowledge engineering process.
Since the major challenge in expert system development is the building of the
knowledge base, it is encouraged that gap and difference between the expert’s
representation of the knowledge and the representation in the knowledge base
should be minimized. With a customized system, the system engineer can
implement a knowledge base whose structures are as close as possible to those
used by the domain expert.
Knowledge-based Expert Systems
Not all expert systems have learning components to adapt in new environments
or to meet new requirements. But a common element each expert system
16 | P a g e
possesses is that once the system is fully developed it will be tested and be
proven by being placed in the same real world problem solving situation,
typically as an aid to human workers or a supplement to some information
system.
Although reference books are able to provide a tremendous amount of
knowledge, users have to read, comprehend and interpret the knowledge for it
to be used. Conventional computer programs are built to perform functions
using conventional decision-making logic — having only little knowledge along
with the basic algorithm for performing the specific functions and fulfill the
necessary boundary conditions.
The so-called “knowledgebase” was created in purpose of utilizing some
knowledge representation formalism to capture and store the Subject Matter
Expert’s (SME) knowledge. The process includes gathering that knowledge from
the SME and codifying it according to a standardized format. Knowledge-based
expert systems collect the small segments of human knowledge and combined
into a set of knowledge-base which is used to aid in solving a complex problem.
Any other problem that is within the range and domain of the knowledge-base
can also be solved using the same program without reprogramming.
Knowledge-based expert systems solve problems which normally require
human intelligence. These said expert systems represent the expertise
knowledge as data or rules within a system. These rules and data can be used
and called upon for reference when needed to solve complex problems.
When compared to conventional programming, the system has the ability to
reason the process with explanations by back-traces and calculate the levels of
confidence and deal with uncertainty. The knowledge has to be codified into
programming code, hence as the knowledge changes, the program has to be
changed accordingly as well and then rebuilt.
Expert System Features
There are a number of features which are commonly used in expert systems.
These features allows the users to fully utilize the expert system’s capability
conveniently in providing the most logical and reasonable decision in a
problematic situation.
Backward chaining – an inference technique which continuously break
a goal into smaller sub-goals which are easier to prove via IF THEN rules
17 | P a g e
Dealing with uncertainties – the system has the capability to handle
and reason with conditions that are uncertain and data which are not
precisely known
Forward chaining – an inference technique which deduce a problem
solution from initial data via IF THEN rules
Data representation – the method where the specific problem data is
stored and accessed in the system
User interface – that portion of the code which creates an easy to use
system;
Explanations – the ability of the system to explain the reasoning process
that it used to reach a recommendation.
The Advantages of Using Expert System
Expert system has been reliably used in the business world to gain tactical
advantages and forecast the market’s condition. In this globalization era where
every decision made in the business world is critical for success, the assistance
provided from an expert system is undoubtedly essential and highly reliable for
an organization to succeed. Examples given below will be the advantages for
the implementation of an expert system in business:
1. Providing consistent solutions – It can provide consistent answers for
repetitive decisions, processes and tasks. As long as the rule base in the
system remains the same, regardless of how many times similar
problems are being tested, the final conclusions drawn will remain the
same.
2. Provides reasonable explanations – It has the ability to clarify the
reasons why the conclusion was drawn and be why it is considered as
the most logical choice among other alternatives. If there are any doubts
in concluding a certain problem, it will prompt some questions for users
to answer in order to process the logical conclusion.
3. Overcome human limitations – It does not have human limitations and
can work around the clock continuously. Users will be able to frequently
use it in seeking solutions. The knowledge of experts is an invaluable
asset for the company. It can store the knowledge and use it as long as
the organization needs.
18 | P a g e
4. Easy to adapt to new conditions – Unlike humans who often have
troubles in adapting in new environments, an expert system has high
adaptability and can meet new requirements in a short period of time. It
also can capture new knowledge from an expert and use it as inference
rules to solve new problems.
The Disadvantages of Using Expert System
Although the expert system does provide many significant advantages, it does
have its drawbacks as well. Examples given below will be the disadvantages for
the implementation of an expert system in business:
1. Lacks common sense – It lacks common sense needed in some decision
making since all the decisions made are based on the inference rules set
in the system. It also cannot make creative and innovative responses as
human experts would in unusual circumstances.
2. High implementation and maintenance cost – The implementation of
an expert system in business will be a financial burden for smaller
organizations since it has high development cost as well as the
subsequent recurring costs to upgrade the system to adapt in new
environment.
3. Difficulty in creating inference rules – Domain experts will not be able
to always explain their logic and reasoning needed for the knowledge
engineering process. Hence, the task of codifying out the knowledge is
highly complex and may require high
4. May provide wrong solutions – It is not error-free. There may be errors
occurred in the processing due to some logic mistakes made in the
knowledge base, which it will then provide the wrong solutions.
Executive Information System
An executive information system (EIS) is a decision support system (DSS) used
to assist senior executives in the decision-making process. It does this by
providing easy access to important data needed to achieve strategic goals in an
organization. An EIS normally features graphical displays on an easy-to-use
interface.
Executive information systems can be used in many different types of
organizations to monitor enterprise performance as well as to identify
opportunities and problems.
19 | P a g e
EIS find application in decision areas of:
Monitor company performance
Track activities of competitors
Spot & foresee problems
Identify opportunities
Forecasting
EIS depend heavily on external sources of data like stock market, economic
databases, news services etc. as well as on internal information.
EIS have highly flexible and user friendly input & output interfaces.
They have drill-down options.
Executive support systems are intended to be used by the senior managers
directly to provide support to non-programmed decisions in strategic
management.
These information are often external, unstructured and even uncertain. Exact
scope and context of such information is often not known beforehand.
This information is intelligence based −
Market intelligence
Investment intelligence
Technology intelligence
Examples of Intelligent Information
Following are some examples of intelligent information, which is often the
source of an ESS −
External databases
Technology reports like patent records etc.
Technical reports from consultants
Market reports
20 | P a g e
Confidential information about competitors
Speculative information like market conditions
Government policies
Financial reports and information
Features of Executive Information System
Advantages of ESS
Easy for upper level executive to use
Ability to analyze trends
Augmentation of managers' leadership capabilities
Enhance personal thinking and decision-making
Contribution to strategic control flexibility
21 | P a g e
Enhance organizational competitiveness in the market place
Instruments of change
Increased executive time horizons.
Better reporting system
Improved mental model of business executive
Help improve consensus building and communication
Improve office automation
Reduce time for finding information
Early identification of company performance
Detail examination of critical success factor
Better understanding
Time management
Increased communication capacity and quality
Disadvantage of ESS
Functions are limited
Hard to quantify benefits
Executive may encounter information overload
System may become slow
Difficult to keep current data
May lead to less reliable and insecure data
Excessive cost for small company
Artificial Intelligence (AI)
Artificial intelligence (AI) is an area of computer sciences that emphasizes the
creation of intelligent machines that work and reacts like humans. Some of the
activities computers with artificial intelligence are designed for include:
Speech recognition
22 | P a g e
Learning
Planning
Problem solving
Research associated with artificial intelligence is highly technical and
specialized. The core problems of artificial intelligence include programming
computers for certain traits such as:
Knowledge
Reasoning
Problem solving
Perception
Learning
Planning
Ability to manipulate and move objects
Knowledge engineering is a core part of AI research. Machines can often act
and react like humans only if they have abundant information relating to the
world. Artificial intelligence must have access to objects, categories, properties
and relations between all of them to implement knowledge engineering.
Initiating common sense, reasoning and problem-solving power in machines is
a difficult and tedious task.
Reasoning, problem solving
Early researchers developed algorithms that imitated step-by-step reasoning
that humans use when they solve puzzles or make logical deductions. By the
late 1980s and 1990s, AI research had developed methods for dealing
with uncertain or incomplete information, employing concepts
from probability and economics. These algorithms proved to be insufficient for
solving large reasoning problems, because they experienced a "combinatorial
explosion": they became exponentially slower as the problems grew larger. In
fact, even humans rarely use the step-by-step deduction that early AI research
was able to model. They solve most of their problems using fast, intuitive
judgments.
23 | P a g e
Machine perception deals with the capability to use sensory inputs to deduce
the different aspects of the world, while computer vision is the power to analyze
visual inputs with a few sub-problems such as facial, object and gesture
recognition.
Machine learning is also a core part of AI. Learning without any kind of
supervision requires an ability to identify patterns in streams of inputs,
whereas learning with adequate supervision involves classification and
numerical regressions.
Classification determines the category an object belongs to and regression
deals with obtaining a set of numerical input or output examples, thereby
discovering functions enabling the generation of suitable outputs from
respective inputs. Mathematical analysis of machine learning algorithms and
their performance is a well-defined branch of theoretical computer science
often referred to as computational learning theory.
Planning
Intelligent agents must be able to set goals and achieve them. They need a way
to visualize the future—a representation of the state of the world and be able to
make predictions about how their actions will change it—and be able to make
choices that maximize the utility (or "value") of available choices.
In classical planning problems, the agent can assume that it is the only system
acting in the world, allowing the agent to be certain of the consequences of its
actions. However, if the agent is not the only actor, then it requires that the
agent can reason under uncertainty. This calls for an agent that can not only
assess its environment and make predictions, but also evaluate its predictions
and adapt based on its assessment.
Robotics is also a major field related to AI. Robots require intelligence to
handle tasks such as object manipulation and navigation, along with sub-
problems of localization, motion planning and mapping.
24 | P a g e
25 | P a g e
Examples of AI technology
AI is incorporated into a variety of different types of technology. Here are seven
examples.
Automation: What makes a system or process function
automatically? For example, robotic process automation (RPA) can be
programmed to perform high-volume, repeatable tasks those humans
normally performed. RPA is different from IT automation in that it can
adapt to changing circumstances.
Machine learning: The science of getting a computer to act without
programming. Deep is a subset of machine learning that, in very simple
terms, can be thought of as the automation of predictive analytics. There
are three types of machine learning algorithms:
o Supervised learning: Data sets are labeled so that patterns can be
detected and used to label new data sets
o Unsupervised learning: Data sets aren't labeled and are sorted
according to similarities or differences
o Reinforcement learning: Data sets aren't labeled but, after
performing an action or several actions, the AI system is given
feedback
Machine vision: The science of allowing computers to see. This
technology captures and analyzes visual information using a camera,
analog-to-digital conversion and digital signal processing. It is often
compared to human eyesight, but machine vision isn't bound by biology
and can be programmed to see through walls, for example. It is used in a
range of applications from signature identification to medical image
analysis. Computer vision, which is focused on machine-based image
processing, is often conflated with machine vision.
Natural language processing (NLP): The processing of human -- and not
computer -- language by a computer program. One of the older and best
known examples of NLP is spam detection, which looks at the subject
line and the text of an email and decides if it's junk. Current approaches
to NLP are based on machine learning. NLP tasks include text
translation, sentiment analysis and speech recognition.
26 | P a g e
Robotics: A field of engineering focused on the design and
manufacturing of robots. Robots are often used to perform tasks that are
difficult for humans to perform or perform consistently. They are used in
assembly lines for car production or by NASA to move large objects in
space. Researchers are also using machine learning to build robots that
can interact in social settings.
Self-driving cars: These use a combination of computer vision, image
recognition and deep learning to build automated skill at piloting a
vehicle while staying in a given lane and avoiding unexpected
obstructions, such as pedestrians.
AI applications
Artificial intelligence has made its way into a number of areas. Here are six examples.
AI in healthcare. The biggest bets are on improving patient outcomes
and reducing costs. Companies are applying machine learning to make
better and faster diagnoses than humans. One of the best known
healthcare technologies is IBM Watson. It understands natural language
and is capable of responding to questions asked of it. The system mines
patient data and other available data sources to form a hypothesis,
which it then presents with a confidence scoring schema. Other AI
applications include chatbots, a computer program used online to
answer questions and assist customers, to help schedule follow-up
appointments or aid patients through the billing process, and virtual
health assistants that provide basic medical feedback.
AI in business. Robotic process automation is being applied to highly
repetitive tasks normally performed by humans. Machine learning
algorithms are being integrated into analytics and CRM platforms to
uncover information on how to better serve customers. Chatbots have
been incorporated into websites to provide immediate service to
customers. Automation of job positions has also become a talking point
among academics and IT analysts.
AI in education. AI can automate grading, giving educators more time.
AI can assess students and adapt to their needs, helping them work at
their own pace. AI tutors can provide additional support to students,
ensuring they stay on track. AI could change where and how students
learn, perhaps even replacing some teachers.
27 | P a g e
AI in finance. AI in personal finance applications, such as Mint or Turbo
Tax, is disrupting financial institutions. Applications such as these
collect personal data and provide financial advice. Other programs, such
as IBM Watson, have been applied to the process of buying a home.
Today, software performs much of the trading on Wall Street.
AI in law. The discovery process, sifting through of documents, in law is
often overwhelming for humans. Automating this process is a more
efficient use of time. Startups are also building question-and-answer
computer assistants that can sift programmed-to-answer questions by
examining the taxonomy and ontology associated with a database.
AI in manufacturing. This is an area that has been at the forefront of
incorporating robots into the workflow. Industrial robots used to perform
single tasks and were separated from human workers, but as the
technology advanced that changed.
Concerns and criticisms
While AI tools present a range of new functionality for businesses, artificial
intelligence also raises some ethical questions. Deep learning algorithms, which
underpin many of the most advanced AI tools, only know what's in the data
used during training. Most available data sets for training likely contain traces
of human bias. This in turn can make the AI tools biased in their function. This
has been seen in the Microsoft chatbot, Tay, which learned a misogynistic and
anti-Semitic vocabulary from Twitter users, and the Google Photo image
classification tool that classified a group of African Americans as gorillas.
The application of AI in the realm of self-driving cars also raises ethical
concerns. When an autonomous vehicle is involved in an accident, liability is
unclear. Autonomous vehicles may also be put in a position where an accident
is unavoidable, forcing it to make ethical decisions about how to minimize
damage.
28 | P a g e
Another major concern is the potential for abuse of AI tools. Hackers are
starting to use sophisticated machine learning tools to gain access to sensitive
systems, complicating the issue of security beyond its current state.
Deep learning-based video and audio generation tools also present bad actors
with the tools necessary to create so-called deepfakes, convincingly fabricated
videos of public figures saying or doing things that never took place.
Data Mining
Data mining is the process of analyzing hidden patterns of data according to different
perspectives for categorization into useful information, which is collected and
assembled in common areas, such as data warehouses, for efficient analysis, data
mining algorithms, facilitating business decision making and other information
requirements to ultimately cut costs and increase revenue.
Data mining is also known as data discovery and knowledge discovery.
The major steps involved in a data mining process are:
Extract, transform and load data into a data warehouse
Store and manage data in a multidimensional databases
Provide data access to business analysts using application software
Present analyzed data in easily understandable forms, such as graphs
The first step in data mining is gathering relevant data critical for business. Company
data is either transactional, non-operational or metadata. Transactional data deals
with day-to-day operations like sales, inventory and cost etc. Non-operational data is
normally forecast, while metadata is concerned with logical database design. Patterns
and relationships among data elements render relevant information, which may
increase organizational revenue. Organizations with a strong consumer focus deal
with data mining techniques providing clear pictures of products sold, price,
competition and customer demographics.
For instance, the retail giant Wal-Mart transmits all its relevant information to a data
warehouse with terabytes of data. This data can easily be accessed by suppliers
enabling them to identify customer buying patterns. They can generate patterns on
29 | P a g e
shopping habits, most shopped days, most sought for products and other data
utilizing data mining techniques.
The second step in data mining is selecting a suitable algorithm - a mechanism
producing a data mining model. The general working of the algorithm involves
identifying trends in a set of data and using the output for parameter definition. The
most popular algorithms used for data mining are classification algorithms and
regression algorithms, which are used to identify relationships among data elements.
Major database vendors like Oracle and SQL incorporate data mining algorithms,
such as clustering and regression tress, to meet the demand for data mining.
Who's using it?
Data mining is at the heart of analytics efforts across a variety of industries and
disciplines.
Communications
In an overloaded market where competition is tight, the answers are often within your
consumer data. Multimedia and telecommunications companies can use analytic
models to make sense of mountains of customers data, helping them predict customer
behavior and offer highly targeted and relevant campaigns.
Insurance
With analytic know-how, insurance companies can solve complex problems
concerning fraud, compliance, risk management and customer attrition. Companies
have used data mining techniques to price products more effectively across business
lines and find new ways to offer competitive products to their existing customer base.
Education
With unified, data-driven views of student progress, educators can predict student
performance before they set foot in the classroom – and develop intervention strategies
to keep them on course. Data mining helps educators access student data, predict
achievement levels and pinpoint students or groups of students in need of extra
attention.
Manufacturing
Aligning supply plans with demand forecasts is essential, as is early detection of
problems, quality assurance and investment in brand equity. Manufacturers can
predict wear of production assets and anticipate maintenance, which can maximize
uptime and keep the production line on schedule.
30 | P a g e
Banking
Automated algorithms help banks understand their customer base as well as the
billions of transactions at the heart of the financial system. Data mining helps
financial services companies get a better view of market risks, detect fraud faster,
manage regulatory compliance obligations and get optimal returns on their marketing
investments.
Retail
Large customer databases hold hidden insights that can help you improve customer
relationships, optimize marketing campaigns and forecast sales. Through more
accurate data models, retail companies can offer more targeted campaigns – and find
the offer that makes the biggest impact on the customer.
How It Works
Data mining, as a composite discipline, represents a variety of methods or techniques
used in different analytic capabilities that address a gamut of organizational needs,
ask different types of questions and use varying levels of human input or rules to
arrive at a decision.
Descriptive Modeling: It uncovers shared similarities or groupings in historical data
to determine reasons behind success or failure, such as categorizing customers by
product preferences or sentiment. Sample techniques include:
Clustering Grouping similar records together.
Anomaly detection Identifying multidimensional outliers.
Association rule learning Detecting relationships between records.
Principal component analysis Detecting relationships between variables.
Grouping people with common interests or similar goals
Affinity grouping
(e.g., people who buy X often buy Y and possibly Z).
31 | P a g e
Predictive Modeling: This modeling goes deeper to classify events in the future or
estimate unknown outcomes – for example, using credit scoring to determine an
individual's likelihood of repaying a loan. Predictive modeling also helps uncover
insights for things like customer churn, campaign response or credit defaults. Sample
techniques include:
A measure of the strength of the relationship between one
Regression
dependent variable and a series of independent variables.
Computer programs that detect patterns, make predictions
Neural networks
and learn.
Tree-shaped diagrams in which each branch represents
Decision trees
a probable occurrence.
Support vector machines Supervised learning models with associated learning algorithms.
Prescriptive Modeling: With the growth in unstructured data from the web, comment
fields, books, email, PDFs, audio and other text sources, the adoption of text mining
as a related discipline to data mining has also grown significantly. You need the
ability to successfully parse, filter and transform unstructured data in order to
include it in predictive models for improved prediction accuracy.
In the end, you should not look at data mining as a separate, standalone entity
because pre-processing (data preparation, data exploration) and post-processing
(model validation, scoring, model performance monitoring) are equally essential.
Prescriptive modeling looks at internal and external variables and constraints to
recommend one or more courses of action – for example, determining the best
marketing offer to send to each customer.
Data Warehousing
Data warehousing is the process of constructing and using a data warehouse. A data
warehouse is constructed by integrating data from multiple heterogeneous sources
that support analytical reporting, structured and/or ad hoc queries, and decision
32 | P a g e
making. Data warehousing involves data cleaning, data integration, and data
consolidations.
Using Data Warehouse Information
There are decision support technologies that help utilize the data available in a data
warehouse. These technologies help executives to use the warehouse quickly and
effectively. They can gather data, analyze it, and take decisions based on the
information present in the warehouse. The information gathered in a warehouse can
be used in any of the following domains −
Tuning Production Strategies − The product strategies can be well
tuned by repositioning the products and managing the product portfolios
by comparing the sales quarterly or yearly.
Customer Analysis − Customer analysis is done by analyzing the
customer's buying preferences, buying time, budget cycles, etc.
Operations Analysis − Data warehousing also helps in customer
relationship management, and making environmental corrections. The
information also allows us to analyze business operations.
Integrating Heterogeneous Databases
To integrate heterogeneous databases, we have two approaches −
Query-driven Approach
Update-driven Approach
Query-Driven Approach
This is the traditional approach to integrate heterogeneous databases. This approach
was used to build wrappers and integrators on top of multiple heterogeneous
databases. These integrators are also known as mediators.
Process of Query-Driven Approach
When a query is issued to a client side, a metadata dictionary translates
the query into an appropriate form for individual heterogeneous sites
involved.
Now these queries are mapped and sent to the local query processor.
33 | P a g e
The results from heterogeneous sites are integrated into a global answer
set.
Disadvantages
Query-driven approach needs complex integration and filtering
processes.
This approach is very inefficient.
It is very expensive for frequent queries.
This approach is also very expensive for queries that require
aggregations.
Update-Driven Approach
This is an alternative to the traditional approach. Today's data warehouse systems
follow update-driven approach rather than the traditional approach discussed earlier.
In update-driven approach, the information from multiple heterogeneous sources are
integrated in advance and are stored in a warehouse. This information is available for
direct querying and analysis.
Advantages
This approach has the following advantages −
This approach provides high performance.
The data is copied, processed, integrated, annotated, summarized and
restructured in semantic data store in advance.
Query processing does not require an interface to process data at local
sources.
Functions of Data Warehouse Tools and Utilities
The following are the functions of data warehouse tools and utilities −
Data Extraction − Involves gathering data from multiple heterogeneous
sources.
Data Cleaning − Involves finding and correcting the errors in data.
Data Transformation − Involves converting the data from legacy format
to warehouse format.
34 | P a g e
Data Loading − Involves sorting, summarizing, consolidating, checking
integrity, and building indices and partitions.
Refreshing − Involves updating from data sources to warehouse.
Note − Data cleaning and data transformation are important steps in improving the
quality of data and data mining results.
Process Flow in Data Warehouse
There are four major processes that contribute to a data warehouse −
Extract and load the data.
Cleaning and transforming the data.
Backup and archive the data.
Managing queries and directing them to the appropriate data sources.
Extract and Load Process
Data extraction takes data from the source systems. Data load takes the extracted
data and loads it into the data warehouse.
Note − before loading the data into the data warehouse, the information extracted
from the external sources must be reconstructed.
35 | P a g e
Controlling the Process
Controlling the process involves determining when to start data extraction and the
consistency check on data. Controlling process ensures that the tools, the logic
modules, and the programs are executed in correct sequence and at correct time.
When to Initiate Extract
Data needs to be in a consistent state when it is extracted, i.e., the data warehouse
should represent a single, consistent version of the information to the user.
For example, in a customer profiling data warehouse in telecommunication sector, it
is illogical to merge the list of customers at 8 pm on Wednesday from a customer
database with the customer subscription events up to 8 pm on Tuesday. This would
mean that we are finding the customers for whom there are no associated
subscriptions.
Loading the Data
After extracting the data, it is loaded into a temporary data store where it is cleaned
up and made consistent.
Note − Consistency checks are executed only when all the data sources have been
loaded into the temporary data store.
Clean and Transform Process
Once the data is extracted and loaded into the temporary data store, it is time to
perform Cleaning and Transforming. Here is the list of steps involved in Cleaning and
Transforming −
Clean and transform the loaded data into a structure
Partition the data
Aggregation
Clean and Transform the Loaded Data into a Structure
Cleaning and transforming the loaded data helps speed up the queries. It can be done
by making the data consistent −
Within itself.
With other data within the same data source.
36 | P a g e
With the data in other source systems.
With the existing data present in the warehouse.
Transforming involves converting the source data into a structure. Structuring the
data increases the query performance and decreases the operational cost. The data
contained in a data warehouse must be transformed to support performance
requirements and control the ongoing operational costs.
Partition the Data
It will optimize the hardware performance and simplify the management of data
warehouse. Here we partition each fact table into multiple separate partitions.
Aggregation
Aggregation is required to speed up common queries. Aggregation relies on the fact
that most common queries will analyze a subset or an aggregation of the detailed
data.
Backup and Archive the Data
In order to recover the data in the event of data loss, software failure, or hardware
failure, it is necessary to keep regular back ups. Archiving involves removing the old
data from the system in a format that allow it to be quickly restored whenever
required.
For example, in a retail sales analysis data warehouse, it may be required to keep
data for 3 years with the latest 6 months data being kept online. In such as scenario,
there is often a requirement to be able to do month-on-month comparisons for this
year and last year. In this case, we require some data to be restored from the archive.
Query Management Process
This process performs the following functions −
Manages the queries.
Helps speed up the execution time of queries.
Directs the queries to their most effective data sources.
Ensures that all the system sources are used in the most effective way.
Monitors actual query profiles.
37 | P a g e
The information generated in this process is used by the warehouse management
process to determine which aggregations to generate. This process does not generally
operate during the regular load of information into data warehouse.
To design an effective and efficient data warehouse, we need to understand and
analyze the business needs and construct a business analysis framework. Each
person has different views regarding the design of a data warehouse. These views are
as follows −
The top-down view − This view allows the selection of relevant
information needed for a data warehouse.
The data source view − This view presents the information being
captured, stored, and managed by the operational system.
The data warehouse view − This view includes the fact tables and
dimension tables. It represents the information stored inside the data
warehouse.
The business query view − It is the view of the data from the viewpoint
of the end-user.
Data Warehouse Models
From the perspective of data warehouse architecture, we have the following data
warehouse models −
Virtual Warehouse
Data mart
Enterprise Warehouse
Virtual Warehouse
The view over an operational data warehouse is known as a virtual warehouse. It is
easy to build a virtual warehouse. Building a virtual warehouse requires excess
capacity on operational database servers.
Data Mart
Data mart contains a subset of organization-wide data. This subset of data is valuable
to specific groups of an organization.
38 | P a g e
In other words, we can claim that data marts contain data specific to a particular
group. For example, the marketing data mart may contain data related to items,
customers, and sales. Data marts are confined to subjects.
Points to remember about data marts −
Window-based or Unix/Linux-based servers are used to implement data
marts. They are implemented on low-cost servers.
The implementation data mart cycles is measured in short periods of
time, i.e., in weeks rather than months or years.
The life cycle of a data mart may be complex in long run, if its planning
and design are not organization-wide.
Data marts are small in size.
Data marts are customized by department.
The source of a data mart is departmentally structured data warehouse.
Data mart are flexible.
Enterprise Warehouse
An enterprise warehouse collects all the information and the subjects
spanning an entire organization
It provides us enterprise-wide data integration.
The data is integrated from operational systems and external information
providers.
This information can vary from a few gigabytes to hundreds of gigabytes,
terabytes or beyond.