Chapter Two

Intelligent Agents

Compiled by Gizachew M.(MSc in CS)

content
✓ Introduction
✓ Agents and Environments
✓ Acting of Intelligent Agents (Rationality)
✓ Structure of Intelligent Agents
✓ Agent Types
Simple reflex agent
Model-based reflex agent
Goal-based agent
Utility-based agent
Learning agent

Compiled by Gizachew M.(MSc in CS)

 Introduction: Intelligent agents
• In artificial intelligence, an intelligent agent is anything which
perceives its environment, takes actions autonomously in order to
achieve goals, and may improve its performance with learning or
may use knowledge. Or
• It is a program that can make decisions or perform a service
based on its environment, user input and experiences.
• It is an entity that makes a decision, that enables artificial
intelligence to be put into action. It can also be described as a
software entity that conducts operations in the place of users or
programs after sensing the environment. It uses actuators to initiate
action in that environment.
Compiled by Gizachew M.(MSc in CS)
 Agent and environment
• Agent is something that perceives/observes its environment through SENSORS and acts upon that
environment through EFFECTORS.
• The agent is assumed to exist in an environment in which it perceives and acts
• An agent is rational/sensible since it does the right thing to achieve the specified goal.
• Human: sensors are eyes, ears,
actuators (effectors) are hands,legs, mouth.
• Robot: sensors are cameras, sonar,
lasers, ladar, bump, effectors are
grippers, manipulators, motors
• The agent’s behavior is described by
its function that maps percept to action.
• Agent = architecture + program

Compiled by Gizachew M.(MSc in CS)

 Rational agent
• In summary what is rational at any given point depends on four things.
– Perception/sensitivity: Everything that the agent has perceived so far concerning
the current scenario in the environment
– Knowledge: What an agent already knows about the environment
– Action: The actions that the agent can perform back to the environment
– Performance measure: The performance measure that defines degrees of success of
the agent.
 Generally, rationality refers to “doing the right thing”.
 So that one key property of Intelligent Agents is being a rational agent,
because rational agents means doing the right thing.

• Therefore in designing an intelligent agent, one has to remember PEAS (Performance,

Environment, Actuators, Sensors) framework.

Compiled by Gizachew M.(MSc in CS)

Cont..
• Design of intelligent agent needs prior knowledge of
◦ Performance measure or Goal the agent supposed
to achieve,
◦ On what kind of Environment it operates
◦ What kind of Actuators it has (what are the possible
Actions),
◦ What kind of Sensors the agent has (what are the
possible Percepts)

Compiled by Gizachew M.(MSc in CS)

 Agent function & program
Agent’s behavior is mathematically described by
 Agent function
 A function mapping any given percept sequence to an
action
Practically it is described by
 An agent program
 The real implementation

Compiled by Gizachew M.(MSc in CS)

 Vacuum-cleaner world
Perception: Clean or Dirty? where it is in?
Actions: Move left, Move right, suck, do nothing

Compiled by Gizachew M.(MSc in CS)

 2.4 Types of Environment
 Based on the portion of the environment observable
 Fully observable: An agent's sensors give it access to the complete state of the
environment at each point in time.
 if the sensors detect all aspects
 That are relevant to the choice of action eg. Chess vs driver
 Partially observable: An environment might be Partially observable because of
noisy and inaccurate sensors or because parts of the state are simply missing
from the sensor data. And that the agent needs a memory in order to make best
possible decisions.
Example:
 A local dirt sensor of the cleaner cannot tell

 Whether other squares are clean or not

 Fully unobservable: when the agent has no sensors in all environments.
Compiled by Gizachew M.(MSc in CS)
 Based on the effect of the agent action
 Deterministic : The next state of the environment is completely determined by
the current state and the action executed by the agent.
 Agents’ action uniquely determine the outcome. E.g. chess
 Strategic: If the environment is deterministic except for the actions of other
agents, then the environment is strategic
 Stochastic or probabilistic: when the agents action not uniquely determine the
outcome. The next state is totally unpredictable for the agent. So randomness
exists in the environment.
E.g. Self deriving car

Compiled by Gizachew M.(MSc in CS)

 Types of Environment
 Based on the number agent involved
 Single agent A single agent operating by itself in an environment.
 Multi-agent: multiple agents are involved in the environment
 Based on the state, action and percept space pattern
 Discrete: A limited number of distinct, clearly defined state, percepts
and actions.
 Continuous: state, percept and action are consciously changing
variables
 Note: one or more of them can be discrete or continuous

Compiled by Gizachew M.(MSc in CS)

 Types of Environment cont …
 Based on the effect of time
 Static: The environment is unchanged while an agent is deliberating.
 Dynamic: The environment changes while an agent is not deliberating.
 semi-dynamic: The environment is semi-dynamic if the environment itself does
not change with the passage of time but the agent's performance score does
 Based on loosely dependent sub-objectives
 Episodic: The agent's experience is divided into atomic "episodes" (each episode
consists of the agent perceiving and then performing a single action), and the
choice of action in each episode depends only on the episode itself. Is an
environment where each state is independent of each other. The action on a state
has nothing to do with the next state.
 Sequential: The agent's experience is a single atomic "episodes“. is an
environment where the next state is dependent on the current action.
Compiled by Gizachew M.(MSc in CS)
 Environment types example
Environment types Chess with a clock Chess without Taxi driving Medical Image-
/agents clock diagnosis analysis
system system

Fully observable Yes Yes No No Yes

Deterministic Strategic Strategic No No Yes

Episodic No No No No Yes
Static Semi Yes No No Semi
Discrete Yes Yes No No No
Single agent No No No No No

Examples of environments and their characteristics.

The environment type largely determines the agent design
The real world is (of course) partially observable, stochastic, sequential, dynamic, continuous, multi-agent
Compiled by Gizachew M.(MSc in CS)
Hierarchy of Agent Types

 Let’s have a closer look on the following five agent

categories:-
I. Simple Reflex Agents
II. Model-Based Reflex Agents
III. Goal-Based Agents
IV. Utility-Based Agents
V. Learning Agents

Compiled by Gizachew M.(MSc in CS)

I. Simple Reflex Agents
 Select action on the basis of current percept, ignoring
all past percepts
 It uses just condition-action rules
 The rules are like the form “if … then …”
 Efficient but have narrow range of applicability
 Because knowledge sometimes cannot be stated
explicitly
 Work only
o If the environment is fully observable
 If car-in-front-is breaking then initiate-braking.
 Blinking when something approaches the eye.

Compiled by Gizachew M.(MSc in CS)

Figure: Structure of a simple reflex agent

function SIMPLE-REFLEX-AGENT(percept) returns action

static: rules, a set of condition-action rules
state  INTERPRET-INPUT (percept)
rule  RULE-MATCH (state,rules)
action  RULE-ACTION [rule]
return action

Compiled by Gizachew M.(MSc in CS)

 II. Model-Based Reflex Agents
 Maintains internal state that keeps track of aspects of the environment
that cannot be currently observed
 For the world that is partially observable
 The agent has to keep track of an internal state
o That depends on the percept history
o Reflecting some of the unobserved aspects
o Example, driving a car and changing lane
 Requiring two types of knowledge
 How the world evolves independently of the agent
 How the agent’s actions affect the world
 The agent is with memory

Compiled by Gizachew M.(MSc in CS)

Figure: Structure of Model-Based reflex agent

function REFLEX-AGENT-WITH-STATE (percept) returns action

static: state, a description of the current world state
rules, a set of condition-action rules
state  UPDATE-STATE (state, percept)
rule  RULE-MATCH (state, rules)
action  RULE-ACTION [rule]
state  UPDATE-STATE (state, action)
return action Compiled by Gizachew M.(MSc in CS)
 Example Table Agent With Internal State

IF THEN
Saw an object ahead, and Go straight
turned right, and it’s now
clear ahead
Saw an object Ahead, Halt
turned right, and object
ahead again
See no objects ahead Go straight
See an object ahead Turn randomly

Compiled by Gizachew M.(MSc in CS)

 Case Example for A Reflex Agent With Internal State
 The Wall-Following

start

 The Actions are:-

 Left
 Right
 Straight
 Open-Door

Compiled by Gizachew M.(MSc in CS)

 The Rules are:-
1) If open(left) & open(right) and open(straight) then
choose randomly between right and left
2) If wall(left) and open(right) and open(straight) then
straight
3) If wall(right) and open(left) and open(straight) then
straight
4) If wall(right) and open(left) and wall(straight) then left
5) If wall(left) and open(right) and wall(straight) then
right
6) If wall(left) and door(right) and wall(straight) then
open-door
7) If wall(right) and wall(left) and open(straight) then
straight.
8) (Default) MoveCompiled
randomly
by Gizachew M.(MSc in CS)
III. Goal-Based Agents

 The agent uses goal information to select between

possible actions in the current state
 Current state of the environment is always not
enough
 The goal is another issue to achieve
 Judgment of rationality / correctness
 Actions chosen  goals, based on
 The current state
 The current percept

Compiled by Gizachew M.(MSc in CS)

 Conclusion
 Goal-based agents are less efficient
 But more flexible
o Agent  Different goals  different tasks
 Search and planning
o Two other sub-fields in AI
o To find out the action sequences to achieve
its goal

Compiled by Gizachew M.(MSc in CS)

Figure: Structure of a Goal-based agent

function GOAL_BASED_AGENT (percept) returns action

state  UPDATE-STATE (state, percept)
action  SELECT-ACTION [state, goal]
state  UPDATE-STATE (state, action)
return action
Compiled by Gizachew M.(MSc in CS)
IV. Utility-Based Agents
 The agent uses a utility function to evaluate the
desirability of states that could result from each
possible action
 Goals alone are not enough
 To generate high-quality behavior
 Example, meals in Canteen, good or not ?
 Many action sequences  the goals
 Some are better and some worse
 If goal means success,
 Then utility means the degree of success (how
successful it is)

Compiled by Gizachew M.(MSc in CS)

 Figure: Structure of a utility-based agent

function UTILITY_BASED_AGENT (percept) returns action

state  UPDATE-STATE (state, percept)

action  SELECT-OPTIMAL_ACTION [state, goal]
state  UPDATE-STATE (state, action)
return action Compiled by Gizachew M.(MSc in CS)
 It is said state A has higher utility
 If state A is more preferred than others
 Utility is therefore a function
 That maps a state onto a real number
 The degree of success

Compiled by Gizachew M.(MSc in CS)

 Utility has several advantages:
 When there are conflicting goals,
o Only some of the goals but not all can be
achieved
o utility describes the appropriate trade-off
 When there are several goals
o None of them are achieved certainly
o Utility provides a way for the decision-
making

Compiled by Gizachew M.(MSc in CS)

V. Learning Agents
 To build learning machines and then to teach them.
 After an agent is programmed, can it work
immediately?
 No, it still need teaching
 In AI,
 Once an agent is done
 We teach it by giving it a set of examples
 Test it by using another set of examples
 We then say the agent learns
 A learning agent

Compiled by Gizachew M.(MSc in CS)

 Four conceptual components
a) Learning Element
 Which is responsible for making improvement
b) Performance Element
 Which is responsible for selecting external actions
c) Critic
 The learning element uses feedback from the critic on how the
agent is doing and determines how the performance element
should be modified to do better in the future.
 Tells the Learning element how well the agent is doing with
respect to fixed performance standard.
o Feedback from user or examples, good or not?
d) Problem Generator
 Responsible for suggesting actions that will lead to new
and informative experiences

Compiled by Gizachew M.(MSc in CS)

Figure: Structure of a Learning agent

Compiled by Gizachew M.(MSc in CS)

 Summery hierarchy in types of agents
• Reflex agents:
 These agents function in a current state, ignoring past history.
 Responses are based on the event-condition-action rule (ECA rule) where a
user initiates an event and the agent refers to a list of pre-set rules and pre-
programmed outcomes.
• Model-based agents:
 These agents choose an action in the same way as a reflex agent, but they
have a more comprehensive view of the environment.
 A model of the world is programmed into the internal system that
incorporates the agent's history.
• Goal-based agents:
 These agents expand upon the information model-based agents store by also
including goal information, or information about desirable situations.

Compiled by Gizachew M.(MSc in CS)

• Utility-based agents:
 These agents are similar to goal-based agents but provide an extra utility
measurement which rates each possible scenario on its desired result and
chooses the action that maximizes the outcome.
 Rating criteria examples could be the probability of success or the resources
required.
• Learning agents:
 These agents have the ability to gradually improve and become more
knowledgeable about an environment over time through an additional
learning element.
 The learning element will use feedback to determine how performance
elements should be changed to improve gradually.

Compiled by Gizachew M.(MSc in CS)