Tsp problem

Ameisen Systeme

Dr. Dagmar Monett Díaz
ba@monettdiaz.com

Fachbereich Berufsakademie, Fachrichtung Informatik

Ant Colony Systems

Ant Colony Optimization

Outline
• What is Swarm Intelligence? Motivations
• Ant Colonies
• Ant Colony Optimization
• Optimization problem
– Characteristics
– Algorithm
– Examples
– Modifications
– Applications

What is Swarm Intelligence?
• “Swarm Intelligence is the property of a
system whereby the collective behaviors of
(unsophisticated) agents interacting locally
with their environment cause coherent
functional global patterns to emerge.”
• Characteristics of a swarm:
– distributed, no central control or data source;
– no (explicit) model of the environment;
– perception of environment, i.e. sensing;
– ability to change environment.

What is Swarm Intelligence?

• Swarm systems are examples of behavior-
based systems exhibiting:
– multiple lower level competences;
– situated in environment;
– limited time to act;
– autonomous with no explicit control provided;
– problem solving is emergent behavior;
– strong emphasis on reaction and adaptation;

Motivations
• Robust nature of animal problem-solving
– simple creatures exhibit complex behavior;
– behavior modified by dynamic environment.
• Emergent behavior observed in:
– bacteria
– ants
– bees
– ...

Motivations
• 1018 living insects (rough estimate)
• ~2% of all insects are social
• Social insects are:
– All ants, all termites
– Some beeps, some wasps
• 50% of all social insects are ants
• Avg weight of one ant between 1 and 5 mg
• Ants have colonized Earth for 100 million
years, Homo sapiens for 50,000 years

Motivations
Each element of the swarm has its own simple
behaviour, and a set of rules for interacting with its
fellows, and with the environment.

Every element is the same – there is no central
controller.

However, X emerges as a result of these local
interactions.

E.g. ants finding food, termites building mounds,
jellyfish.

Motivations
Ant colony size: from as few as 30 to millions of
workers

Coordination of activities:
Set of dynamical mechanisns whereby structure
appears at the global level as the result of interactions
among lower-level components
The rules specifying the interactions among the
system‘s constituent units are executed on the basis of
purely local information, without reference to the global
pattern, which is an emergent property of the system
rather than a property imposed upon the system by an
external ordering influence.

Stigmergy

Indirect communication via interaction
with environment [Gassé, 59]

I.e. swarm behaviour emerges from the way individuals
communicate through and affect their environment

Self-organization
• How do social insects achieve self-
organization?
– Communication is necessary
– Two types of communication:
Direct: antennation, trophallaxis (food or liquid
exchange), mandibular contact, visual contact,
chemical contact, etc.
Indirect: two individuals interact indirectly when
one of them modifies the environment and the
other responds to the new environment at a later
time (stigmergy!!)

Ant Colonies
• Ants are behaviorally unsophisticated;
collectively perform complex tasks.
• Ants have highly developed sophisticated
sign-based stigmergy
– communicate using pheromones;
– trails are laid that can be followed by other
ants.

Pheromone Trails
• Species lay pheromone trails traveling from
nest to nest, or possibly in both directions
• pheromones accumulate with multiple ants
using path. Pheromones also evaporate
• helps in avoiding suboptimal solutions –
local optima
• In ACO: may differ from how it takes places
in the real world
Food source
Nest

Introduction ACO

• Ant Colony Optimization (ACO)
algorithms attempt to imitate or to
simulate the process of collective ants
behavior;
Important:
– to understand how do ant colonies behave,
– Mechanisms and strength of stigmergy
– collective intelligent behavior and how to use it.

History of Ant Algorithms

• Goss et al. 1989,
Deneuborg et al. 1990,
experiments with
Argentine ants
• Dorigo et al. 1991,
applications to shortest
path problems
• Now: established
method for various
optimization problems

What are ant algorithms?

“Ant algorithms are multi-agent systems
that exploit artificial stigmergy as a means for
coordinating artificial ants for the solution of
computational problems”

Ants in action
(The history of „Anton Ameise“)

*???* Ameisenhügel

*???* Anton Ameise (grosser Entdecker)
*???*

*gelangweilt entscheidet sich für einen Spaziergang*

Pheromonpfad – Folge der Autokatalyse

Heureka!

Keks, der mir aus der Hand gefallen ist

Ants in Action (Stigmergy)

E E E

D D
C C
H H
B B

A A A

Use of tour A-B-C-D-E increase, A-B-H-D-E decline with time

Ants in Action (discrete)

15 15 10 20

15 15 10 20

Assumptions: discrete time intervals, at t=0 no pheromone on edges
30 ants both A->B & E->D
ants move at 1 unit per timestep
strong visited/marked routes synonym with smallest path

ACO

• ACO is a meta-heuristic that uses strategies of real ants
to solve optimization problems
• ACO was initially proposed by Colorni, Dorigo and
Maniezzo
• The main underlying idea was that of parallelizing
search over several constructive computational threads,
all based on a dynamic memory structure incorporating
information on the effectiveness of previously obtained
results and in which the behavior of each single agent is
inspired by the behavior of real ants

Optimization problem for ACO
• Optimization problem in general:
– given: X, f:X  ℝ, f:X  {True, False}
– find: x  X, so that f(x) minimal (or maximal) and c(x) feasible.

• Optimization problem for ACO:
– find
• basic components C = {c1,...,cn}, so that
• partial solution subsets S are in C,
• feasible (partial) solution F are in C,
• solution s in C
• cost function f.
– then
• iterative extend (feasible) partial solutions with basic components
in order to find a solution s, so that f(s) is minimal (or maximal).
• Pheromone deposit on each component ci to control the search

TSP: The problem
A salesman must visit n cities, passing through
each city only once, beginning from one of them
which is considered as his base, and returning to it.

The cost of the transportation among the cities
(whichever combination possible) is given.

The program of the journey is requested, that is the
order of visiting the cities in such a way that the
cost is the minimum.

TSP: Examples

From http://www.tsp.gatech.edu

TSP: Examples
By George Dantzig, Ray Fulkerson, and Selmer Johnson (1954)
Original instance = 49 cities (one city from each of the 48 states in
the U.S.A. and adding Washington, D.C.). Costs of travel = road
distances
Solved instance: 42-city problem obtained by removing Baltimore,
Wilmington, Philadelphia, Newark, New York, Hartford, and
Providence.

TSP: Examples

By Groetschel and Holland (1987)

Solved instance: 666 interesting places in the world

TSP: Examples

By Applegate, Bixby,
Chvátal, and Cook (2001)

Solved instance:
15,112 German cities

TSP: Examples

By Applegate, Bixby, Chvátal,
Cook, and Helsgaun (2004)

Solved instance:
24,978 cities in Sweden

TSP: Examples
Current best:
by Yuichi Nagata (2009)

Solved instance:
100,000 cities
(Mona Lisa TSP)

TSP: Examples

By Helsgaun (2009)

Solved instance:
1,904,711 cities
(World TSP)

TSP: Examples

By Groetschel

Solved instance: 52 locations in Berlin

(See berlin52:
- Concorde program
- berlin52.tsp
- optimal tour)

E.g. A 4-city TSP
Initially, random levels of pheromone are scattered on the edges

A B

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

E.g. A 4-city TSP
An ant is placed at a random node

A B

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

E.g. A 4-city TSP
The ant decides where to go from that node, based on probabilities
calculated from:
- pheromone strengths, A B
- next-hop distances.

Suppose this one
chooses BC

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

E.g. A 4-city TSP

The ant is now at AC, and has a `tour memory’ = {B, C} – so he cannot
visit B or C again. A B
Again, he decides next hop
(from those allowed) based
on pheromone strength
and distance;
Suppose he chooses
CD

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

E.g. A 4-city TSP
The ant is now at D, and has a `tour memory’ = {B, C, D}
There is only one place he can go now:
A B

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

E.g. A 4-city TSP
So, he finished his tour, having gone over the links:
BC, CD, and DA. AB is added to complete the tour.
A B

Now, pheromone on the tour
is increased, in line with the
fitness of that tour.

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

E.g. A 4-city TSP
Next, pheromone everywhere is decreased a little, to model
decay of trail strength over time
A B

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

E.g. A 4-city TSP
We start again, with another ant in a random position.

A B
Where will he go?

D
Pheromone C

AB: 10, AC: 10, AD, 30, BC, 40, CD 20
Ant

Optimization problem for ACO

• more rigorous mathematical models.
• TSP has been a popular problem for the ACO models.
- several reasons why TSP is chosen…..

Key concepts:
• Positive feedback – build a solution using local
solutions, by keeping good solutions in memory
• Negative feedback – want to avoid premature
convergence, evaporate the pheromone.
• Time scale – number of runs are also critical.

Design choices

• Ants are given a memory of visited nodes
• Ants build solutions probabilistically without
updating pheromone trails
• Ants deterministically backward retrace the
forward path to update pheromone
• Ants deposit a quantity of pheromone function
of the quality of the solution they generated

Ant System

• Developed 1991 by Marco Dorigo
• Used to solve TSP
• Transition from city i to j depends on:
– Tabu list – list of cities not visited
– Visibility = 1/lenghtij; represents local information –
heuristic desirability to visit city j when in city i.
– Pheromone trail Tij(t) for each edge – represents the
learned desirability to visit city j when in city i.

• Generally, have several ants searching the
solution space
Nr. cities = Nr. ants

General Ant Colony Heuristic
• Ants generation and
activity: compute
make decision
transition
• while resources and move
probabilities
available: create ant
• for each ant:
evaluate
1. initialize update the
the current
state
2. let ant run until a state
solution is found possibly
update
3. possibly: update
pheromone
pheromone and
routing table

Transition Rule

• Probability of ant k going from city i to j:

k
p (t ) 
 ij  

(t ) .  ij


ij
  il (t ) . il 
 J ik

• Alpha and beta are adjustable parameters.

Transition Rule

k
pij (t ) 
  

ij (t ) .  ij


  il (t ) . il 
J ik

• Alpha = 0 : represents a greedy approach
• Beta = 0 : represents rapid selection of
tours that may not be optimal.
• Thus, a tradeoff is necessary.

Pheromone update
• Pheromone update :

k k k
  Q / L (t ) if (i, j )  T (t ) else 0
ij

• T is the tour done at time t by ant k, L is the
length, Q is a heuristic parameter.
• Pheromone decay:

 ij (t )  (1   ). ij (t )   ij (t )

ACO - Metaheuristic

init pheromone τi:=const for each component ci;
while termination condition not met: „pheromone persistence“
0<ρ≤1
for all ants i: construct_solution(i);
for all ants i: global_pheromone_update(i);
for all pheromones i: evaporate: τi:=(1-ρ)∙τi; Constraint „c(x)=True“?

construct_solution(i); init s:={ }; „trail intensities“

while s is not a solution:  0 j not allowed
 
choose cj with probability p =     j
j
„visibility“
otherwise
  
 j '  j '
expand s by cj;  j ' allowed

global_pheromone_update(i);
for all cj in the solution s:
increase pheromone: τj:=τj+ const / f(s); Cost function

Ant-Cycle for TSP

• Tabu List
5 1
4
• Random Walk 3 1 1
2

• Priorities

• Return and evaluate tour length

Tabu List

n : Nr. of cities = a tour
m : Nr. of ants
k : Ant index
s : pointer to Tabu list (current city)

For each action on each iteration from the Ant-Cycle:
Insert for each ant k the visited city

For k := 1 to m do
insert town of ant k in Tabuk(s)
od

Random Walk & Priorities

i, j : edge between nodes i, j allowedk : for k in i feasible,
nij : visibility: 1/distance(i, j) adjacent, not visited cities
 : Weights for marking
 : Weights for close nodes From Ant-Routing-Table:

Transition probability from city i to city j for ant k

k [ij (t )] [nij ]
pij (t )   
if j  allowedk , else 0
 [il (t )] [ nil ]
l  allowed k

,  are control parameters that determine the sensitivity
of the algorithm to distance and pheromone

Return and Evaluate

i,j : Edges between nodes i, j Q/Lk : Const/tour length ant k
ij (t): Marks at time t
 : evaporation each (t, t+n) With Tabu List, Ant-Routing-Table:

Mark_delta = Marks Sum for all ants k that have walked the edge (i, j)

m
k Q
 ij    ij
 ijk  if (i, j) Tour k , else 0
k 1 Lk

Marks tour = evaporation * Mark_alt + Mark_delta

 ij (t  n)     ij (t )   ij

Ant Systems Algorithm for TSP
Initialize

Place each ant in a randomly chosen city

For Each Ant

Choose NextCity(For Each Ant)

yes more cities
to visit

no
Return to the initial cities

Update pheromone level using the tour cost for each ant

no Stopping
criteria

yes
Print Best tour

TSP Models

• Ant density model
–  ijk = Q
– Pheromone increase in trail is independent of
lengthij
• Ant quantity model
–  ijk = Q / lengthij
– Shorter edges made more desirable by making
trail inversely proportional to lengthij

Experimental studies

• 30 city problem, NC = 5000 cycles
• Q found to be (relatively) unimportant

Best Parameter Set Average Result Best Result
Ant- =1, =5, =0.99
426.740 424.635
density
Ant- =1, =5, =0.99
427.315 426.635
quantity
Ant- =1, =5, =0.5
424.250 423.741
cycle

Parameter Sensitivity
• Bad solutions and stagnation
– For high values of  the algorithm enters
stagnation behavior very quickly without finding
very good solutions

• Bad solutions and no stagnation
–  too low, insufficient importance associated
with trail
• Good solutions
–  ,  in the central area (1,1), (1,2), (1,5), (0.5, 5)

Exploiting ant synergy

• In original algorithm, all ants start from one
town. Modify algorithm to distribute ants
amongst nodes
– Better than “one town” algorithm.
– Approximately n = m proved optimal.
– Allow communication between ants I.e.
pheromone sensing (0<1)

Exploiting ant synergy

• Initialization
– Placing ants uniformily (rather than aggregated
on individual nodes) resulted in superior
performance.
• Employ ‘elitest’ (GAs) strategy
– best-so-far trail is reinforced more than in the
standard algorithm (ne - Q/L);
– found optimal number of elitest ants.

Modifications
• New transition rules
• New pheromone update rules
• Candidate lists of closest cities
• Local search methods in conjunction with ACO
(Hybrid ACO)
• Elitism, worst tours (pheromone removed), local
search enhancement
• Diversification: All pheromone trail values are
reinitialized if no improvement is made in S
generations
• Intensification – keeping new best solutions in
memory and replacing the current ones with them;
again similar to elitism

Artificial vs. real ants
Main similarities:
• Colony of individuals
• Exploitation of stigmergy & pheromone
trail
– Stigmergic, indirect communication
– Pheromone evaporation
– Local access to information
• Shortest path & local moves (no jumps)
• Stochastic state transition

Artificial vs. real ants
Main differences:
• Artificial ants:
– Live in a discrete world
– Deposit pheromone in a problem dependent way
– Can have extra capabilities (local search,
lookahead, etc.)
– Exploit an internal state (memory)
– Deposit an amount of pheromone function of the
solution quality
– Can use local heuristic information

Applications of ACO
ACO algorithms have been applied to several
optimization problems now.
Some of them are:
– Job-scheduling problem
– TSP
– Graph-coloring
– Vehicle Routing
– Routing in telecommunication networks
– Sequential ordering
– Multiple knapsack problem

Some references
• Dorigo M. and G. Di Caro (1999). The Ant Colony Optimization Meta-Heuristic. In D.
Corne, M. Dorigo and F. Glover, editors, New Ideas in Optimization, McGraw-Hill, 11-32.
• M. Dorigo and L. M. Gambardella. Ant colonies for the traveling salesman problem.
BioSystems, 43:73–81, 1997.
• M. Dorigo and L. M. Gambardella. Ant Colony System: A cooperative learning approach to
the traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1):53–
66, 1997.
• G. Di Caro and M. Dorigo. Mobile agents for adaptive routing. In H. El-Rewini, editor,
Proceedings of the 31st International Conference on System Sciences (HICSS-31), pages 74–
83. IEEE Computer Society Press, Los Alamitos, CA, 1998.
• M. Dorigo, V. Maniezzo, and A. Colorni. The Ant System: An autocatalytic optimizing
process. Technical Report 91-016 Revised, Dipartimento di Elettronica,Politecnico di Milano,
Italy, 1991.
• L. M. Gambardella, ` E. D. Taillard, and G. Agazzi. MACS-VRPTW: A multiple ant colony
system for vehicle routing problems with time windows. In D. Corne, M. Dorigo, and F.
Glover, editors, New Ideas in Optimization, pages 63–76. McGraw Hill, London, UK, 1999.
• L. M. Gambardella, ` E. D. Taillard, and M. Dorigo. Ant colonies for the quadratic
assignment problem. Journal of the Operational Research Society,50(2):167–176, 1999.
• V. Maniezzo and A. Colorni. The Ant System applied to the quadratic assignment problem.
IEEE Transactions on Data and Knowledge Engineering, 11(5):769–778, 1999.
• Gambardella L. M., E. Taillard and M. Dorigo (1999). Ant Colonies for the Quadratic
Assignment Problem. Journal of the Operational Research Society, 50:167-176.

Tsp problem

More Related Content

What's hot

Similar to Tsp problem

Recently uploaded

Tsp problem