DATA STRUCTURES & ALGORITHMS MINIMUM SPANNING TREE

DATA STRUCTURES & ALGORITHMS
MINIMUM SPANNING TREE
Ngo Quoc Viet-2023 1

Contents
+Minimum Spanning Tree problem
+Kruskal algorithm
+Prim algorithm
+Summary
2

Lecture outcomes
1. Understanding Kruskal and Prim algorithms to determine
minimum spanning tree.
2. Implementing the Kruskal and Prim using C++/Python.
3. Using the Python/C++ packages to implement Kruskal and
Prim.
3

The Spanning Tree
+Given an undirected connected graph G, a Spanning Tree is
defined as a tree subgraph (without cycles) having all vertices
of G and all vertices connected. A graph can have many
spanning trees
or or or
Some Spanning Trees from A
Graph A
4

The Spanning Tree
+The number of spanning trees of the graph G
𝑡 𝐺 = ൞
1 𝐺 𝑖𝑠 𝑡𝑟𝑒𝑒
𝑛 𝐺 𝑖𝑠 𝑐𝑦𝑐𝑙𝑒 𝑔𝑟𝑎𝑝ℎ 𝑐𝑎𝑙𝑙𝑒𝑑 𝐶𝑛
𝑛𝑛−2 𝐺 𝑖𝑠 𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒 𝑔𝑟𝑎𝑝ℎ 𝑐𝑎𝑙𝑙𝑒𝑑 𝐾𝑛
+Complete graph: every pair of vertices is connected by a
unique edge
+Bigraph: vertex set in G divided into two disjoint sets U, V.
Each edge connects only point in U with point in V
5

The Minimum Spanning Tree
+The minimum spanning tree is the spanning tree whose sum
of the weights of the edges is less than all other spanning
trees
+Some algorithms to find MST on graph with or without weight:
Prim, Kruskal, Boruvka.
5
7
2
1
3
4
2
1
3
Complete Graph Minimum Spanning Tree
6

Applications of MST
+Network design: direct applications in the design of networks,
including computer networks, telecommunications networks,
transportation networks, water supply networks, and electrical
grids. We try to determine set of lines that connects all your offices
with a minimum total cost.
+Cluster analysis: k clustering problem can be viewed as finding an
MST and deleting the k-1 most expensive edges. Clustering points
in the plane, single-linkage clustering, graph-theoretic clustering,
and clustering gene expression data
+Traveling salesman problem: use the MST to get a 2-approximation
to the optimal solution to the traveling salesman problem.
7

Applications of MST
+Image registration and segmentation
+Curvilinear feature extraction in computer vision
+Handwriting recognition of mathematical expressions
+Circuit design: implementing efficient multiple constant
multiplications, as used in finite impulse response filters
+Regionalization of socio-geographic areas, the grouping of
areas into homogeneous, contiguous regions
+Comparing ecotoxicology data
+Measuring homogeneity of two-dimensional materials
8

Prim’s Algorithm
+Input: A non-empty connected weighted graph with
vertices V and edges E (the weights can be negative).
+Initialize: Vnew = {x}, where x is an arbitrary node (starting point)
from V, Enew = {}
+Repeat until Vnew = V:
+Choose an edge {u, v} with smallest weight such that u is
in Vnew and v is not (if there are multiple edges with the same weight,
any of them may be picked)
+Add v to Vnew, and {u, v} to Enew
+Output: Vnew and Enew describe a minimal spanning tree
9

Prim’s Algorithm Example
1 0
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 1
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

MST-Giải thuật Prim
1 2
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 3
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 4
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 5
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 6
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 7
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 8
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

1 9
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J
4
1
2 3
2 1
3
5
3
4
2
5 6
4
4
10
A
B C
D
E F
G
H
I
J

Prim’s Algorithm
+Input: A non-empty connected weighted graph with
vertices V and edges E (the weights can be negative).
+Initialize: Vnew = {x}, where x is an arbitrary node (starting point)
from V, Enew = {}
+Repeat until Vnew = V:
+Choose an edge {u, v} with smallest weight such that u is
in Vnew and v is not (if there are multiple edges with the same weight,
any of them may be picked)
+Add v to Vnew, and {u, v} to Enew
+The problem: how to find smallest edge?
2 0

Prim’s Algorithm with priority queue
+We know: stack is first, last out. Queue is first in, first out.
+Priority queue is least-first out.
+The smallest element is the first one removed. Of course, we can
define a largest first out priority queue.
+The definition of smallest is up to the problem/programmer
+If there are several smallest elements, the implementer must decide
which to remove first
+ Remove any smallest element (don’t care which)
+ Remove the first one added
+Priority queue can help to find smallest in Prim’s MST
algorithm
2 1

2 2

2 3

2 4

Prim’s Algorithm implemetation
+Edge structure
struct Edge {
int w = INF; //weight. INF (INT_MAX) no edge between 2 vertices
to = -1; //to vertice
};
+Adj matrix for graph: using <vector <vector>> or 2D array.
+ vector <vector <int>> G; // or G[][]: adjacency matrix of graph
+ n: number of vertices
+One 2D vector to keep selected vertices in MST
+ vector <bool> selected(n, false); // indicates which vertices we already
have selected
+MST tree: vector <Edge> mst(n);
2 5

Prim’s Algorithm implemetation
Initialize: mst[0].w = 0;
for (int i=0; i<n; ++i) { // all vertices
for (int j = 0; j < n; ++j) { // find the vertex with minimum weight value, from the set of vertices
v = -1; // not yet included in MST
if (!selected[j] && (v == -1 || mst[j].w < mst[v].w))
v = j;
}
selected[v] = true; total_weight += mst.w;
for (int to = 0; to < n; ++to) { // Update weight value and parent index of the adjacent vertices of
// the picked vertex. Consider only those vertices which are not yet
// included in MST
if (adj[v][to] < mst[to].w)
mst[to] = {adj[v][to], v};
}
}
2 6

Prim’s Algorithm Complexity
+Running Time: 𝑂(𝑚 + 𝑛 log 𝑛) (𝑚 = 𝑒𝑑𝑔𝑒𝑠, 𝑛 = 𝑛𝑜𝑑𝑒𝑠)
+If not used heap, the run time will be 𝑂(𝑛2).
+No need to sort by edge weight first
+Using the vertices, there is no need to check the possibility of
creating a cycle
2 7

MST-Kruskal’s Algorithm
1. Sort ascending by edge weight
2. Draw the edge with the least weight. Check if it generates a
cycle with the spanning-tree formed till now. If the cycle is
not formed, include this edge. Else, discard it and move to
the next.
3. Repeat step 2 until there are (𝑉−1) edges in the spanning
tree
2 8

MST- Kruskal’s Algorithm
2 9
edge dv
(D,E) 1
(D,G) 2
(E,G) 3
(C,D) 3
(G,H) 3
(C,F) 3
(B,C) 4
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

MST- Kruskal’s Algorithm
3 0
edge dv
(D,E) 1 
(D,G) 2
(E,G) 3
(C,D) 3
(G,H) 3
(C,F) 3
(B,C) 4
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

MST- Kruskal’s Algorithm Example
3 1
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3
(C,D) 3
(G,H) 3
(C,F) 3
(B,C) 4
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

3 2
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3
(G,H) 3
(C,F) 3
(B,C) 4
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10
Accepting edge (E,G) would create a cycle

3 3
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3
(C,F) 3
(B,C) 4
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

3 4
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3
(B,C) 4
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

3 5
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3 
(B,C) 4
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

3 6
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3 
(B,C) 4 
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

3 7
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3 
(B,C) 4 
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4 
(B,F) 4
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

3 8
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3 
(B,C) 4 
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4 
(B,F) 4 
(B,H) 4
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

3 9
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3 
(B,C) 4 
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4 
(B,F) 4 
(B,H) 4 
(A,H) 5
(D,F) 6
(A,B) 8
(A,F) 10

4 0
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3 
(B,C) 4 
5
1
A
H
B
F
E
D
C
G 3
2
4
6
3
4
3
4
8
4
3
10 edge dv
(B,E) 4 
(B,F) 4 
(B,H) 4 
(A,H) 5 
(D,F) 6
(A,B) 8
(A,F) 10

4 1
edge dv
(D,E) 1 
(D,G) 2 
(E,G) 3 
(C,D) 3 
(G,H) 3 
(C,F) 3 
(B,C) 4 
5
1
A
H
B
F
E
D
C
G
2
3
3
3
edge dv
(B,E) 4 
(B,F) 4 
(B,H) 4 
(A,H) 5 
(D,F) 6
(A,B) 8
(A,F) 10
Done
Total Cost =  dv = 21
4
}

Kruskal’s Algorithm implemetation
4 2

Problem of detecting cycle
+Edge checking creates cycles that can be slow. In the worst
case, the edge causing cycle may be the last edge to be
examined in the for loop (loop to add edge by edge). We
need to check if there is cycle in spanning-tree formed till now
with the new edge being examined.
+We can use BFS or DFS to detect cycle in graph. It takes O(V+E) to
traverse graph and detect cycle.
+In the worst case, it takes 𝐸 × 𝑂 𝑉 + 𝐸 to build MST from
graph.
4 3

Check cycle in Undirected graph
+Cycle: the path which starts from any given vertex and ends at
the same vertex is called the cycle of the graph
4 4

+Given a graph and an unvisited node, run a Depth First Search
traversal from the unvisited node to detect cycles in the graph
+A DFS of a connected graph produces a tree
+A graph has a cycle if and only if it contains a back edge
+A back edge is an edge joining a node to itself or one of its
ancestors in the depth-first search tree produced by the DFS
algorithm.
4 5

Now for node B, the only neighboring node
is A which is visited and also the parent of
node B, so we backtrack to node A
Since node A is unvisited, we visit it and its
neighbor node are B and C. We go to node
B first.
4 6

We visit C, since we are coming from node
A, the parent remains equal to node A
Now from C, we have node 3 neighbors. First
is node A which is visited but is the parent of
C, and the other is D and E which are
unvisited. Therefore, we move to node D
4 7

Now from D, we have C and E as
neighbors. Here, C is visited but the parent
of D so we move to the unvisited neighbor
of E. At last, we mark E as visited. Since we
are from D, we update our parent variable
to D
From E, we have a neighbor i.e., D which is
visited but the parent of E, and another is
C which is visited but is not a parent of E.
Cycle C-D-E-C is detected .
4 8

MST-Kruskal’s Algorithm using Disjoin set
1. Sort ascending by edge weight
2. Draw the edge with the least weight. Check if it generates a
cycle with the spanning-tree formed till now using a union-
find algorithm. If the cycle is not formed, include this edge.
Else, discard it and move to the next
3. Repeat step 2 until there are (𝑉−1) edges in the spanning
tree
4 9

MST-Kruskal’s Algorithm using Disjoin set
1. Initialize an array for storing the groups of the vertices. Initialize an array for storing
the edges of the graph with their weights. Initialize the spanning tree as an empty
array.
2. Put all vertices of the graph into different groups (disjoint sets)
3. Sort the array of edges in increasing order of weights.
4. Continue step 5 till there is no more edge remaining in the sorted array of edges
5. If the group of one node of an edge does not match the group of the other
node of the edge, add them both to the same group and add the edge to the
spanning tree array. (Find & Union)
6. Iterate through the spanning-tree array and add the weights of the edges.
7. The sum of the edges thus obtained is the weight of the minimum spanning tree,
while the spanning tree array contains the edges for the same
5 0

Kruskal’s implementation using disjoint set
+A disjoint set data structure is used to track the division of
elements into different disjoint subsets
+Disjoint data structure is used to find if two nodes are in the
same subset or not.
+The union-find algorithm performs two operations on this data
structure, namely, find and union.
5 1

Disjoint sets in Kruskal’s algorithm
+A disjoint-set data structure is defined as a data structure that
keeps track of a set of elements partitioned into a number of
disjoint (non-overlapping) subsets.
+The disjoint set data structure is also known as Union-find data
structure and Merge-find set. It is a data structure that contains a
collection of disjoint or non-overlapping sets.
+The union-find algorithm performs two operations on this data
structure, namely, find (this can be used for determining if two
elements are in the same subset) and union (Join two subsets into
a single subset. Here first we have to check if the two subsets
belong to same set. If no, then we cannot perform union).
5 2

Kruskal’s Algorithm Complexity
+Running Time = 𝑂(𝑚 log 𝑛) (m = edges, n = nodes).
QuickSort algorithm
+Edge checking creates cycles that can be slow. However, using the
data structure “Union-find” will overcome the disadvantage
+After sorting, we iterate through all edges and apply the find-union
algorithm. The find and union operations can take at most 𝑂(𝑙𝑜𝑔𝑛).
So overall complexity is 𝑂(𝑚𝑙𝑜𝑔𝑚 + 𝑚𝑙𝑜𝑔𝑛).
+The value of m can be at most 𝑂 𝑛2 , so 𝑂 log 𝑛 , 𝑂 log 𝑚 the
same. Therefore, the overall time complexity 𝑂 nlog 𝑛 or
𝑂 mlog 𝑚 .
5 3

Summary
+The minimum spanning tree is the spanning tree whose sum
of the weights of the edges is less than all other spanning
trees
+The Prim’s Algorithm loop by vertices of graph to determine
MST
+The Kruskal s Algorithm loop by edges of graph to determine
MST.
5 4

DATA STRUCTURES & ALGORITHMS MINIMUM SPANNING TREE

More Related Content

Similar to DATA STRUCTURES & ALGORITHMS MINIMUM SPANNING TREE

Recently uploaded

DATA STRUCTURES & ALGORITHMS MINIMUM SPANNING TREE