Lecture 7: Minimum Spanning Trees and Prims Algorithm

CLRS Chapter 23 Outline of this Lecture

Spanning trees and minimum spanning trees.

The minimum spanning tree (MST) problem.

The generic algorithm for MST problem.

Prims algorithm for the MST problem. The algorithm Correctness Implementation + Running Time
1

Spanning Trees Spanning Trees: A subgraph of a undirected graph

is a spanning tree of if it is a tree and contains every vertex of . Example:
              d         e c       a      b          b  a                  d         c e
spanning tree 1         b  a            d              c e spanning tree 3

Graph         a b               d        c e spanning tree 2

Spanning Trees Theorem: Every connected graph has a spanning tree. Question: Why is this true? Question: Given a connected graph nd a spanning tree of ?

, how can you

Weighted Graphs Weighted Graphs: A weighted graph is a graph, in which each edge has a weight (some real number). Weight of a Graph: The sum of the weights of all edges. Example:
  10         c 9     a   7 32         d 23     e     b      a       10      d     c 9
Tree 1.      a  7   10       d     c

     b   32   23    e
w=74     b 32    23       e

weighted graph         a b 32 7           d   23    c 9 e

Tree 2, w=71 Minimum spanning tree

Tree 3, w=72

Minimum Spanning Trees A Minimum Spanning Tree in an undirected connected weighted graph is a spanning tree of minimum weight (among all spanning trees). Example:
  10         c 9     a   7 32         d 23     e     b      a       10      d     c 9
Tree 1.      a  7   10       d     c

     b   32   23    e
w=74     b 32      23      e

weighted graph         a b 32 7           d   23    c 9 e

Tree 2, w=71 Minimum spanning tree

Tree 3, w=72

Minimum Spanning Trees Remark: The minimum spanning tree may not be unique. However, if the weights of all the edges are pairwise distinct, it is indeed unique (we wont prove this now). Example:

2 24

2 67 24

2 67

2 24 67

100 weighted graph

MST1

MST2

Minimum Spanning Tree Problem MST Problem: Given a connected weighted undi rected graph , design an algorithm that outputs a minimum spanning tree (MST) of . Question: What is most intuitive way to solve? Generic approach: A tree is an acyclic graph. The idea is to start with an empty graph and try to add edges one at a time, always making sure that what is built remains acyclic. And if we are sure every time the resulting graph always is a subset of some minimum spanning tree, we are done.

Generic Algorithm for MST problem Let be a set of edges such that , where
is a MST. An edge is a safe edge for , if
is also a subset of some MST. , we If at each step, we can nd a safe edge can grow a MST. This leads to the following generic approach: Generic-MST(G, w) Let A=EMPTY; while A does not form a spanning tree find an edge (u, v) that is safe for A add (u, v) to A return A How can we nd a safe edge?
8

How to nd a safe edge We rst give some denitions. Let connected and undirected graph. We dene:

be a

Cut A cut

of G is a partition of V.

crosses the cut Cross An edge

if one of its endpoints is in , and the other is in .

Respect A cut respects a set in A crosses the cut.

of edges if no edge

Light edge An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut.
9

How to nd a safe edge Lemma Let be a connected, undirected graph with a real-valued weight function dened on . Let be a subset of that is included in some minimum
spanning tree for , let be any cut of that
respects , and let be a light edge crossing the

cut . Then, edge is safe for .

It means that we can nd a safe edge by

1. rst nding a cut that respects ,

2. then nding the light edge crossing that cut.

That light edge is a safe edge.

10

Proof

1. Let .

, where

is a MST. Suppose

2. The trick is to construct another MST that con

tains both and , thereby showing is a safe edge for .

11

3. Since , and are on opposite sides of the cut

, there is at least one edge in on the
be path from to that crosses the cut. Let
. such edge. Since the cut respects ,

Since is a light edge crossing the cut, we

. have

MST T

another MST T

y x v u

y x v u

a cut respects A

12

4. Add to , it creates a cycle. By removing an edge from the cycle, it becomes a tree again.
In particular, we remove ( ) to make a new tree .

5. The weight of

is

6. Since is a MST, we must have hence is also a MST.

7. Since is also a subset of

is safe for .

(a MST),

13

Prims Algorithm The generic algorithm gives us an idea how to grow a MST. If you read the theorem and the proof carefully, you will notice that the choice of a cut (and hence the corresponding light edge) in each iteration is immaterial. We can select any cut (that respects the selected edges) and nd the light edge crossing that cut to proceed. The Prims algorithm makes a nature choice of the cut in each iteration it grows a single tree and adds a light edge in each iteration.

14

Prims Algorithm : How to grow a tree Grow a Tree

Start by picking any vertex to be the root of the tree.

While the tree does not contain all vertices in the graph nd shortest edge leaving the tree and add it to the tree .

Running time is

15

More Details
Step 0: Choose any element ; set and . (Take as the root of our spanning tree.)

Step 1: Find a lightest edge such that one endpoint . Add this edge to is in and the other is in and its (other) endpoint to .

Step 2: If , then stop & output (minimum)

. Otherwise go to Step 1. spanning tree

The idea: expand the current tree by adding the lightest (shortest) edge leaving it and its endpoint.

b a r d

26 c 16 14 e

24 20 g 8 f

h 12 12 i r a

b 26 c

24 20 16 14 8 e g f

h 12 23 new new edge

16

12 i

23

Prims Algorithm Worked Example

b 4 a 8 c 9 2 1 10 8 9 8 c 2 1 d 9 f 7 5 2 f 8 d 10 7 9 5 2 e 6 g Connected graph

b 4 a

e 6 g

Step 0 S={a} V \ S = {b,c,d,e,f,g} lightest edge = {a,b}

17

Prims Algorithm Prims Example Continued

b 4 a 8 c 9 2 8

10 7 d 9

e 6 5 2 f g

Step 1.1 before S={a} V \ S = {b,c,d,e,f,g} A={} lightest edge = {a,b}

1 10 8 7 d 2 9 1

b 4 a 8 c 9

e 6 5 2 f g

Step 1.1 after S={a,b} V \ S = {c,d,e,f,g} A={{a,b}} lightest edge = {b,d}, {a,c}

18

Prims Algorithm Prims Example Continued

b 4 a 8 c 9 2 8

10 7 d 9

e 6 5 2 f g

Step 1.2 before S={a,b} V \ S = {c,d,e,f,g} A={{a,b}} lightest edge = {b,d}, {a,c}

1 10 8 7 d 2 9 1

b 4 a 8 c 9

e 6 5 2 f g

Step 1.2 after S={a,b,d} V \ S = {c,e,f,g} A={{a,b},{b,d}} lightest edge = {d,c}

19

Prims Algorithm Prims Example Continued

b 4 a 8 c 9 2 1 10 8 9 8 c 2 1 d 9 f 7 5 2 f 8 d 10 7 9 5 2 e 6 g

Step 1.3 before S={a,b,d} V \ S = {c,e,f,g} A={{a,b},{b,d}} lightest edge = {d,c}

b 4 a

e 6 g

Step 1.3 after S={a,b,c,d} V \ S = {e,f,g} A={{a,b},{b,d},{c,d}} lightest edge = {c,f}

20

Prims Algorithm Prims Example Continued

b 4 a 8 c 9 2 8

10 7 d 9

e 6 5 2 f g

1 10 8 7 d 2 9 1

Step 1.4 before S={a,b,c,d} V \ S = {e,f,g} A={{a,b},{b,d},{c,d}} lightest edge = {c,f}

b 4 a 8 c 9

e 6 5 2 f g

Step 1.4 after S={a,b,c,d,f} V \ S = {e,g} A={{a,b},{b,d},{c,d},{c,f}} lightest edge = {f,g}

21

Prims Algorithm Prims Example Continued

b 4 a 8 c 9 2 8

10 7 d 9

e 6 5 2 f g

Step 1.5 before S={a,b,c,d,f} V \ S = {e,g} A={{a,b},{b,d},{c,d},{c,f}} lightest edge = {f,g}

1 10 8 7 d 2 9 1

b 4 a 8 c 9

e 6 5 2 f g

Step 1.5 after S={a,b,c,d,f,g} V \ S = {e} A={{a,b},{b,d},{c,d},{c,f}, {f,g}} lightest edge = {f,e}

22

Prims Algorithm Prims Example Continued

b 4 a 8 c 9 2 8

10 7 d 9

e 6 5 2 f g

1 10 8 7 d 2 9 1

Step 1.6 before S={a,b,c,d,f,g} V \ S = {e} A={{a,b},{b,d},{c,d},{c,f}, {f,g}} lightest edge = {f,e} Step 1.6 after S={a,b,c,d,e,f,g} V \ S = {} A={{a,b},{b,d},{c,d},{c,f}, {f,g},{f,e}} MST completed

b 4 a 8 c 9

e 6 5 2 f g

23

Recall Idea of Prims Algorithm

Step 0: Choose any element and set (Take as the root of our spanning tree.)

and

Step 1: Find a lightest edge such that one endpoint is in and the other is in . Add this edge to and its (other) endpoint to . Step 2: If  , then stop and output the minimum span . ning tree
Otherwise go to Step 1.

Questions:

Why does this produce a Minimum Spanning Tree?

How does the algorithm nd the lightest edge and update efciently? How does the algorithm update

efciently?
24

Prims Algorithm Question: How does the algorithm update

efciently?

Answer: Color the vertices. Initially all are white. Change the color to black when the vertex is moved to . Use color[ ] to store color.

Question: How does the algorithm nd the lightest edge and update efciently? Answer: (a) Use a priority queue to nd the lightest edge. (b) Use pred[ ] to update .

25

Reviewing Priority Queues Priority Queue is a data structure (can be implemented as a heap) which supports the following operations:

insert( ): Insert with the key value in .

u = extractMin(): Extract the item with the minimum key value in .

decreaseKey( ): Decrease s key value to

Remark: Priority Queues can be implemented so that

each operation takes time . See CLRS!
26

Using a Priority Queue to Find the Lightest Edge

Each item of the queue is a triple , where is a vertex in , is the weight of the lightest edge from to any vertex in , and is the endpoint of this edge in . The array is used to build the MST tree.

b a r d

26 c 16 14 e

24 20 g 8 f

h 12 12 i r a

b 26 c

24 20 16 14 8 e g f

h 12 23 12 i

23

key[f] = 8, pred[f] = e key[i] = infinity, pred[i] = nil key[g] = 16, pred[g] = c key[h] = 24, pred[h] = b f has the minimum key

new edge key[i] = 23, pred[i] = f After adding the new edge and vertex f, update the key[v] and pred[v] for each vertex v adjacent to f
27

Description of Prims Algorithm Remark: is given by adjacency lists. The vertices in are stored in a priority queue with key=value of lightest edge to vertex in .
) Prim( for each
     

initialize ; ;

 
    ; ! 
 ;  " new PriQueue( ); " while( is nonempty) "$# u= extraxtMin(); )(* & % ' for each (  )    ! % if (( )&&( -.   %/ %! ; -. "$#  decreaseKey( % %! );  0 %! ;  
  

start at root put vertices in until all vertices in MST lightest edge
%! ,+ "

)) new lightest edge

%!

When the algorithm terminates,

3

"

0

and the MST is

%! $ 4/%&

The pred pointers dene the MST as an inverted tree rooted at .

28

Example for Running Prims Algorithm

a 1 b 2 c u key[u] pred[u] a 3 4 10

3 4 d 1 5

f b c d e f

29

Analysis of Prims Algorithm Let and . The data structure PriQueue supports the following two operations: (See CLRS)

to extract each vertex from the queue.

Done once for each vertex

time to decrease the key value of neighboring vertex.

Done at most once for each edge

Total cost is then

30

Analysis of Prims Algorithm Continued

Prim(G, w, r) { for each (u in V) { key[u] = +infinity; color[u] = white; } key[r] = 0; pred[r] = nil; Q = new PriQueue(V); while (Q. nonempty()) { u = Q.extractMin(); for each (v in adj[u]) { if ((color[v] == white) & (w(u,v) < key[v]) { key[v] = w(u, v); Q.decreaseKey(v, key[v]); pred[v] = u; } } color[u] = black; } } [O(log n) + O(deg(u) log n)] u in V
31

2n

1 1 n 1 O(log n) 1 1 O(deg(u) log n) 1 O(log n) 1 1

Analysis of Prims Algorithm Continued So the overall running time is

32