Graphs

Data Structures and Algorithms

Outline
 Non-Linear Data Structures
 Graphs
 Traversal
 BFS
 DFS
 Topological Sort
 Shortest Path
 Dijkstra’s Algorithm

2
Graph Traversal
 Visiting all vertices one bye one
 Just like we did in tree but remember tree has no cycles, so its bit different in
graph scenario
 We need to keep track of nodes that have been visited/discovered
 And if graph is disconnected
 We cannot traverse each vertex from just a single vertex
 Two ways:
 Breadth First Traversal
 Visit nodes that are closest to starting node before other nodes
 Depth First Traversal
 Visit recursively one side before going back to other side

3
Breadth First Search
 Given a graph G and a start vertex, breadth first search systematically explores
edges of G to discover vertices that are reachable from start vertex.
 It is named as breadth first because as it traverse the vertices at distance K
before the vertices at distance k+1, where k represents number of edges.
 BFS order of vertices From 1: 1 , 2, 3, 4, 5, 6, 7

 Vertices at edge distance of 1are visited before vertices at edge distance of 2.

 So, BFS discovers each vertex by exploring minimum possible number of edges.
4
Breadth First Search
 Progress can be tracked by maintaining a state for each vertex. Vertices can be
in three distinguished states:
 Not discovered
 Partially discovered: a vertex is discovered first time but not fully explored.
 Finished/Fully explored: vertex that has been fully explored, all its adjacent nodes have
been discovered.
 1 2
Tri-Coloring
 Vertices are given colors according to state
 3 4
White : un discovered
 Grey: partially discovered
 Black: finish/fully explored

5
Breadth First Search
 BFS: from 1  1’s adjacency list is fully explored now.
 All vertices are undiscovered at start.
1 2 1 2

3 3
∅ 2 3
 Go to 2’s adjacency list. 2 edge distance
 Its vertices are already discovered
 Mark start vertex as discovered 1 2
1 2 1 3
3
3
 Go to 3’s adjacency list. 2 edge distance
 Its vertices are already discovered
1 2

 Go to its adjacency1list 1 edge2distance 3

 first node is 2 and its un-discovered 2 3 ∅

3  Graph is fully explored
2nd node is 3 and it’s undiscovered
6
Breadth First Search (BFS)
 Algorithm: BFS(G, start)
 Input: Graph and start vertex of graph.
 Output: list of vertices reachable from start in order of their discovery time
 Steps:

1. Q = new Queue()
2. For each vertex v in G
3. color[v]=white
4. color[start]=grey
5. Q.enqueue(start)
6. While Q is not empty
7. u= Q.dequeue()
8. print(u)
9. For each vertex v adjacent to u
10. if color[v] is white //undiscovered
11. color[v]=grey //discovered
12. Q.enqueue(v)
13. End if
14. End For
15. color[u]=black //fully explored or finished
16. End While

7
Breadth First Search
 BFS: start vertex=1
1 1 2

3 4

∅ 1 2 3 4

1 2 1 2 1 2

3 4 5 3 4 5 3 4 5

3 4 5 4 5 6 6 5 6 7 6 7

8
Breadth First Search
 BFS: start vertex=1
1 2 1 2 11 2

3 4 5 3 4 5 3 4 5

6 7 6 7 6 7
6 7 7 ∅

 BFS Order: 1 2 3 4 5 6 7

9
Breadth First Search (Alternate Approach)
 Algorithm: BFS(G, start)
 Rather than using tri-coloring, just  Input: Graph and start vertex of graph.
maintain two states.  Output: list of vertices reachable from
start
 If a vertex is undiscovered  Steps:

 Q = new Queue()
It will be marked as visited=false 1.
2. For each vertex v in G
 If a vertex is pushed to queue it is 3. visited[v]=false
visited[start]=true
discovered 4.
5. Q.enqueue(start)
 Mark it visited = true 6. While Q is not empty
 u= Q.dequeue()
If a vertex is popped off it is fully 7.
8. print(u)
explored 9. For each node v adjacent to u
 if !(visited[v])
No marking 10.
11. visited[v]=true
12. Q.enqueue(v)
13. End if
14. End For
15. End While

10
Breadth First Search
 Start vertex=1

 Order: 1, 5, 3, 6, 2, 4
11
Breadth First Search
 BFS Tree
 If graph is connected, by discarding the edges that were not explored during BFS, we
can get a BFS tree of graph
 Which does not have cycle
1 2

3 4 5

6 7

12
Breadth First Search
 BFS Forest
 If graph is not connected, then by traversing each connected component separately a
forest of trees can be formed.
a b a b

c d g c d g

e f e f

a b a b

c d g c d g

e f e f
13
Depth First Search
 From start vertex traverse the vertices at one path in depth till last reachable
vertex before traversing other paths. Then go back to last visited node to explore
other paths recursively un till all reachable vertices have been traversed.
 DFS order of nodes from 1: 1 2 4 3 6 7 5

 DFS as it name suggest discover a vertex in depth. When it first discovers a vertex, it will
go to its adjacent vertices [adjacency list] and before discovering all its adjacent vertices
at once (like BFS does) it will just discover first (undiscovered) vertex, and then will go
to this vertex’s adjacent vertices. And process goes one.

14
Depth First Search
 DFS: from 1 
1 2 3’s adjacency list is fully explored
now.
3
1 2
 Mark start vertex as discovered
1 2 3
 We discovered 3 from 2parent
3
 Go back to 2 1 2
 Its fully explored now
 1
Go to its adjacency list, first node is 2 and2its 3
un-discovered
3
 Repeat process of going back, till start
vertex 1 2
1 2
 Go to adjacency list of 2, 1 is already
3
discovered, discover 3 3
15
Depth First Search
 Difference from BFS:
 3 is discovered from 2 and not 1
 Before discovering all adjacent vertices of 1, DFS has went to adjacent vertices of 2
d=0, p=∅ d=1, p=1

1 2
d=2, p=2

 Two different time stamps of visit? 3

 Discovery time: Vertex becomes grey
 Explore/Finish time: Vertex becomes black 1/6 2/5

1 2
3/4

3
 discovery/finish-time
16
Depth First Search
 Algorithm: DFS(G, start)
 Input: Graph and start vertex of graph.
 Output: list of vertices reachable from start with their discovery and finish time labels
 Steps:
1. For each vertex v in G
2. color[v]=white
3. DFS_Visit(start)
4. time=0 //a global timer
 DFS_Visit (start)
1. color[start]=grey //discovered
2. d[start]=time+1 //discovery time
3. time=time+1
4. print(start)
5. for each node v adjacent to start
6. if color[v]=white
7. DFS_Visit (v)
8. color[start]=black //finished
9. f[start]=time+1 //finish time
10. time=time+1

17
Depth First Search
 Discovery/explore time
1/ 1/ 2/ 1/ 2/
1 1 2 1 2
3/
4

1/ 2/ 18 1/ 2/ 1/ 2/
1 2 1 2 1 2
4/ 3/ 4/ 3/ 4/ 3/
3 4 3 4 3 4
5/ 5/ 6/
6 6 7

18
Depth First Search
1/ 2/ 1/ 2/ 1/ 2/
1 2 1 2 1 2
4/ 3/ 7/ 4/ 3/ 7/8 4/ 3/ 7/8
3 4 5 3 4 5 3 4 5
5/ 6/ 5/ 6/ 5/ 6/9
6 7 6 7 6 7

1/ 2/ 1/ 2/ 1/ 2/
1 2 1 2 1 2
4/ 3/ 7/8 4/11 3/ 7/8 4/11 3/12 7/8
3 4 5 3 4 5 3 4 5
5/10 6/9 5/10 6/9 5/10 6/9
6 7 6 7 6 7

19
Depth First Search
1/ 2/13 1/14 2/13
1 2 1 2
4/11 3/12 7/8 4/11 3/12 7/8
3 4 5 3 4 5
5/10 6/9 5/10 6/9
6 7 6 7

 DFS Oder: 1 2 4 3 6 7 5
 PreOrdering: vertices in order of discovery time
 1243675 1
 PostOrdering: vertices in order of their finish time
 5763421 2 3
 PreOrdering and Postordering are not reverse of each other. 4
 Check on the graph at right.
20
Depth First Search
 Start vertex=1
0 0

1 2 1 2

3 4 3 4

5 6 5 6

7 7

 without tri/coloring Order: 1, 2, 5, 3, 7, 6

21
Depth First Search
 DFS Tree
 If a graph is connected, then by discarding the un-explored edges, resultant set of
vertices and edges forms a tree
1 2

3 4 5

6 7

 Multiple trees are possible depending upon choice of start vertex

 And how adjacent vertices are picked

22
Depth First Search
 DFS Forest
 If graph is not connected, then by running DFS on all un-discovered nodes gives a DFS
tree forest.
a b a b
c d g c d g
e f e f

a b a b

c d g c d g

e f e f
23
Useful Links
 A different approach
 http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/GraphAlgor/bre
adthSearch.htm
 http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/GraphAlgor/de
pthSearch.htm
 Visualization
 https://www.cs.usfca.edu/~galles/visualization/BFS.html
 https://www.cs.usfca.edu/~galles/visualization/DFS.html

24
BFS vs. DFS
 Complexity?
 Adjacency List?
 Adjacency Matrix?
 Helps to solve certain problems?
 Cycle detection
 both
 Reachability of vertices
 both
 Is the graph connected?
 both
 Shortest path in non-weighted graphs
 BFS

25
Shortest Path
 Given a graph, find the shortest path from start node to end node
 Applications:
 Going from one location to other
 Routing phone calls from one network to other network
 GPS applications for navigation
 And many more
 Single source vs. All pair shortest paths
 SSSP: single source shortest paths
 Shortest path from one vertex to all other vertices
 APSP: all pair shortest paths
 Shortest path from all vertices to all vertices

26
BFS-Shortest Path
 Non-Weighted Graph:
 Let say we want to find shortest paths to each vertex from source vertex?
 How BFS can help?

 Every edge is considered as having unit weight, means edges are equal in terms of cost.
 BFS discovers each vertex by exploring minimum possible number of edges.
 That is shortest path
 Path length= sum of edges
 What information needs to be maintained?
 Record distance Path Length

 Record parent/prev vertex of discovered vertex It is the vertex from which you discovered the
current vertex.
 To track the path from start vertex to destination vertex
27
BFS-Shortest Path
d=∞, p=∅ d=∞, p=∅ d=0, p=∅ d=∞, p=∅ d=0, p=∅ d=1, p=1
1 1 2
d=∞, p=∅ d=∞, p=∅ d=∞, p=∅ d=∞, p=∅ d=1, p=1 d=1, p=1
3 4
d=∞, p=∅ d=∞, p=∅ d=∞, p=∅

∅ d=∞, p=∅ d=∞, p=∅ 1 d=∞, p=∅ d=∞, p=∅ 2 3 4 d=∞, p=∅ d=∞, p=∅

d=0, p=∅ d=1, p=1 d=0, p=∅ d=1, p=1 d=0, p=∅ d=1, p=1

1 2 1 2 1 2
d=1, p=1 d=1, p=1 d=1, p=1 d=1, p=1 d=1, p=1 d=1, p=1

3 4 5 3 4 5 3 4 5
d=2, p=2 d=2, p=2 d=2, p=2

3 4 5 4 5 6 6 5 6 7 6 7
d=∞, p=∅ d=∞, p=∅ d=2, p=3 d=∞, p=∅ d=2, p=3 d=2, p=4
28
BFS-Shortest Path
d=0, p=∅ d=1, p=1 d=0, p=∅ d=1, p=1 d=0, p=∅ d=1, p=1
1 2 1 2 11 2
d=1, p=1 d=1, p=1 d=1, p=1 d=1, p=1 d=1, p=1 d=1, p=1
3 4 5 3 4 5 3 4 5
d=2, p=2 d=2, p=2 d=2, p=2

6 7 6 7 6 7
6 7 d=2, p=3 d=2, p=4 7 d=2, p=3 d=2, p=4 ∅ d=2, p=3 d=2, p=4

 Back track paths from destination to start:

 2-1
 3-1
 4-1
 5-2-1
 6-3-1
 7-4-1
29
Shortest Path(Non-Weighted Graph)
 Algorithm: BFS(G, start)
 Input: Graph and start vertex of graph.
 Output: list of vertices reachable from start along with their paths and distances
 Steps:
1. Q = new Queue()
2. For each vertex v in G
3. d[v]=infinity
4. p[v]=null
5. End For
6. d[start]=0
7. Q.enqueue(start)
8. While Q is not empty
9. u= Q.dequeue()
10. For each node v adjacent to u
11. if d[v] is infinity //undiscovered
12. d[v]=d[u]+1
13. p[v]=u
14. Q.enqueue(v)
15. End if
16. End For
17. End While

30
Shortest Path
 Weighted Graph?
 Shortest path between A to other nodes
B: [A, B]= 2 D: [A, B, D] = 5
C: [A, C]= 3 E: [A, B, E] = 6

B 3 D
 How to find? 3
5
A
2 1 6
4
1 C E
7

31
Shortest Path
 Weighted Graph?
 Shortest path between A to other nodes
B: [A, B]= 2 D: [A, B, D] = 6
C: [A, C]= 3 E: [A, B, E] = 7

B 3 D
 How to find? 3
 Can we use BFS still? 5
 BFS shortest path from A to B is 3 A

2 1 6
Actual shortest path from A to B is 2
 So simple BFS will not work 4
 But why it works flawlessly with unit weights
1 C E
7

32
Shortest Path
 Modify BFS Shortest Path Algorithm to handle weights
 Instead of finalizing distance of a nodes when we simply visit them, we should only finalize node
which has minimum distance so far.
 The node that is popped out has been finalized but nodes inside queue will continue to get updated when ever we
can visit them from popped out node
 So, a priority queue is needed instead of queue, to pop out the node with minimum distance so far from start node
 See the example:
 A is start vertex, it will be pushed to queue with distance 0
 Now check adjacent nodes, B and C will be pushed to queue with distance 3 and 1
 But now instead of popping out B which comes first order wise, we will pop C
 Because it’s distance is 1 from A
 Now B is also adjacent node of C, so we will see if its distance can be updated which was 3 when it was discovered
from A. 3 B
 it will become 2 now
 Keep on updating distances of adjacent nodes, when a node is popped out.
A
2 1
 Untill queue becomes empty
1 C
33
Dijkstra’s Algorithm
 This is how Dijkstra’s algorithm works.
 Idea is to maintain two set of nodes
 Nodes whose distance have been found, have been popped off from queue
 Nodes whose best shortest path has been found so far but not finalized, may be it can be further
reduced yet, so they have been discovered, but still in queue
 Node with minimum distance so far will be popped from queue( Priority queue will help here)

34
Dijkstra’s Algorithm
 Step 1: Initialize Graph
 Mark all node’s prev pointer to NULL
 Mark all node’s distance to infinity
p=∅ p=∅
d=∞ d=∞

B 3 D
p=∅ 3
d=∞ 5
A
2 1 6
4
1 C E
7
p=∅ p=∅
d=∞ d=∞
35
Dijkstra’s Algorithm
 STEP 2:
 Initialize distance of start node 0
 Push start node to queue
p=∅ p=∅
d=∞ d=∞

B 3 D
p=∅ 3
d=0 5
A
2 1 6
4
A 1 C E
7
0 p=∅ p=∅
d=∞ d=∞
36
Dijkstra’s Algorithm
 Now B and C are adjacent nodes of A, they will be pushed to queue, with their
respective distances. Distance is calculated as:
 d[v]=d[u]+weight(u,v)
 u and v are two connected nodes, p=A p=∅
d=3 d=∞
 weight(u,e) is edge weight between u and v
B 3 D
 Prev pointer will point to A p=∅ 3
d=0 5
A A
0
2 1 6
4
B C 1 C E
7
3 1 p=A p=∅
d=1 d=∞
37
Dijkstra’s Algorithm
 C has minimum distance so far, so it will be popped out.
 Two new nodes D and E are pushed to queue, and B’s distance and prev pointer is
updated
 Node’s distance is updated according to following: p=C p=C
d=2 d=6
 If d[v] > d[u]+weight(u,v)
B 3 D
 d[v]=d[u]+weight(u,v)
 p[v]=u
p=∅ 3
d=0 5
B C A
3 1
2 1 6
4
B D E 1 C E
7
2 6 8 p=A p=C
d=1 d=8
38
Dijkstra’s Algorithm
 Now B will be popped out, as it has minimum distance of 2
 Is it possible to reduce distance of D and E, as they are adjacent nodes of B
 If Yes, update them
p=C p=B
d=2 d=5

B 3 D
p=∅ 3
d=0 5
B D E A
2 6 8
2 1 6
4
D E 1 C E
7
5 6 p=A p=B
d=1 d=6
39
Dijkstra’s Algorithm
 Now D will be popped out, as it has minimum distance of 5
 No node will be updated

p=C p=B
d=2 d=5

B 3 D
p=∅ 3
d=0 5
D E A
5 6
2 1 6
4
E 1 C E
7
6 p=A p=B
d=1 d=6
40
Dijkstra’s Algorithm
 Finally pop E
 Queue has become empty and all distance and paths have been found

p=C p=B
d=2 d=5

B 3 D
p=∅ 3
d=0 5
A
2 1 6
4
1 C E
7
p=A p=B
d=1 d=6
41
Dijkstra’s Algorithm
 Dijkstra’s Shortest Path Tree

B 3 D

A
1
4
1 C E

42
Directed Graph
 Start vertex: 1

 Path is alternate for parent

 Cost is alternate for distance

43
Dijkstra’s Shortest Path Algorithm
 Algorithm: DIJKSTRA(G, start)
 Input: Graph G and start vertex of graph G
 Steps:
1. PQ = new PriorityQueue(V) //where V is number of vertices
2. Mark start’s dist=0
3. For each node v in G //initialization of all nodes
4. if start != v
5. d[v]= infinity
6. p[v]=null
7. end if
8. PQ.add(v,d[v])
9. End For
10. while PQ is not empty
11. u = PQ.removeMin()
12. For each node v adjacent to u that is in PQ //updating distances of adjacent nodes
13. if d[v] > d[u]+ cost(u,v) //cost mean edge weight between u and v
14. d[v] = d[u] + cost(u,v) // update distance with new value
15. p[v] = u // update prev pointers to maintain path
16. PQ.decrease_Priority(v,d[v])
17. End if
18. End For
19. End While
20. return d[] and p[]

44
Time Complexity?
 Depends upon implementation of priority queue
 With array/ linked list implementation
 removeMin() will take O(V) time
 Priority update at each distance update O(1) time
 With binary heap implementation
 removeMin() will take O(logV) time
 Priority update at each distance update O(logV) time

 Time complexity with array or list as priority queue will be O(V 2)

 Time complexity with heap as priority queue will be O(E+ VlogV)
 For details please visit:
 http://en.wikipedia.org/wiki/Dijkstra%27s_algorithm
 https://www.cs.cornell.edu/courses/cs312/2002sp/lectures/lec20/lec20.htm
45
Dijkstra’s Shortest Path Algorithm
 Run Dijkstra’s algorithm on following graph, taking 0 as start node.

46
Where Dijkstra’s fails?
 Run Dijkstra’s algorithm on following graph taking A as start vertex:
 Is there any problem with final answers?
 What is shortest path distance from A to B according to Dijkstra’s?
 What is actual shortest path distance from A to B

47
Topological Ordering/Sort
 Topological ordering of a directed graph is a linear ordering of its vertices such that for
every directed edge (u, v) from vertex u to vertex v, u comes before v in the ordering.
Vertices are placed in an ordering according to edges.

 Order the vertices from left to right

B D
Source Sink
A A B C E D

Topological Ordering
C E ABCED

 Very helpful to decide relative ordering of tasks.

 Which tasks can be ignored.
 Which must be accomplished in order to complete another task
48
Topological Sort/Ordering
 Applications:
 Scheduling Dependency
 Which task should be performed in which order
 Which tasks depends upon other to be completed first
 Course pre-requisites
 Class order in object oriented design
 Which tasks are independent

CS22
CS19

CS16 CS141 CS242

CS15

CS17 CS32
CS18 CS123 CS224

49
Topological Ordering
 What is topological  What is topological  What is topological
ordering? ordering? ordering?
B D B D B

A A A

C E C E C

 Any observation?  Any observation?  Any observation?

50
Topological Ordering
 What is topological  What is topological  What is topological
ordering?
ordering? ordering?
B D B D B

A A A

C E C E C

 Fact 3:
 Fact 2:
  For any DAG, a
Fact 1:  Multiple topological
 Graph must be acyclic- orderings are possible. topological ordering
DAG
 ABCDE must exist.
 ABCED
 You can’t define order in  ACBDE
case of cycle  ACBED
51
Topological Ordering
 How to get topological ordering of vertices?
B D

C E
 Will DFS help here?
 What is the order of vertices according to discovery time?
 What is the order of vertices according to finish time?
 How to detect Cycle?

52
Topological Ordering
 Using DFS 2/9 4/5
 Perform DFS on graph G B D
 When a vertex is finished, put it on start of a linked list 1/10
 List is topological ordering of vertices. A

C E
3/8 6/7
 Why DFS will produce correct ordering? A B C E D
 DFS ensures that if there is a directed edge (u, v)
 Then F[u] > F[v]
 Here u is ancestor of v.
 u is a pre-req for v
 This relationship can be used to classify edges into different types during DFS traversal.
53
DFS Edge Classification
 During DFS traversal of a directed graph G, edges can be classified into four
different types when explored. This classification of edge (u, v), depends on
whether we have visited v before u in the DFS and if so, the relationship between u
and v.
 If there is an edge from u to v then edge (u, v) can be classified as:
 Tree edge: if (u,v) is present in dfs tree p[v]=u
 If V has already been visited from some other vertex
 Backward edge: if v is ancestor of u in dfs tree
 Forward edge: if v is descendent of u in dfs tree
 Cross edge: v is neither ancestor nor the descendent in dfs tree

 This classification is very helpful to solve certain problems

 Cycle detection
54
DFS Edge Classification
 Graph and its DFS tree
1/10
A A
Backward-edge Tree-edge

C B 3/8 C B 2/9

Forward-edge

E D D 4/5
6/7 E Cross-edge

 If a graph has a back edge, it contains a cycle.

55
DFS Edge Classification
 How to detect if an edge is back edge?
 See the following graphs, which one contains a back edge
1/ 1/ 1/ 1/
A A A A

3/ C B 2/ 3/4 C B 2/5 3/8 C B 2/ 3/ C B 2/

 6/ E
D 4/5 D 4/
Is there any specific observation?
 Edge type and vertex color?

56
DFS Edge Classification
 How to detect back edge?
 If an edge is going back to a grey vertex
 It’s a back edge.

57