Shortest path algorithms Variants: 1) single-pair shortest path 2) single-source shortest path 3) all-pairs shortest path All of these run on (optionally directed) graphs with non-negative edge weights. Applications? Network routing! Assuming edge weights have something to do with costs associated with transmitting data, then shorter routes yield faster networks. DIJKSTRA'S ALGORITHM (single-source) (borrowed from Wikipedia) 1 function Dijkstra(Graph, source): 2 for each vertex v in Graph: // Initializations 3 dist[v] := infinity // Unknown distance function from source to v 4 previous[v] := undefined 5 dist[source] := 0 // Distance from source to source 6 Q := copy(Graph) // All nodes in the graph are unoptimized - thus are in Q 7 while Q is not empty: // The main loop 8 u := extract_min(Q) // Remove and return best vertex from nodes in two given nodes // we would use a path finding algorithm on the new graph, such as depth-first search. 9 for each neighbor v of u: // where v has not yet been removed from Q. 10 alt = dist[u] + length(u, v) 11 if alt < dist[v] // Relax (u,v) 12 dist[v] := alt 13 previous[v] := u 14 return previous[] In short, it looks a lot like the breadth-first algorithm we saw earlier in the semester. The only trick is that we always have an estimate of the distance to every node, and we "relax" it every time we find a better path. Still, just like BFS, we're always looking at shortest paths first. Runtime analysis: just like BFS. The heap is O(log V) per loop iteration, or O(V log V) total. Then, for each vertex, we have to consider each adjacent vertex. How many of those? O(E). Each edge will be considered exactly once (in a directed graph, anyway), so the total runtime is O(E + V log V). BELLMAN-FORD (single-source) This one can deal with negative edge weights, so long as there isn't a cycle of negative weight. (Then you could just keep going around and around and get a cheaper total path every time...) procedure BellmanFord(list vertices, list edges, vertex source) (borrowed from Wikipedia) // This implementation takes in a graph, represented as lists of vertices // and edges, and modifies the vertices so that their distance and // predecessor attributes store the shortest paths. // Step 1: Initialize graph for each vertex v in vertices: if v is source then v.distance := 0 else v.distance := infinity v.predecessor := null // Step 2: relax edges repeatedly for i from 1 to size(vertices)-1: for each edge uv in edges: u := uv.source v := uv.destination // uv is the edge from u to v if v.distance > u.distance + uv.weight: v.distance := u.distance + uv.weight v.predecessor := u // Step 3: check for negative-weight cycles for each edge uv in edges: u := uv.source v := uv.destination if v.distance > u.distance + uv.weight: error "Graph contains a negative-weight cycle" The general idea: Much like the dichotomy between Prim's and Kruskal's algorithms (where you either build your MST out from a starting point, or you build it piecemeal as you go), Dijkstra is building it out while B-F is building it piecemeal. B-F keeps looping over all the edges O(V) times. Whenever it finds a distance on the source side of an edge where that distance plus the weight of the edge is less than the weight on the target side of the edge, then we now have a better estimate of the weight on the target side, so we can update it. Why run V times? Consider a worst-case graph (basically, a linked list). It will take V iterations for the system to converge. Also, notice how much "redundant" work you're doing. Runtime: O(V * E) Cleverness: a negative edge is no big deal. A negative cycle will be detected if present. Extreme cleverness: this parallelizes really well. Chop the problem up into tiny pieces and spread it around. Exercise for the reader: you can also make it *incremental*. Say you've got a solution already, and then one of the edge weights changes (e.g., a network route just disappeared). You can now start reseting some distances to infinity while leaving others alone. FLOYD-WARSHALL (all-pairs shortest path) (pseudocode from Wikipedia) 1 /* Assume a function edgeCost(i,j) which returns the cost of the edge from i to j 2 (infinity if there is none). 3 Also assume that n is the number of vertices and edgeCost(i,i)=0 4 */ 5 6 int path[][]; 7 /* A 2-Dimensional matrix. At each step in the algorithm, path[i][j] is the shortest path 8 from i to j using intermediate values in (1..k-1). Each path[i][j] is initialized to 9 edgeCost(i,j). 10 */ 11 12 procedure FloydWarshall () 13 for k: = 1 to n 14 begin 15 for each (i,j) in (1..n) 16 begin 17 path[i][j] = min ( path[i][j], path[i][k]+path[k][j] ); 18 end 19 end 20 endproc Note that when we're done, all we know is the *cost* of the path between nodes i and j, but we don't actually know the whole path. This algorithm is constantly hunting for any possible case where some intermediate node yields a shorter path between two nodes than the length of the shortest path that we've found so far. The inner loop (line 15) does the obvious thing, looping over all pairs of vertices, regardless of whether there happens to be an edge between them. The outer loop (line 13) is a little bit funkier. We take exactly n trips around this loop (n == the number of vertices). Is that enough? Too much? Again, think about a worst-case linked-list sort of arrangement. Each pass around the inner loop is only going to propagate data from one side toward the other side. The chain has n elements, so it will take n loops. Runtime: O(V^3) That seems expensive (it is), but again, note how trivially parallelizable it is. Snide (?) side commentary: Since Dijkstra's algorithm is so fast, why not just use it for every vertex? Running Dijkstra for one vertex costs O(E + V log V). Running it for every vertex costs O(V E + V^2 log V). Okay, what's the worst case size of E? V^2. In that case, we get the same runtime as F-W. If the graph has very few vertices (i.e., if E is the same size as V), then the V^2 log V term dominates. And you can even still parallelize it. (This is the essential idea behind Johnson's algorithm.) General principles: Both Floyd-Warshall and Bellman-Ford have these loops that just go around and around doing the same simple thing, over and over again. So long as each iteration helps us converge to the proper answer, we have some pretty powerful algorithms. This same style of problem solving shows up in other places, like solving large systems of linear equations or... computer graphics. One of the holy grails in computer graphics is called global illumination. Look under your table. See how there's light there? It didn't get there from the ceiling lights because the table is in the way. Instead, it had to bounce there from somewhere else in the room. One of the ways that graphics programs solve this problem is by dicing up the world into tiny slices and computing how much of the light from this slice over here ultimately hits that slice over there. Then, you just loop over the world, making sure that the sum of the light going into someplace is equal to the sum of the light going back out again. Repeat. Eventually you converge on an answer. http://en.wikipedia.org/wiki/Radiosity