In this lecture we look at greedy algorithms—an algorithm design technique for optimization problems. To solve these problems, a number of choices have to be made to arrive at an optimal solution. After making each choice we are left with a smaller subproblem. A greedy algorithm makes "the best" choice in each step. This is contrast to dynamic programming algorithms where, at each step, a number of choices are considered together with solutions to the subproblems that these choices generate. For a given problem, usually there are a number of possible interpretations of "the best". Which one of them leads to an optimal solution is the main challenge when designing greedy algorithms. For some problems, there are no greedy algorithms that always lead to optimal solutions, but such algorithms may be an efficient way to produce good-enough solutions.
In this lecture we look at two problems that can be solved with greedy algorithms: an activity-selection problem and data compression with Huffman codes. We also revisit the problem of finding a minimum spanning tree of a weighted graph. For each greedy algorithm we prove that it finds an optimal solution. We also explore other ideas for solving these problems using greedy algorithms. In each case we try to find counter-examples to show why these ideas do not work.
Goals of the lecture:
- to understand the principles of the greedy algorithm design technique;
- to understand the example greedy algorithms for activity selection and Huffman coding, to be able to prove that these algorithms find optimal solutions;
- to be able to apply the greedy algorithm design technique.
Tuesday, the 8th of Februar, 2011 at 14:25.
 |
Read Sections 16.1-16.3 of CLRS. When reading, make sure at
least that you find answers to the following questions:
- What is the general structure of a greedy optimization algorithm? - Which two properties have to be proven to prove that a greedy algorithm finds an optimal solution? - What is the greedy-choice property? - What is the greedy choice in the Huffman algorithm? |
|
Background reading: If you feel that you need to refresh your knowledge of the principles of dynamic programming, read section 15.3 of CLRS; minimum spanning trees are covered in Chapter 23 of CLRS (specifically, we will refer to the algorithm of Kruskal in our lecture, if we have time...) |
[pdf]
|
Solve CLRS 16.3-3 (16.3-2 in the 2nd edition). By how much the text with the frequences of characters as given in the exercise would be compressed? |
|
Finding shortest paths from a single source vertex and
finding a minimum spanning tree are graph optimization
problems solvable by greedy algorithms. Consider Dijkstra's,
Prim-Jarnik's, and Kruskal's algorithms. For each of these
algorithms, describe:
- what is the greedy choice made in each step of the algorithm? - what is the sub-problem that remains after each step of the algorithm? (Remember, the sub-problem should "look" like the original problem, only smaller) |
|
Solve CLRS 16.1-4 (16.1-3 in the 2nd edition). Argue that your algorithm correctly finds an optimal solution. (Hint: Prove by contradiction. Assume that the algorithm's solution with k lecture halls is not optimal and the k-th lecture hall is not necessary, then, investigate the first activity scheduled in this lecture hall.) |