Your Progress
63% complete. Continue learning to master DSA!
Greedy Algorithms
Greedy algorithms make locally optimal choices at each step with the hope of finding a global optimum. They are simple, intuitive, and efficient for many optimization problems.
Key Takeaways
- Greedy algorithms solve problems by making the best choice at each step
- They are efficient and easy to implement but don't always guarantee the optimal solution
- Work well for problems with "greedy choice property" and "optimal substructure"
- Common applications include scheduling, path finding, and data compression
- Understanding when to use greedy vs. dynamic programming is crucial
What are Greedy Algorithms?
A greedy algorithm is a simple, intuitive algorithm that builds up a solution piece by piece, always choosing the next piece that offers the most immediate benefit. Greedy algorithms are used for optimization problems where the goal is to find the maximum or minimum element according to some criteria.
Key Characteristics
- Greedy Choice Property: A global optimum can be reached by selecting a local optimum at each step
- Optimal Substructure: An optimal solution to the problem contains optimal solutions to its subproblems
- Irrevocable Decisions: Once a choice is made, it's never reconsidered
- Simplicity: Generally easier to design and implement than other techniques
Important Note: While greedy algorithms are efficient, they don't always produce the optimal solution for every problem. It's crucial to verify that the greedy approach is valid for your specific problem.
When to Use Greedy Algorithms
Greedy algorithms are most suitable when:
✓ Good Use Cases
- The problem has optimal substructure
- The problem has the greedy choice property
- Local optimal choices lead to a global optimal solution
- You need a reasonably good solution with minimal computation
✗ Poor Use Cases
- The problem requires considering all possible solutions
- Local choices don't lead to a global optimum
- Past decisions affect future optimal choices
- The absolutely optimal solution is required
Common Greedy Algorithms
1. Activity Selection Problem
Select the maximum number of activities that can be performed by a single person, assuming the person can only work on a single activity at a time.
Greedy Approach:
- Sort activities by finish time
- Select the first activity (earliest finish)
- For remaining activities, select the next compatible activity with earliest finish time
function activitySelection(start, finish) { // Assuming activities are sorted by finish time const n = start.length; let result = []; // First activity is always selected let i = 0; result.push(i); // Consider rest of the activities for (let j = 1; j < n; j++) { // If this activity has start time greater than or // equal to the finish time of previously selected // activity, then select it if (start[j] >= finish[result[result.length - 1]]) { result.push(j); } } return result; } // Example usage const start = [1, 3, 0, 5, 8, 5]; const finish = [2, 4, 6, 7, 9, 9]; // Sort activities by finish time (already sorted in this example) console.log(activitySelection(start, finish)); // [0, 1, 3, 4]
2. Fractional Knapsack Problem
Fill a knapsack with fractional parts of items to maximize the total value, given weight constraints.
Greedy Approach:
- Calculate value-to-weight ratio for all items
- Sort items by value-to-weight ratio in descending order
- Take as much as possible of the highest ratio items first
function fractionalKnapsack(weights, values, capacity) { const n = weights.length; // Create array of objects with value-to-weight ratios let items = []; for (let i = 0; i < n; i++) { items.push({ weight: weights[i], value: values[i], ratio: values[i] / weights[i] }); } // Sort by value-to-weight ratio in descending order items.sort((a, b) => b.ratio - a.ratio); let totalValue = 0; let currentWeight = 0; // Take items with highest value-to-weight ratio first for (let i = 0; i < n; i++) { if (currentWeight + items[i].weight <= capacity) { // Take the whole item currentWeight += items[i].weight; totalValue += items[i].value; } else { // Take a fraction of the item const remainingCapacity = capacity - currentWeight; totalValue += items[i].value * (remainingCapacity / items[i].weight); break; } } return totalValue; } // Example usage const weights = [10, 20, 30]; const values = [60, 100, 120]; const capacity = 50; console.log(fractionalKnapsack(weights, values, capacity)); // 240
3. Huffman Coding
A lossless data compression algorithm that assigns variable-length codes to input characters based on their frequencies.
Greedy Approach:
- Build a frequency table for the input characters
- Create a leaf node for each character and add them to a priority queue
- While there is more than one node in the queue:
- Remove the two nodes with lowest frequency
- Create a new internal node with these two nodes as children and frequency equal to the sum of their frequencies
- Add the new node to the queue
- The remaining node is the root of the Huffman tree
More Greedy Algorithm Examples
Coin Change Problem
Find the minimum number of coins needed to make a given amount of change (works for some coin systems).
Greedy Approach:
Always take the largest coin that does not exceed the remaining amount.
Dijkstra's Algorithm
Find the shortest path from a source vertex to all other vertices in a weighted graph.
Greedy Approach:
Always choose the next vertex with the smallest distance from the source.
Prim's Algorithm
Find a minimum spanning tree for a weighted undirected graph.
Greedy Approach:
Start from an arbitrary vertex and always add the edge with minimum weight that connects a vertex in the tree to a vertex outside.
Kruskal's Algorithm
Another algorithm to find a minimum spanning tree for a weighted graph.
Greedy Approach:
Sort all edges by weight and add them one by one to the spanning tree if they don't form a cycle.
Greedy vs. Dynamic Programming
Both greedy algorithms and dynamic programming are used to solve optimization problems, but they differ in their approach:
Aspect | Greedy Algorithms | Dynamic Programming |
---|---|---|
Approach | Makes the locally optimal choice at each step | Breaks down problem into overlapping subproblems and solves each once |
Decision Making | Makes irrevocable decisions | Considers all possible decisions and their outcomes |
Optimality | Not always optimal | Always finds the optimal solution |
Time Complexity | Generally more efficient | Often higher due to solving all subproblems |
Implementation | Usually simpler | Often more complex |
Example: Coin Change Problem
Greedy (Works for some coin systems)
function coinChangeGreedy(coins, amount) { // Sort coins in descending order coins.sort((a, b) => b - a); let count = 0; let remainingAmount = amount; for (const coin of coins) { // Take as many as possible of current coin count += Math.floor(remainingAmount / coin); remainingAmount %= coin; } return remainingAmount === 0 ? count : -1; }
Note: This greedy approach works for some coin systems (like US coins) but fails for others.
Dynamic Programming (Works for all coin systems)
function coinChangeDP(coins, amount) { // Initialize array with infinity const dp = Array(amount + 1).fill(Infinity); dp[0] = 0; for (const coin of coins) { for (let i = coin; i <= amount; i++) { dp[i] = Math.min(dp[i], dp[i - coin] + 1); } } return dp[amount] === Infinity ? -1 : dp[amount]; }
This DP approach works for any valid coin system and always finds the optimal solution.
Proving Greedy Algorithm Correctness
To prove that a greedy algorithm produces the optimal solution, you typically need to:
- Prove the Greedy Choice Property: Show that a locally optimal choice at each step leads to a globally optimal solution.
- Prove Optimal Substructure: Show that an optimal solution to the problem contains optimal solutions to its subproblems.
- Exchange Argument: Often used to prove that any non-greedy solution can be transformed into a greedy solution without worsening the result.
- Induction: Use mathematical induction to prove that the algorithm works for all problem sizes.
Advanced Topic: Proving the correctness of greedy algorithms often requires mathematical rigor and can be challenging. For practical purposes, it's sometimes sufficient to test the algorithm on various test cases and compare with known optimal solutions.
Common Pitfalls with Greedy Algorithms
1. Assuming Greedy Always Works
The biggest mistake is assuming a greedy approach will always yield the optimal solution without proof.
Example: Non-Standard Coin System
For coin denominations [1, 3, 4] and amount 6, the greedy approach gives 3 coins (4+1+1) while the optimal solution is 2 coins (3+3).
2. Incorrect Greedy Choice
Choosing the wrong criterion for local optimization can lead to suboptimal solutions.
3. Not Considering All Constraints
Sometimes a greedy choice might seem optimal initially but violates some constraint later in the solution.
Conclusion
Greedy algorithms provide a powerful approach for solving optimization problems with their simplicity and efficiency. While they don't guarantee optimal solutions for all problems, they work remarkably well for many practical applications and often serve as good approximation algorithms when finding the exact optimal solution is computationally expensive.
Remember
When approaching a new problem, consider whether a greedy approach might work. Verify the greedy choice property and optimal substructure. If in doubt, compare against dynamic programming or other approaches to ensure you're getting the optimal solution.
Next Steps
To continue your learning journey on algorithm design paradigms, explore these related topics:
Related Tutorials
Related Tutorials
Dynamic Programming
Explore another approach for solving optimization problems.
Learn moreMinimum Spanning Tree
Learn about a classic application of greedy algorithms.
Learn moreShortest Path Algorithms
Study Dijkstra's algorithm and other path-finding techniques.
Learn more