Your Progress

19/30 completed

63% complete. Continue learning to master DSA!

Greedy Algorithms

Greedy algorithms make locally optimal choices at each step with the hope of finding a global optimum. They are simple, intuitive, and efficient for many optimization problems.

Key Takeaways

  • Greedy algorithms solve problems by making the best choice at each step
  • They are efficient and easy to implement but don't always guarantee the optimal solution
  • Work well for problems with "greedy choice property" and "optimal substructure"
  • Common applications include scheduling, path finding, and data compression
  • Understanding when to use greedy vs. dynamic programming is crucial

What are Greedy Algorithms?

A greedy algorithm is a simple, intuitive algorithm that builds up a solution piece by piece, always choosing the next piece that offers the most immediate benefit. Greedy algorithms are used for optimization problems where the goal is to find the maximum or minimum element according to some criteria.

Key Characteristics

  • Greedy Choice Property: A global optimum can be reached by selecting a local optimum at each step
  • Optimal Substructure: An optimal solution to the problem contains optimal solutions to its subproblems
  • Irrevocable Decisions: Once a choice is made, it's never reconsidered
  • Simplicity: Generally easier to design and implement than other techniques

Important Note: While greedy algorithms are efficient, they don't always produce the optimal solution for every problem. It's crucial to verify that the greedy approach is valid for your specific problem.

When to Use Greedy Algorithms

Greedy algorithms are most suitable when:

✓ Good Use Cases

  • The problem has optimal substructure
  • The problem has the greedy choice property
  • Local optimal choices lead to a global optimal solution
  • You need a reasonably good solution with minimal computation

✗ Poor Use Cases

  • The problem requires considering all possible solutions
  • Local choices don't lead to a global optimum
  • Past decisions affect future optimal choices
  • The absolutely optimal solution is required

Common Greedy Algorithms

1. Activity Selection Problem

Select the maximum number of activities that can be performed by a single person, assuming the person can only work on a single activity at a time.

Greedy Approach:

  1. Sort activities by finish time
  2. Select the first activity (earliest finish)
  3. For remaining activities, select the next compatible activity with earliest finish time
function activitySelection(start, finish) {
  // Assuming activities are sorted by finish time
  const n = start.length;
  let result = [];
  
  // First activity is always selected
  let i = 0;
  result.push(i);
  
  // Consider rest of the activities
  for (let j = 1; j < n; j++) {
    // If this activity has start time greater than or
    // equal to the finish time of previously selected
    // activity, then select it
    if (start[j] >= finish[result[result.length - 1]]) {
      result.push(j);
    }
  }
  
  return result;
}

// Example usage
const start = [1, 3, 0, 5, 8, 5];
const finish = [2, 4, 6, 7, 9, 9];
// Sort activities by finish time (already sorted in this example)
console.log(activitySelection(start, finish)); // [0, 1, 3, 4]

2. Fractional Knapsack Problem

Fill a knapsack with fractional parts of items to maximize the total value, given weight constraints.

Greedy Approach:

  1. Calculate value-to-weight ratio for all items
  2. Sort items by value-to-weight ratio in descending order
  3. Take as much as possible of the highest ratio items first
function fractionalKnapsack(weights, values, capacity) {
  const n = weights.length;
  
  // Create array of objects with value-to-weight ratios
  let items = [];
  for (let i = 0; i < n; i++) {
    items.push({
      weight: weights[i],
      value: values[i],
      ratio: values[i] / weights[i]
    });
  }
  
  // Sort by value-to-weight ratio in descending order
  items.sort((a, b) => b.ratio - a.ratio);
  
  let totalValue = 0;
  let currentWeight = 0;
  
  // Take items with highest value-to-weight ratio first
  for (let i = 0; i < n; i++) {
    if (currentWeight + items[i].weight <= capacity) {
      // Take the whole item
      currentWeight += items[i].weight;
      totalValue += items[i].value;
    } else {
      // Take a fraction of the item
      const remainingCapacity = capacity - currentWeight;
      totalValue += items[i].value * (remainingCapacity / items[i].weight);
      break;
    }
  }
  
  return totalValue;
}

// Example usage
const weights = [10, 20, 30];
const values = [60, 100, 120];
const capacity = 50;
console.log(fractionalKnapsack(weights, values, capacity)); // 240

3. Huffman Coding

A lossless data compression algorithm that assigns variable-length codes to input characters based on their frequencies.

Greedy Approach:

  1. Build a frequency table for the input characters
  2. Create a leaf node for each character and add them to a priority queue
  3. While there is more than one node in the queue:
    • Remove the two nodes with lowest frequency
    • Create a new internal node with these two nodes as children and frequency equal to the sum of their frequencies
    • Add the new node to the queue
  4. The remaining node is the root of the Huffman tree

More Greedy Algorithm Examples

Coin Change Problem

Find the minimum number of coins needed to make a given amount of change (works for some coin systems).

Greedy Approach:

Always take the largest coin that does not exceed the remaining amount.

Dijkstra's Algorithm

Find the shortest path from a source vertex to all other vertices in a weighted graph.

Greedy Approach:

Always choose the next vertex with the smallest distance from the source.

Prim's Algorithm

Find a minimum spanning tree for a weighted undirected graph.

Greedy Approach:

Start from an arbitrary vertex and always add the edge with minimum weight that connects a vertex in the tree to a vertex outside.

Kruskal's Algorithm

Another algorithm to find a minimum spanning tree for a weighted graph.

Greedy Approach:

Sort all edges by weight and add them one by one to the spanning tree if they don't form a cycle.

Greedy vs. Dynamic Programming

Both greedy algorithms and dynamic programming are used to solve optimization problems, but they differ in their approach:

AspectGreedy AlgorithmsDynamic Programming
ApproachMakes the locally optimal choice at each stepBreaks down problem into overlapping subproblems and solves each once
Decision MakingMakes irrevocable decisionsConsiders all possible decisions and their outcomes
OptimalityNot always optimalAlways finds the optimal solution
Time ComplexityGenerally more efficientOften higher due to solving all subproblems
ImplementationUsually simplerOften more complex

Example: Coin Change Problem

Greedy (Works for some coin systems)

function coinChangeGreedy(coins, amount) {
  // Sort coins in descending order
  coins.sort((a, b) => b - a);
  
  let count = 0;
  let remainingAmount = amount;
  
  for (const coin of coins) {
    // Take as many as possible of current coin
    count += Math.floor(remainingAmount / coin);
    remainingAmount %= coin;
  }
  
  return remainingAmount === 0 ? count : -1;
}

Note: This greedy approach works for some coin systems (like US coins) but fails for others.

Dynamic Programming (Works for all coin systems)

function coinChangeDP(coins, amount) {
  // Initialize array with infinity
  const dp = Array(amount + 1).fill(Infinity);
  dp[0] = 0;
  
  for (const coin of coins) {
    for (let i = coin; i <= amount; i++) {
      dp[i] = Math.min(dp[i], dp[i - coin] + 1);
    }
  }
  
  return dp[amount] === Infinity ? -1 : dp[amount];
}

This DP approach works for any valid coin system and always finds the optimal solution.

Proving Greedy Algorithm Correctness

To prove that a greedy algorithm produces the optimal solution, you typically need to:

  1. Prove the Greedy Choice Property: Show that a locally optimal choice at each step leads to a globally optimal solution.
  2. Prove Optimal Substructure: Show that an optimal solution to the problem contains optimal solutions to its subproblems.
  3. Exchange Argument: Often used to prove that any non-greedy solution can be transformed into a greedy solution without worsening the result.
  4. Induction: Use mathematical induction to prove that the algorithm works for all problem sizes.

Advanced Topic: Proving the correctness of greedy algorithms often requires mathematical rigor and can be challenging. For practical purposes, it's sometimes sufficient to test the algorithm on various test cases and compare with known optimal solutions.

Common Pitfalls with Greedy Algorithms

1. Assuming Greedy Always Works

The biggest mistake is assuming a greedy approach will always yield the optimal solution without proof.

Example: Non-Standard Coin System

For coin denominations [1, 3, 4] and amount 6, the greedy approach gives 3 coins (4+1+1) while the optimal solution is 2 coins (3+3).

2. Incorrect Greedy Choice

Choosing the wrong criterion for local optimization can lead to suboptimal solutions.

3. Not Considering All Constraints

Sometimes a greedy choice might seem optimal initially but violates some constraint later in the solution.

Conclusion

Greedy algorithms provide a powerful approach for solving optimization problems with their simplicity and efficiency. While they don't guarantee optimal solutions for all problems, they work remarkably well for many practical applications and often serve as good approximation algorithms when finding the exact optimal solution is computationally expensive.

Remember

When approaching a new problem, consider whether a greedy approach might work. Verify the greedy choice property and optimal substructure. If in doubt, compare against dynamic programming or other approaches to ensure you're getting the optimal solution.

Next Steps

To continue your learning journey on algorithm design paradigms, explore these related topics:

Related Tutorials

Related Tutorials

Dynamic Programming

Explore another approach for solving optimization problems.

Learn more

Minimum Spanning Tree

Learn about a classic application of greedy algorithms.

Learn more

Shortest Path Algorithms

Study Dijkstra's algorithm and other path-finding techniques.

Learn more