Your Progress

18/30 completed

60% complete. Continue learning to master DSA!

Heap Sort

Heap sort is an efficient comparison-based sorting algorithm that uses a binary heap data structure to sort elements in an array with guaranteed O(n log n) time complexity.

Key Takeaways

  • Heap sort uses a binary heap data structure to efficiently sort elements
  • Time complexity is O(n log n) for all cases (best, average, worst)
  • Space complexity is O(1) as it sorts in-place
  • It's not a stable sorting algorithm but offers consistent performance
  • Heap sort has better worst-case performance than quicksort but is often slower in practice

Understanding Heap Sort

Heap sort is a comparison-based sorting algorithm that uses a binary heap data structure. The algorithm divides the input into a sorted region and an unsorted region, and iteratively shrinks the unsorted region by extracting the largest element from it and inserting it into the sorted region.

How Binary Heaps Work

A binary heap is a complete binary tree where each node satisfies the heap property:

  • Max Heap: Each parent node is greater than or equal to its children
  • Min Heap: Each parent node is less than or equal to its children

Heap sort typically uses a max heap to sort elements in ascending order. In a binary heap represented as an array:

  • For a node at index i, its left child is at index 2i + 1
  • Its right child is at index 2i + 2
  • Its parent is at index floor((i-1)/2)

Algorithm Steps

  1. Build a max heap from the input array
  2. Swap the root (maximum element) with the last element of the heap
  3. Reduce the heap size by 1 (exclude the last element)
  4. Heapify the root to maintain the max heap property
  5. Repeat steps 2-4 until the heap size becomes 1

Visual Example

Let's sort the array [4, 10, 3, 5, 1] using heap sort:

Original array: [4, 10, 3, 5, 1]

Step 1: Build a max heap
  - First, we represent the array as a complete binary tree:
      4
     / \
   10   3
  / \
 5   1

  - Heapify the tree starting from the last non-leaf node:
      4              10             10
     / \           / \           / \
   10   3    →    4   3    →     5   3
  / \           / \           / \
 5   1         5   1         4   1

Final Max Heap: [10, 5, 3, 4, 1]

Step 2: Sort the array
  - Swap root (10) with last element (1): [1, 5, 3, 4, 10]
  - Reduce heap size to 4 and heapify: [5, 4, 3, 1, 10]
  
  - Swap root (5) with last element (1): [1, 4, 3, 5, 10]
  - Reduce heap size to 3 and heapify: [4, 1, 3, 5, 10]
  
  - Swap root (4) with last element (3): [3, 1, 4, 5, 10]
  - Reduce heap size to 2 and heapify: [3, 1, 4, 5, 10]
  
  - Swap root (3) with last element (1): [1, 3, 4, 5, 10]
  - Reduce heap size to 1 (only one element left)

Final sorted array: [1, 3, 4, 5, 10]

Implementing Heap Sort

JavaScript Implementation

function heapSort(arr) {
  const n = arr.length;
  
  // Build max heap
  for (let i = Math.floor(n / 2) - 1; i >= 0; i--) {
    heapify(arr, n, i);
  }
  
  // Extract elements from heap one by one
  for (let i = n - 1; i > 0; i--) {
    // Move current root to end
    [arr[0], arr[i]] = [arr[i], arr[0]];
    
    // Call heapify on the reduced heap
    heapify(arr, i, 0);
  }
  
  return arr;
}

// Heapify a subtree rooted at node i
function heapify(arr, n, i) {
  let largest = i;        // Initialize largest as root
  const left = 2 * i + 1; // Left child
  const right = 2 * i + 2; // Right child
  
  // If left child is larger than root
  if (left < n && arr[left] > arr[largest]) {
    largest = left;
  }
  
  // If right child is larger than largest so far
  if (right < n && arr[right] > arr[largest]) {
    largest = right;
  }
  
  // If largest is not root
  if (largest !== i) {
    // Swap and continue heapifying
    [arr[i], arr[largest]] = [arr[largest], arr[i]];
    heapify(arr, n, largest);
  }
}

// Example usage
const array = [4, 10, 3, 5, 1];
console.log(heapSort(array)); // Output: [1, 3, 4, 5, 10]

Python Implementation

def heap_sort(arr):
    n = len(arr)
    
    # Build a max heap
    for i in range(n // 2 - 1, -1, -1):
        heapify(arr, n, i)
    
    # Extract elements one by one
    for i in range(n - 1, 0, -1):
        # Move current root to end
        arr[i], arr[0] = arr[0], arr[i]
        
        # Call heapify on the reduced heap
        heapify(arr, i, 0)
    
    return arr

def heapify(arr, n, i):
    largest = i       # Initialize largest as root
    left = 2 * i + 1  # Left child
    right = 2 * i + 2 # Right child
    
    # If left child exists and is greater than root
    if left < n and arr[left] > arr[largest]:
        largest = left
    
    # If right child exists and is greater than largest so far
    if right < n and arr[right] > arr[largest]:
        largest = right
    
    # If largest is not root
    if largest != i:
        # Swap
        arr[i], arr[largest] = arr[largest], arr[i]
        
        # Recursively heapify the affected sub-tree
        heapify(arr, n, largest)

# Example usage
array = [4, 10, 3, 5, 1]
print(heap_sort(array))  # Output: [1, 3, 4, 5, 10]

C++ Implementation

#include <iostream>
#include <vector>

void heapify(std::vector<int>& arr, int n, int i) {
    int largest = i;      // Initialize largest as root
    int left = 2 * i + 1; // Left child
    int right = 2 * i + 2; // Right child
    
    // If left child is larger than root
    if (left < n && arr[left] > arr[largest]) {
        largest = left;
    }
    
    // If right child is larger than largest so far
    if (right < n && arr[right] > arr[largest]) {
        largest = right;
    }
    
    // If largest is not root
    if (largest != i) {
        std::swap(arr[i], arr[largest]);
        
        // Recursively heapify the affected sub-tree
        heapify(arr, n, largest);
    }
}

void heapSort(std::vector<int>& arr) {
    int n = arr.size();
    
    // Build max heap
    for (int i = n / 2 - 1; i >= 0; i--) {
        heapify(arr, n, i);
    }
    
    // Extract elements from heap one by one
    for (int i = n - 1; i > 0; i--) {
        // Move current root to end
        std::swap(arr[0], arr[i]);
        
        // Call heapify on the reduced heap
        heapify(arr, i, 0);
    }
}

int main() {
    std::vector<int> array = {4, 10, 3, 5, 1};
    heapSort(array);
    
    // Print the sorted array
    for (int num : array) {
        std::cout << num << " ";
    }
    // Output: 1 3 4 5 10
    
    return 0;
}

Time and Space Complexity

Time Complexity

  • Build Heap: O(n) - This is not the obvious O(n log n) that you might expect, but can be proven to be O(n)
  • Heapify: O(log n) - Each call to heapify takes O(log n) time
  • Extract from Heap: O(n log n) - We call heapify n times

Overall time complexity for all cases (best, average, worst):

  • Best Case: O(n log n)
  • Average Case: O(n log n)
  • Worst Case: O(n log n)

This consistent performance across all scenarios is a significant advantage of heap sort.

Space Complexity

  • Space Complexity: O(1) - Heap sort is an in-place sorting algorithm
  • The recursive implementation of heapify can use O(log n) stack space, but it can be implemented iteratively to use O(1) space

Advantages and Disadvantages

Advantages

  • Consistent performance: O(n log n) in all cases
  • In-place sorting: Requires only O(1) extra space
  • Efficient for large datasets: Guaranteed O(n log n) time complexity makes it suitable for large arrays
  • Better worst-case: Has better worst-case performance than quick sort
  • No dependence on input: Performance is independent of the initial order of elements

Disadvantages

  • Not stable: Does not preserve the relative order of equal elements
  • Not adaptive: Performance doesn't improve for partially sorted arrays
  • Slower in practice: Often outperformed by quick sort in practice due to better cache locality
  • Complex implementation: More complex than simple algorithms like insertion sort
  • Poor locality of reference: Doesn't access memory in a predictable pattern, causing more cache misses

Comparison with Other Sorting Algorithms

AlgorithmBest CaseAverage CaseWorst CaseSpaceStable
Heap SortO(n log n)O(n log n)O(n log n)O(1)No
Quick SortO(n log n)O(n log n)O(n²)O(log n)No
Merge SortO(n log n)O(n log n)O(n log n)O(n)Yes
Insertion SortO(n)O(n²)O(n²)O(1)Yes
Selection SortO(n²)O(n²)O(n²)O(1)No

Compared to other sorting algorithms, heap sort has the following characteristics:

  • Better worst-case performance than quick sort (O(n log n) vs O(n²))
  • Uses less memory than merge sort (O(1) vs O(n))
  • Unlike quick sort and merge sort, heap sort doesn't require recursion
  • Often slower in practice than quick sort due to cache locality issues
  • Doesn't preserve stability like merge sort and insertion sort do

Applications of Heap Sort

When to Use Heap Sort

  • Limited memory environments: When memory usage needs to be minimized (O(1) extra space)
  • Guaranteed performance: When a consistent O(n log n) performance is required regardless of input distribution
  • Worst-case scenarios: When the worst-case performance is critical (better than quick sort)
  • Priority queue operations: When building a heap is part of other operations, like in a priority queue implementation
  • K-way merging: When you need to merge k sorted arrays (the heap can store the smallest elements from each array)

Real-World Applications

  • K largest/smallest elements: Finding the k largest or smallest elements in an array
  • Priority scheduling: CPU scheduling and other priority-based systems
  • Graph algorithms: Dijkstra's shortest path and Prim's minimum spanning tree algorithms
  • External sorting: Sorting large files that don't fit in memory
  • Media streaming: Buffering and managing media data for streaming applications

Optimizations and Variations

Iterative Heapify

The recursive heapify function can be implemented iteratively to save stack space:

function heapifyIterative(arr, n, i) {
  let largest = i;
  let complete = false;
  
  while (!complete) {
    const left = 2 * largest + 1;
    const right = 2 * largest + 2;
    let newLargest = largest;
    
    if (left < n && arr[left] > arr[newLargest]) {
      newLargest = left;
    }
    
    if (right < n && arr[right] > arr[newLargest]) {
      newLargest = right;
    }
    
    if (newLargest !== largest) {
      [arr[largest], arr[newLargest]] = [arr[newLargest], arr[largest]];
      largest = newLargest;
    } else {
      complete = true;
    }
  }
}

Bottom-up Heap Construction

Starting from the bottom of the heap and working up (as shown in our implementation) is more efficient than inserting elements one by one. This approach has O(n) complexity rather than O(n log n).

Other optimizations include:

  • Cache optimization: Modifying the heap structure to improve cache locality
  • Parallel heap sort: Using multiple threads to process different parts of the heap simultaneously
  • Weak heaps: A variant that reduces the number of comparisons needed
  • Smoothsort: A variation that performs better on already sorted or nearly-sorted arrays

Conclusion

Heap sort is a powerful, in-place sorting algorithm with guaranteed O(n log n) time complexity. While it may not be as commonly used as quick sort in practice due to cache locality issues, its consistent performance and low memory requirements make it valuable in specific scenarios.

Key Points to Remember

  • Heap sort has O(n log n) time complexity for all cases, making it reliable in worst-case scenarios
  • It sorts in-place with O(1) extra space, but is not a stable sorting algorithm
  • The algorithm involves two main phases: building a max heap and repeatedly extracting the maximum element
  • Heap sort is less commonly used in practice than quick sort due to cache performance
  • The binary heap data structure used in heap sort is also fundamental to priority queue implementations

Next Steps

To deepen your understanding of efficient sorting algorithms and related data structures, explore these topics:

Related Tutorials

Related Tutorials

Heaps & Priority Queues

Learn about the binary heap data structure that powers heap sort.

Learn more

Quick Sort

Compare with another efficient sorting algorithm with O(n log n) average time complexity.

Learn more

Merge Sort

Explore another divide-and-conquer sorting algorithm with O(n log n) complexity.

Learn more